Love systemd timers

I haven't used systemd timers enough to disagree, but

> Ambiguous $PATH settings make cron script execution difficult to predict.

What makes you say that? You can set the PATH right in the crontab. Is that harder to "predict" than it being set in /etc/bashrc, ~/.bashrc, ~/.profile, ~/.bash_profile, /etc/systemd/…, or wherever else?

> You might feel cool knowing the scheduling grammar by heart

I've used Linux since 1994 and I don't know it by heart. But luckily it's pre-printed in the crontab as comments:

    # For more information see the manual pages of crontab(5) and cron(8)
    # 
    # m h  dom mon dow   command

You just put numbers aligned with the titles.

The rest of the complaints, sure. Next time I need a cronjob, I'll try it out.

I have a Canon printer, I actually can't trust that their print nozzle won't get jammed up after sitting idle for a while. So I had claude setup a systemd script to print a picture of my dog every week, I ensure it has enough CMYK spectrum to stress the printer. Its a nice surprise every monday as I sit on my desk to see a sudden picture pop up from the printer :)

Moved from cronie to systemd timers because they are resilient to system startup times. My backup strategy is to create a borg archive entry every day at a fixed time. With cronie the system needs to be running at the scheduled time, but systemd timer tolerates this and runs the service as soons as the system is available.

Btw this is my repo for the backup automation: https://github.com/gchamon/borg-automated-backups

I love systemd timers! I've slowly moved all of my ansible-deployed cron jobs to timers (now just an ansible copy!). The integration with journalctl, especially in a newer OS like Debian 13 where syslog is gone, is really nice. It's also really nice to be able to start the service manually for debug. Having a cron job that didn't work was an annoying exercise in copy/pasting or writing an extra shell script. Don't even get me started on the black hole of cron job stdout. I can monitor systemd services like I already do and get a notification on failure.

I've noticed more and more open source projects recommending timers as a deployment method and I think that's great!

Timers can work with arbitrary units (not just a similarly-named service unit) so they can be surprisingly flexible. I have a timer on my servers that starts a backup.target that fires off a full "restic backup","restic prune", "restic forget" backup cycle each morning with randomized start times and notifications. The actual restic-* units are Podman Quadlets so the whole setup runs agnosticaly of what's on the server, just as long as it has Podman and Systemd installed.

I will admit thought, timers are up there in terms of being the clunkiest systemd unit type to use on a regular basis. I get why they're split up into two files and require different start vs enable syntax's, but man sometimes I just want to create a file that runs a script and be done with it.

I wonder what happened with the heading, it was okay before, and then was mutilated since.

It may be a disastrous comment to make but I think I like cron better! A tool designed for a particular job etc.... :/

NixOS comes with systemd, so I've been using it as a first-class part of managing stuff. It's great, especially coming from macOS' launchd.

Which makes it nice to distribute a tool for NixOS so that it can lean into systemd instead of as some bolted-on afterthought.

Makes me wonder what you'd do if you were distributing a lifecycle-heavy tool for Linux users in general since systemd isn't ubiquitous.

I use a systemd timer to run a monthly scrub for my btrfs pool. Kinda cool how you can do increasingly useful things like skip the next scheduled event if the user initiates a scrub, do or don't accumulate tasks if you have a monthly task but the machine was offline for 6 months -- or fold them into a single task, etc.

I believe one of the major distro lines (redhat or debian, I forget which) uses systemd-cron, where cron is just a thin wrapper around systemd. You get more power from writing the unit files directly, but if all you ever need is a simple cron job, you have the old interface still available.

We have used cron perfectly fine for decades and it served us well within its very clear limitations.

But now obviously we were so blind and wrong all this time and the only true solution is of course systemd.

I will use what I am comfortable with and so should others. CronD, SystemD, atD, multiple conditional checks in a shell script, whatever tickles your fancy. There is no wrong answer, just document what you did and add a comment. Comments are permitted in cron. If someone keeps putting complex obfuscated time structures into cron make them decipher their incantation and keep nagging them until they keep it simple, comment their cron entries or until they and their manager resign.

For what it's worth there are usually web apps popping up that can decipher goofy cron time/date incantations. [1] This one has a git repo in the top right, not my repo. Maybe clone it just in case their site goes away some day.

[1] - https://crontab.cronhub.io/

I've been using Linux for over 20 years, systemd for over 10.

Yet there's always something new to learn and actually consider as another useful tool.

I haven't always been the biggest fan of systemd in some regards, but I will say that I mostly agree with this sentiment. I've almost completely quit using cron, and now favor systemd timers for scheduled jobs - at the "system" level anyway. I might still embed Quartz for scheduling that's scoped to a particular application or something.

Why? It's one of those fuzzy and somewhat hard to explain things. The systemd approach just maps more cleanly to my mental model of "how things should work" I guess. And maybe some of it is that I did indeed experience plenty of " Ambiguous $PATH settings make cron script execution difficult to predict" in the past, although it's not just that.

I won't sit here and claim that systemd timers are necessarily better than cron in any universal / objective sense. But they've won me over, for what it's worth.

I've converted all my crons to systemd timers+services over the past year but cant help but think it's sort of.. less tangible than cron

Like imagine trying to explain systemd timers and services and unit files to a beginner.

This is a very good intro to systemd timers -- I think you convinced me to finally start using them. Love the "list-timers" thing as well. With cron, it never seemed easy to me to get a picture of all the cron jobs running on a box. I'd need to check crontab for all users, as well as /etc/cron.d/, as well as the daily/hourly/monthly directories.

And in fact I do have a use-case for needing to run something ~5 minutes after the system boots and then every ~12 hours onward from there. It's great that systemd timers has me covered!

systemd is complex on first view, but after using it you didn't want to use anything else. It's handy to manage everything using systemctl

After years of using orchestration tools like airflow and dagster so many lightbulbs have just lit up in my head.

I wish documentation for tools would explain their abstractions concepts in terms of its primitives.

Great post, thanks!

Oh I love them quite a lot! I use them to run all of our backup jobs, easy to set up and have never had an issue.

And you immediately lose the ability to do `crontab -l` on any server to know its scheduled tasks.

Now you get to look around the myriad of places where you can put systemd files, and figure out which ones are base services and which ones are custom, with no general convention to go about it. Nope.

> Prime Time for a Timer Primer

It's pronounced, "primmer."

I've been almost convinced by systemd (and have switched to using it), but God the syntax of those service files is so ugly ...

Does systemd ship with something to upgrade your cron jobs for you? That would be the friendly way. Write your old school cron jobs, and then a script that converts them to do things the systemd way, documenting its steps, i.e. I created this file and this is why. Friendly "I help you do things better" rather than standoffish "your way is obsolete, you need to do it our way". Oh wait. I get it. LLM agents can do exactly that for you can't they. Another way I'm behind the curve.

I have knocked together a systemd service or three based on google copypasta. But generally, for cron jobs, why make it complicated? One line in /etc/crontab and done. I generally call an encapsulation script that sets the right environment variables, uses absolute paths, captures stdout/stderr if required and so on. I just want the simplest possible way to launch that script on a schedule.

I setup a few systemd timers last year and created https://systemd.guru to play with different options in OnCalendar expressions.

That would assume I like systemd in the first place.

Is there a way yet to force-trigger a timer? There wasn't the last time I used them, which I found to be super annoying for testing them.

I'm fully ready to drink the "just let systemd do all the things" kool-aid, but I would love to see some sort of introductory/tutorial info into some of the things it can do other than services - i.e. containers and timers. I know man pages exist, but it would be nice if there was more scannable intro out there.

There's another big feature: You're not relying on the time zone to which the server was set (like with cron) but can explicitly specify a time zone:

https://www.freedesktop.org/software/systemd/man/latest/syst...

> stdout and stderr output often ends up in a black hole

Ain't that the truth. Literally every crontab I've written for the last 10 years has had this in it:

2>&1 | logger -t cron-WHATEVER

...and that does a pretty good job of capturing anything that the script emits and making it easy to grep for in syslog the following morning.

But I'm still amazed at how many crontabs I run across that don't capture any output at all.

Even better is systemd socket activation.

> humble systemd

This is actually something that I like in systemd.

I am dealing with mostly non systemd system: BSD, Alpine, termux On BSD anacron works well, but I do not why I am always running into problems with the cronie anacron implementation. And it is very hard to debug.

I would really like a simple modern cron/anacron alternative.

Cronicle looked cool but it is node.js, a bit heavy and being replace now by their new product called xyOps anyway.

In decades of trying, I do not believe there was one time that I ever got a cron job to work properly in the first attempt. Systemd timers are a godsend.

You will love SystemD [0] timers until they fuck you over in an entirely inscrutable way and the SystemD maintainers don't care to either fix the problem or update the docs to warn of the shortcoming.

One of our customers called in with a production down incident caused by a full disk. We got a copy of the VM and took a look. Investigation revealed that / was full because /var/log was full and that our 'logrotate' timer unit that was scheduled to run once a day had run either exactly never or exactly once... I can't remember which. Further investigation revealed no difference in software load or configuration between this VM and a VM that had a functional logrotate timer unit. Exactly one VM out of hundreds of identical VMs at this site (and many multiples of that at other customer's sites) were affected by this. Advising the customer to clear out /var/log and reboot did not unstick 'logrotate', and none of the diagnostics or fixes we could find anywhere unstuck it. Once "systemd-crond" decided to never schedule this job ever again, it stuck to that decision.

After a lot of searching, we found an open bug report from a year or three prior where someone reported exactly the same symptoms and was scheduling a unit with pretty much the same set of unit configuration flags that we were using. The conversation from the core devs ran through the pattern that one gets used to seeing when one runs into SystemD bugs that are caused by extremely complex unanticipated interactions between parts of the project: "That's not a bug, only an idiot would want that to work.", "Oh, we don't document that that's not supposed to work?", "Wow, okay, yeah, I can see how that maybe should work. That it doesn't sure does seem weird.", "Having said that, I don't know if it's supposed to work, or if it's unsupported. Someone should really either document that or fix it."... and then the behavior is neither fixed nor documented. [1] Absent any actual explanation for the failure, we ended up swizzling the options in our 'logrotate' unit and praying that satisfied whatever gremlin arose from the depths to trouble our customer.

SystemD contains an enormous -and ever-growing- amount of accidental complexity, and has a set of core maintainers who are generally disinterested in either documenting the places where one or more complex systems bind together to cause stop-the-world problems or fixing the systems involved so that they don't bind up. It's a fine project until it's very, very suddenly not, and then you're absolutely SOL. If you're lucky, you can shuffle around what you're doing [2] and hope that avoids the problem. [3]

[0] Some folks use the spelling "SystemD" to mock the project. I use the spelling "SystemD" to distinguish between "the entire systemd project" and systemd(1). I do this because some folks will make a claim like "systemd is very, very small and self-contained. I don't understand why anyone would say otherwise.", but what they are actually saying is that systemd(1) is a fairly small program that doesn't do all that much when run as PID 1. It sucks minor amounts of ass that the project and the program it runs as PID 1 share the same name, but what can you do?

[1] No, I don't have a link to the open bug report. This was more than a year ago, so the bug ID has been long forgotten.

[2] The term of art for this practice is "wave a dead chicken at it".

[3] Plus, like, even disregarding most of the rest of my report... how in the hell do you design a cron that knows a job is scheduled to be run periodically, can tell you how long it has been since it last ran, but never manages to run it? To me, that's unforgivable. It's a "You had one job!"-tier cockup.

systemd is great all around. Don’t listen to the boomers complaining about it because their cheese was loved.

[dead]