How to monitor cron jobs (and know when they silently fail)

Cron has one job: start a command at a time. It doesn’t check whether the command worked, and it can’t tell you about runs that never started. Nobody looks for those, so they cost the most. The backup that stopped in March gets noticed in June, when someone needs a restore.

The ways a cron job fails

There are two kinds of failure, and you catch them in different ways.

The job runs and fails. The script exits non-zero, a dependency is down, the disk is full or a credential expired. At least something got written somewhere, even if it’s just cron’s mail or a log file.

The job doesn’t run at all. Nothing gets written anywhere. The usual causes:

• The crontab was lost when the server was rebuilt, migrated or replaced by a container that doesn’t run cron.
• Someone edited the crontab and broke the line, or commented it out “for a minute”.
• The machine was off, asleep or out of memory at the scheduled time. Standard cron doesn’t catch up on missed runs.
• The scheduler itself stopped: a disabled systemd timer, a suspended Kubernetes CronJob, or a GitHub Actions schedule that GitHub turned off after 60 days without activity in a public repository.
• The job is still running from last time, and an overlap lock (correctly) skipped this run. Every run after that is skipped too.

And a third, sneakier one: the job runs, exits 0 and does the wrong thing. The backup finished in two seconds because the source directory was empty. The sync processed zero rows.

Why logs and MAILTO aren’t enough

Cron’s MAILTO sends you a job’s output. Log alerts fire on error lines. Both only work when the job runs and writes something, so they can’t see the second kind of failure at all. They also fail open. If mail delivery breaks or the log shipper stops, you get silence, and silence looks exactly like success.

What you want is a signal that has to arrive for everything to be fine. That’s a heartbeat (also known as a dead man’s switch).

How a heartbeat check works

1. You create a check with the job’s schedule and a grace period.
2. The job calls the check’s private URL when it succeeds.
3. If the call doesn’t arrive by the next expected time plus the grace period, the check alerts you.

Now silence is an alert. The job not running, the server being gone, the crontab getting deleted and the job failing all look the same from outside: no ping. You find out within one schedule interval.

Set it up so it can’t lie to you

I’ve seen heartbeat setups go wrong in the same few places over and over.

Ping only after success

A common first attempt pings no matter what:

# Wrong: pings even when the backup fails
0 3 * * * /opt/jobs/backup.sh; curl -fsS https://pingstack.io/ping/YOUR_TOKEN

Use && so the ping only runs if the job exited 0:

0 3 * * * /opt/jobs/backup.sh && curl -fsS -m 10 --retry 3 https://pingstack.io/ping/YOUR_TOKEN > /dev/null

For anything longer than one command, use a wrapper script with set -euo pipefail, so a failure anywhere stops the script before the ping. Without pipefail, pg_dump mydb | gzip > backup.gz succeeds whenever gzip does, even if pg_dump failed. The crontab recipe has a complete wrapper.

Keep the ping URL out of the crontab

Anyone with the URL can report a run, so treat it like a password. Keep it in a file only the job’s user can read, or in your scheduler’s secret store. Keep it out of crontabs, repositories and shell history.

Match the schedule and the time zone

Use the same cron expression for the check as for the job, and the same time zone. Cron runs in the server’s local time unless your cron supports and sets CRON_TZ. Most cloud schedulers, including GitHub Actions and Vercel, run in UTC. A check that expects 03:00 UTC watching a job that runs at 03:00 in New York will alert every night.

Not sure what an expression means? Paste it into the cron expression explainer to see it in plain English with its next 10 run times in any time zone.

Set a grace period longer than the slowest normal run

The ping arrives when the job finishes. If a backup usually takes 20 minutes but sometimes 50, set the grace period to an hour. Check it again as your data grows. A job that slowly gets longer will eventually start alerting, and that’s a good thing, as long as you know that’s why.

Stop runs from overlapping

If a run can take longer than the gap between runs, two copies can fight over the same files. Use flock so a second copy exits instead:

*/15 * * * * flock -n /tmp/sync.lock /opt/jobs/sync-monitored.sh

A skipped run doesn’t ping. So a job that’s stuck holding the lock shows up as a missed heartbeat, and you don’t end up with a pile of stuck processes.

Report a number to catch “succeeded but wrong”

For the third kind of failure, send a value with the ping and alert when it’s out of range. That could be the backup size, rows synced or queue length:

SIZE_MB=$(du -sm /backups/latest | cut -f1)
curl -fsS -m 10 --retry 3 --get --data-urlencode "value=$SIZE_MB" \
  "$PINGSTACK_PING_URL" > /dev/null

In Pingstack, set warning and critical thresholds on the check, for example warning below 500 MB and critical below 100 MB. A backup that “succeeds” with an empty dump then raises an alert.

Beyond crontab

The pattern is the same everywhere: report success at the end, and give the check the scheduler’s schedule and time zone.

• systemd timers: add the ping as ExecStartPost= or in the service’s script. See the systemd recipe.
• Kubernetes CronJobs: ping from the container after the main command succeeds, and remember a CronJob without timeZone uses the controller’s time zone. See the Kubernetes recipe.
• GitHub Actions schedules: add a final step with if: success(). Scheduled runs can start late under load, so allow a generous grace period. See the GitHub Actions recipe.
• Framework schedulers: Laravel, Rails, Celery beat, Sidekiq and others have hooks for after-success callbacks. The recipes index covers 25 schedulers.

Try it

Pingstack’s heartbeat checks support interval and cron schedules, time zones, grace periods, numeric thresholds and named locations for jobs that run on several servers. Free covers 15 checks. Start free and add one to the job you’d least like to fail silently. Send alerts to a person who can fix the job (a shared inbox nobody reads doesn’t count).