From a polling Bash script to an event-driven Swift daemon on macOS

The first version of MacSentry's root daemon was a 1,110-line Bash script that polled the system in a loop. It worked, but on my Apple M4 development Mac it was measured at about 10% of one core at one point, and even after two rounds of optimization it still used about 4.5%, all day. The Swift rewrite runs at about 0.4%. This post covers what actually made the difference: forks, polling, cheap change detection, letting macOS deprioritize the daemon, and an undocumented netstat behavior that silently hides connections.

The numbers come from the commit history and were measured on that one machine. Treat them as orders of magnitude, not benchmarks.

Measure per check before rewriting anything

Before the rewrite, the Bash version got two rounds of optimization, and both were guided by timing each check rather than guessing:

• Skipping work when nothing changed (details below) cut the time spent on checks per cycle from about 16.6 s to 8.1 s.
• A fork-free fast path when a list had not changed, no more date and cksum forks, lsof only for suspicious programs, slower polling for checks that rarely change, and no wazuh-control call every 30 seconds brought the daemon from 12.0 s of CPU per 120 s to 4.6 s, roughly from 10% of one core to 4%.

Those rounds showed where the cost was, and that the remaining cost was structural: a shell script cannot read process tables or wait for events without forking something.

Forks were the cost, so read data through APIs

Each ps, lsof or codesign call starts a process, loads it, and produces text that the script then parses with more processes. The Swift daemon reads the same data directly:

Commands are still used where there is no API, such as netstat, profiles, kmutil and sfltool. They run rarely, and the slow ones run on a separate low-priority queue so they never block the main loop. sfltool dumpbtm alone takes 3–5 seconds.

Subscribe to events instead of polling

Polling means paying every few seconds to learn that nothing changed. The daemon now subscribes to what macOS already announces, and every callback only enqueues work:

• FSEvents for LaunchAgents and LaunchDaemons, /etc, ~/.ssh and other persistence folders.
• IOKit for USB devices (IOServiceAddMatchingNotification), power source changes, and sleep and wake (IORegisterForSystemPower).
• DiskArbitration for external volumes, and SCDynamicStore for network, proxy and DNS changes.
• A dispatch source on the spool directory for messages from the Endpoint Security helper.

Two IOKit details are easy to get wrong. A matching notification fires once and then only re-arms after you drain its iterator with IOIteratorNext, including right after registering. And a power callback that receives "can the system sleep?" or "the system will sleep" must answer with IOAllowPowerChange, or macOS waits for it before sleeping.

Events also come in bursts: plugging in one USB hub produces several notifications. A small debounce merges them into one check a second later.

What remains is a 2-second timer for things with no event, such as new TCP connections, and a fallback scan for everything else at slower intervals: 10 seconds, 30 seconds, a minute, five minutes. When the Endpoint Security helper is running, it already reports SSH logins and reverse shell commands as they happen, so the timer polls those less often.

netstat hides sockets when called from a non-Apple binary

This one is not documented anywhere I could find, so take it as observed behavior on recent macOS versions. When netstat -an -p tcp is started by a binary that is not Apple's, even as a child process, sockets owned by other processes are left out of its output. Nothing reports an error; the list is just shorter.

Two changes work around it:

• Combine two sources. The daemon reads sockets per process through libproc, which sees everything when run as root, and also parses netstat, which sees sockets that belong to no process, such as connections launchd holds for sshd before it starts. A connection counts if either source reports it.
• Let launchd run netstat for the app. When the menu bar app has to scan by itself (without the daemon), it submits a tiny launchd job that runs netstat every two seconds and writes the output to a file. Jobs started by launchd were not affected, and the app only reads the file. When the daemon is running, it publishes its scan in a world-readable file under /var/run, so the app does not scan at all.

The general lesson: when a tool's output looks plausible but incomplete, compare it against a second source before trusting it.

Detect change cheaply, then do the expensive work

Most checks ask "did this set of files change?". Hashing every file each time is wasteful, so the daemon first compares a cheap signature: ctime, size and path of each file. Only when that differs does it hash, diff and alert.

ctime instead of mtime is deliberate. utimes(2) lets the owner set the access and modification times to any value, but the inode change time is always set to the current time, so a modified file cannot pretend to be untouched by restoring its old timestamp.

Two more rules prevent alert floods:

• The first run only records a baseline. Installing MacSentry should not report every LaunchAgent that was already there. The same applies when the daemon's state format changes: a version marker triggers a new baseline instead of a burst of false "new" items.
• An empty read is an error, not "everything was removed". Lists that can never be empty (users, applications) skip the pass when they come back empty, and a browser profile that cannot be read is skipped instead of being recorded as an empty baseline.

Let macOS deprioritize the daemon

A security monitor should not compete with the user. Both root jobs are declared as background work in their LaunchDaemon plist:

<key>ProcessType</key><string>Background</string>
<key>LowPriorityIO</key><true/>

launchd.plist(5) describes Background jobs as "processes that do work that was not directly requested by the user", with resource limits applied, and LowPriorityIO marks the daemon's filesystem I/O as low priority. The trade-off is explicit: when the machine is busy, an alert may arrive a few seconds late.

One serial queue removes a class of bugs

Every check and every piece of state runs on one serial dispatch queue. Event callbacks, the timer and the spool watcher only enqueue work. That removes the need for locks entirely, which matters in a program whose whole job is comparing "before" and "after".

It also removed the most frustrating bugs of the Bash version. Shell functions share variables unless every local is declared, and two of them didn't: a helper that read new log lines overwrote a loop variable, so remote-control alerts showed the wrong log path, and the throttling helper overwrote a variable in the connection check. Neither produced an error.

Results

On the same machine, the whole of MacSentry now uses about 0.4–0.8% of one core for the daemon, about 2% for the Endpoint Security helper with exec monitoring on, and about 0.2% for the menu bar app. The Endpoint Security post covers how the helper keeps its share down.

The rewrite was worth it, but most of the gain came from two ideas that work in any language: don't start a process to read data the kernel can give you directly, and don't ask "did anything change?" when the system can tell you.

Summary

• Time each check before rewriting; two rounds of optimizing the Bash daemon showed the remaining cost was structural, not a slow function.
• Forks were the main cost, so read process, socket, login and signing data through libproc, sysctl, utmpx, Security and IOKit instead of parsing command output.
• Subscribe to FSEvents, IOKit, DiskArbitration and SCDynamicStore instead of polling, debounce bursts, and keep slower polling only as a fallback.
• netstat started by a non-Apple binary silently leaves out other processes' sockets on recent macOS versions, as far as I observed, so combine it with a second source before trusting its output.
• Compare a cheap ctime, size and path signature before hashing, record a baseline on the first run, and treat an empty read as an error.
• Declare the daemon as Background with LowPriorityIO, and run all checks and state on one serial queue so no locks are needed.

If you've moved a polling daemon to events on macOS, or found other tools that quietly return partial output, I'd love to compare notes. My email is on the home page.