Save Agent Job Progress on a Hosted Mac Guide

Long-running automation jobs on a hosted Mac fail for dozens of reasons: network blips, app crashes, OS updates, operator mistakes. Without saved progress, every failure restarts from scratch, burning time, API quota, and money. The fix is a local checkpoint store: each task commits its result before the next one starts, and the resume logic skips anything already marked done. SQLite in WAL mode is the right tool for this on a Mac — it survives process crashes without corruption, needs no server, and ships with macOS.

Which Figment Mac Plan Fits This Job

Start on the Figment virtual Intel macOS VM at $29/month (2 vCPU, 8 GiB RAM, 80 GiB disk). It runs real macOS, so your Python runner, sqlite3, launchd, and compatible Intel Xcode-26-or-earlier toolchains all work. For a checkpointed automation agent that drives HTTP APIs, writes files, and persists state in SQLite, this is the plan to pick — the 80 GiB disk keeps the checkpoint database next to your working files, which is both fast and the only supported SQLite setup (WAL mode does not work over network filesystems).

Move up to an M-series Mac from $95/month if your job must execute Apple-silicon-only binaries, use Xcode 27, drive the iOS Simulator on Apple silicon, or run GPU/ML workloads. Exact chip, RAM and disk at the $95 entry are confirmed on the quote. If you want the checkpointing pattern plus managed CI setup, backups, monitoring and 30 minutes of monthly support on one M-series Mac, the Managed Build Mac is $249/month. Physical M-series availability is request-based; ask for a configuration quote at /mac-hosting/start and we will confirm current supply before you pay.

Inputs and Required Permissions

The agent user only needs read/write access to the working directory. No elevated privileges are required. Keep the database on a local volume — WAL mode requires shared memory between processes on the same host, so SMB/NFS paths are not supported.

The SQLite Checkpoint Schema

-- checkpoint.sql: run once at job startup via `sqlite3 checkpoint.db < checkpoint.sql`

PRAGMA journal_mode = WAL;

CREATE TABLE IF NOT EXISTS job_progress (
    task_id     TEXT PRIMARY KEY,
    status      TEXT NOT NULL
                CHECK(status IN ('pending', 'running', 'done', 'failed')),
    result_json TEXT,                 -- per-task output (JSON text)
    started_at  TEXT,                 -- ISO-8601 UTC, e.g. 2026-11-10T14:00:00Z
    finished_at TEXT,
    attempt     INTEGER DEFAULT 1
);

CREATE TABLE IF NOT EXISTS job_meta (
    key   TEXT PRIMARY KEY,
    value TEXT
);

INSERT OR IGNORE INTO job_meta (key, value) VALUES ('schema_version', '1');

Why WAL? SQLite writes committed changes to a separate -wal file and moves them into the main database during a checkpoint. If the process is killed, the first new connection runs a recovery pass under an exclusive lock and replays committed WAL frames, so the main database file is never left partially written. The default auto-checkpoint fires when the WAL reaches 1,000 pages (~4 MB), and PRAGMA wal_checkpoint(TRUNCATE) forces a full checkpoint and truncates the WAL file to zero. See the SQLite Write-Ahead Logging documentation and PRAGMA wal_checkpoint for the authoritative reference.

Implement the Checkpoint Workflow

At startup, initialize the schema and seed pending tasks. In the main loop, skip done tasks and mark each task before and after execution.

#!/usr/bin/env python3
"""
agent_runner.py — checkpoint-aware job runner
Usage: python3 agent_runner.py --tasks tasks.json [--resume]
"""
import argparse, json, sqlite3, sys
from datetime import datetime, timezone

DB_PATH = "checkpoint.db"

def now_utc():
    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")

def init_db(conn):
    with open("checkpoint.sql") as f:
        conn.executescript(f.read())
    conn.commit()

def seed_tasks(conn, tasks):
    """Idempotent on resume: existing rows are untouched."""
    conn.executemany(
        "INSERT OR IGNORE INTO job_progress (task_id, status) VALUES (?, 'pending')",
        [(t["id"],) for t in tasks],
    )
    conn.commit()

def run_task(task_id: str) -> dict:
    """Replace with your actual work. Must return a JSON-serialisable dict."""
    return {"processed": task_id, "ts": now_utc()}

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--tasks", required=True)
    ap.add_argument("--resume", action="store_true")
    args = ap.parse_args()

    with open(args.tasks) as fh:
        tasks = json.load(fh)
    conn = sqlite3.connect(DB_PATH)
    conn.execute("PRAGMA journal_mode = WAL")

    init_db(conn)
    seed_tasks(conn, tasks)

    rows = conn.execute(
        "SELECT task_id FROM job_progress WHERE status != 'done' ORDER BY rowid"
    ).fetchall()

    if not rows:
        print("All tasks already done.", flush=True)
        sys.exit(0)

    processed_since_checkpoint = 0
    for (task_id,) in rows:
        conn.execute(
            "UPDATE job_progress SET status='running', started_at=? WHERE task_id=?",
            (now_utc(), task_id),
        )
        conn.commit()

        try:
            result = run_task(task_id)
            conn.execute(
                """UPDATE job_progress
                   SET status='done', result_json=?, finished_at=?
                   WHERE task_id=?""",
                (json.dumps(result), now_utc(), task_id),
            )
        except Exception as exc:
            conn.execute(
                "UPDATE job_progress SET status='failed', result_json=? WHERE task_id=?",
                (json.dumps({"error": str(exc)}), task_id),
            )
        conn.commit()

        processed_since_checkpoint += 1
        if processed_since_checkpoint >= 50:
            conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
            processed_since_checkpoint = 0

    conn.close()
    print("Job complete.", flush=True)

if __name__ == "__main__":
    main()

The INSERT OR IGNORE in seed_tasks makes startup safe to re-run. The explicit wal_checkpoint(TRUNCATE) every 50 commits bounds WAL growth so a backup window sees a small, consistent file. This snippet is an illustrative example, not a benchmark — time it against your own tasks before you rely on throughput numbers.

Crash-Replay Acceptance Test

Verify the resume path before you run against production data. The test needs only sqlite3 and the runner above. For the mid-run kill to interrupt deterministically, add import time; time.sleep(0.1) to run_task while testing, then remove it.

#!/usr/bin/env bash
# test_checkpoint_resume.sh
set -euo pipefail

DB=checkpoint.db
TASKS=test_tasks.json
rm -f "$DB" "$DB-wal" "$DB-shm" "$TASKS"

python3 -c "
import json
tasks = [{'id': f'task-{i:02d}'} for i in range(10)]
print(json.dumps(tasks))
" > "$TASKS"

python3 agent_runner.py --tasks "$TASKS" &
RUNNER_PID=$!
while :; do
    done=$(sqlite3 "$DB" "SELECT COUNT(*) FROM job_progress WHERE status='done';" 2>/dev/null || echo 0)
    [[ "$done" -ge 5 ]] && break
    sleep 0.2
done
kill "$RUNNER_PID" 2>/dev/null || true
wait "$RUNNER_PID" 2>/dev/null || true

done_before=$(sqlite3 "$DB" \
    "SELECT task_id FROM job_progress WHERE status='done' ORDER BY task_id;")

python3 agent_runner.py --tasks "$TASKS" --resume

total=$(sqlite3 "$DB" "SELECT COUNT(*) FROM job_progress WHERE status='done';")
if [[ "$total" -eq 10 ]]; then
    echo "PASS: all 10 tasks done"
else
    echo "FAIL: only $total done"; exit 1
fi

done_after=$(sqlite3 "$DB" \
    "SELECT task_id FROM job_progress WHERE status='done' ORDER BY task_id;")
while IFS= read -r tid; do
    grep -qx "$tid" <<<"$done_after" || { echo "FAIL: $tid lost on resume"; exit 1; }
done <<<"$done_before"
echo "PASS: all pre-crash task IDs preserved"

This script asserts two things that matter: every task finishes, and no task completed before the kill was re-executed. Run it before every migration or plan change.

Cost and Operating Limits

A row of job_progress with a small result_json is roughly 100–200 bytes, so a million tasks fit comfortably in well under 1 GiB on the $29 VM's 80 GiB disk. If you store raw HTML or binary blobs in result_json the database grows far faster — write those to files and store only the path.

Because WAL mode keeps recent commits in the -wal file, your backup script must copy checkpoint.db and checkpoint.db-wal together, or run PRAGMA wal_checkpoint(TRUNCATE) first and then copy only the main file.

Start or Migrate the Workload

Pick your plan at Figment Mac Hosting and start a quote at /mac-hosting/start. Once the Mac is ready:

1. Copy agent_runner.py and checkpoint.sql to your working directory on the hosted Mac.
2. Run test_checkpoint_resume.sh and confirm both PASS lines before any production data.
3. Wire the runner into launchd so it restarts automatically after a crash or reboot.

To migrate a job already in progress, stop the runner cleanly, run PRAGMA wal_checkpoint(TRUNCATE), then copy the single checkpoint.db file to the new Mac. If you cannot stop cleanly, copy checkpoint.db, checkpoint.db-wal and checkpoint.db-shm together — losing the WAL loses the most recent committed frames.

Related guides:

• Mac Agent Action Approvals
• Mac Agent Network Scope
• GitHub Actions Self-Hosted Runner on Mac