Resume Large File Uploads from a Hosted Mac

A large upload that fails at 90% and restarts from zero wastes a day of bandwidth. The fix is to checkpoint at the protocol level and resume from the last confirmed byte. This guide sets that up on a hosted Mac for three common cases: AWS S3 (native multipart), general cloud storage via rclone, and a direct HTTP endpoint driven by a checkpoint script.

Pick a Figment Mac plan for the job

For a pure network-bound push — video exports, archives, dataset snapshots that are already prepared — Figment's $29/month virtual Intel macOS environment (2 vCPU, 8 GiB RAM, 80 GiB disk) is the useful starting point. It is a real macOS environment, not an SSH tunnel to another machine, and it runs compatible Intel Mac toolchains including supported Xcode 26 or earlier. Access is through a browser console; the AWS CLI, rclone, and curl all install with Homebrew the same as on a laptop. See the current plan listing on Figment Mac Hosting.

Pick a larger tier when the Mac itself is doing work before the upload:

• Local rendering, encoding, or packaging before the push → $49 i3 Mac or $95+ M-series (both are request-based; the exact chip, RAM, and disk on the $95 entry tier are set in the quote, not pre-advertised).
• Signed CI artifacts uploaded as part of a build pipeline → $249 Managed Build Mac (one M-series Mac with CI setup, backups, monitoring, and 30 minutes of monthly support).

The $29 plan is billed monthly in advance; a two-week transfer still pays one month. Size the plan around the longest job you expect to run, not the shortest.

Inputs, credentials, and the variables you will reuse

Pin these down on paper before you write any shell:

Keep credentials in the macOS Keychain or in your CI secret manager. If you shell them out, write ~/.aws/credentials with chmod 600 or inject AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY at job start from the secret store — do not dump the whole environment to a file and do not paste secret literals into shell history.

Multipart upload checkpoint and recovery checklist

Run these before you start a long transfer:

Pre-upload

• Free disk space is at least 10% above the source size, for any temp packaging.
• Credentials work: aws s3 ls s3://your-bucket/ returns without error.
• Destination bucket and prefix exist and are writable.
• Checkpoint path is writable: touch "$CHECKPOINT_FILE".
• Part size chosen: 64 MB for files over 1 GB on a reliable link; 8–16 MB on constrained links.
• If you are driving s3api by hand, the upload ID from create-multipart-upload is saved.

After an interruption

• List incomplete uploads: aws s3api list-multipart-uploads --bucket your-bucket.
• List completed parts: aws s3api list-parts --bucket your-bucket --key your-key --upload-id YOUR_UPLOAD_ID.
• Either resume part-by-part with s3api upload-part + complete-multipart-upload, or start again with aws s3 cp (it will open a new multipart upload).
• Verify the final size: aws s3 ls s3://bucket/key against wc -c < source-file.
• Abort stale incomplete uploads older than 48 h so they stop accruing storage: aws s3api abort-multipart-upload.

Three concrete upload patterns

Pattern 1 — AWS S3 with automatic multipart

The AWS CLI's high-level aws s3 commands automatically perform a multipart upload for large objects. Per the AWS CLI User Guide, "you can't resume a failed upload when using these aws s3 commands"; a re-run opens a new multipart upload rather than continuing the previous one. For true resumption of a specific upload ID, drive s3api directly.

Set chunk size and parallelism in ~/.aws/config:

[default]
s3 =
  multipart_threshold = 64MB
  multipart_chunksize = 64MB
  max_concurrent_requests = 4

Then upload:

aws s3 cp /path/to/large-file.zip s3://your-bucket/backups/large-file.zip \
  --no-progress \
  2>> ~/Library/Logs/upload.log

If it fails, re-run — or, if you need to pick up exactly where the previous attempt stopped, list the parts with aws s3api list-parts and upload only the missing part numbers against the same upload ID, then call complete-multipart-upload.

Pattern 2 — rclone for S3, GCS, B2, SFTP, and friends

rclone supports dozens of backends and exposes explicit retry controls. The rclone global flags include --retries (high-level operation retries), --retries-sleep (duration between retries), and --low-level-retries (individual HTTP request retries). The S3 backend's chunk size flag is --s3-chunk-size. rclone does not resume an in-progress multipart upload at a byte offset; what it does well is retry transfers and, for directory copies, skip files already present at the destination with matching size and mtime.

rclone copy /path/to/large-file.zip remote:your-bucket/backups/ \
  --s3-chunk-size 64M \
  --retries 5 \
  --retries-sleep 30s \
  --low-level-retries 10 \
  --progress \
  2>> ~/Library/Logs/rclone-upload.log

Run rclone help flags on your installed version before trusting a script you copied — flag names and defaults do move between releases.

Pattern 3 — a byte-offset checkpoint script for HTTP endpoints

When the destination is your own HTTP API that accepts chunked PUTs, keep the last confirmed part number on disk and resume from it. The script stops immediately on the first failure so a failed job is never counted as a completed one.

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:?Usage: upload.sh <file> <url>}"
DEST_URL="${2:?}"
PART_SIZE=$(( 64 * 1024 * 1024 ))   # 64 MB
CHECKPOINT="${SOURCE}.upload_state"
LOG="${HOME}/Library/Logs/upload.log"

TOTAL=$(wc -c < "$SOURCE" | tr -d ' ')
PARTS=$(( (TOTAL + PART_SIZE - 1) / PART_SIZE ))

LAST_PART=0
[[ -f "$CHECKPOINT" ]] && LAST_PART=$(cat "$CHECKPOINT")

echo "$(date -u +%FT%TZ) start $SOURCE bytes=$TOTAL parts=$PARTS resume_from=$LAST_PART" >> "$LOG"

for (( part=LAST_PART; part<PARTS; part++ )); do
  OFFSET=$(( part * PART_SIZE ))
  HTTP_CODE=$(curl -sS -o /dev/null -w "%{http_code}" \
    -X PUT "${DEST_URL}/part?n=${part}&offset=${OFFSET}" \
    -H "Content-Type: application/octet-stream" \
    --data-binary "@-" \
    < <(dd if="$SOURCE" bs="$PART_SIZE" skip="$part" count=1 2>/dev/null))

  if [[ "$HTTP_CODE" =~ ^2 ]]; then
    echo "$((part + 1))" > "$CHECKPOINT"
    echo "$(date -u +%FT%TZ) part $part ok ($HTTP_CODE)" >> "$LOG"
  else
    echo "$(date -u +%FT%TZ) part $part FAILED ($HTTP_CODE) stopping" >> "$LOG"
    exit 1
  fi
done

rm -f "$CHECKPOINT"
echo "$(date -u +%FT%TZ) complete $SOURCE" >> "$LOG"

Adapt the URL, headers, and completion call to your API. The checkpoint file persists across runs on the same host, so a re-run picks up at the last confirmed part. Note the dd form uses bs=$PART_SIZE skip=$part count=1 — this is orders of magnitude faster than bs=1 with a large skip.

A small worked worksheet

Say the source is a 42 GB archive and the link holds 200 Mbit/s steady, and you set PART_SIZE = 64 MB.

• Parts: ceil(42 * 1024 / 64) = ceil(672) = 672 parts.
• Theoretical transfer time at 200 Mbit/s ≈ 42 * 1024 * 8 / 200 = 1720 s ≈ 28.7 min (ignoring TLS and per-request overhead).
• A single-part restart after a drop at 90% would re-send ~38 GB. With 64 MB parts and a checkpoint, a restart re-sends at most one part — about 64 MB, roughly 2.6 s of the same link.

That ratio is the whole reason multipart exists. The $29 VM is fine for the push itself; your bottleneck will be the link, the API's per-part rate limits, and the destination's part-size minimums (S3 requires 5 MB minimum per part except the last).

Schedule recurring uploads with launchd

For nightly exports, drop a LaunchAgent plist in ~/Library/LaunchAgents/com.example.nightly-upload.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.example.nightly-upload</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/bash</string>
    <string>/Users/agent/bin/upload.sh</string>
    <string>/Users/agent/exports/nightly.zip</string>
    <string>https://api.example.com/uploads/nightly</string>
  </array>
  <key>StartCalendarInterval</key>
  <dict>
    <key>Hour</key><integer>2</integer>
    <key>Minute</key><integer>0</integer>
  </dict>
  <key>StandardOutPath</key>
  <string>/Users/agent/Library/Logs/nightly-upload.log</string>
  <key>StandardErrorPath</key>
  <string>/Users/agent/Library/Logs/nightly-upload-error.log</string>
</dict>
</plist>

Load it with launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.example.nightly-upload.plist. See Apple's Creating launchd Jobs for StartCalendarInterval semantics; consult the launchd.plist man page on the target macOS version for the current behavior around sleep and missed intervals rather than assuming it.

Start on Figment or migrate the workload

To start fresh: open Mac Hosting, pick the $29 virtual Intel macOS plan (or a larger tier if the Mac also packages the payload), open the browser console, install the AWS CLI or rclone via Homebrew, load credentials from your secret manager, and test a 1 GB transfer before you point the real job at it.

To migrate from another host: copy the upload script and the launchd plist, re-register credentials on the new host, run a small test upload end to end, then switch the cron/launchd source and point your producers at the new destination.

Start your Mac →

Related guides:

• Network recovery for Mac automation agents
• Human review steps in Mac automation workflows
• GitHub Actions self-hosted runner on a Mac