iOS Simulator Screenshot Regression Archive

Short answer: Boot each simulator with xcrun simctl boot, capture with xcrun simctl io <UDID> screenshot, compare against a per-alias baseline with ImageMagick's SSIM metric, and attach any diff as an XCTAttachment with .keepAlways lifetime so Xcode keeps the artefact in passing bundles. For hosted execution, Figment's M-series Mac from $95/month is the sensible starting point for a multi-device simulator matrix; exact chip, RAM and disk are confirmed in the quote. Apple documents the simctl surface at Running your app in Simulator or on a device and attachments at XCTAttachment.

Choosing Simulator Device UDIDs for a Regression Matrix

A meaningful matrix covers the screen sizes and OS versions your app ships against. List the simulators your host currently has installed:

xcrun simctl list devices available --json | \
  python3 -c "
import json, sys
data = json.load(sys.stdin)
for runtime, devices in data['devices'].items():
    for d in devices:
        print(runtime.split('.')[-1], d['udid'], d['name'])
"

Pick one device per canonical viewport to keep runs tractable:

Record the pairs in sim-matrix.txt at the project root, one tab-separated alias and UDID per line (se3\t<UDID>). Adding a device later means editing one line instead of the pipeline.

Screenshot Command and File-Naming Convention

xcrun simctl io captures the current framebuffer of a booted simulator:

xcrun simctl io <UDID> screenshot <output-path>.png

For an archive that sorts chronologically in a directory listing, encode project slug, build identifier, device alias and orientation:

screenshots/<slug>/<build>/<alias>_<orientation>_<epoch>.png

Boot the device explicitly, wait for boot completion, launch the app and give it a moment to render before capturing. Shut down afterwards so the next run starts from a clean simulator state:

xcrun simctl boot "$UDID"
xcrun simctl bootstatus "$UDID" -b          # blocks until fully booted
xcrun simctl launch "$UDID" "com.example.MyApp"
sleep 2                                     # tune per your app's first-frame time
xcrun simctl io "$UDID" screenshot "$OUT_PATH"
xcrun simctl shutdown "$UDID"

Baseline Storage: git-lfs vs S3

Under about 100 baseline images, git-lfs is simpler. Add a .gitattributes rule and a dedicated directory:

# .gitattributes
baselines/**/*.png filter=lfs diff=lfs merge=lfs -text

For many device targets or heavy branching, store baselines at s3://your-bucket/baselines/<slug>/<branch>/<alias>.png and download only the matching baseline per device alias before comparing.

The Regression Pipeline Script

This is the core artefact: a single shell script that reads sim-matrix.txt, captures fresh screenshots, diffs each against its baseline with ImageMagick's SSIM metric, and writes per-run results. Save as scripts/screenshot-regression.sh; it is an unexecuted example — run it against your own project and review before adoption.

#!/bin/bash
set -euo pipefail

SLUG="${1:?Usage: $0 <slug> <build-id> <baselines-dir>}"
BUILD="${2:?missing build-id}"
BASELINE_DIR="${3:?missing baselines-dir}"
THRESHOLD="${THRESHOLD:-0.985}"
RESULTS_DIR="screenshot-results/${BUILD}"
FAILURES=0

mkdir -p "$RESULTS_DIR"

while IFS=$'\t' read -r ALIAS UDID; do
  [[ -z "${ALIAS:-}" || "$ALIAS" =~ ^# ]] && continue

  OUT="$RESULTS_DIR/${ALIAS}_portrait.png"
  BASE="$BASELINE_DIR/${ALIAS}_portrait.png"

  echo "Booting $ALIAS ($UDID)..."
  xcrun simctl boot "$UDID" 2>/dev/null || true
  xcrun simctl bootstatus "$UDID" -b
  xcrun simctl launch "$UDID" "com.example.${SLUG}"
  sleep 3
  xcrun simctl io "$UDID" screenshot "$OUT"
  xcrun simctl shutdown "$UDID"

  if [[ ! -f "$BASE" ]]; then
    echo "  No baseline for $ALIAS; seeding from this run"
    mkdir -p "$(dirname "$BASE")"
    cp "$OUT" "$BASE"
    continue
  fi

  # magick compare writes the metric to stderr; keep exit status separate
  SSIM=$(magick compare -metric SSIM "$BASE" "$OUT" null: 2>&1 || true)
  echo "  $ALIAS SSIM=$SSIM (threshold=$THRESHOLD)"

  if python3 -c "import sys; sys.exit(0 if float('${SSIM}'.split()[0]) >= float('${THRESHOLD}') else 1)"; then
    echo "  PASS"
  else
    echo "  FAIL -- writing diff image"
    magick compare "$BASE" "$OUT" "$RESULTS_DIR/${ALIAS}_diff.png" || true
    FAILURES=$((FAILURES + 1))
  fi

done < <(awk -F'\t' 'NF==2 && $1 !~ /^#/' sim-matrix.txt)

echo "Regression complete: $FAILURES failure(s)"
# Cap exit code at 255 so shells report it correctly
if (( FAILURES > 255 )); then exit 255; else exit "$FAILURES"; fi

Keep the SSIM tolerance in a separate screenshot-regression.conf that CI sources before invocation, so per-project tuning does not require script edits.

Surfacing Failures as Named XCTest Attachments

Wrap the shell step in a test and attach the diff so it appears in Xcode's Report Navigator. Confirm the initializer overload for your Xcode version against XCTAttachment — contentsOfFile takes a String path, so pass .path from a URL:

func attachRegressionDiff(named alias: String, diffPath: URL) {
    guard FileManager.default.fileExists(atPath: diffPath.path) else { return }
    let image = XCTAttachment(contentsOfFile: diffPath.path)
    image.name = "RegressionDiff-\(alias)"
    image.lifetime = .keepAlways
    add(image)
}

func testScreenshotRegression() throws {
    let result = try runShellScript("scripts/screenshot-regression.sh",
                                    args: ["myapp", buildID, baselinesDir])
    if result.exitCode != 0 {
        for alias in deviceAliases {
            let diff = resultsDir.appendingPathComponent("\(alias)_diff.png")
            attachRegressionDiff(named: alias, diffPath: diff)
        }
        XCTFail("Screenshot regression: \(result.exitCode) failure(s). See attachments.")
    }
}

The attachment lands in the .xcresult bundle and can be extracted with xcrun xcresulttool get --type file on the host.

Scheduling Nightly Runs with launchd

A launchd user agent triggers the pipeline each night on the hosted Mac without pulling in cron:

<!-- ~/Library/LaunchAgents/com.example.screenshot-regression.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key><string>com.example.screenshot-regression</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/bash</string>
    <string>/Users/ci/project/scripts/screenshot-regression.sh</string>
    <string>myapp</string>
    <string>nightly</string>
    <string>/Users/ci/project/baselines</string>
  </array>
  <key>StartCalendarInterval</key>
  <dict><key>Hour</key><integer>2</integer><key>Minute</key><integer>0</integer></dict>
  <key>StandardOutPath</key><string>/tmp/screenshot-regression.log</string>
  <key>StandardErrorPath</key><string>/tmp/screenshot-regression-err.log</string>
</dict>
</plist>

Load it with launchctl load ~/Library/LaunchAgents/com.example.screenshot-regression.plist. The agent persists across reboots while the user session is active.

Hosted Mac Sizing and a Price Worksheet

Each booted iOS Simulator typically takes roughly 1–2 GB of RAM and one active core during capture. A four-device matrix run sequentially fits on modest hardware; parallelising beyond four devices turns RAM into the first constraint.

For a full iOS/iPadOS simulator matrix and ARM-only runtimes, pick Figment's M-series Mac from $95/month — chip, RAM and disk are confirmed when you request the configuration. If your matrix only needs older iOS runtimes under a compatible Intel Xcode (Xcode 26 or earlier, per Apple's Porting your macOS apps to Apple silicon), the $29 virtual Intel macOS environment can cross-compile and run supported Intel toolchains with 2 vCPU, 8 GiB RAM and 80 GiB disk; it has no GPU acceleration or audio. The Managed Build Mac at $249/month adds one M-series Mac, CI setup, backups, monitoring and 30 minutes of monthly support, which can replace the launchd wiring above when you want the pipeline handed off.

Quick entry-price comparison for a single hosted Mac running this pipeline:

CloudMac's annual rate is lower than 12×$95 on entry price; Figment's $95 is an entry M-series without a published chip, so treat this as an entry-price line, not an equal-hardware comparison. If you need the specific managed CI work — baseline seeding, launchd wiring, nightly artefact review — that sits inside the Managed Build plan, not on a bare rental. Tax and fees apply as stated in each offer.

Request a configuration quote at /mac-hosting or begin setup at /mac-hosting/start.

Related guides:

• Safari Sequoia vs Ventura Comparison
• WebKit Pixel Diff Regression Testing
• GitHub Actions Self-Hosted Runner on Mac