Run Xcode Performance Tests on a Remote Mac

Performance budgets only work if the machine running your tests is consistent. A developer laptop drifts: background apps, OS updates, and thermal throttling all change wall-clock times from run to run. A dedicated remote Mac gives you the same hardware state on every CI push, so a baseline you record on Monday is still meaningful on Friday.

This guide walks through writing an XCTestCase measure block with XCTClockMetric, XCTCPUMetric, and XCTMemoryMetric, recording a baseline on a remote Mac, committing it to the repository, and running xcodebuild to surface regressions. For tight performance budgets pick a Figment M-series Mac (from $95/month, specific chip and RAM confirmed in the quote); for coarse regression catching the $29 virtual Intel macOS plan is enough. Start a request at /mac-hosting/start.

How XCTest Measure Works

XCTestCase.measure(metrics:block:) runs the closure multiple times and reports aggregate statistics across those iterations. Each XCTMetric collects data independently during the run. Iteration count and manual control are configured through XCTMeasureOptions, whose iterationCount and invocationOptions properties are documented in Apple's framework reference; check the current default for your installed Xcode rather than assuming a fixed number.

After the first passing run in Xcode, you can click Set Baseline in the test results UI. Xcode stores the baseline values inside the project (under the scheme's test action settings, or in a .xctestplan depending on project structure) and associates them with the device or Simulator destination the measurement ran against. On subsequent runs, XCTest compares each metric against its baseline and reports a regression when the configured maximum deviation is exceeded. Confirm the deviation threshold in your own scheme or test plan — do not assume a fixed default.

The three most useful metrics for a CPU-bound path:

A fourth option, XCTStorageMetric, tracks logical bytes written — useful for caching layers but rarely needed for parsing tests.

Writing the Measure Block

The snippet below measures a JSON parsing function. Replace FeedItem with your own Decodable type and provide a sample_feed.json fixture representative of your production payload size. This is an unexecuted example, not a benchmark result.

import XCTest
@testable import MyApp

final class FeedParserPerformanceTests: XCTestCase {

    private var sampleData: Data!

    override func setUpWithError() throws {
        let bundle = Bundle(for: type(of: self))
        guard let url = bundle.url(forResource: "sample_feed", withExtension: "json") else {
            throw XCTSkip("Missing sample_feed.json — add a representative fixture to the test bundle")
        }
        sampleData = try Data(contentsOf: url)
    }

    func testFeedParsingPerformance() throws {
        let data = try XCTUnwrap(sampleData, "sampleData must be loaded in setUp")

        measure(metrics: [
            XCTClockMetric(),
            XCTCPUMetric(),
            XCTMemoryMetric()
        ]) {
            do {
                _ = try JSONDecoder().decode([FeedItem].self, from: data)
            } catch {
                XCTFail("Decoder failed: \(error)")
            }
        }
    }

    // Manual start/stop lets you exclude fixture work from the metric.
    func testFeedParsingPerformanceTight() throws {
        let data = try XCTUnwrap(sampleData)
        let options = XCTMeasureOptions()
        options.iterationCount = 10
        options.invocationOptions = [.manuallyStart, .manuallyStop]

        measure(metrics: [XCTClockMetric()], options: options) {
            self.startMeasuring()
            do {
                _ = try JSONDecoder().decode([FeedItem].self, from: data)
            } catch {
                XCTFail("Decoder failed: \(error)")
            }
            self.stopMeasuring()
        }
    }
}

Crucially, the measure block catches decoder errors and calls XCTFail. A failed build or a throwing decoder must not be silently counted as a fast run.

Establishing a Baseline on the Remote Mac

1. Open the Figment remote Mac through the browser console (verified access path for the $29 VM; the M-series physical machines are quoted per request).
2. Open your project in Xcode on that machine, select the scheme, and run testFeedParsingPerformance with Product → Test.
3. In the test results, expand the performance entry and click Set Baseline.
4. Xcode writes the baseline into the scheme's test action (or into the .xctestplan if you use one) under xcshareddata.
5. Commit the updated scheme or test-plan file so CI reads the same baseline:

git add MyApp.xcodeproj/xcshareddata/xcschemes/MyApp.xcscheme MyApp.xctestplan 2>/dev/null
git commit -m "chore: record XCTest performance baseline on remote Mac"

Baselines are tied to the destination they were recorded on. If you later move from the Intel VM to an M-series Mac, re-record the baseline on the new machine rather than scaling numbers by hand.

Running xcodebuild and Reading Results

The command below runs only the performance test and writes a result bundle for inspection. Pick a destination that actually exists on the host: on the $29 Intel VM use an Intel-compatible Simulator shipped with your Xcode version; on an M-series Mac the ARM iOS Simulator runs natively.

xcodebuild test \
  -project MyApp.xcodeproj \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -only-testing:MyAppTests/FeedParserPerformanceTests \
  -resultBundlePath ./PerfResults.xcresult \
  | tee xcodebuild.log

# Fail the CI job explicitly on xcodebuild's exit status.
test ${PIPESTATUS[0]} -eq 0

When a measurement exceeds the configured maximum relative deviation, Xcode marks the measure test as failed and the result bundle records the measured average, standard deviation, and baseline for each metric. Example shape of the measurement line Xcode prints in the log (your exact wording varies by Xcode version):

Test Case '-[MyAppTests.FeedParserPerformanceTests testFeedParsingPerformance]'
  measured [Clock Monotonic Time, s] average: 0.142, relative standard deviation: 3.2%

Inspect the bundle with xcresulttool:

# Xcode 16+: the legacy JSON "get" invocation still works with --legacy.
xcrun xcresulttool get --format json --path ./PerfResults.xcresult --legacy > PerfResults.json

# Preferred newer flow: export individual payloads by id.
# xcrun xcresulttool export --type file --id <id> --path ./PerfResults.xcresult --output-path ./payload

The structured JSON is not a documented stable contract, so grep for your test name and iterate from there, or open the .xcresult in Xcode for a graphical diff against the stored baseline.

VM vs M-Series: Picking the Right Measurement Host

Virtual Intel macOS, $29/month. 2 vCPU, 8 GiB RAM, 80 GiB disk, running on Linux/KVM with no GPU acceleration and no audio. It is a real macOS environment and runs compatible Intel toolchains (Xcode 26 and earlier per Apple's porting guide), so you can build, sign, and run Intel or universal targets and Simulator runtimes that your Xcode supports. Because the host is shared and virtualised, wall-clock variance is higher than a dedicated machine, which makes it a weak choice for tight ±5% budgets but a reasonable place to catch coarse 2–3× regressions.

M-series Mac from $95/month. An Apple silicon machine; specific chip, RAM, and disk are confirmed in the quote and physical plans collect requests until supply is confirmed. Pick this when you need ARM execution (Xcode 27, ARM-only Simulator runtimes, Metal, Core ML on-device paths), more RAM than 8 GiB, or stricter baselines than the VM can hold. Configuration quote: /mac-hosting/start.

Worked example — which plan pays off? If your performance suite runs in CI about 40 times a day and a 90-second wall-clock swing from VM scheduling causes roughly 1 in 20 runs to need a retry, that is about 2 retries/day × 90 s = 3 minutes of extra CI time per day. Against the $66/month difference between $29 and $95, the M-series only pays back on CI time if your team's rebuild cost (developer wait + runner minutes) exceeds roughly $2.20/day. Use your own retry rate and hourly cost; do not assume the VM is always cheaper in practice.

Combining Performance Tests with Sanitizers

Thread Sanitizer and Address Sanitizer add significant overhead and distort clock metrics, so run them separately. Keep performance tests in their own test plan and invoke xcodebuild twice:

# 1. Performance baseline check (no sanitizers).
xcodebuild test -scheme MyApp \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -only-testing:MyAppTests/FeedParserPerformanceTests \
  -resultBundlePath ./PerfResults.xcresult

# 2. Correctness check with Thread Sanitizer on the rest of the suite.
# Verify the current sanitizer flag with `xcodebuild -help` on your Xcode;
# a dedicated test plan that toggles sanitizers via Scheme → Test → Diagnostics
# is the most portable way to express this.
xcodebuild test -scheme MyApp \
  -destination 'platform=iOS Simulator,name=iPhone 16' \
  -enableThreadSanitizer YES \
  -skip-testing:MyAppTests/FeedParserPerformanceTests \
  -resultBundlePath ./TSanResults.xcresult

A TSan-induced 10× slowdown then cannot contaminate your performance baseline, and a data race in the parsing path still surfaces in the second job.

Related reading:

• Test App Clip Experiences on a Remote Mac
• Audit Xcode Build Settings on a Rented Mac
• Run GitHub Actions on a Self-Hosted Mac Runner