Safari iframe Sandbox Policy Regression Test

Build the regression suite as a Playwright test.each matrix, run it against WebKit on a macOS host, and put the runner on Figment's virtual Intel macOS plan at $29/month. Sandbox enforcement is engine-level, not CPU-level, so the Intel VM runs the same WebKit test logic you would run on an M-series machine. Move to an M-series Mac from $95/month only if you also need Xcode 27, iOS Simulator runs, or heavier parallelism — chip, RAM and macOS image are confirmed in the quote.

Why Safari Silently Breaks Sandboxed iframes

The HTML Living Standard defines 13 sandbox tokens for <iframe>. Every current browser implements them, but WebKit's enforcement details diverge from Chromium in ways that surface in production without a dedicated regression suite.

The spec is explicit about two traps that show up often:

• Sandbox flags only take effect when the iframe navigates. Removing or changing the sandbox attribute on an already-loaded frame has no effect, so your fixture must set the attribute before the iframe is inserted or its src changes.
• Setting both allow-scripts and allow-same-origin on a same-origin child lets that child remove the sandbox attribute and reload itself, effectively escaping the sandbox. The spec calls this out as a security antipattern; your matrix should cover it as a row, not rely on it as a default.

A parameterized Playwright run against WebKit on macOS turns this class of bug into a one-line CI failure the moment a token combination regresses.

The 13 Sandbox Tokens — Five That Matter Most in Safari

The spec lists allow-downloads, allow-forms, allow-modals, allow-orientation-lock, allow-pointer-lock, allow-popups, allow-popups-to-escape-sandbox, allow-presentation, allow-same-origin, allow-scripts, allow-top-navigation, allow-top-navigation-by-user-activation and allow-top-navigation-to-custom-protocols.

Five of those carry most of the real-world regression risk:

Because post-load changes are a no-op, your fixture must build the iframe element, set sandbox, set src, then append — in that order.

Building the HTML Test Fixtures

Serve two static files from the same origin so allow-same-origin tests exercise a real same-origin relationship, not an about:blank host. Fixture 1 is the iframe content; fixture 2 is the host page the test loads.

<!-- fixtures/frame.html -->
<!DOCTYPE html>
<html>
<head><title>Sandbox fixture</title></head>
<body>
  <form id="f" action="/echo" method="POST">
    <input name="x" value="1">
    <button type="submit" id="submit-btn">Submit</button>
  </form>
  <script>
    window.parent.postMessage({ event: 'script-executed' }, '*');
  </script>
</body>
</html>

<!-- fixtures/host.html -->
<!DOCTYPE html>
<html>
<head><title>Sandbox host</title></head>
<body></body>
</html>

The host page is intentionally empty — the test creates the iframe in-page so it controls the exact attribute order.

The Parameterized test.each Matrix

Each row specifies the sandbox attribute string, whether a script-executed postMessage should arrive, and whether a POST to /echo should fire. The test clicks the submit button directly rather than calling form.submit() from inside the frame, so the form row is not entangled with the script row.

// tests/sandbox-regression.spec.ts
import { test, expect } from '@playwright/test';

type SandboxRow = {
  sandbox: string;
  expectScript: boolean;
  expectForm: boolean;
  label: string;
};

const matrix: SandboxRow[] = [
  { sandbox: '',                                            expectScript: true,  expectForm: true,  label: 'no-sandbox'          },
  { sandbox: 'allow-scripts allow-forms',                   expectScript: true,  expectForm: true,  label: 'scripts-and-forms'   },
  { sandbox: 'allow-scripts',                               expectScript: true,  expectForm: false, label: 'scripts-only'        },
  { sandbox: 'allow-forms',                                 expectScript: false, expectForm: true,  label: 'forms-only'          },
  { sandbox: 'allow-same-origin',                           expectScript: false, expectForm: false, label: 'same-origin-only'    },
  { sandbox: 'allow-same-origin allow-scripts',             expectScript: true,  expectForm: false, label: 'same-origin-scripts' },
  { sandbox: 'allow-scripts allow-forms allow-same-origin', expectScript: true,  expectForm: true,  label: 'all-three'           },
];

for (const row of matrix) {
  test(`[${row.label}] script=${row.expectScript} form=${row.expectForm}`, async ({ page }, testInfo) => {
    let formPosted = false;
    await page.route('**/echo', route => {
      formPosted = true;
      return route.fulfill({ status: 200, body: 'ok' });
    });

    await page.goto('http://localhost:3000/host.html');

    const scriptPromise = page.evaluate((sandboxAttr: string) => {
      return new Promise<boolean>(resolve => {
        const handler = (e: MessageEvent) => {
          if ((e.data as { event?: string })?.event === 'script-executed') {
            window.removeEventListener('message', handler);
            resolve(true);
          }
        };
        window.addEventListener('message', handler);

        const iframe = document.createElement('iframe');
        iframe.id = 'sandbox-frame';
        if (sandboxAttr) iframe.setAttribute('sandbox', sandboxAttr);
        iframe.src = '/frame.html';
        document.body.appendChild(iframe);

        setTimeout(() => resolve(false), 2500);
      });
    }, row.sandbox);

    try {
      await page.frameLocator('#sandbox-frame')
        .locator('#submit-btn')
        .click({ timeout: 1500 });
    } catch {
      // A missing allow-forms (or allow-scripts, depending on the row) can block the
      // navigation; formPosted stays false unless the POST actually reaches the route.
    }

    await page.waitForTimeout(500);
    const scriptFired = await scriptPromise;

    expect(scriptFired, `script signal for [${row.label}]`).toBe(row.expectScript);
    expect(formPosted, `form POST for [${row.label}]`).toBe(row.expectForm);

    const screenshotPath = testInfo.outputPath(`${row.label}.png`);
    await page.screenshot({ path: screenshotPath });
    await testInfo.attach(`screenshot-${row.label}`, {
      path: screenshotPath,
      contentType: 'image/png',
    });
  });
}

This is reference code to adapt, not a benchmark we executed. Run it against your real fixtures and only adjust expectations when your installed WebKit version demonstrably disagrees — the rows above encode the spec, which is the correct starting point. Any deviation you then observe is the regression this suite exists to catch.

Manual Acceptance Worksheet

Before you trust the automated matrix, walk one row by hand in Safari so you have a human baseline:

1. Open the fixtures/ directory served on http://localhost:3000/ in the Mac VM's browser console session.
2. Load host.html. Open the Web Inspector console (Develop menu).
3. Paste the iframe-creation snippet for sandbox="allow-same-origin" and press Enter. Confirm no script-executed postMessage logs and the form submit button does not navigate when clicked.
4. Repeat for allow-scripts allow-forms. Confirm the message does log and the submit button triggers a POST (the Playwright route is not active here, so you will see a 404 to /echo — that is the signal the POST fired).
5. Save a screenshot of each Web Inspector run and compare to the Playwright per-row screenshots. Any row where manual Safari and Playwright WebKit disagree is a lead on a Playwright-vs-Safari divergence worth filing.

Playwright Config: WebKit on macOS

Target the webkit project on macOS. Per Playwright's browsers documentation, Playwright's WebKit is built from the upstream WebKit main branch with Playwright patches; it is not the branded Safari binary, and Playwright explicitly recommends running WebKit on macOS for the closest-to-Safari behavior because media codecs and platform integrations differ by OS. If you need byte-identical shipping Safari (for example to reproduce a bug a customer only sees in Safari.app), drive the real Safari binary on the same Mac over WebDriver and keep the Playwright suite as your fast first line.

Pin the Playwright version in package.json and record npx playwright --version in each CI log so a WebKit bump is visible when a matrix row flips.

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  webServer: {
    command: 'npx serve ./fixtures -p 3000',
    port: 3000,
    reuseExistingServer: !process.env.CI,
  },
  projects: [
    { name: 'webkit-macos', use: { ...devices['Desktop Safari'] } },
  ],
});

Install the WebKit browser once with npx playwright install webkit. A Linux WebKit run is useful as a smoke check but is not a substitute for macOS; platform-level WebKit behavior is what breaks Safari users.

Choosing a Mac Tier and Worked Cost

Start with the Figment virtual Intel macOS plan at $29/month (2 vCPU, 8 GiB RAM, 80 GiB disk). It runs macOS, installs Node and Playwright normally, and the Playwright WebKit build for Intel macOS runs on it. Access is through a browser-based desktop console; a CI agent installed on the VM dials outbound to GitHub Actions, GitLab or Buildkite, so pushes trigger the suite without exposing inbound SSH. There is no GPU acceleration or audio on this VM, and it is not an Apple silicon Mac — fine for a sandbox matrix, not fine for Xcode 27 or iOS Simulator.

Move to M-series from $95/month if you also need iOS Simulator, Xcode 27 (Apple silicon only), or parallel browser projects. M-series is request-based — request a configuration quote and confirm the chip, RAM and macOS image in writing before you commit.

Annual math for a single-runner WebKit suite at headline monthly rates:

If this matrix is your only macOS workload, the $29 VM is $792/year less than the M-series entry at headline monthly rates. These are monthly figures; annual prepayment, setup fees and taxes are not stated and belong in the quote. $95 is an entry price for an unspecified M-series configuration, not a committed M4/16GB — equivalent hardware is agreed in writing.

Two Safari Failures the Matrix Catches First

Row allow-same-origin with no allow-scripts. Parent-page handlers that assume they can drive the iframe through same-origin DOM access fail here. Fix: add allow-scripts if the parent legitimately needs scripted interaction, or stop reaching into contentDocument and use postMessage with allow-scripts.

Row allow-scripts without allow-forms. Inline scripts run, but a programmatic form.submit() and a user click on the submit button both stop at the navigation boundary. Fix: add allow-forms, or POST via fetch from the parent frame on the child's behalf.

Add a new row every time your app embeds a new third-party widget — payment iframe, survey, comment thread. Each embed gets its own token combination so a vendor change cannot regress your form flow silently.

Running the Suite in CI

# .github/workflows/safari-sandbox.yml
name: Safari sandbox regression
on: [push, pull_request]
jobs:
  webkit:
    runs-on: [self-hosted, macos]   # your Figment Mac runner
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx playwright install webkit
      - run: npx playwright test --project=webkit-macos --reporter=html
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/

The HTML report uploads on every run, including failures, with per-row screenshots. Keep this runner behind a private repo or restrict pull_request triggers to trusted forks — a self-hosted macOS runner executes whatever workflow a PR brings in, so public-repo exposure is a direct code-execution risk on the VM. If the repo must be public, prefer ephemeral runners that are destroyed after each job.

Set up a Mac for your WebKit regression suite or request a configuration.

Related testing guides:

• Safari ITP Cookie Block Test
• macOS Notification Permission UITest
• GitHub Actions Self-Hosted Runner on Mac