Give an Agent Permission to Run Home Scenes
The safest way to give an AI agent access to HomeKit is to decide its limits before the app requests permission. Build an explicit allowlist — scenes the agent runs freely, scenes it asks about first, and scenes that stay human-only — then enforce that classification in code. The matrix below guides those decisions.
How HomeKit permission works for an agent
Apple's HomeKit framework grants access at the app level through a standard iOS or Mac Catalyst permission dialog. The entry point, HMHomeManager, is available on iOS, iPadOS, Mac Catalyst, tvOS, visionOS, and watchOS — not on native macOS outside of Mac Catalyst. An agent embedded in an iOS app or a Mac Catalyst target can request HomeKit access; a plain macOS daemon cannot call this API directly (HMHomeManager).
After the user grants permission, the app can read homes, accessories, and action sets (scenes). It cannot act on scenes beyond what the user configured in the Home app, and it cannot change home structure or add accessories without additional entitlements. The agent's reach is bounded by what is already in the Home app and what the agent code explicitly calls.
Remote access requires a home hub. For background or off-device use, the user's home needs a compatible hub — HomePod, HomePod mini, or Apple TV — signed into the same Apple Account and connected to the home Wi-Fi network. iCloud Keychain and two-factor authentication are prerequisites for remote access to work. iPad is no longer supported as a home hub (Apple home hub requirements). Without a hub, scenes only trigger locally.
There is no Figment-supplied HomeKit connector or scope. The integration below is a developer-owned design using Apple's public HomeKit framework.
Build the allowlist before requesting permission
Define the allowed set before the app calls HMHomeManager, not after. This keeps the agent's reach explicit in code rather than assumed at runtime.
Sort every scene in the home by what it actually does:
- Reversible convenience — lights, speakers, non-critical climate. Undone in seconds; no security surface.
- Mixed — includes at least one lock, garage door, or security-arm action alongside convenience devices.
- Physical access or alert — the primary effect is unlocking a door, triggering a siren, or escalating a security state.
Convenience-only scenes are candidates for free agent execution. Mixed scenes need a confirmation step. Physical-access and alert scenes stay off the agent's list entirely.
Scene and confirmation matrix
Use this matrix to classify your scenes before setup. The three levels map directly to agent behavior: Permit means the agent calls the scene on its own judgment; Confirm means it asks the user first; Exclude means the scene is not in the agent's allowed set at all.
| Scene type | Typical accessories | Level | Reason |
|---|---|---|---|
| Ambience — Movie Night, Reading, Dinner | Lights (dim/color), speakers | Permit | Fully reversible; no security surface |
| Morning Routine | Lights, coffee maker, thermostat | Permit | Reversible convenience; no locks |
| Arrive Home (lights + thermostat only) | Lights, thermostat | Permit | Reversible; exclude this scene if it includes a lock |
| Away / Leave Home (mixed) | Lights off, thermostat setback, door lock | Confirm | Lock state change; agent asks first |
| Good Night (mixed) | Lights off, door lock, arm security | Confirm | Security-critical; requires human confirmation |
| Guest Access | Front door lock or garage door | Exclude | Physical access; always human-initiated |
| Security / Alarm | Siren, alarm arm, cameras | Exclude | Irreversible alert; must stay human-only |
If a scene spans two categories, split it. Create a lights-only "Leave Home – lights" version for the agent and keep the full version — locks and all — for yourself.
Enforcing the allowlist in code
Persist the allowlist as a set of scene identifiers. Use the uniqueIdentifier property of each HMActionSet the user has approved for agent use. At runtime:
- Check
HMHomeManagerAuthorizationStatusbefore attempting any scene call. If access is not.authorized, surface the system permission prompt; do not proceed silently. - Filter through the allowlist. The agent only sees scenes whose identifiers appear in the Permit or Confirm set. Exclude-tier scenes are never surfaced to the agent.
- For Confirm-tier scenes, describe what the scene will do and require an explicit user acknowledgment before calling
HMHome.executeActionSet(_:completionHandler:). - Handle unknown outcomes. The HomeKit completion handler can return an error even when a hub is reachable. Surface failures clearly rather than retrying silently.
Locking a scene out of the allowlist does not prevent you from running it manually in the Home app. It only means the agent cannot reach it.
Narrowing or revoking access
HomeKit permission is app-scoped. To narrow the agent's reach, update the allowlist in your app and redeploy — no Home app change is needed to tighten the confirmation tier. To stop the app from accessing HomeKit entirely, the user revokes access in Settings → Privacy & Security → Home on iOS.
If a scene ends up in the wrong tier — the agent ran an "Away" scene that also locked the front door — update the allowlist and tighten the confirmation logic before the next release.
Figment currently provides iMessage REST access and a synthetic sandbox for Contacts, Calendar, and Reminders for connected Apple Accounts. To explore available scopes and start a connection, see Connect Apple Account.
