Turn New iCloud Photos into Useful Agent Inputs

iCloud Photos gives each asset a full metadata record before the original image file downloads to the device. A Mac helper that queries Apple's Photos framework can read that record—creation date, GPS coordinates, media subtype, favorite flag—and hand a structured JSON object to your agent without loading a single pixel. This page shows the field map, explains how to detect cloud-only originals correctly, and walks through a polling design and three workflows you can build from that foundation.

Figment does not currently offer a public Photos REST endpoint. The design below is a developer-owned Mac process that uses Apple's PhotoKit API (available on macOS 10.13+). The structured-record format is the core artifact; the Swift snippets are design references, not production-tested code.

Photos carry more signal than a filename

Apple's PHAsset class represents each library asset as an object whose metadata properties are available locally even before the full image file is present. For most agent tasks—routing, tagging, filtering—you never request the image at all.

mediaSubtypes is the fastest routing signal. A photoScreenshot flag identifies expense candidates before any OCR runs. sourceType lets you watch only shared-album contributions without touching the personal camera roll.

localIdentifier is device-local: it identifies an asset on the Mac where the fetch runs but is not a portable cross-device ID. For cross-device stable references, map localIdentifier to a PHCloudIdentifier using PHPhotoLibrary.cloudIdentifierMappingsForLocalIdentifiers(_:). Cloud-identifier lookups can fail; always handle the error case.

The photo-to-structured-record recipe

This JSON shape is the artifact your Mac helper produces. Each field maps back to its PHAsset source.

{
  "assetId":          "ABC123…",             // PHAsset.localIdentifier (device-local)
  "mediaType":        "image",               // PHAsset.mediaType → "image" | "video" | "audio"
  "mediaSubtype":     "photoScreenshot",     // PHAsset.mediaSubtypes, first matched flag
  "creationDate":     "2026-11-10T14:32:00Z",// PHAsset.creationDate, ISO-8601 UTC
  "modificationDate": "2026-11-10T14:35:00Z",// PHAsset.modificationDate
  "addedDate":        "2026-11-10T14:32:01Z",// PHAsset.addedDate
  "location": {                              // PHAsset.location; null when absent
    "latitude":  48.8566,
    "longitude":  2.3522
  },
  "pixelWidth":       1290,                  // PHAsset.pixelWidth
  "pixelHeight":      2796,                  // PHAsset.pixelHeight
  "isFavorite":       false,                 // PHAsset.isFavorite
  "isHidden":         false,                 // PHAsset.isHidden
  "durationSeconds":  0,                     // PHAsset.duration (0 for still images)
  "sourceType":       "userLibrary",         // PHAsset.sourceType, normalized to lowercase
  "fetchedAt":        "2026-11-12T09:00:00Z" // wall-clock poll time
}

This record is the minimum viable input for most agent tasks. Add image data only when the workflow needs pixels.

Detecting cloud-only originals

A PHAsset metadata fetch always works locally—the fields above are available regardless of whether the original file has downloaded. Requesting image data for a cloud-only asset returns a placeholder unless the helper explicitly allows network access.

Use PHImageRequestOptions to detect and control this behavior:

// Integration design — developer-owned Mac process. Not production-tested.
let options = PHImageRequestOptions()
options.isNetworkAccessAllowed = false   // Default: prevents automatic download.
options.isSynchronous = false

PHImageManager.default().requestImage(
    for: asset,
    targetSize: CGSize(width: 256, height: 256),
    contentMode: .aspectFit,
    options: options
) { image, info in
    // PHImageResultIsInCloudKey is true when the original is not on this device.
    let isInCloud = info?[PHImageResultIsInCloudKey] as? Bool ?? false

    if isInCloud {
        // Schedule a download pass; metadata is still usable now.
    } else {
        // Full original is locally available; proceed with pixel-data task.
    }
}

PHImageResultIsInCloudKey is documented in Apple's PHImageManager result info dictionary. Set isNetworkAccessAllowed = true only when you actively want to trigger a download for pixel-data tasks. Metadata-only tasks—routing, tagging, filtering on the fields above—do not require a download.

Polling for new arrivals

A time-windowed PHFetchOptions query is the simplest new-asset trigger. Persist the watermark date between runs; query on each polling cycle.

// Integration design — developer-owned Mac process. Not production-tested.
// Requires Photos library authorization and NSPhotoLibraryUsageDescription in Info.plist.

import Photos

func fetchNewAssets(after watermark: Date) -> [PHAsset] {
    let options = PHFetchOptions()
    options.predicate = NSPredicate(
        format: "creationDate > %@", watermark as CVarArg
    )
    options.sortDescriptors = [
        NSSortDescriptor(key: "creationDate", ascending: true)
    ]
    let result = PHAsset.fetchAssets(with: .image, options: options)
    var assets: [PHAsset] = []
    result.enumerateObjects { asset, _, _ in assets.append(asset) }
    return assets
}

Deduplicate on localIdentifier, not on fetchedAt. Advance the watermark only after a record is successfully persisted. For reactive change detection, implement PHPhotoLibraryChangeObserver; for batch pipelines, a scheduled fetch is simpler to reason about.

Three workflows you can build today

Travel log. Filter assets where location is non-null and creationDate falls within a trip window. Group by day, then pass each GPS cluster to a geocoding step. The agent writes one journal entry per location without looking at the photos themselves.

Receipt scanner. Filter on mediaSubtype == "photoScreenshot". Pass the localIdentifier to an OCR step that requests the original only when it needs pixels. Store the extracted merchant name, amount, and date alongside the structured record. This narrows a large library to a tractable daily inbox on metadata alone.

Shared-library archivist. Filter sourceType == "cloudShared" to watch only photos contributed by other family members. Emit a notification or write an annotation—see Read Apple Notes with a Scoped Mac Helper—each time a new shared photo arrives, without touching the personal camera roll.

Connect and start

Figment Apple Connect—OAuth for iCloud—gives developer apps user-approved, scoped access to Apple data through a hosted connected Mac. The iMessage read, search, and send API is available today, as is a synthetic Contacts, Calendar, and Reminders sandbox for testing. The same connected Mac is the natural host for a developer-owned Photos helper following the design above.

A live connection costs $29/month; sandbox access is free. Register your app and start building at /apple-connect/developers. Manage existing grants at /apple-connect/account.

Connect Apple Account to register and get started.

For the broader data picture, see Connect Apple Notes, Photos, Files, and Safari. To build alongside photo records, Create Apple Reminders with EventKit on Mac covers triggering reminders from the same connection.