# Turn New iCloud Photos into Useful Agent Inputs

Canonical: https://figment.so/blog/icloud-photos-agent-workflow

iCloud Photos gives each asset a full metadata record before the original image file downloads to the device. A Mac helper that queries Apple's Photos framework can read that record—creation date, GPS coordinates, media subtype, favorite flag—and hand a structured JSON object to your agent without loading a single pixel. This page shows the field map, explains how to detect cloud-only originals correctly, and walks through a polling design and three workflows you can build from that foundation.

Figment does not currently offer a public Photos REST endpoint. The design below is a developer-owned Mac process that uses Apple's PhotoKit API (available on macOS 10.13+). The structured-record format is the core artifact; the Swift snippets are design references, not production-tested code.

## Photos carry more signal than a filename

Apple's [`PHAsset`](https://developer.apple.com/documentation/photos/phasset) class represents each library asset as an object whose metadata properties are available locally even before the full image file is present. For most agent tasks—routing, tagging, filtering—you never request the image at all.

| PHAsset property | Type | What it tells your agent |
|---|---|---|
| `localIdentifier` | String | Device-local unique identifier. Valid only on the device where the asset was fetched; not portable across devices. |
| `mediaType` | PHAssetMediaType | `image`, `video`, or `audio` |
| `mediaSubtypes` | PHAssetMediaSubtype | `photoScreenshot`, `photoLive`, `photoPanorama`, `photoHDR`, `photoDepthEffect`, and others |
| `creationDate` | Date? | Capture timestamp |
| `modificationDate` | Date? | Last edit timestamp |
| `addedDate` | Date? | Date the asset was added to the library on this device |
| `location` | CLLocation? | GPS coordinate from EXIF or device at capture |
| `pixelWidth` / `pixelHeight` | Int | Image dimensions |
| `isFavorite` | Bool | User has starred this asset |
| `isHidden` | Bool | User has hidden this asset |
| `duration` | TimeInterval | Non-zero for video and audio assets |
| `sourceType` | PHAssetSourceType | How the asset entered the library: `.userLibrary`, `.cloudShared`, or `.iTunesSynced` |

`mediaSubtypes` is the fastest routing signal. A `photoScreenshot` flag identifies expense candidates before any OCR runs. `sourceType` lets you watch only shared-album contributions without touching the personal camera roll.

`localIdentifier` is device-local: it identifies an asset on the Mac where the fetch runs but is not a portable cross-device ID. For cross-device stable references, map `localIdentifier` to a [`PHCloudIdentifier`](https://developer.apple.com/documentation/photos/phcloudidentifier) using `PHPhotoLibrary.cloudIdentifierMappingsForLocalIdentifiers(_:)`. Cloud-identifier lookups can fail; always handle the error case.

## The photo-to-structured-record recipe

This JSON shape is the artifact your Mac helper produces. Each field maps back to its `PHAsset` source.

```json
{
  "assetId":          "ABC123…",             // PHAsset.localIdentifier (device-local)
  "mediaType":        "image",               // PHAsset.mediaType → "image" | "video" | "audio"
  "mediaSubtype":     "photoScreenshot",     // PHAsset.mediaSubtypes, first matched flag
  "creationDate":     "2026-11-10T14:32:00Z",// PHAsset.creationDate, ISO-8601 UTC
  "modificationDate": "2026-11-10T14:35:00Z",// PHAsset.modificationDate
  "addedDate":        "2026-11-10T14:32:01Z",// PHAsset.addedDate
  "location": {                              // PHAsset.location; null when absent
    "latitude":  48.8566,
    "longitude":  2.3522
  },
  "pixelWidth":       1290,                  // PHAsset.pixelWidth
  "pixelHeight":      2796,                  // PHAsset.pixelHeight
  "isFavorite":       false,                 // PHAsset.isFavorite
  "isHidden":         false,                 // PHAsset.isHidden
  "durationSeconds":  0,                     // PHAsset.duration (0 for still images)
  "sourceType":       "userLibrary",         // PHAsset.sourceType, normalized to lowercase
  "fetchedAt":        "2026-11-12T09:00:00Z" // wall-clock poll time
}
```

This record is the minimum viable input for most agent tasks. Add image data only when the workflow needs pixels.

## Detecting cloud-only originals

A `PHAsset` metadata fetch always works locally—the fields above are available regardless of whether the original file has downloaded. Requesting image data for a cloud-only asset returns a placeholder unless the helper explicitly allows network access.

Use `PHImageRequestOptions` to detect and control this behavior:

```swift
// Integration design — developer-owned Mac process. Not production-tested.
let options = PHImageRequestOptions()
options.isNetworkAccessAllowed = false   // Default: prevents automatic download.
options.isSynchronous = false

PHImageManager.default().requestImage(
    for: asset,
    targetSize: CGSize(width: 256, height: 256),
    contentMode: .aspectFit,
    options: options
) { image, info in
    // PHImageResultIsInCloudKey is true when the original is not on this device.
    let isInCloud = info?[PHImageResultIsInCloudKey] as? Bool ?? false

    if isInCloud {
        // Schedule a download pass; metadata is still usable now.
    } else {
        // Full original is locally available; proceed with pixel-data task.
    }
}
```

`PHImageResultIsInCloudKey` is documented in Apple's [`PHImageManager`](https://developer.apple.com/documentation/photos/phimagemanager) result info dictionary. Set `isNetworkAccessAllowed = true` only when you actively want to trigger a download for pixel-data tasks. Metadata-only tasks—routing, tagging, filtering on the fields above—do not require a download.

## Polling for new arrivals

A time-windowed `PHFetchOptions` query is the simplest new-asset trigger. Persist the watermark date between runs; query on each polling cycle.

```swift
// Integration design — developer-owned Mac process. Not production-tested.
// Requires Photos library authorization and NSPhotoLibraryUsageDescription in Info.plist.

import Photos

func fetchNewAssets(after watermark: Date) -> [PHAsset] {
    let options = PHFetchOptions()
    options.predicate = NSPredicate(
        format: "creationDate > %@", watermark as CVarArg
    )
    options.sortDescriptors = [
        NSSortDescriptor(key: "creationDate", ascending: true)
    ]
    let result = PHAsset.fetchAssets(with: .image, options: options)
    var assets: [PHAsset] = []
    result.enumerateObjects { asset, _, _ in assets.append(asset) }
    return assets
}
```

Deduplicate on `localIdentifier`, not on `fetchedAt`. Advance the watermark only after a record is successfully persisted. For reactive change detection, implement `PHPhotoLibraryChangeObserver`; for batch pipelines, a scheduled fetch is simpler to reason about.

## Three workflows you can build today

**Travel log.** Filter assets where `location` is non-null and `creationDate` falls within a trip window. Group by day, then pass each GPS cluster to a geocoding step. The agent writes one journal entry per location without looking at the photos themselves.

**Receipt scanner.** Filter on `mediaSubtype == "photoScreenshot"`. Pass the `localIdentifier` to an OCR step that requests the original only when it needs pixels. Store the extracted merchant name, amount, and date alongside the structured record. This narrows a large library to a tractable daily inbox on metadata alone.

**Shared-library archivist.** Filter `sourceType == "cloudShared"` to watch only photos contributed by other family members. Emit a notification or write an annotation—see [Read Apple Notes with a Scoped Mac Helper](/blog/apple-notes-export-integration)—each time a new shared photo arrives, without touching the personal camera roll.

## Connect and start

Figment [Apple Connect—OAuth for iCloud](/oauth-for-icloud)—gives developer apps user-approved, scoped access to Apple data through a hosted connected Mac. The iMessage read, search, and send API is available today, as is a synthetic Contacts, Calendar, and Reminders sandbox for testing. The same connected Mac is the natural host for a developer-owned Photos helper following the design above.

A live connection costs $29/month; sandbox access is free. Register your app and start building at [/apple-connect/developers](https://figment.so/apple-connect/developers). Manage existing grants at [/apple-connect/account](https://figment.so/apple-connect/account).

**[Connect Apple Account](https://figment.so/apple-connect/developers)** to register and get started.

For the broader data picture, see [Connect Apple Notes, Photos, Files, and Safari](/apple-data-connectors). To build alongside photo records, [Create Apple Reminders with EventKit on Mac](/blog/apple-reminders-eventkit) covers triggering reminders from the same connection.