Skip to content

Beacon Sync Protocol

Jump to bottom
Danny Collier edited this page Jun 7, 2026 · 1 revision

Beacon Sync Protocol

(Canonical protocol for content synchronization between SkillRX (server) and a Beacon (device). Companion to the Requirements [draft] §3, which stays a product-level summary and links here. This page is matched to the implemented code and should be updated alongside it. Derived from the lead developer's data_exchange.md / json_data_exchange.md, reconciled with Beacons::ManifestBuilder and config/routes.rb.)

Terminology

Manifest means the manifest of files exchanged during a content sync — the list of providers, topics, tags, and the files (with checksums and sizes) a Beacon should hold. It is the communication artifact, not a description of the device. An endpoint like .../manifest for an authenticated Beacon returns "the file manifest for this device," not "a device manifest." (The separate software-update mechanism in json_data_exchange.md reuses the word "manifest" for a release descriptor; that is a different concept, out of scope here, and should be renamed to avoid collision.)

1. Identity & authentication

Each Beacon authenticates with a revocable per-device API key, sent as Authorization: Bearer {api_key}. The server identifies the Beacon from the key and scopes every response to that Beacon's assigned content. Keys are stored hashed on the server (api_key_digest + api_key_prefix) and shown once at creation; they can be regenerated or revoked from the SkillRX admin UI. A 401 Unauthorized means the key is invalid or revoked — the device clears credentials and prompts for re-registration.

2. What a Beacon receives (content scoping)

A Beacon's content is defined by cumulative (AND) filters set in SkillRX (Requirements §2):

  • Language — required, exactly one per Beacon.
  • Region — optional, exactly one if set.
  • Provider(s) — optional; one or more.

Topics and tags are derived, not assigned. The admin sets the filters above; the topic set is the materialized result of those filters, and the tag set is derived from those topics' tags. There is no manual per-device topic or tag selection. (This corrects the dev docs, which described assigning topics and tags directly.)

3. The manifest

Request: GET /api/v1/beacons/manifest with the Bearer key.

Shape (as produced by Beacons::ManifestBuilder):

{
  "manifest_version": "v42",
  "manifest_checksum": "sha256:abc123...",
  "generated_at": "2026-01-15T12:00:00Z",
  "language": { "id": 1, "code": "en", "name": "English" },
  "region":   { "id": 5, "name": "East Region" },
  "tags": [ { "id": 201, "name": "Prenatal" } ],
  "providers": [
    {
      "id": 10,
      "name": "Health Ministry",
      "topics": [
        {
          "id": 100,
          "name": "Maternal Health",
          "tag_ids": [201],
          "files": [
            {
              "id": 1001,
              "filename": "prenatal_care_guide.pdf",
              "path": "providers/10/topics/100/prenatal_care_guide.pdf",
              "checksum": "sha256:def456...",
              "size_bytes": 2457600,
              "content_type": "application/pdf",
              "updated_at": "2026-01-10T08:00:00Z"
            }
          ]
        }
      ]
    }
  ],
  "total_size_bytes": 156000000,
  "total_files": 47
}

Notes and decisions baked in here:

  • region is a single object (one optional region), not an array. The dev docs' regions: [...] array and the per-region code field are dropped.
  • File id is the Active Storage blob.id. Change detection keys on blob.id + checksum (§4).
  • path is advisory under the recommended Active Storage approach (§7); it is only load-bearing if a filesystem layout is chosen instead.
  • Proposed addition — topic published_at. The manifest topic node currently carries only id, name, tag_ids, files. The Beacon needs published_at to build the Provider → Month/Year → Topic hierarchy (Requirements §4.3) and Latest Content (§4.7); description is optional for the metadata sidebar / semantic search. This is a required SkillRX-side change (extend ManifestBuilder#build_topic).

4. Change detection (conditional requests)

The manifest is versioned (manifest_version) and carries a manifest_checksum. The device uses ETag conditional requests against the manifest endpoint:

  • Before sync: If-None-Match: "v42" → 304 Not Modified if nothing changed; 200 OK with a new ETag if it did.
  • During sync: If-Match: "v42" → 412 Precondition Failed if the manifest changed mid-sync, so the device aborts and restarts. (Optimization: check every N files rather than every file.)

Per-file change detection keys on blob.id + checksum: the device re-downloads a file only when its checksum differs from what it already holds, even if the manifest version didn't bump.

Open question for Dmitry — checksum algorithm. The manifest examples label checksums sha256:..., but ManifestBuilder#build_file emits blob.checksum, which Active Storage computes as base64-encoded MD5, not SHA256. The device must verify against whatever the server actually sends. Decide: (a) add and populate a real SHA256 column per the dev's Task #10, or (b) accept Active Storage's MD5 and correct the sha256: labels. The manifest-level manifest_checksum is genuinely SHA256 of the JSON and is unaffected.

5. File download

Request: GET /api/v1/beacons/files/:id with the Bearer key. Access is scoped to the Beacon's assigned content; a file outside that scope returns 404.

Resumable downloads: the endpoint supports HTTP Range requests and returns 206 Partial Content, so an interrupted download resumes from the last byte rather than restarting — important for the low-connectivity, large-media reality (broken connections, slow links).

6. Applying a sync (differential update)

  1. Fetch the manifest; compare against local state.
  2. Compute the diff: new (id not held), changed (checksum differs), deleted (held locally but absent from the manifest).
  3. Download each new/changed file to a temp location; verify its checksum before adopting it.
  4. Reconcile the database against the manifest, keying entirely on SkillRX IDs: find_or_create_by(id:) for providers, topics, tags; attach/replace files by blob.id; handle deletes by removing local records absent from the manifest.
  5. Report status (§8); return to serving mode.

Robustness: retry transient download failures with exponential backoff; on 412 (manifest changed mid-sync) abort and restart; never leave the served dataset in a partially-updated state visible to users.

7. On-device storage

Recommended: Active Storage on the Beacon (Disk service), with sync implemented as the differential reconcile above — not the versioned-directory + symlink-swap model from the dev's storage doc.

Rationale, on two practical criteria:

  • Usability by the local app: Active Storage gives native in-app serving, content-type handling, and HTTP Range/streaming for MP4/MP3 with no custom code — directly serving the §4.4 viewer requirements. The filesystem model would require hand-built file-serving and range logic.
  • Manageability during sync: Active Storage keeps file bookkeeping and relational metadata together, so a sync reconciles in a single DB transaction. The filesystem model is clean for raw files but leaves the SQLite metadata (providers/topics/tags/files) to be coordinated separately, since a symlink swap doesn't cover the database. Per-file safety (temp + verify + attach) provides the robustness a differential sync needs without a whole-dataset atomic swap.

The cache/hard-link reuse idea from the dev's doc remains a viable optimization if re-download cost proves high, but isn't required for correctness.

Gut-check for Dmitry. This overrides the versioned-filesystem approach in the storage doc. Confirm there's no constraint behind that design (e.g., whole-dataset instant rollback, or a SQLite/disk limitation with large media) that should change the call.

8. Sync status reporting

Request: POST /api/v1/beacons/sync_status with the Bearer key:

{
  "status": "synced",
  "manifest_version": "v43",
  "manifest_checksum": "sha256:xyz...",
  "synced_at": "2026-01-15T14:30:00Z",
  "files_count": 47,
  "total_size_bytes": 156000000,
  "device_info": { "hostname": "clinic-pc-001", "os_version": "Ubuntu 22.04", "app_version": "1.0.0" }
}

Status values: synced (complete, up to date), syncing (in progress — include progress_percent, supporting the N-4 real-time update experience), outdated (knows it's behind), error (last sync failed — include error_message). The server updates last_seen_at/last_sync_at from this call.

There is also a device-facing GET /api/v1/beacons/status endpoint in the code. Open question for Dmitry: confirm its intended purpose (e.g., the device querying the server's view of its own up-to-date-ness) and document it here.

9. Error handling

Scenario Device behavior
Network timeout during file download Retry 3× with exponential backoff, then abort sync
Checksum mismatch after download Delete file, re-download; if it fails again, abort sync
Manifest changed during sync (412) Abort immediately, restart with the new manifest
401 Unauthorized Clear credentials, prompt for re-registration
404 on a file Log, abort sync (manifest inconsistency)
Disk full Abort sync, report error status

10. Endpoint summary (authoritative — matches routes.rb)

Method Endpoint Purpose
GET /api/v1/beacons/manifest Get the file manifest for the authenticated Beacon
HEAD /api/v1/beacons/manifest Conditional check via ETag (If-None-Match / If-Match)
GET /api/v1/beacons/files/:id Download a file (supports Range)
POST /api/v1/beacons/sync_status Report sync status
GET /api/v1/beacons/status Device queries its server-side status (purpose to confirm)

(The dev docs' /api/v1/devices/me/... paths are superseded by these. The earlier requirements draft's plural manifests / sync_statuses were incorrect — the routes are singular Rails resources.)

11. Security

HTTPS for all endpoints; API keys hashed at rest on the server and stored securely on the device (encrypted at rest); file downloads verified by checksum to detect tampering or corruption; keys revocable from the admin UI.

Out of scope: software/application updates

The application-update protocol (layered runtime/gems/app, binary deltas, release symlinks, app-manifest + /releases/... endpoints) in json_data_exchange.md is a separate concern mapping to Requirements N-3 (packaging) and N-4 (connect-to-update). It is not built (no routes exist) and is pending the N-3 deployment decision (Docker images vs. tarball-delta-releases). When taken up, its "app-manifest" should be renamed so the system has only one "manifest" (the content/file manifest defined here).

Open questions for Dmitry

  1. Checksum algorithm — real SHA256 column, or accept Active Storage MD5 and relabel? (§4)
  2. On-device storage — confirm Active Storage over the versioned-filesystem model, or name the constraint that argues for the filesystem approach. (§7)
  3. GET /api/v1/beacons/status — confirm purpose and document. (§8)
  4. Topic published_at (± description) in the manifest — confirm the SkillRX-side addition needed for Month/Year browse and Latest Content. (§3)

Clone this wiki locally