|
| 1 | +# Operator Design Decisions: Comic Artifact Encapsulation |
| 2 | + |
| 3 | +**Authority**: First-mate operator (role: operator) |
| 4 | +**Scope**: Resolves 6 adversarial bouncer challenges |
| 5 | +**Effect**: Gates code generation, binds sub-agent 2 implementation |
| 6 | + |
| 7 | +--- |
| 8 | + |
| 9 | +## Challenge 1: LLM Non-Determinism vs. Content-Hash Idempotency |
| 10 | + |
| 11 | +**Problem**: LLMs are probabilistic. Same input → different output. Content-hash model assumes determinism. |
| 12 | + |
| 13 | +**Operator Decision**: **Accept Non-Determinism. Artifacts are outputs, not specifications.** |
| 14 | + |
| 15 | +### Rationale |
| 16 | + |
| 17 | +**The artifact IS the output**, not a promise. If Gemma4 produces two different scripts from "robot humor": |
| 18 | +- Script A: "A robot walks into a bar, orders a beer. Bartender: 'We don't serve your kind here.' Robot: 'I accept.'" |
| 19 | +- Script B: "A robot enters a bar. Bartender stares. Robot: 'I'm just here to understand human comedy.'" |
| 20 | + |
| 21 | +Both are **valid, distinct artifacts**. They have different IDs (content-hashes): |
| 22 | +- artifact_id_A = blake3("variant_a" || script_A) = "abc123" |
| 23 | +- artifact_id_B = blake3("variant_a" || script_B) = "def456" |
| 24 | + |
| 25 | +**No idempotency promise at the artifact level.** Idempotency exists at the **workflow invocation level**: |
| 26 | +- Invoke script_generation(topic, cast) → returns artifact_id_X (whatever LLM produced) |
| 27 | +- Same invocation → same artifact_id (Rhai code is deterministic given fixed seed/model) |
| 28 | +- **Different invocation** (new model, new seed, new day) → potentially different artifact_id |
| 29 | + |
| 30 | +### Implementation |
| 31 | + |
| 32 | +- Artifact ID includes: `blake3(variant || model || seed || script_text)` |
| 33 | +- Store model + seed in artifact.attrs for reproducibility |
| 34 | +- Rhai script_generation() captures: `seed = context.seed || get_default_seed()` |
| 35 | +- **Lineage shows branching**: Comic → [Script_A1 (model:Gemma4, seed:123), Script_A2 (model:Gemma4, seed:456)] |
| 36 | + |
| 37 | +### Governance Value |
| 38 | + |
| 39 | +✅ Full provenance: "This script came from Gemma4 with seed 123 on 2026-05-10" |
| 40 | +✅ Reproducibility: Given same model + seed, same artifact_id regenerates identical script |
| 41 | +✅ Deduplication: Two runs with same (model, seed, prompt) → same artifact (reuse) |
| 42 | +✅ Auditability: Variants track which model/seed produced them |
| 43 | + |
| 44 | +--- |
| 45 | + |
| 46 | +## Challenge 2: Voter Changed Mind (Vote Deduplication) |
| 47 | + |
| 48 | +**Problem**: If vote ID = blake3(day || voter_hash || choice), changing choice creates duplicate votes with contradictory intents. |
| 49 | + |
| 50 | +**Operator Decision**: **Use (day, voter_hash) as primary key. Latest vote wins. Keep audit history via Relations.** |
| 51 | + |
| 52 | +### Rationale |
| 53 | + |
| 54 | +**Voter voting twice is not an artifact problem, it's a **state management problem**.** |
| 55 | + |
| 56 | +Use a hybrid model: |
| 57 | +1. **Mutable state layer** (votes table, existing schema): |
| 58 | + ```sql |
| 59 | + CREATE TABLE votes ( |
| 60 | + day TEXT, voter_hash TEXT, choice TEXT, created_at INT, |
| 61 | + PRIMARY KEY(day, voter_hash) -- One vote per voter per day |
| 62 | + ); |
| 63 | + ``` |
| 64 | + This stays. ON CONFLICT → UPDATE choice. |
| 65 | + |
| 66 | +2. **Immutable audit layer** (ontology): |
| 67 | + Each vote creates an AuditEvent artifact: |
| 68 | + ``` |
| 69 | + Artifact(AuditEvent) { |
| 70 | + id: blake3(day || voter_hash || choice || timestamp), |
| 71 | + attrs: {day, choice, voter_hash, timestamp} |
| 72 | + } |
| 73 | + ``` |
| 74 | + NEW vote → NEW artifact (different timestamp, different ID). |
| 75 | + |
| 76 | +3. **Relation tracking the state change**: |
| 77 | + ``` |
| 78 | + Relation { |
| 79 | + from: Comic.id, |
| 80 | + to: VoteAuditEvent_v1.id, |
| 81 | + relation: "ReviewedByOperator", |
| 82 | + provenance: {timestamp, choice: "a", voter: voter_hash} |
| 83 | + } |
| 84 | +
|
| 85 | + Relation { |
| 86 | + from: Comic.id, |
| 87 | + to: VoteAuditEvent_v2.id, -- LATER VOTE |
| 88 | + relation: "ReviewedByOperator", |
| 89 | + provenance: {timestamp: later, choice: "b", voter: voter_hash, reason: "changed mind"} |
| 90 | + } |
| 91 | + ``` |
| 92 | + |
| 93 | +### Implementation |
| 94 | + |
| 95 | +- **Frontend logic**: POST /api/vote still simple. Rhai receives {choice} only. |
| 96 | +- **Backend logic**: Rhai invokes record_vote(), which: |
| 97 | + 1. Checks votes table for existing (day, voter_hash) |
| 98 | + 2. If exists: UPDATE votes SET choice=... (mutable) |
| 99 | + 3. Create NEW AuditEvent artifact (immutable record of this vote moment) |
| 100 | + 4. Create Relation showing the audit trail |
| 101 | + 5. Return {variant: "Ok", value: {artifact: AuditEvent, previous_choice: old_choice}} |
| 102 | + |
| 103 | +### Governance Value |
| 104 | + |
| 105 | +✅ Vote counts stay simple: `SELECT choice, COUNT(*) FROM votes WHERE day=? GROUP BY choice` |
| 106 | +✅ Audit trail preserved: All vote_audit_events visible, timestamped |
| 107 | +✅ Intent captured: Provenance shows "voter changed mind from a→b at timestamp" |
| 108 | +✅ Compliance: "Was this voter eligible to vote? When did they vote? Did they change?" = queryable |
| 109 | + |
| 110 | +--- |
| 111 | + |
| 112 | +## Challenge 3: HTTP Layer Complexity |
| 113 | + |
| 114 | +**Problem**: Voting now involves artifacts. Does /api/vote endpoint scale? |
| 115 | + |
| 116 | +**Operator Decision**: **Thin HTTP layer. Rhai handles complexity. MCP invocation transparent to HTTP.** |
| 117 | + |
| 118 | +### Rationale |
| 119 | + |
| 120 | +HTTP layer stays **functionally unchanged**: |
| 121 | +```typescript |
| 122 | +// Frontend still sends |
| 123 | +POST /api/vote { |
| 124 | + day: "2026-05-10", |
| 125 | + choice: "a" | "b" | null // null = abstain |
| 126 | +} |
| 127 | + |
| 128 | +// Frontend still receives |
| 129 | +{ |
| 130 | + success: true, |
| 131 | + votes: {a: 42, b: 38, abstain: 2} |
| 132 | +} |
| 133 | +``` |
| 134 | + |
| 135 | +**Backend changes** (internal to handlers, not visible to HTTP): |
| 136 | +```typescript |
| 137 | +// TypeScript handler (image-generate.ts or new vote.ts) |
| 138 | +export async function onRequestPost(context: any) { |
| 139 | + const {request, env} = context; |
| 140 | + const {day, choice} = await request.json(); |
| 141 | + |
| 142 | + // Generate voter hash (same as before) |
| 143 | + const voter_hash = sha256(ip + ua + day); |
| 144 | + |
| 145 | + // NEW: Invoke Rhai via MCP instead of raw DB insert |
| 146 | + const vote_result = await invokeWorkflow('record_vote', { |
| 147 | + day, |
| 148 | + voter_hash, |
| 149 | + choice // "a", "b", or null |
| 150 | + }); |
| 151 | + |
| 152 | + // NEW: Handle {variant: "Ok"|"Err"} response |
| 153 | + if (vote_result.variant === 'Ok') { |
| 154 | + const artifact = vote_result.value.artifact; |
| 155 | + // Artifact is stored in DB (MCP invocation handles it) |
| 156 | + |
| 157 | + // Get vote counts (same query as before) |
| 158 | + const counts = await env.DB.prepare( |
| 159 | + 'SELECT choice, COUNT(*) as count FROM votes WHERE day=? GROUP BY choice' |
| 160 | + ).bind(day).all(); |
| 161 | + |
| 162 | + return Response.json({ |
| 163 | + success: true, |
| 164 | + votes: { |
| 165 | + a: counts.find(c => c.choice === 'a')?.count || 0, |
| 166 | + b: counts.find(c => c.choice === 'b')?.count || 0, |
| 167 | + abstain: counts.find(c => c.choice === 'abstain')?.count || 0 |
| 168 | + } |
| 169 | + }); |
| 170 | + } else { |
| 171 | + return Response.json({ |
| 172 | + success: false, |
| 173 | + error: vote_result.reason |
| 174 | + }, {status: 400}); |
| 175 | + } |
| 176 | +} |
| 177 | +``` |
| 178 | + |
| 179 | +### Governance Value |
| 180 | + |
| 181 | +✅ HTTP API stable (no breaking changes for frontend) |
| 182 | +✅ Complexity encapsulated in Rhai (governance layer) |
| 183 | +✅ MCP invocation is internal plumbing (transparent to callers) |
| 184 | +✅ Vote counts queryable from existing votes table |
| 185 | + |
| 186 | +--- |
| 187 | + |
| 188 | +## Challenge 4: Script Uniqueness & Day Context |
| 189 | + |
| 190 | +**Problem**: Should script artifact ID include the day (origin context)? |
| 191 | + |
| 192 | +**Operator Decision**: **YES. Include day in artifact ID. Scripts are day-specific proposals.** |
| 193 | + |
| 194 | +### Rationale |
| 195 | + |
| 196 | +**Principle**: Artifacts represent **instances**, not archetypes. |
| 197 | + |
| 198 | +Same joke told on different days is a **different proposal**: |
| 199 | +- Day 1: "A robot walks into a bar" (timely, novel, fits audience) |
| 200 | +- Day 2: "A robot walks into a bar" (repetition, same audience tires, context matters) |
| 201 | + |
| 202 | +**Artifact ID should reflect instance context**: |
| 203 | +``` |
| 204 | +script_artifact_id = blake3( |
| 205 | + "variant_a" || // variant |
| 206 | + "2026-05-10" || // day (context) |
| 207 | + "Gemma4" || // model |
| 208 | + script_text // content |
| 209 | +) |
| 210 | +``` |
| 211 | + |
| 212 | +**Result**: Same script text on different days → **different artifact IDs** |
| 213 | + |
| 214 | +### Benefits |
| 215 | + |
| 216 | +✅ **Deduplication within a day**: Rerun script_generation(day=2026-05-10) → same artifact_id |
| 217 | +✅ **Separation across days**: Day 1 script ≠ Day 2 script (even if identical) |
| 218 | +✅ **Lineage clarity**: Comic(2026-05-10) references Script(2026-05-10), not Script(2026-05-01) |
| 219 | +✅ **Governance**: Audit trail shows "This comic used this script on this day" |
| 220 | + |
| 221 | +### Implementation |
| 222 | + |
| 223 | +Rhai script_generation(): |
| 224 | +```rhai |
| 225 | +let artifact_id = blake3( |
| 226 | + "variant_a" || context.day || context.model_a || context.script_a |
| 227 | +); |
| 228 | +``` |
| 229 | + |
| 230 | +--- |
| 231 | + |
| 232 | +## Challenge 5: Provenance Immutability Paradox |
| 233 | + |
| 234 | +**Problem**: Relations are content-hashed (immutable). How do we add post-hoc audit annotations? |
| 235 | + |
| 236 | +**Operator Decision**: **Annotations are NEW relations, not mutations. Keep original relation immutable. Chain via provenance.** |
| 237 | + |
| 238 | +### Rationale |
| 239 | + |
| 240 | +**Original relation stays immutable**: |
| 241 | +``` |
| 242 | +Relation V1 { |
| 243 | + id: blake3(comic_id || script_id || "DerivedFrom" || {}) |
| 244 | + from: comic_id, |
| 245 | + to: script_id, |
| 246 | + relation: "DerivedFrom", |
| 247 | + provenance: {timestamp: "2026-05-10T10:00:00Z", generator: "rhai"} |
| 248 | +} |
| 249 | +``` |
| 250 | + |
| 251 | +**Later operator review creates NEW relation**: |
| 252 | +``` |
| 253 | +Relation V2 { |
| 254 | + id: blake3(operator_id || relation_v1_id || "ValidatedBy" || {notes: "..."}) |
| 255 | + from: operator_id, |
| 256 | + to: relation_v1_id, // Relation refers to another relation! |
| 257 | + relation: "ValidatedBy", |
| 258 | + provenance: { |
| 259 | + timestamp: "2026-05-10T14:30:00Z", |
| 260 | + reviewer: "operator_alice", |
| 261 | + notes: "Script quality approved, ready for publishing" |
| 262 | + } |
| 263 | +} |
| 264 | +``` |
| 265 | + |
| 266 | +**Audit trail shows the chain**: |
| 267 | +- Comic → (DerivedFrom) → Script (original, immutable) |
| 268 | +- Operator → (ValidatedBy) → [Comic → Script relation] (annotation, immutable) |
| 269 | + |
| 270 | +### Governance Value |
| 271 | + |
| 272 | +✅ Original provenance never lies (immutable) |
| 273 | +✅ Post-hoc annotations are separate artifacts (don't rewrite history) |
| 274 | +✅ Full audit trail visible: decision → review → outcome |
| 275 | +✅ Compliance: "Who reviewed this? When? What did they conclude?" = queryable |
| 276 | + |
| 277 | +--- |
| 278 | + |
| 279 | +## Challenge 6: Abstain Vote Semantics |
| 280 | + |
| 281 | +**Problem**: How should "neither is funny" votes affect popularity counting? |
| 282 | + |
| 283 | +**Operator Decision**: **Abstain votes are VISIBLE but EXCLUDED from popularity ratio. Reported separately.** |
| 284 | + |
| 285 | +### Rationale |
| 286 | + |
| 287 | +**Goal**: Measure which variant is funnier **among engaged voters**. |
| 288 | + |
| 289 | +Abstain is a **signal**: |
| 290 | +- High abstain rate → "Neither variant resonated with audience" → Fitness function penalty |
| 291 | +- Low abstain rate → "Audience had clear preference" → Fitness function reward |
| 292 | + |
| 293 | +### Vote Response Model |
| 294 | + |
| 295 | +```typescript |
| 296 | +{ |
| 297 | + success: true, |
| 298 | + votes: { |
| 299 | + a: 42, |
| 300 | + b: 38, |
| 301 | + abstain: 4, |
| 302 | + engagement: 0.96 // (42+38) / (42+38+4) = 0.95 |
| 303 | + }, |
| 304 | + fitness_signal: { |
| 305 | + variant_a_score: 42 / (42+38) = 0.525, // 52.5% prefer A |
| 306 | + variant_b_score: 38 / (42+38) = 0.475, // 47.5% prefer B |
| 307 | + engagement_score: 0.95, // 95% voted, didn't abstain |
| 308 | + popularity: 0.525 * 0.95 = 0.498 // A wins slightly, but low engagement hurts |
| 309 | + } |
| 310 | +} |
| 311 | +``` |
| 312 | + |
| 313 | +### Implementation |
| 314 | + |
| 315 | +Rhai record_vote(): |
| 316 | +```rhai |
| 317 | +fn record_vote(comic_id, voter_hash, vote_choice) { |
| 318 | + // vote_choice: {variant: "Some", value: "a"|"b"} or {variant: "None"} |
| 319 | +
|
| 320 | + if vote_choice.variant == "None" { |
| 321 | + return #{ |
| 322 | + variant: "Ok", |
| 323 | + value: #{ |
| 324 | + artifact: #{ |
| 325 | + kind: "AuditEvent", |
| 326 | + attrs: { |
| 327 | + choice: "abstain", |
| 328 | + voter_hash: voter_hash |
| 329 | + } |
| 330 | + } |
| 331 | + } |
| 332 | + }; |
| 333 | + } |
| 334 | +
|
| 335 | + // ... normal voting logic |
| 336 | +} |
| 337 | +``` |
| 338 | + |
| 339 | +Vote counting in HTTP handler: |
| 340 | +```typescript |
| 341 | +const votes = await env.DB.prepare( |
| 342 | + 'SELECT choice, COUNT(*) as count FROM votes WHERE day=? GROUP BY choice' |
| 343 | +).bind(day).all(); |
| 344 | + |
| 345 | +const voteA = votes.find(v => v.choice === 'a')?.count || 0; |
| 346 | +const voteB = votes.find(v => v.choice === 'b')?.count || 0; |
| 347 | +const abstain = votes.find(v => v.choice === 'abstain')?.count || 0; |
| 348 | +const totalEngaged = voteA + voteB; |
| 349 | + |
| 350 | +const engagement = totalEngaged > 0 ? totalEngaged / (voteA + voteB + abstain) : 0; |
| 351 | +const variantAScore = totalEngaged > 0 ? voteA / totalEngaged : 0; |
| 352 | +const variantBScore = totalEngaged > 0 ? voteB / totalEngaged : 0; |
| 353 | + |
| 354 | +return Response.json({ |
| 355 | + success: true, |
| 356 | + votes: {a: voteA, b: voteB, abstain: abstain}, |
| 357 | + engagement: engagement, |
| 358 | + fitness: { |
| 359 | + variant_a: variantAScore * engagement, |
| 360 | + variant_b: variantBScore * engagement |
| 361 | + } |
| 362 | +}); |
| 363 | +``` |
| 364 | + |
| 365 | +### Governance Value |
| 366 | + |
| 367 | +✅ **Popularity measured where it matters**: Among engaged voters |
| 368 | +✅ **Disengagement visible**: High abstain rate signals weakness |
| 369 | +✅ **Fitness function precision**: Rewards engagement + preference clarity |
| 370 | +✅ **Internet fitness challenge**: "This comic engaged 95% of voters and won 55/45" > "This comic engaged 60% and won 60/40" |
| 371 | + |
| 372 | +--- |
| 373 | + |
| 374 | +## Summary: Design Positions Locked |
| 375 | + |
| 376 | +| Challenge | Decision | Rationale | |
| 377 | +|-----------|----------|-----------| |
| 378 | +| **LLM Non-Determinism** | Accept variation. Content-hash per output. | Artifacts are instances, not specs. Lineage shows model+seed. | |
| 379 | +| **Vote Dedup** | Hybrid: mutable votes table + immutable AuditEvent artifacts | Simple vote counts. Full audit trail. Change-of-mind tracked. | |
| 380 | +| **HTTP Complexity** | Thin HTTP layer. Rhai handles governance. MCP transparent. | No breaking changes to API. Complexity encapsulated in workflow. | |
| 381 | +| **Script Uniqueness** | Include day in artifact ID. Scripts are day-specific. | Dedup within day, separate across days. Lineage clarity. | |
| 382 | +| **Provenance Immutability** | Annotations are NEW relations, not mutations. | Original provenance never lies. Chain via relations. | |
| 383 | +| **Abstain Votes** | Visible. Excluded from ratio. Reported separately. | Engagement signal. Fitness function penalty for low engagement. | |
| 384 | + |
| 385 | +--- |
| 386 | + |
| 387 | +## Code Generation Gates (Locked In) |
| 388 | + |
| 389 | +Sub-agent 2 will patch `comic-generation.rhai` with: |
| 390 | + |
| 391 | +1. ✅ Include day/model/seed in artifact IDs (Challenge 4) |
| 392 | +2. ✅ record_vote() returns {variant: "Ok"|"Err"} with AuditEvent artifact (Challenge 2) |
| 393 | +3. ✅ Abstain encoded as {choice: "abstain"} in AuditEvent (Challenge 6) |
| 394 | +4. ✅ No changes to HTTP API contract (Challenge 3) |
| 395 | +5. ✅ Provenance in relations, not mutations (Challenge 5) |
| 396 | +6. ✅ Accept LLM variance; lineage shows model+seed (Challenge 1) |
| 397 | + |
| 398 | +Code review will verify compliance with these positions. |
| 399 | + |
0 commit comments