Summary
The "Exporting Results" section of docs/concepts/annotations/queues.md is out of date. The annotation queue export format (both CSV and JSONL) has been pivoted to one column/entry per annotator, and the is_authoritative boolean has been replaced by an authoritative_annotator field. The current docs still describe the old flat, one-row-per-annotation shape.
This is stale for two changes:
- CSV (already merged in OCS #3832): pivoted to one row per (item, schema field), with one column per annotator email, plus
authoritative_annotator, annotated_at, and a field column. This shipped without a docs update, so the CSV description is already inaccurate on the live docs.
- JSONL (in progress, OCS #3837): pivoted to one record per item, with per-field values nested per annotator (
fields: {field: {annotator_email: value}}), plus top-level authoritative_annotator and annotated_at.
What is inaccurate today
| Doc says (current) |
Reality after the changes |
| "CSV: one row per annotation" |
CSV is one row per (item, schema field), one column per annotator |
| "includes the item details, reviewer, and all field values" |
No single reviewer column: CSV has one column per annotator email; JSONL nests values by annotator |
Field table lists is_authoritative (boolean) |
Replaced by authoritative_annotator (annotator email, blank if none picked) in both formats |
| (not documented) |
New fields: CSV field + annotated_at; JSONL fields: {field: {email: value}} + annotated_at |
| JSONL "one JSON object per line" (per annotation) |
One JSON object per item, values nested per field by annotator |
Also: the field table calls the column flagged_reason, but the actual export column is flags. Worth correcting while updating this section.
Proposed shape (JSONL, for reference)
{
"item_id": 67, "item_type": "session", "session_id": "d99c...",
"flagged": false, "flags": [],
"authoritative_annotator": "marcus@example.com", "annotated_at": "2026-07-15T13:17:02Z",
"fields": {
"tone": {"fatima@example.com": "friendly", "marcus@example.com": "too casual"},
"accuracy": {"fatima@example.com": "accurate", "marcus@example.com": "missed grammar"}
}
}A field an annotator did not answer is null; values keep their JSON types (ints stay ints). CSV carries the same information with one column per annotator email.
Scope
- Update the "Exporting Results" section of
docs/concepts/annotations/queues.md to describe both the new CSV and JSONL shapes, replace is_authoritative with authoritative_annotator, add the new fields, and correct flagged_reason -> flags.
- The docs PR should cross-reference the OCS JSONL PR (#3837 follow-up) and vice versa.
Related
Summary
The "Exporting Results" section of
docs/concepts/annotations/queues.mdis out of date. The annotation queue export format (both CSV and JSONL) has been pivoted to one column/entry per annotator, and theis_authoritativeboolean has been replaced by anauthoritative_annotatorfield. The current docs still describe the old flat, one-row-per-annotation shape.This is stale for two changes:
authoritative_annotator,annotated_at, and afieldcolumn. This shipped without a docs update, so the CSV description is already inaccurate on the live docs.fields: {field: {annotator_email: value}}), plus top-levelauthoritative_annotatorandannotated_at.What is inaccurate today
reviewercolumn: CSV has one column per annotator email; JSONL nests values by annotatoris_authoritative(boolean)authoritative_annotator(annotator email, blank if none picked) in both formatsfield+annotated_at; JSONLfields: {field: {email: value}}+annotated_atAlso: the field table calls the column
flagged_reason, but the actual export column isflags. Worth correcting while updating this section.Proposed shape (JSONL, for reference)
{ "item_id": 67, "item_type": "session", "session_id": "d99c...", "flagged": false, "flags": [], "authoritative_annotator": "marcus@example.com", "annotated_at": "2026-07-15T13:17:02Z", "fields": { "tone": {"fatima@example.com": "friendly", "marcus@example.com": "too casual"}, "accuracy": {"fatima@example.com": "accurate", "marcus@example.com": "missed grammar"} } }A field an annotator did not answer is
null; values keep their JSON types (ints stay ints). CSV carries the same information with one column per annotator email.Scope
docs/concepts/annotations/queues.mdto describe both the new CSV and JSONL shapes, replaceis_authoritativewithauthoritative_annotator, add the new fields, and correctflagged_reason->flags.Related