Separating provenance data from answers#1230
Merged
Merged
Conversation
|
🪓 PR closed, deleted preview. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the study runtime + storage layer to keep Trrack provenance graphs out of the per-trial answers payload and instead persist provenance as separate per-participant task assets (similar to audio/screen recordings), improving performance and export/download speed while maintaining backward compatibility for legacy inline provenance.
Changes:
- Removed inline
provenanceGraphfromStoredAnswerand added a dedicatedStoredProvenancemodel plus normalization/migration helpers. - Added
StorageEnginesupport for saving/loading provenance under a newprovenance/asset namespace, including migration behavior when legacy inline provenance is encountered. - Updated replay/analysis UI to fetch provenance via
getProvenance(...)(falling back to legacy inline provenance when present) and added/updated tests around provenance splitting and persistence.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/store/types.ts | Introduces StoredProvenance and removes provenance from StoredAnswer. |
| src/store/store.tsx | Stops initializing stored answers with inline provenance. |
| src/store/provenance.ts | Adds helpers to normalize/extract/strip legacy inline provenance and split it from answers. |
| src/store/provenance.spec.ts | Unit tests for provenance helper behaviors (null/non-object tolerance and stripping). |
| src/store/hooks/useNextStep.ts | Persists provenance separately via storageEngine.saveProvenance(...) alongside saving answers. |
| src/store/hooks/useNextStep.spec.tsx | Updates mocks/fixtures to accommodate separate provenance persistence. |
| src/storage/types.ts | Updates documentation to reflect provenance being stored separately from answers. |
| src/storage/tests/highLevel.spec.ts | Adds coverage ensuring saveAnswers strips inline provenance and stores it as a provenance asset. |
| src/storage/engines/utils/participantDataRecovery.spec.ts | Updates fixtures for the removed inline provenance field. |
| src/storage/engines/types.ts | Implements saveProvenance/getProvenance, migrates legacy inline provenance in saveAnswers, and includes provenance in copy/delete/snapshot flows. |
| src/components/audioAnalysis/TaskProvenanceTimeline.tsx | Switches the timeline input from answers-derived provenance to an explicit StoredProvenance prop. |
| src/components/audioAnalysis/AudioProvenanceVis.tsx | Loads provenance via storageEngine.getProvenance(...) with legacy fallback and updates all provenance reads accordingly. |
| src/analysis/individualStudy/thinkAloud/ThinkAloudFooter.tsx | Fetches provenance via storage engine for legend building, with legacy fallback. |
| src/analysis/individualStudy/summary/utils.test.ts | Updates fixtures for the removed inline provenance field. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
12 tasks
JackWilb
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Give a longer description of what this PR addresses and why it's needed
We decided to separate out the provenance data (which can get quite long) from the answers data as a way to ensure the answers doesnt grow and cause slow downs in studies with increased provenance data.
Added a few tests, but this is a bit difficult to test. I did make sure that its backwards compatible, so anything with old provenance format will still work, with it favoring the new format. Replays still work, which afaik is the only place we actually use provenance. The exported json now will not include provenance data, and a separate button is added to download provenance (similar to the audio). This might make analysis slightly more difficult since wed have to merge in provenance if you actually wanted to analyse it but, by default I think makes sense, to keep json download speeds fast.