|
1 | 1 | # Raincloud conformance matrix — one line per vortex-enabled corpus slug: <slug>,<status> |
2 | | -# status: ok — vortex-java must read the file and match the parquet oracle exactly |
3 | | -# gap:<n> — known reader gap tracked by issue #<n>; the test asserts the failure |
4 | | -# still occurs, so a fix forces flipping the entry to ok in the same change |
5 | | -# untriaged — not yet run; the test executes it and reports the outcome as an |
6 | | -# aborted (skipped) test instead of failing the build — flip to ok/gap |
7 | | -# as hydration coverage grows |
8 | | -# A `#`-comment directly above an `untriaged` line notes WHY it hasn't been hydrated yet |
9 | | -# (e.g. missing credentials for a gated source) — status stays literal "untriaged" (the |
10 | | -# harness string-matches it exactly), the comment is for triagers only. |
| 2 | +# status: ok — vortex-java must read the file and match the parquet oracle exactly |
| 3 | +# gap:<n> — known reader gap tracked by issue #<n>; the test asserts the failure |
| 4 | +# still occurs, so a fix forces flipping the entry to ok in the same change |
| 5 | +# untriaged — not yet run; the test executes it and reports the outcome as an |
| 6 | +# aborted (skipped) test instead of failing the build — flip to ok/gap |
| 7 | +# as hydration coverage grows |
| 8 | +# missing_auth — can't even hydrate: the source (Kaggle, gated HuggingFace, …) needs |
| 9 | +# credentials this environment doesn't have. Distinguishes "blocked on |
| 10 | +# a credential" from plain untriaged ("just hasn't been attempted yet") |
| 11 | +# so a future triager doesn't waste time rediscovering the same blocker. |
| 12 | +# An optional `#`-comment above the line can name the credential. |
11 | 13 | # Corpus: all slugs with a Vortex artifact in the Raincloud catalog (spiraldb/raincloud), |
12 | 14 | # written by vortex-data 0.69.0. Hydrate any subset with scripts/hydrate-raincloud-corpus.sh. |
13 | 15 | 120-years-of-olympic-history-athletes-and-results,ok |
14 | 16 | ai2-arc,ok |
15 | | -# needs Kaggle credentials to hydrate (kaggle auth login / KAGGLE_API_TOKEN) — not attempted |
16 | | -airbnb-prices-in-european-cities,untriaged |
17 | | -# needs Kaggle credentials to hydrate (kaggle auth login / KAGGLE_API_TOKEN) — not attempted |
18 | | -airbnbopendata,untriaged |
| 17 | +# kaggle auth login / KAGGLE_API_TOKEN |
| 18 | +airbnb-prices-in-european-cities,missing_auth |
| 19 | +# kaggle auth login / KAGGLE_API_TOKEN |
| 20 | +airbnbopendata,missing_auth |
19 | 21 | amazon-reviews-2023-subscription-boxes,ok |
20 | 22 | ampds-whe,ok |
21 | 23 | anthropic-economic-index,ok |
22 | 24 | anthropic-hh-rlhf-helpful-base,ok |
23 | 25 | anthropic-interviewer,ok |
24 | 26 | aya-collection-templated,ok |
25 | 27 | aya-dataset,ok |
26 | | -# needs Kaggle credentials to hydrate (kaggle auth login / KAGGLE_API_TOKEN) — not attempted |
27 | | -bank-account-fraud-dataset-neurips-2022,untriaged |
| 28 | +# kaggle auth login / KAGGLE_API_TOKEN |
| 29 | +bank-account-fraud-dataset-neurips-2022,missing_auth |
28 | 30 | basketball,untriaged |
29 | 31 | behavioral-risk-factor-surveillance-system,ok |
30 | 32 | beir-msmarco,untriaged |
|
0 commit comments