Skip to content

Commit 4f8f693

Browse files
committed
feat: add storage-finder CSV importer and hourly sync
1 parent 5dbf673 commit 4f8f693

12 files changed

Lines changed: 809 additions & 11 deletions

File tree

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
name: Sync Storage Finder Data
2+
3+
on:
4+
schedule:
5+
- cron: "0 * * * *"
6+
workflow_dispatch:
7+
8+
jobs:
9+
sync:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Checkout
13+
uses: actions/checkout@v4
14+
with:
15+
fetch-depth: 0
16+
17+
- name: Set up Bun
18+
uses: oven-sh/setup-bun@v2
19+
20+
- name: Install dependencies
21+
run: bun install --frozen-lockfile
22+
23+
- name: Generate storage finder data from sheet
24+
run: bun scripts/storage-finder-data-generator/generate.ts --output src/data/storage-finder
25+
26+
- name: Lint and format
27+
run: |
28+
bun lint --fix
29+
bun format
30+
31+
- name: Commit changes
32+
uses: stefanzweifel/git-auto-commit-action@v5
33+
with:
34+
commit_message: "chore: sync storage finder data from sheet"

docusaurus.config.ts

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -167,8 +167,7 @@ const config: Config = {
167167
items: [
168168
{
169169
label: "Feedback",
170-
to:
171-
"https://docs.google.com/forms/d/e/1FAIpQLSeHnmkPdR_IvWnT6a7U_V3RpfmQrpS8hjxI11FNnsZMlrBa4g/viewform",
170+
to: "https://docs.google.com/forms/d/e/1FAIpQLSeHnmkPdR_IvWnT6a7U_V3RpfmQrpS8hjxI11FNnsZMlrBa4g/viewform",
172171
},
173172
{
174173
label: "Announcements",

eslint.config.mjs

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,14 @@ const eslintConfig = tseslint.config(
6666
reactPlugin.configs.flat["jsx-runtime"],
6767
jsxA11y.flatConfigs.recommended,
6868

69+
{
70+
settings: {
71+
react: {
72+
version: "detect",
73+
},
74+
},
75+
},
76+
6977
{
7078
rules: {
7179
"@typescript-eslint/consistent-type-imports": [

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@
4343
"@docusaurus/types": "^3.9.2",
4444
"@eslint/compat": "^1.4.1",
4545
"@types/react": "^19.2.2",
46+
"csv-parse": "^6.1.0",
4647
"eslint": "^9.39.0",
4748
"eslint-config-prettier": "^10.1.8",
4849
"eslint-import-resolver-typescript": "^4.4.4",

pnpm-lock.yaml

Lines changed: 13 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Storage Finder CSV Importer: Maintainer Guide
2+
3+
This approach aims to balance user-friendliness (content editors update the Google Sheet) with functionality (developers tune `config.ts` so the JSON stays in sync).
4+
5+
## What the CLI does
6+
7+
- Reads a CSV export of the shared Google Sheet (either via `STORAGE_FINDER_SHEET_URL` or `--csv <path>`).
8+
- Applies the questions/choices/matchers defined in `scripts/storage-finder-data-generator/config.ts`.
9+
- Generates `facet-tree.json` and `service-list.json` with slug-based IDs.
10+
11+
## How to regenerate data
12+
13+
```sh
14+
bun scripts/storage-finder-data-generator/generate.ts \
15+
--csv "Datafinder Data - Sheet1.csv" \
16+
--output src/data/storage-finder
17+
```
18+
19+
- Omit `--csv` to download from `STORAGE_FINDER_SHEET_URL`.
20+
- Use `--output` to write elsewhere (defaults to `src/data/storage-finder/generated` in the code).
21+
- If you do not have Bun, install it from <https://bun.sh/> or run with `pnpm dlx tsx scripts/storage-finder-data-generator/generate.ts ...`. Node does not run TypeScript by default; `tsx` provides the TypeScript loader.
22+
23+
## How to add or change a question
24+
25+
1. Add a column to the sheet that contains the signals you want to match.
26+
2. In `scripts/storage-finder-data-generator/config.ts`, add a new facet entry to `FACET_CONFIGS`:
27+
- Set `id` (slug), `name`, `controlType` (`radio` or `checkbox`), `column` (sheet column name), and `choices` (labels the app should show).
28+
- Add `matchers`: regex patterns that map cell text to choice IDs. Include `allowMultipleMatches: true` if a radio question legitimately matches more than one choice.
29+
- If no regex matches, `fallback: "all"` keeps the service visible; otherwise supply an explicit array of choice IDs.
30+
3. Regenerate with the CLI and verify in the app.
31+
32+
## Service fields
33+
34+
Field definitions live in `scripts/storage-finder-data-generator/config.ts` (`FIELD_DEFINITIONS`). They map sheet columns to service detail rows (Links, Use Case, Limitations, Permission Settings, Eligibility, Synchronous Access, Alumni Access, Backup). Adjust labels or formatters there if the sheet schema changes.
35+
36+
## Naming and IDs
37+
38+
- Services are slugged from the `Title` column; duplicates get `-2`, `-3`, etc.
39+
- Facet and choice IDs are slugs defined in `config.ts`; keep them stable to avoid breaking references.
40+
41+
## Validation rules
42+
43+
- Radio facets throw if more than one choice matches unless `allowMultipleMatches` is set.
44+
- Blank cells render as “Not Available” in service fields.
45+
- Regexes match against raw cell text; use clear keywords in the sheet for deterministic mapping.

0 commit comments

Comments
 (0)