Skip to content

Commit 906d552

Browse files
authored
py dependency updates using pipenv (#346)
Update python tools to use pipenv and update dependencies. Try to get ward leader export and import working, but still WIP --------- Co-authored-by: Peter Wolanin <107691+pwolanin@users.noreply.github.com>
1 parent aed6de6 commit 906d552

9 files changed

Lines changed: 476 additions & 41 deletions

File tree

AGENTS.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# AGENTS.md
2+
3+
## Project Overview
4+
5+
Philly Ward Leaders is a civic transparency web app showing Philadelphia's 69 Ward Leaders and their political influence. The frontend is a Vue 3 SPA that pulls data from Contentful CMS and static GeoJSON files. The `data-scripts/` subdirectory contains Python CLI tools for cleaning source data and importing it into Contentful.
6+
7+
## Common Commands
8+
9+
### Frontend (Node.js 20, run from repo root)
10+
11+
```bash
12+
npm ci # Install dependencies
13+
npm run dev # Dev server on localhost:8080 (hot reload)
14+
npm run build # Production build to ./build
15+
npm run lint # ESLint (JS + Vue files)
16+
npm run lint:fix # ESLint auto-fix
17+
npm run format:check # Prettier check
18+
npm run format # Prettier auto-fix
19+
npm run cy:run # Cypress component tests (headless)
20+
npm run cy:e2e # Cypress e2e tests (needs serve-http running)
21+
npm run cy:open # Cypress interactive UI
22+
```
23+
24+
### Data Scripts (Python 3 + pipenv, run from `data-scripts/`)
25+
26+
```bash
27+
pipenv install # Install dependencies
28+
pipenv sync # Install exact locked versions
29+
pipenv run python cli.py --help # Show available commands
30+
pipenv run python cli.py leaders <csv> # Convert leaders CSV to JSON
31+
pipenv run python cli.py committee <csv> # Clean committee person data
32+
pipenv run python cli.py divisions -o <outdir> <geojson> # Split ward boundary GeoJSON
33+
pipenv run python cli.py voters -r <registry> -t <turnout> # Combine voter data
34+
pipenv run python cli.py import <json> --space ID --content-type ID --apikey KEY
35+
```
36+
37+
### Docker (alternative local dev)
38+
39+
```bash
40+
docker compose build && docker compose up -d
41+
docker compose exec ward-leaders bash # Then run npm/pipenv commands inside
42+
```
43+
44+
## CI Pipeline
45+
46+
GitHub Actions on push/PR to `master` (`.github/workflows/node.js.yml`):
47+
1. `npm run format:check`
48+
2. `npm run build`
49+
3. `npm run cy:run` (component tests)
50+
4. Cypress e2e tests (starts `serve-http`, runs e2e suite)
51+
52+
## Architecture
53+
54+
### Frontend (Vue 3 + Vite)
55+
56+
- **Entry:** `src/main.js` creates the Vue app with Vue Router and Vuex store
57+
- **Views** (`src/views/`): Page-level components for each route — splash, ward-leader-list, ward-leader (detail), city-map, content-page (CMS-driven), feedback
58+
- **Components** (`src/components/`): Reusable UI — ward-map (Leaflet), baseball-card, geocoder, nav-bar, stats-bar, notification
59+
- **Store** (`src/store/`): Vuex with actions (async API calls), mutations, getters. Key state: leaders list, currentLeader, wardBoundaries, citywideBoundaries, contentPage
60+
- **API** (`src/api/index.js`): Contentful SDK + axios wrapper
61+
- **Config** (`src/config.js`): Public Contentful space/access token, AIS API key
62+
- **Routing** (`src/router/index.js`): 7 routes including `/leaders/:party`, `/leaders/:party/:ward/:slug`, `/map`, and dynamic `/:slug` for CMS content
63+
- **Styling:** SCSS with Bulma CSS framework
64+
- **Maps:** Leaflet with vue-leaflet for interactive ward boundary maps
65+
- **Build output:** `./build` directory
66+
67+
### Data Scripts (Python CLI)
68+
69+
- **`cli.py`**: Click CLI entry point grouping all subcommands
70+
- **Processing modules**: `leaders.py`, `committee.py`, `divisions.py`, `registry.py`, `turnout.py` — each uses PETL for ETL pipelines
71+
- **`contentful.py`**: Contentful Management API integration for import/drop operations
72+
- **`input_data/`**: Source CSV and GeoJSON files
73+
74+
### Data Flow
75+
76+
Raw CSV/GeoJSON in `data-scripts/input_data/` -> Python scripts transform -> JSON output or Contentful import -> Frontend fetches from Contentful API at runtime + static GeoJSON from `public/data/`
77+
78+
## Linting
79+
80+
ESLint config (`.eslintrc.json`) extends: `plugin:vue/strongly-recommended`, `airbnb`, `plugin:cypress/recommended`, `prettier`. Prettier has final say on formatting.

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

data-scripts/Pipfile

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
[[source]]
2+
url = "https://pypi.org/simple"
3+
verify_ssl = true
4+
name = "pypi"
5+
6+
[packages]
7+
click = ">=8.3.1"
8+
petl = ">=1.7.17"
9+
ratelimit = ">=2.2.1"
10+
tqdm = ">=4.67.3"
11+
contentful-management = ">=2.15.1"
12+
13+
[dev-packages]
14+
15+
[requires]
16+
python_version = "3"

data-scripts/Pipfile.lock

Lines changed: 231 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

data-scripts/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,29 @@
22
Handy little python scripts to clean and merge voter turnout
33
and registration data. Also a script to migrate content.
44

5+
## Install dependencies
6+
7+
This project uses [pipenv](https://pipenv.pypa.io/) to manage Python dependencies.
8+
9+
```bash
10+
# Install pipenv if you don't have it
11+
pip install --user pipenv
12+
13+
# Install dependencies
14+
pipenv install
15+
16+
# Or to install the versions in the lock file without updating
17+
pipenv sync
18+
19+
# Run a script
20+
pipenv run python cli.py --help
21+
22+
# Start a virtual environment to run multiple commands
23+
pipenv shell
24+
```
25+
26+
## Usage
27+
528
```
629
Usage: cli.py [OPTIONS] COMMAND [ARGS]...
730

0 commit comments

Comments
 (0)