Skip to content

Commit 2f6fff4

Browse files
abdusabriabdusabri
committed
Prepare project for public release
0 parents  commit 2f6fff4

34 files changed

Lines changed: 6355 additions & 0 deletions

‎.github/workflows/ci.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main, master]
6+
pull_request:
7+
branches: [main, master]
8+
9+
permissions:
10+
contents: read
11+
12+
concurrency:
13+
group: ci-${{ github.workflow }}-${{ github.ref }}
14+
cancel-in-progress: true
15+
16+
jobs:
17+
validate:
18+
name: Format, lint, and build
19+
runs-on: ubuntu-latest
20+
timeout-minutes: 10
21+
22+
steps:
23+
- name: Check out repository
24+
uses: actions/checkout@v6
25+
26+
- name: Set up Node.js
27+
uses: actions/setup-node@v6
28+
with:
29+
node-version: 22.14.0
30+
cache: npm
31+
32+
- name: Install dependencies
33+
run: npm ci
34+
35+
- name: Check formatting
36+
run: npm run format:check
37+
38+
- name: Lint
39+
run: npm run lint
40+
41+
- name: Build
42+
run: npm run build

‎.gitignore‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
pnpm-debug.log*
8+
lerna-debug.log*
9+
10+
node_modules
11+
dist
12+
dist-ssr
13+
.playwright-cli/
14+
*.local
15+
16+
# Editor directories and files
17+
.vscode/*
18+
!.vscode/extensions.json
19+
.idea
20+
.DS_Store
21+
*.suo
22+
*.ntvs*
23+
*.njsproj
24+
*.sln
25+
*.sw?

‎.prettierrc‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"singleQuote": true,
3+
"tabWidth": 2,
4+
"plugins": ["prettier-plugin-tailwindcss"],
5+
"tailwindStylesheet": "./src/index.css"
6+
}

‎README.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# High-Performance CSV Plotter
2+
3+
This project contains a React and TypeScript app for plotting large CSV datasets in the browser.
4+
5+
Live app: [plotting.abdusabri.com](https://plotting.abdusabri.com/)
6+
7+
## Overview
8+
9+
- React 18 + TypeScript app built with Vite
10+
- Browser-based CSV upload and parsing
11+
- Web Worker-based parsing to keep the UI responsive during file loading
12+
- Sliding window playback with configurable start position, window size, update interval, and increment size
13+
- `uPlot`-based chart rendering with downsampling for large windows
14+
- Window-level `min`, `max`, `average`, and `variance` calculations
15+
16+
## Prerequisites
17+
18+
- Node.js: `22.14.0` or newer
19+
- npm: `10.9.2` or newer
20+
21+
## Quick Start
22+
23+
### 1. Install dependencies
24+
25+
```bash
26+
npm install
27+
```
28+
29+
### 2. Start the development server
30+
31+
```bash
32+
npm run dev
33+
```
34+
35+
The app runs at [http://localhost:5173](http://localhost:5173).
36+
37+
### 3. Load a dataset
38+
39+
Use the file picker in the app to load a local CSV file, or use the linked 2M-row sample file exposed in the UI.
40+
41+
## Data Format
42+
43+
The current parser expects a simple two-column numeric CSV:
44+
45+
```csv
46+
0,12.4
47+
1,12.9
48+
2,11.7
49+
```
50+
51+
- Each row should contain `x,y`
52+
- Rows that do not parse into numeric pairs are skipped
53+
- The parser does not currently support headers or column selection
54+
55+
## Scripts
56+
57+
| Script | Description |
58+
| :--------------------- | :--------------------------------------------------------------- |
59+
| `npm run dev` | Starts the Vite development server. |
60+
| `npm run build` | Runs TypeScript build checks and bundles the app for production. |
61+
| `npm run lint` | Runs ESLint. |
62+
| `npm run preview` | Serves the production build locally. |
63+
| `npm run format` | Formats the repository with Prettier. |
64+
| `npm run format:check` | Checks repository formatting without writing changes. |
65+
66+
## Project Structure
67+
68+
```text
69+
src/
70+
App.tsx # Main page layout and top-level composition
71+
main.tsx # App entrypoint
72+
index.css # Global styles
73+
constants/ # Default plotting and playback settings
74+
types/ # Shared TypeScript types
75+
hooks/
76+
useDataStream.ts # File loading, worker lifecycle, and progress state
77+
usePlotWindow.ts # Sliding window state and playback loop
78+
utils/
79+
dataProcessor.ts # Window aggregates and downsampling logic
80+
workers/
81+
csvParser.worker.ts # Chunked CSV parsing in a Web Worker
82+
components/
83+
FileSelector.tsx # File upload and sample data entry point
84+
PlotControls.tsx # Window and playback controls
85+
DataPlot.tsx # uPlot chart wrapper
86+
Aggregates.tsx # Aggregate metrics display
87+
ui/ # Shared UI primitives
88+
```
89+
90+
## Implementation Notes
91+
92+
- Vite, React, and TypeScript keep the app small and straightforward while still making the data flow easy to reason about.
93+
- `uPlot` and `uplot-react` are used for chart rendering because the app is centered on plotting large datasets efficiently.
94+
- Tailwind CSS and `shadcn/ui` are used for the controls and layout so the implementation can stay focused on the plotting behavior.
95+
- CSV parsing runs in [`src/workers/csvParser.worker.ts`](/home/abdu/dev/abdu-fe-data-plot/src/workers/csvParser.worker.ts), where the file is read in chunks and posted back to the main thread in batches.
96+
- The playback loop in [`src/hooks/usePlotWindow.ts`](/home/abdu/dev/abdu-fe-data-plot/src/hooks/usePlotWindow.ts) uses `requestAnimationFrame` while still respecting the configured update interval.
97+
- [`src/utils/dataProcessor.ts`](/home/abdu/dev/abdu-fe-data-plot/src/utils/dataProcessor.ts) calculates aggregates and downsampled chart data together for the active window.
98+
99+
## Performance Notes
100+
101+
- File parsing happens off the main thread, so loading large files does not block React rendering.
102+
- The worker reads the file in chunks instead of decoding the whole file in one synchronous pass.
103+
- The chart renders a downsampled view of the active window rather than every point in that window.
104+
- Aggregate metrics are calculated from the full active window, not from the downsampled subset.
105+
- In practice, the active window size is the main performance lever once the file has been loaded.
106+
107+
## Limitations
108+
109+
- The parser currently expects a narrow two-column numeric CSV format.
110+
- The full parsed dataset is kept in browser memory after loading.
111+
- Aggregate calculations are recomputed for each active window.

‎components.json‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{
2+
"$schema": "https://ui.shadcn.com/schema.json",
3+
"style": "new-york",
4+
"rsc": false,
5+
"tsx": true,
6+
"tailwind": {
7+
"config": "",
8+
"css": "src/index.css",
9+
"baseColor": "neutral",
10+
"cssVariables": true,
11+
"prefix": ""
12+
},
13+
"aliases": {
14+
"components": "@/components",
15+
"utils": "@/lib/utils",
16+
"ui": "@/components/ui",
17+
"lib": "@/lib",
18+
"hooks": "@/hooks"
19+
},
20+
"iconLibrary": "lucide"
21+
}

‎eslint.config.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import js from '@eslint/js';
2+
import globals from 'globals';
3+
import reactHooks from 'eslint-plugin-react-hooks';
4+
import reactRefresh from 'eslint-plugin-react-refresh';
5+
import tseslint from 'typescript-eslint';
6+
7+
export default tseslint.config(
8+
{ ignores: ['dist'] },
9+
{
10+
extends: [js.configs.recommended, ...tseslint.configs.recommended],
11+
files: ['**/*.{ts,tsx}'],
12+
languageOptions: {
13+
ecmaVersion: 2020,
14+
globals: globals.browser,
15+
},
16+
plugins: {
17+
'react-hooks': reactHooks,
18+
'react-refresh': reactRefresh,
19+
},
20+
rules: {
21+
...reactHooks.configs.recommended.rules,
22+
'react-refresh/only-export-components': [
23+
'warn',
24+
{ allowConstantExport: true },
25+
],
26+
},
27+
},
28+
);

‎index.html‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
7+
<title>High-Performance CSV Plotter</title>
8+
</head>
9+
<body>
10+
<div id="root"></div>
11+
<script type="module" src="/src/main.tsx"></script>
12+
</body>
13+
</html>

0 commit comments

Comments
 (0)