Skip to content

Commit f08204d

Browse files
sbasu107sbasu107
authored
Merge pull request #111 from badgerloop-software/data-replayer
Added Data replayer functionality
2 parents e8b8f11 + c1b64aa commit f08204d

7 files changed

Lines changed: 84087 additions & 5 deletions

File tree

‎DataGenerator/README.md‎

Lines changed: 137 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,137 @@
1-
-Generate data
2-
-Store data in bufferarray
3-
-Send data through TCP to
1+
# DataGenerator
2+
3+
The DataGenerator simulates live solar car telemetry data by generating realistic sensor readings and GPS coordinates, then transmitting them via UDP packets to the backend dashboard system.
4+
5+
## Features
6+
7+
- **Realistic Data Simulation**: Generates mathematically varied sensor readings using sine wave functions
8+
- **GPS Route Simulation**: Cycles through real GPS coordinates from `gps_dataset/dataset1.csv`
9+
- **Real-time Timestamps**: Automatically generates current timestamps for live dashboard compatibility
10+
- **UDP Transmission**: Sends data packets at 31Hz (32ms intervals) to match real telemetry rates
11+
- **Binary Protocol**: Uses the same binary format as live car data (`<bsr>` header/footer)
12+
- **Comprehensive Sensors**: Simulates all solar car systems (battery, motor, GPS, temperatures, etc.)
13+
14+
## Data Sources
15+
16+
### GPS Dataset
17+
- **File**: `gps_dataset/dataset1.csv` (21,435 coordinate points)
18+
- **Format**: CSV with columns: `time`, `latitude`, `longitude`, `elevation`
19+
- **Coverage**: Real GPS route data for realistic car movement simulation
20+
- **Cycling**: Automatically loops through coordinates when reaching the end
21+
22+
### Sensor Simulation
23+
- **Algorithm**: Mathematical sine wave generation for realistic sensor variation
24+
- **Scope**: All fields defined in `../../Backend/Data/sc1-data-format/format.json`
25+
- **Timestamps**: Live system time (hour, minute, second, millisecond, Unix timestamp)
26+
27+
## Usage
28+
29+
1. Ensure GPS dataset is available in `gps_dataset/dataset1.csv`
30+
2. Navigate to the DataGenerator directory and run:
31+
```bash
32+
cd DataGenerator
33+
npm install
34+
npm start
35+
```
36+
3. The generator will:
37+
- Build the project using Babel
38+
- Start generating and sending UDP packets to `localhost:4003`
39+
- Cycle through GPS coordinates continuously
40+
- Generate varied sensor readings in real-time
41+
42+
## Installation
43+
44+
Install the required dependencies:
45+
```bash
46+
npm install
47+
```
48+
49+
### Dependencies
50+
- **luxon**: Date/time manipulation for accurate timestamps
51+
- **express**: Web framework (used for potential future HTTP endpoints)
52+
53+
### Development Dependencies
54+
- **@babel/cli**: Command-line interface for Babel transpilation
55+
- **@babel/core**: Core Babel functionality
56+
- **@babel/preset-env**: Smart preset for modern JavaScript features
57+
58+
## Data Flow
59+
60+
```
61+
GPS Dataset + Sensor Simulation → Binary Buffer → UDP Packet → Backend (port 4003)
62+
```
63+
64+
The generator:
65+
1. Loads GPS coordinates from the CSV dataset
66+
2. Generates sensor readings using mathematical functions
67+
3. Applies current timestamps to all time-related fields
68+
4. Packs data into binary format matching the solar car protocol
69+
5. Transmits UDP packets at 31Hz to simulate real telemetry
70+
71+
## Binary Protocol
72+
73+
The DataGenerator follows the exact binary format used by the actual solar car:
74+
75+
- **Header**: `<bsr>` (5 bytes)
76+
- **Data**: Binary encoded according to field types defined in the format specification
77+
- **Footer**: `</bsr>` (6 bytes)
78+
79+
### Supported Data Types
80+
- `float`: 32-bit floating point (Little Endian)
81+
- `uint8`: 8-bit unsigned integer
82+
- `uint16`: 16-bit unsigned integer (Little Endian)
83+
- `uint64`: 64-bit unsigned integer (Little Endian)
84+
- `char`: 8-bit character
85+
- `bool`: Boolean (0 or 1)
86+
87+
## Special Field Handling
88+
89+
### GPS Coordinates
90+
- `lat`: Latitude from GPS dataset
91+
- `lon`: Longitude from GPS dataset
92+
- `elev`: Elevation from GPS dataset
93+
94+
### Timestamps
95+
- `tstamp_hr`: Current hour (0-23)
96+
- `tstamp_mn`: Current minute (0-59)
97+
- `tstamp_sc`: Current second (0-59)
98+
- `tstamp_ms`: Current millisecond (0-999)
99+
- `tstamp_unix`: Current Unix timestamp
100+
101+
### Sensor Values
102+
All other fields are generated using `Math.abs(Math.sin(nextValue)) * 100` to create realistic, continuously varying sensor readings.
103+
104+
## Build Process
105+
106+
The project uses Babel to transpile modern JavaScript:
107+
1. Source code in `src/` is transpiled to `dist/`
108+
2. Node.js executes the transpiled code from `dist/server.js`
109+
3. Babel configuration in `.babelrc` targets current Node.js version
110+
111+
## Architecture
112+
113+
### Core Components
114+
- **server.js**: Main UDP server and data generation logic
115+
- **car.js**: Car simulation class (future expansion capability)
116+
- **constants.json**: Configuration constants (port numbers, etc.)
117+
118+
### Data Format Integration
119+
- Reads format specification from `../../Backend/Data/sc1-data-format/format.json`
120+
- Dynamically calculates buffer sizes based on data types
121+
- Ensures perfect compatibility with backend data parsing
122+
123+
## Development
124+
125+
### Adding New Sensors
126+
1. Update the format specification in the backend
127+
2. Add special handling in `server.js` if needed
128+
3. The generator will automatically include new fields
129+
130+
### Modifying GPS Routes
131+
1. Replace `gps_dataset/dataset1.csv` with new GPS data
132+
2. Ensure CSV format: `time,latitude,longitude,elevation`
133+
3. Update `dataset1.json` if using JSON format instead
134+
135+
## Stopping
136+
137+
Press `Ctrl+C` to gracefully stop the data generator.

‎DataReplayer/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# DataReplayer
2+
3+
The DataReplayer reads recorded CSV data and replays it by sending UDP packets to simulate live car data, similar to the DataGenerator.
4+
5+
## Features
6+
7+
- Loads CSV data from `raw_data/2024-4-7rawdata.csv`
8+
- Parses CSV headers and maps them to the solar car data format
9+
- **Generates current timestamps** for live graphing compatibility
10+
- Sends data packets via UDP at 31Hz (32ms intervals) to match real-time telemetry
11+
- Automatically loops through the dataset when it reaches the end
12+
- Uses the same binary format as the DataGenerator (`<bsr>` header/footer)
13+
14+
## Usage
15+
16+
1. Make sure you have the CSV file in `raw_data/2024-4-7rawdata.csv`
17+
2. Navigate to the DataReplayer directory and run:
18+
```bash
19+
cd DataReplayer
20+
npm start
21+
```
22+
3. The replayer will:
23+
- Load all CSV data into memory
24+
- Start sending UDP packets to `localhost:4003`
25+
- Display progress updates every 100 packets
26+
27+
## Installation
28+
29+
No additional dependencies are needed since the DataReplayer uses only Node.js built-in modules:
30+
- `fs` - File system operations
31+
- `readline` - CSV parsing
32+
- `dgram` - UDP socket communication
33+
- `buffer` - Binary data handling
34+
35+
## Data Flow
36+
37+
```
38+
CSV File → Parse → Binary Buffer → UDP Packet → Backend (port 4003)
39+
```
40+
41+
The replayer:
42+
- Reads the CSV headers and maps them to the solar car data format
43+
- **Replaces old timestamps with current time** to enable live graphing
44+
- Converts each row to a binary buffer using the same format as DataGenerator
45+
- Sends packets at the same rate as live telemetry (31Hz)
46+
- Loops through the dataset continuously
47+
48+
## Timestamp Handling
49+
50+
The DataReplayer automatically replaces timestamp fields from the CSV with current timestamps:
51+
- `tstamp_hr`, `tstamp_mn`, `tstamp_sc` → Current hour, minute, second
52+
- `tstamp_ms` → Current milliseconds
53+
- `tstamp_unix` → Current Unix timestamp
54+
55+
This ensures the frontend's live graphing feature works correctly, as it expects "now" timestamps rather than historical ones from 2024.
56+
57+
## Data Format
58+
59+
The binary packets follow the format defined in `../../Backend/Data/sc1-data-format/format.json`:
60+
- Header: `<bsr>`
61+
- Data: Binary encoded according to field types (float, uint8, etc.)
62+
- Footer: `</bsr>`
63+
64+
## Stopping
65+
66+
Press `Ctrl+C` to gracefully stop the replayer.

‎DataReplayer/package.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"name": "chase-car-dashboard-datareplayer",
3+
"version": "1.0.0",
4+
"description": "Replays recorded CSV data by sending UDP packets to simulate live car data",
5+
"main": "src/server.js",
6+
"scripts": {
7+
"start": "node src/server.js",
8+
"dev": "node src/server.js"
9+
},
10+
"keywords": [
11+
"solar-car",
12+
"telemetry",
13+
"replay",
14+
"udp",
15+
"data"
16+
],
17+
"author": "BSR"
18+
}

0 commit comments

Comments
 (0)