Skip to content

Fix perf runner json#7

Open
Chandanydv1234 wants to merge 29 commits into
zinggAI:mainfrom
Chandanydv1234:fix-perf-runner-json
Open

Fix perf runner json#7
Chandanydv1234 wants to merge 29 commits into
zinggAI:mainfrom
Chandanydv1234:fix-perf-runner-json

Conversation

@Chandanydv1234

@Chandanydv1234 Chandanydv1234 commented Jun 24, 2026

Copy link
Copy Markdown

Previously after the initial run the report was being over written every single execution. This meant loosing the older record data.
This PR modifies the script to store all the performance records and maintain a complete history of every single execution.
Changes made:-

  1. Storing as a list of dicts instead of a single dictionary ensuring every single execution report can be appended to the list. Raw appending of dictionaries would create multiple separate json elements, which breaks the file as two json elements cannot be present.
  2. Added safety checks in save_results to prevent FileNotFoundError and empty file crashes.
    3.Retrieves the last run (data[-1]) in load_results for the threshold comparison.
    4.Enables backward compatibility for the old dictionary formatted reports to automatically wrap them inside a list.

#2 Changes Made:-

  1. instead of appending to JSON list, reports are now being updated individually, on the basis of phase, in csv format. and also implemented yearly roll over logic, whenever year changes the past report get's archived in the past year's report(eg - train_report_2025.csv) and then starts a fresh file.
  2. Added a python script for plotting the reports. it reads all the reports dynamically ( *reports.csv). It compares run duration over time.
  3. The generated chart is embedded in readme as png. as performance_chart.png
  4. all of this automated using GitHub action. It automatically executes the plotting script using the updated csv and commits the new graph back to repo.

#3 Changes Made:-

  1. Now ploting multiple datasets on the same chart dynamically, styling each dataset with unique lines (solid/dashed) and symbols (circles/triangles).
  2. displaying PC Specs (dynamically parsed from the latest JSON report), Data Specs (parsed from the active config), and a Run Summary (aggregate averages, best times, and ratios).
  3. Made some ui changes to better align with the default Zingg.ai theme. Switched to dark theme. added shaded gradients under the curve.
  4. Displays the runtime at each point.

@sonalgoyal sonalgoyal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this into a csv file? I dont want json.

Comment thread .github/workflows/updated_graph.yml Outdated

- name: Plot results
run: |
python3 plot_result.py

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u send me the pr against oss action and not here. this is not the place we run our actions

Comment thread dummy_config1.json Outdated
@@ -0,0 +1,21 @@
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please clear out dummy configs

Comment thread perfTestRunner.py
WINDOW_THRESHOLD = 10 #set 10 minutes window threshold

def detect_pc_specs():
specs = {}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please move all stuff to the script, leave only running part and passing params in action in respective code repors of oss and ent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants