feat(analysis): live stage progress + packet-based time estimate#567
Conversation
The analysis loading view previously showed a size-only time estimate
(~0.5 s/MB) that ignored the nDPI / Suricata / file-extraction options,
and a progress bar driven by a local timer rather than real work.
Backend:
- Report live stage progress from the pipeline via an in-memory
AnalysisProgressService (out-of-band, since analyzeFile runs in one
long transaction), exposed at GET /analysis/{id}/progress. Bar advances
on true milestones and holds during opaque external stages.
- Capture packet count at upload via `capinfos -M -c -` (best-effort,
stdin stream, ~ms) and base the time estimate on packet count — cost
tracks packets, not bytes — weighted per enabled stage. Falls back to
the size-based formula when the count is unknown. Coefficients
calibrated against a clean offline run (69s est vs 65s actual).
Frontend:
- AnalysisLoadingView polls progress, shows "Stage X of Y", drives the
bar off real percent, and lists which stages the estimate includes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 42 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughChangesAnalysis progress and estimation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant AnalysisService
participant AnalysisProgressService
participant AnalysisController
participant AnalysisLoadingView
AnalysisService->>AnalysisProgressService: Publish weighted stage progress
AnalysisLoadingView->>AnalysisController: GET /analysis/{fileId}/progress
AnalysisController->>AnalysisProgressService: Read latest progress
AnalysisProgressService-->>AnalysisController: 200 progress or 204 no content
AnalysisController-->>AnalysisLoadingView: AnalysisProgressResponse
AnalysisLoadingView->>AnalysisLoadingView: Render stage or estimate fallback
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces live analysis progress tracking and more accurate analysis time estimates. It adds an in-memory AnalysisProgressService to track progress through weighted pipeline stages, exposes a new progress endpoint, and integrates it into the frontend's loading view. Additionally, it utilizes capinfos to count packets up front for a packet-count-based time estimate. The review feedback highlights critical issues where calling readAllBytes() before process.waitFor() can block threads indefinitely and bypass timeouts. It also identifies a potential memory leak in AnalysisService if temp file creation fails, and suggests adding defensive guard clauses to prevent index out of bounds errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@backend/src/main/java/com/tracepcap/file/service/FileServiceImpl.java`:
- Around line 91-101: The synchronous countPackets call currently runs inside
the transactional uploadFile and mergeFiles flows, holding database and request
resources during capinfos execution. Move packet-count computation outside the
transaction or schedule it asynchronously after commit, while preserving the
metadata update and using a shorter best-effort timeout so packet counting
cannot block these transactions for the full analysis timeout.
- Around line 417-486: Update both countPackets overloads to execute capinfos
against a file path, writing MultipartFile content to a temporary file and
cleaning it up afterward instead of piping stdin. In each method, drain stdout
and stderr concurrently while enforcing the 60-second process timeout, then
parse stdout only after successful completion; forcibly terminate timed-out
processes and preserve the existing best-effort null behavior.
In `@frontend/src/services/api/endpoints.ts`:
- Around line 10-12: Update the section comment above ANALYSIS_SUMMARY,
ANALYSIS_PROGRESS, and PROTOCOL_STATS to remove the claim that the analysis
endpoints are not implemented in the backend, while leaving the endpoint
definitions unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 880ad3b4-e0de-4131-9a81-64c69f122070
📒 Files selected for processing (9)
backend/src/main/java/com/tracepcap/analysis/controller/AnalysisController.javabackend/src/main/java/com/tracepcap/analysis/dto/AnalysisProgressResponse.javabackend/src/main/java/com/tracepcap/analysis/service/AnalysisProgressService.javabackend/src/main/java/com/tracepcap/analysis/service/AnalysisService.javabackend/src/main/java/com/tracepcap/file/dto/FileMetadataDto.javabackend/src/main/java/com/tracepcap/file/mapper/FileMapper.javabackend/src/main/java/com/tracepcap/file/service/FileServiceImpl.javafrontend/src/pages/Analysis/AnalysisLoadingView.tsxfrontend/src/services/api/endpoints.ts
Address PR review + schema-drift workflow:
- countPackets: drain capinfos stdout on a daemon thread so a hung
process can't block past waitFor's timeout; use UTF-8 (both overloads).
- reportStage: guard out-of-range stage index instead of crashing.
- Clear live progress on early-failure paths that skip the inner finally
(e.g. createTempFile throwing after the first stage report).
- Regenerate openapi/baseline.json for the new /analysis/{id}/progress
endpoint + estimatedAnalysisSeconds field (additive only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Shorten the best-effort capinfos packet-count timeout 60s -> 15s so it can't hold a request thread / DB connection long inside the upload/merge transaction (CodeRabbit). capinfos -c only scans record headers; on timeout we fall back to the size-based estimate. - Drop the stale "Not yet implemented in backend" comment on the analysis endpoints — summary/progress/protocols are all implemented. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 participant
Problem
While a capture is analysing, the loading view showed:
~0.5 s/MB) that ignored the nDPI / Suricata / file-extraction choices made at upload, andWhat changed
Real stage progress
AnalysisProgressServicepublishes per-stage progress from the pipeline. It's out-of-band becauseanalyzeFileruns in one long transaction, so nothing it writes is visible to pollers until commit.GET /analysis/{id}/progress(200 with{stageIndex, totalStages, stage, percent}, 204 when nothing is tracked).Packet-based time estimate
capinfos -M -c -(best-effort stdin stream, ~ms;null→ size-based fallback) and stored onFileEntity.packetCount.FileMappercomputes the estimate from packet count, weighted per enabled stage, and it's surfaced on the file metadata the loading view already fetches.Frontend
AnalysisLoadingViewpolls progress, shows "Stage X of Y: …", drives the bar off real percent, and notes which stages the estimate includes.Testing
Verified end-to-end on the running stack with
editcap.427fbe9e.pcap(13 MB, 21,364 packets):capinfoscount matched tshark's post-analysis count exactly (both 21364).GEO_FORCE_OFFLINE=true, so geo uses the bundled MMDB): per-stage logs showed Extract 55s dominant, geo just 1.5s; tuned estimate 69s vs 65s actual (~6% conservative).Notes / follow-ups
[1/7]…[7/7]logs across more captures over time.packetCountcolumn).🤖 Generated with Claude Code
Summary by CodeRabbit