Skip to content

chore: clean up and document TODOs in processing module#3

Open
devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1777288837-resolve-processing-todos
Open

chore: clean up and document TODOs in processing module#3
devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1777288837-resolve-processing-todos

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Apr 27, 2026

Copy link
Copy Markdown

Reference Issues/PRs

N/A — GitHub issues are disabled on this repo. See TODO inventory below.

What does this implement or fix?

Audits and cleans up all 18 TODOs across clause.cpp (14), clause.hpp (2), clause_utils.cpp (1), and unsorted_aggregation.cpp (1) in the processing module.

Changes made:

  1. Removed 1 obsolete TODO (clause.cpp:2168): The comment requested exception handling for duplicate source-to-target row matches. This is already implemented at lines 2296-2311 via a matched_target_rows BitSet that raises E_INVALID_USER_ARGUMENT. Replaced with a brief note pointing to the existing validation.

  2. Clarified 17 remaining TODOs with improved context:

    • Stripped internal Monday.com ticket references from all comments
    • Fixed misleading wording ("update must be done" → correctly says "insert")
    • Added specific descriptions of what each TODO requires and why

No functional code changes. All edits are comment-only.


TODO Inventory (for future tracking)

Category Files / Lines Summary
Merge update insert support clause.cpp:140, 151, 2125, 2386, 2399 Source rows with no target match should be inserted. Currently skipped. Affects both timeseries and rowrange paths, string and non-string columns.
Sparse target columns clause.cpp:144 for_each_enumerated skips sparse positions, so matched source rows targeting sparse target positions won't be updated.
Parallel MergeClause clause.cpp:1563 MergeClause flattens all entities into one group (serial). Should be parallelized like ResampleClause.
Arrow input for merge update clause.cpp:2105 Arrow format not yet supported as merge update input; error raised in process().
Dynamic schema support clause.cpp:2183, 2533, 2544 Source/target descriptors, field indexes, and type constraints assume static schema.
Performance optimization clause.cpp:2298, 2453 Duplicate-check loop could be inlined; index matching loop could be inverted when source > target.
PassthroughClause structuring clause.hpp:118, 124 Two overloads use inconsistent structuring strategies (row-slice vs. passthrough).
Arrow + multi-index concat clause_utils.cpp:321 Blocked on arrow normalization metadata finalization.
EMPTYVAL in MeanAggregator unsorted_aggregation.cpp:492 Fully sparse column approach works but may not be optimal; revisit when feature flag is removed.

Any other comments?

These are purely comment/documentation changes — no logic was modified.

Checklist

Checklist for code changes...
  • Have you updated the relevant docstrings, documentation and copyright notice?
  • Is this contribution tested against all ArcticDB's features? (comment-only changes, no functional impact)
  • Do all exceptions introduced raise appropriate error messages? (N/A)
  • Are API changes highlighted in the PR description? (N/A — no API changes)
  • Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? (N/A — housekeeping)

Link to Devin session: https://app.devin.ai/sessions/7b17aae56da9472ba724c9658ee07783
Requested by: @chrischappell-cog


Open in Devin Review

- Remove obsolete TODO at clause.cpp:2168 (duplicate source row check
  already implemented at lines 2296-2311 via matched_target_rows BitSet)
- Clarify merge update insert TODOs with specific context about what
  insertion means and how empty vectors signal unmatched source rows
- Document sparse target column limitation in merge_update_string_column_timeseries
- Improve MergeClause parallel processing TODO with reference to ResampleClause
- Strip internal Monday.com ticket references from all TODO comments
- Fix misleading 'update must be done' wording to correctly say 'insert'
- Add context to dynamic schema, Arrow support, and performance TODOs

Files changed:
  clause.cpp (14 TODOs: 1 removed, 13 clarified)
  clause.hpp (2 TODOs clarified)
  clause_utils.cpp (1 TODO clarified)
  unsorted_aggregation.cpp (1 TODO clarified)

Co-Authored-By: Chris Chappell <chris.chappell@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant