Backfill parquet versions for 4 datasets added after #241#245
Merged
Conversation
The submissions merged after the parquet conversion (#235 catechol transient flow, #238 decarboxylative olefination x2) only had .pb.gz. Generated 1:1 parquet siblings with scripts/convert_to_parquet.py (all singletons; no de-shard merge applies). Reaction counts match the originals (1227/120/136) and validate_dataset.py passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add README maintainer notes for backfilling Parquet with convert_to_parquet.py and for routing new-LFS-object pushes to GitHub (since .lfsconfig points reads at the HF mirror and has no pushurl). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bdeadman
approved these changes
Jun 16, 2026
The Cernak 50,688-reaction Pd/Ni/Cu C–N coupling dataset (#243) was merged after the parquet conversion (#241), so it only had .pb.gz. Generated its 1:1 parquet sibling with scripts/convert_to_parquet.py (singleton path; the name matches no MERGE_SPEC, so it keeps its existing dataset_id). Reaction count matches the original (50688) and the parquet round-trips. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Backfills parquet versions for the four datasets submitted after the parquet conversion (#241), which therefore only had
.pb.gz:data/1e/ord_dataset-1ec2807f…— catechol transient flow (#233 catechol transient flow dataset from SOLVE #235), 1227 reactionsdata/c7/ord_dataset-c703268e…— decarboxylative olefination, Bayesian optimization (Data submission - Decarboxylative Olefination (Machine learning optim… #238), 120 reactionsdata/dc/ord_dataset-dc024993…— decarboxylative olefination, transfer learning (Data submission - Decarboxylative Olefination (Machine learning optim… #238), 136 reactionsdata/80/ord_dataset-805ad863…— Cernak 50k Pd/Ni/Cu C–N coupling (Cernak 50k CN Coupling #243), 50688 reactionsHow
Generated 1:1 parquet siblings with
scripts/convert_to_parquet.py(the singleton path — none of the four match a de-shardMERGE_SPEC, so no merging applies and each keeps its existingdataset_id). Reaction counts match the originals exactly (1227 / 120 / 136 / 50688), and the parquet outputs round-trip with matching row counts.mainis merged in so the branch carries the #243.pb.gzalongside its new parquet sibling.The
skip-update-submissionlabel is set because these are format conversions of already-finalized datasets that should not be re-processed byprocess_dataset --update. Validation runs against the full dataset on merge tomain.🤖 Generated with Claude Code
Greptile Summary
This PR backfills Parquet siblings for datasets that were merged to
mainafter the initial Parquet-conversion PR (#241) and therefore only had.pb.gzfiles. It also adds a maintainer-facing README section documenting the conversion and LFS-push workflow..parquetfiles are added: the catechol transient flow dataset (1227 reactions), two decarboxylative olefination datasets (120 and 136 reactions), and the Cernak C–N coupling dataset — all 1:1 singletons carrying their existingdataset_ids.--dry-runflag exists inscripts/convert_to_parquet.py, andord-schema==0.6.3matchesORD_SCHEMA_TAG: v0.6.3used in the CI workflow files.data/80/ord_dataset-805ad863…) for the Cernak dataset; the description should be updated before merge.Confidence Score: 5/5
Safe to merge — adds LFS-tracked Parquet siblings for four already-finalised datasets and accurate maintainer documentation; no logic changes.
All changes are additive: four new Parquet LFS pointers and README documentation. The conversion script is idempotent and was run with validate_dataset.py passing locally. The README instructions accurately reflect the script flags and pinned schema version used in CI.
No files require special attention; the discrepancy between the PR description ('3 datasets') and the four added files is cosmetic.
Important Files Changed
--dry-runflag exists in the script,ord-schema==0.6.3matchesORD_SCHEMA_TAG: v0.6.3in the workflow files..pb.gz..pb.gz..pb.gz.Reviews (4): Last reviewed commit: "Backfill parquet version for Cernak C–N ..." | Re-trigger Greptile