Skip to content

Commit 2ebccb1

Browse files
authored
Update OG2 README convergence metrics and image, fix DATASET.md path (#1562)
Update convergence benchmarks table with final training results (train loss 0.9444, test CE loss 0.9204, test perplexity 2.51), replace convergence plot with updated curve (that uses the final sharded dataset), and add missing data_files: null to DATASET.md custom sharded parquet config example. ### Description <!-- Provide a detailed description of the changes in this PR --> #### Usage <!--- How does a user interact with the changed code --> ```python TODO: Add code snippet ``` ### Type of changes <!-- Mark the relevant option with an [x] --> - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Refactor - [ ] Documentation update - [ ] Other (please describe): ### CI Pipeline Configuration Configure CI behavior by applying the relevant labels. By default, only basic unit tests are run. - [ciflow:skip](https://github.com/NVIDIA/bionemo-framework/blob/main/docs/docs/main/contributing/contributing.md#ciflow:skip) - Skip all CI tests for this PR - [ciflow:notebooks](https://github.com/NVIDIA/bionemo-framework/blob/main/docs/docs/main/contributing/contributing.md#ciflow:notebooks) - Run Jupyter notebooks execution tests - [ciflow:slow](https://github.com/NVIDIA/bionemo-framework/blob/main/docs/docs/main/contributing/contributing.md#ciflow:slow) - Run slow single GPU integration tests marked as @pytest.mark.slow - [ciflow:all](https://github.com/NVIDIA/bionemo-framework/blob/main/docs/docs/main/contributing/contributing.md#ciflow:all) - Run all tests (unit tests, slow tests, and notebooks). This label can be used to enforce running all framework tests. - [ciflow:all-recipes](https://github.com/NVIDIA/bionemo-framework/blob/main/docs/docs/main/contributing/contributing.md#ciflow:all-recipes) - Run tests for all recipes (under bionemo-recipes). This label can be used to enforce running tests for all recipes. Unit tests marked as `@pytest.mark.multi_gpu` or `@pytest.mark.distributed` are not run in the PR pipeline. For more details, see [CONTRIBUTING](CONTRIBUTING.md) > [!NOTE] > By default, only basic unit tests are run. Add appropriate labels to enable an additional test coverage. #### Authorizing CI Runs We use [copy-pr-bot](https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/#automation) to manage authorization of CI runs on NVIDIA's compute resources. - If a pull request is opened by a trusted user and contains only trusted changes, the pull request's code will automatically be copied to a pull-request/ prefixed branch in the source repository (e.g. pull-request/123) - If a pull request is opened by an untrusted user or contains untrusted changes, an NVIDIA org member must leave an `/ok to test` comment on the pull request to trigger CI. This will need to be done for each new commit. #### Triggering Code Rabbit AI Review To trigger a code review from code rabbit, comment on a pull request with one of these commands: - @coderabbitai review - Triggers a standard review - @coderabbitai full review - Triggers a comprehensive review See https://docs.coderabbit.ai/reference/review-commands for a full list of commands. ### Pre-submit Checklist <!--- Ensure all items are completed before submitting --> - [ ] I have tested these changes locally - [ ] I have updated the documentation accordingly - [ ] I have added/updated tests as needed - [ ] All existing tests pass successfully Signed-off-by: Savitha Srinivasan <savithas@nvidia.com>
1 parent 85d0c16 commit 2ebccb1

3 files changed

Lines changed: 2 additions & 1 deletion

File tree

bionemo-recipes/recipes/opengenome2_llama_native_te/DATASET.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ TO 'output' (FORMAT PARQUET, PER_THREAD_OUTPUT true, FILE_SIZE_BYTES '200MB');
162162
```yaml
163163
dataset:
164164
load_dataset_kwargs:
165+
data_files: null
165166
path: "/path/to/your/resharded_parquet_dir"
166167
split: "train"
167168
streaming: true

bionemo-recipes/recipes/opengenome2_llama_native_te/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ which may also contribute to its lower test performance.
2626

2727
| Model | Step / checkpoint | Train loss | Mean Test loss | Mean Test Perplexity |
2828
| -------------------------- | ----------------- | ---------- | -------------- | -------------------- |
29-
| LlaMA3 Recipe (OG2 7B) | 182313 | 0.95 | 0.92 | 2.50 |
29+
| LlaMA3 Recipe (OG2 7B) | 182313 | 0.9444 | 0.9204 | 2.51 |
3030
| Megatron baseline (OG2 7B) | 182313 | 1.01 | 1.019 | 2.80 |
3131

3232
> **Evaluation methodology:** Test losses are average NLL (negative log-likelihood) computed using
-3.48 KB
Loading

0 commit comments

Comments
 (0)