test: expand coverage for duplicate entry checks#103
test: expand coverage for duplicate entry checks#103gbemike wants to merge 2 commits intoevaleval:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Expands the test suite around every_eval_ever.check_duplicate_entries to better cover CLI behavior and edge cases, supporting the broader goal of increasing repo-wide test coverage (Issue #88).
Changes:
- Add coverage for
main()raising on invalid JSON input. - Add a “no duplicates found” CLI output case and strengthen duplicate-output assertions.
- Add coverage for rejecting direct non-JSON file paths in
expand_paths().
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| assert check_module.main([str(file_a), str(file_b)]) == 1 | ||
| captured = capsys.readouterr().out | ||
| assert 'Found duplicate entries' in captured | ||
| assert 'Found duplicate entries (ignoring keys: `evaluation_id`, `retrieved_timestamp`)' in captured |
There was a problem hiding this comment.
The assertion for the duplicate warning hard-codes the ignored key list. Since the CLI message is derived from check_module.IGNORE_KEYS, consider building the expected substring from check_module.IGNORE_KEYS (sorted) to keep the test in sync if the ignore set changes.
| assert 'Found duplicate entries (ignoring keys: `evaluation_id`, `retrieved_timestamp`)' in captured | |
| ignored_keys = ', '.join( | |
| f'`{key}`' for key in sorted(check_module.IGNORE_KEYS) | |
| ) | |
| assert f'Found duplicate entries (ignoring keys: {ignored_keys})' in captured |
|
|
||
| with pytest.raises(Exception, match='Could not find file or directory'): |
There was a problem hiding this comment.
For the .txt path case, the test matches the same 'Could not find file or directory' message used for missing paths. That message is a bit misleading for an existing-but-unsupported file, and matching on it makes the intent (unsupported extension rejection) less clear. Consider either asserting only that an exception is raised for non-JSON files, or (longer-term) updating expand_paths to raise a distinct exception/message for unsupported extensions.
| with pytest.raises(Exception, match='Could not find file or directory'): | |
| with pytest.raises(Exception): |
|
|
||
| payload_a = clone_payload(payload) | ||
| payload_a['evaluation_id'] = 'eval-c' | ||
| payload_a['retrieved_timestamp'] = '2024-01-03' | ||
| if ( | ||
| isinstance(payload_a.get('evaluation_results'), list) | ||
| and payload_a['evaluation_results'] | ||
| ): | ||
| payload_a['evaluation_results'][0]['score_details']['score'] = ( | ||
| payload_a['evaluation_results'][0]['score_details']['score'] + 0.001 | ||
| ) | ||
|
|
||
| write_json(file_c, payload_a) | ||
|
|
There was a problem hiding this comment.
In test_main_reports_no_duplicates, the variable payload_a is actually the modified payload written to file_c, which is a bit confusing when reading the test. Renaming it to something like payload_c (and cleaning up the extra blank lines/trailing whitespace in this block) would improve readability and keep formatting consistent with the rest of the tests.
| payload_a = clone_payload(payload) | |
| payload_a['evaluation_id'] = 'eval-c' | |
| payload_a['retrieved_timestamp'] = '2024-01-03' | |
| if ( | |
| isinstance(payload_a.get('evaluation_results'), list) | |
| and payload_a['evaluation_results'] | |
| ): | |
| payload_a['evaluation_results'][0]['score_details']['score'] = ( | |
| payload_a['evaluation_results'][0]['score_details']['score'] + 0.001 | |
| ) | |
| write_json(file_c, payload_a) | |
| payload_c = clone_payload(payload) | |
| payload_c['evaluation_id'] = 'eval-c' | |
| payload_c['retrieved_timestamp'] = '2024-01-03' | |
| if ( | |
| isinstance(payload_c.get('evaluation_results'), list) | |
| and payload_c['evaluation_results'] | |
| ): | |
| payload_c['evaluation_results'][0]['score_details']['score'] = ( | |
| payload_c['evaluation_results'][0]['score_details']['score'] + 0.001 | |
| ) | |
| write_json(file_c, payload_c) |
Description
This is my first attempt at issue #88 . Further commits/PRs will look to cover more functionality and untested cases across the repo.