Skip to content

test: expand coverage for duplicate entry checks#103

Open
gbemike wants to merge 2 commits intoevaleval:mainfrom
gbemike:add-new-tests
Open

test: expand coverage for duplicate entry checks#103
gbemike wants to merge 2 commits intoevaleval:mainfrom
gbemike:add-new-tests

Conversation

@gbemike
Copy link
Copy Markdown
Contributor

@gbemike gbemike commented Apr 6, 2026

Description

  • adds coverage for invalid JSON input
  • adds a no-duplicates output case
  • adds direct non-JSON path rejection coverage
  • strengthens duplicate output assertions

This is my first attempt at issue #88 . Further commits/PRs will look to cover more functionality and untested cases across the repo.

@gbemike gbemike closed this Apr 7, 2026
@gbemike gbemike reopened this Apr 7, 2026
@gbemike gbemike marked this pull request as draft April 7, 2026 20:13
@gbemike gbemike marked this pull request as ready for review April 7, 2026 20:13
@gbemike gbemike closed this Apr 7, 2026
@gbemike gbemike reopened this Apr 7, 2026
@tommasocerruti tommasocerruti requested a review from Copilot April 17, 2026 19:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands the test suite around every_eval_ever.check_duplicate_entries to better cover CLI behavior and edge cases, supporting the broader goal of increasing repo-wide test coverage (Issue #88).

Changes:

  • Add coverage for main() raising on invalid JSON input.
  • Add a “no duplicates found” CLI output case and strengthen duplicate-output assertions.
  • Add coverage for rejecting direct non-JSON file paths in expand_paths().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

assert check_module.main([str(file_a), str(file_b)]) == 1
captured = capsys.readouterr().out
assert 'Found duplicate entries' in captured
assert 'Found duplicate entries (ignoring keys: `evaluation_id`, `retrieved_timestamp`)' in captured
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assertion for the duplicate warning hard-codes the ignored key list. Since the CLI message is derived from check_module.IGNORE_KEYS, consider building the expected substring from check_module.IGNORE_KEYS (sorted) to keep the test in sync if the ignore set changes.

Suggested change
assert 'Found duplicate entries (ignoring keys: `evaluation_id`, `retrieved_timestamp`)' in captured
ignored_keys = ', '.join(
f'`{key}`' for key in sorted(check_module.IGNORE_KEYS)
)
assert f'Found duplicate entries (ignoring keys: {ignored_keys})' in captured

Copilot uses AI. Check for mistakes.
Comment on lines +94 to +95

with pytest.raises(Exception, match='Could not find file or directory'):
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the .txt path case, the test matches the same 'Could not find file or directory' message used for missing paths. That message is a bit misleading for an existing-but-unsupported file, and matching on it makes the intent (unsupported extension rejection) less clear. Consider either asserting only that an exception is raised for non-JSON files, or (longer-term) updating expand_paths to raise a distinct exception/message for unsupported extensions.

Suggested change
with pytest.raises(Exception, match='Could not find file or directory'):
with pytest.raises(Exception):

Copilot uses AI. Check for mistakes.
Comment on lines +126 to +139

payload_a = clone_payload(payload)
payload_a['evaluation_id'] = 'eval-c'
payload_a['retrieved_timestamp'] = '2024-01-03'
if (
isinstance(payload_a.get('evaluation_results'), list)
and payload_a['evaluation_results']
):
payload_a['evaluation_results'][0]['score_details']['score'] = (
payload_a['evaluation_results'][0]['score_details']['score'] + 0.001
)

write_json(file_c, payload_a)

Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In test_main_reports_no_duplicates, the variable payload_a is actually the modified payload written to file_c, which is a bit confusing when reading the test. Renaming it to something like payload_c (and cleaning up the extra blank lines/trailing whitespace in this block) would improve readability and keep formatting consistent with the rest of the tests.

Suggested change
payload_a = clone_payload(payload)
payload_a['evaluation_id'] = 'eval-c'
payload_a['retrieved_timestamp'] = '2024-01-03'
if (
isinstance(payload_a.get('evaluation_results'), list)
and payload_a['evaluation_results']
):
payload_a['evaluation_results'][0]['score_details']['score'] = (
payload_a['evaluation_results'][0]['score_details']['score'] + 0.001
)
write_json(file_c, payload_a)
payload_c = clone_payload(payload)
payload_c['evaluation_id'] = 'eval-c'
payload_c['retrieved_timestamp'] = '2024-01-03'
if (
isinstance(payload_c.get('evaluation_results'), list)
and payload_c['evaluation_results']
):
payload_c['evaluation_results'][0]['score_details']['score'] = (
payload_c['evaluation_results'][0]['score_details']['score'] + 0.001
)
write_json(file_c, payload_c)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants