Skip to content

fix: resolve integration test failures by migrating COCO image loading to HF Hub fixtures#47219

Closed
abhishekkapoorx wants to merge 12 commits into
huggingface:mainfrom
abhishekkapoorx:fix/test-image-fixtures-for-coco-images
Closed

fix: resolve integration test failures by migrating COCO image loading to HF Hub fixtures#47219
abhishekkapoorx wants to merge 12 commits into
huggingface:mainfrom
abhishekkapoorx:fix/test-image-fixtures-for-coco-images

Conversation

@abhishekkapoorx

@abhishekkapoorx abhishekkapoorx commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

CI

What does this PR do?

This PR resolves the integration test failures triaged in #47205.

The domain images.cocodataset.org has been unstable, resulting in connection timeouts and failing integration tests across multiple model pipelines. To address this, we have migrated COCO-dataset image loading in our test suites to use the mirrored fixtures on the Hugging Face Hub under hf-internal-testing/fixtures-coco and utilizing local path redirection.

Key Changes:

  1. Added test helper: Introduced load_test_image in tests/test_processing_common.py which wraps load_image and automatically processes remote URLs with url_to_local_path.
  2. Refactored tests: Converted occurrences of direct requests.get or httpx.get targeting http://images.cocodataset.org/val2017/000000039769.jpg to the new load_test_image helper pointing to the Hugging Face mirror.
  3. Cleaned up CI downloads: Removed direct references to stale images.cocodataset.org URLs from the URLS_FOR_TESTING_DATA registry in utils/fetch_hub_objects_for_ci.py.

This prevents flaky network issues and leverages local caching during CI execution.

Related Issues

Fixes #47205

Code Agent Policy

The Transformers repo is currently being overwhelmed by a large number of PRs and issue comments written by
code agents. These often are low-quality, or fix extremely minor issues that occur rarely or never in practice.
As a result, we're instituting a rule that first-time contributors should not use code agents to submit PRs or issues.
We'd also ask autonomous "OpenClaw"-like agents not to open any PRs or issues.

Issues/PRs from first-time contributors that violate this rule will probably just be closed without review, and we
might block you, especially if you open more than one or appear to be deliberately ignoring this. We especially do not
want new contributors to jump in on random issues to contribute an agent-written fix. This creates lots of noise
for reviewers and other users and will almost certainly get you blocked.

For more information, please read CONTRIBUTING.md.

  • (First-time contributors only): I confirm that this PR description and code is not written by an LLM or code agent

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline and the
    Pull Request checks?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes according to the guidelines?
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@tarekziade tarekziade requested a review from ydshieh July 11, 2026 07:06
@github-actions

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 29143766271:2
Result: failure | Jobs: 2 | Tests: 21 | Failures: 1 | Duration: 2m 18s

Code quality check failed: test jobs were skipped. Fix the code quality issues and push again to run tests.

Comment thread tests/models/layoutlmv2/test_image_processing_layoutlmv2.py Outdated
Comment thread tests/models/eomt_dinov3/test_modeling_eomt_dinov3.py Outdated
Comment thread tests/models/eomt/test_modeling_eomt.py Outdated
@github-actions

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: aimv2, aria, deepseek_vl_hybrid, eomt, eomt_dinov3, fast_vlm, flava, fuyu, idefics, imagegpt, kimi_k25, layoutlmv2, lfm2_vl, llava, minimax_m3_vl, pix2struct

@tarekziade tarekziade self-assigned this Jul 11, 2026
@abhishekkapoorx

abhishekkapoorx commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

@ydshieh In src/transformers/modeling_utils.py, we are using the axis but latest documentation of torch uses dim. I think we should fix this as well.

index 3f14d74979..f27ae1d884 100644
--- a/src/transformers/modeling_utils.py
+++ b/src/transformers/modeling_utils.py
@@ -982,7 +982,7 @@ class ModuleUtilsMixin:
                     torch.ones((batch_size, seq_length, prefix_seq_len), device=device, dtype=causal_mask.dtype),
                     causal_mask,
                 ],
-                axis=-1,
+                dim=-1,
             )
 
         extended_attention_mask = causal_mask[:, None, :, :] * attention_mask[:, None, None, :]

This is causing local typing check to fail.

@tarekziade

Copy link
Copy Markdown
Collaborator

run-slow: aimv2, aria, deepseek_vl_hybrid, eomt, eomt_dinov3, fast_vlm, flava, fuyu, idefics, imagegpt, kimi_k25, layoutlmv2, lfm2_vl, llava, minimax_m3_vl, pix2struct

@github-actions

Copy link
Copy Markdown
Contributor

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/aimv2", "models/aria", "models/deepseek_vl_hybrid", "models/eomt", "models/eomt_dinov3", "models/fast_vlm", "models/flava", "models/fuyu", "models/idefics", "models/imagegpt", "models/kimi_k25", "models/layoutlmv2", "models/lfm2_vl", "models/llava", "models/minimax_m3_vl", "models/pix2struct"]
quantizations: []

@tarekziade

Copy link
Copy Markdown
Collaborator

Sorry I just realized that work was started earlier at #47218
I messed up when I did the triaging, sorry about that

@tarekziade tarekziade closed this Jul 11, 2026
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions

Copy link
Copy Markdown
Contributor

CI Results

Workflow Run ⚙️

Commit Info

Context Commit Description
RUN 61e10d08 workflow commit (merge commit)
PR 041d01ac branch commit (from PR)
main 1f8daee0 base commit (on main)

✅ No failing test specific to this PR 🎉 👏 !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standardize test image fixture loading for COCO images

4 participants