Skip cloud credential method when NO_GCE_CHECK=true

[0x2b3bfa0]

For consistency with googleapis/google-auth-library-python#1610, see fsspec/filesystem_spec#1768 for context.

Summary

• When auto-detecting credentials (method=None), skip the "cloud" method if the NO_GCE_CHECK environment variable is set to "true"
• Avoids unnecessary GCE metadata server requests in environments where GCE is known to be unavailable
• Only affects the auto-detection path; explicit token="cloud" still works

Test plan

• Added test_no_gce_check_skips_cloud to verify _connect_cloud is not called when NO_GCE_CHECK=true
• Existing credential tests still pass

🤖 Generated with Claude Code

[martindurant]

While this clearly gives the option to skip the slow creds step, if a user knows to pass this environment variable, then they probably should know which of the credentials types they need, no?

[0x2b3bfa0]

if a user knows to pass this environment variable, then they probably should know which of the credentials types they need, no?

Indeed! However, there might be cases where the choice of enabling this environment variable isn't up to the user, but up to the platform running the user's code, or to some intermediary library. 🫣

[martindurant]

Fair enough. @ankitaluthra1 , any thoughts on yet-another-envvar? Perhaps this can be a simple keyword argument (in init), so it can be set via the fsspec config system?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.34%. Comparing base (e70bc65) to head (0ecaefb).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #803      +/-   ##
==========================================
+ Coverage   75.98%   76.34%   +0.36%     
==========================================
  Files          14       14              
  Lines        2665     2668       +3     
==========================================
+ Hits         2025     2037      +12     
+ Misses        640      631       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[martindurant]

If we added a kwarg to init like skip_gke_check=False, then we could document this in a sensible place, and invoke it with env vars alone like:

FSSPEC_GCS='{"skip_gke_check": true}'
or
FSSPEC_GCS_SKIP_GKE_CHECK=true

or these same settings could be put in an fsspec config file.

What do you think, is this more discoverable? It at least fits in better with the fsspec configuration.

[0x2b3bfa0]

My point for using NO_GCE_CHECK was that the google-auth library already exposes this variable (googleapis/google-auth-library-python#1610).

Given that the current logic relies on both the default google-auth behavior, and an extra hardcoded check, the final consumer of gcsfs would only be able to avoid this check completely by either:

1. Setting both FSSPEC_GCS_SKIP_GKE_CHECK=true and NO_GCE_CHECK=true
2. Setting FSSPEC_GCS_SKIP_GKE_CHECK=true and modifying gcsfs to also set NO_GCE_CHECK=true internally when calling google-auth
3. Setting NO_GCE_CHECK=true and modifying gcsfs to honor NO_GCE_CHECK=true internally (what this pull request currently does)

I don't mind 2 or 3, but would rather prefer to avoid 1.

[martindurant]

/gcbrun

[martindurant]

OK, now I follow.

I note you had the following in the original description:

Existing credential tests still pass

I take it you have verified this? Of course, it's not something we can test in CI.

[0x2b3bfa0]

I take it you have verified this?

Yes; better said, Claude Code has 🤖 😇