Skip to content

[Refactor/Chore] Centralize remote file retrieval for signed file URLs #36397

@laipz8200

Description

@laipz8200

Self Checks

  • I have read the Contributing Guide and Language Policy.
  • This is only for refactors or chores; if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report, otherwise it will be closed.
  • Please do not modify this template :) and fill in all the required fields.

Description

Several backend paths fetch URL-backed inputs as remote files through the generic outbound HTTP client. This makes first-party Dify signed file URLs follow the same network path as arbitrary external URLs, even when those URLs point back to files already stored by the current Dify deployment.

Propose introducing a single remote-file retrieval boundary that:

  • handles remote file GET/HEAD operations in one module
  • recognizes valid Dify-signed file URLs for upload, tool, and datasource files
  • loads first-party files through database records and configured storage
  • delegates ordinary external URLs to the existing SSRF-protected HTTP client
  • documents when callers should use the remote-file fetcher versus the generic SSRF proxy

Motivation

Self-hosted deployments commonly configure file URLs that point back to the Dify service or to local network addresses. Treating those first-party file URLs as ordinary outbound HTTP can make remote-file workflows depend on network loopback behavior and duplicate signed-file validation logic across call sites.

Centralizing the behavior keeps remote file handling consistent while preserving the existing SSRF-protected path for true external HTTP requests.

Additional Context

Related historical reports include #30589 and #30780, but this proposal focuses on the backend remote-file retrieval boundary rather than a single workflow surface.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions