Skip to content

feat(unstructured): add Transform MCP tool#3438

Open
SudSampath wants to merge 2 commits into
langgenius:mainfrom
SudSampath:agent/add-transform-tool
Open

feat(unstructured): add Transform MCP tool#3438
SudSampath wants to merge 2 commits into
langgenius:mainfrom
SudSampath:agent/add-transform-tool

Conversation

@SudSampath

@SudSampath SudSampath commented Jul 15, 2026

Copy link
Copy Markdown

Summary

  • add a Transform Document tool to the existing official Unstructured plugin
  • accept Dify files or public URLs and return Markdown, JSON, HTML, or text
  • support parsing, OCR/enrichment, chunking, and embedding for RAG preprocessing
  • preserve the existing Partition tool for legacy and self-hosted deployments

Unstructured Transform converts PDFs, DOCX, PPTX, HTML, images, and scanned pages into clean structured output.

Marketplace behavior

This updates the existing langgenius/unstructured Marketplace plugin. Once approved and merged, Dify's repository workflow packages and uploads version 0.0.10 to the Marketplace. Users will discover and install it from the existing Unstructured listing, then authenticate at runtime with their own Unstructured account and API key. No shared Unstructured credential is included in the plugin.

Validation

  • uv run pytest -q (3 passed)
  • uvx ruff format --check .
  • uvx ruff check .
  • parsed all plugin YAML
  • dify plugin package ./unstructured

@SudSampath
SudSampath marked this pull request as ready for review July 15, 2026 00:33
Copilot AI review requested due to automatic review settings July 15, 2026 00:33
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. enhancement New feature or request labels Jul 15, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the hosted Unstructured Transform MCP workflow while retaining the legacy Partition tool.

Changes:

  • Adds document transformation with multiple output formats and RAG preprocessing.
  • Expands provider credentials, documentation, and privacy disclosures.
  • Adds MCP dependencies and focused helper tests.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
.difyignore Excludes tests from packages.
PRIVACY.md Documents data handling.
README.md Documents Transform and Partition tools.
main.py Extends request timeout.
manifest.yaml Releases version 0.0.10.
provider/unstructured.py Routes credential validation.
provider/unstructured.yaml Registers Transform credentials and tool.
pyproject.toml Adds MCP and test dependencies.
tests/test_transform.py Tests payload and stage helpers.
tools/transform.py Implements the Transform workflow.
tools/transform.yaml Defines the Transform interface.
uv.lock Locks added dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/unstructured/tools/transform.py Outdated
Comment thread tools/unstructured/tools/transform.py Outdated
Comment thread tools/unstructured/tools/transform.py Outdated
Comment thread tools/unstructured/tools/transform.py Outdated
Comment thread tools/unstructured/tools/transform.py Outdated
Comment thread tools/unstructured/provider/unstructured.yaml
@crazywoola

Copy link
Copy Markdown
Member

Please resolve the comments if you think it's irrelvant.

Copilot AI review requested due to automatic review settings July 17, 2026 18:35

Copy link
Copy Markdown
Author

Thanks! I addressed all six review comments in 000a715 and resolved the threads. The update enforces HTTPS, validates the full four-tool MCP protocol, redacts signed transfer URLs, retries the job_not_complete consistency window, and prevents Partition from using Transform credentials. Validation: 9 tests pass, Ruff format/lint pass, YAML parses, and the plugin packages successfully.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.

meta={"filename": filename, "mime_type": mime_type},
)
yield self.create_variable_message("job_id", result["job_id"])
yield self.create_variable_message("output_ref", result.get("output_ref", ""))
from dify_plugin import Plugin, DifyPluginEnv

plugin = Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=120))
plugin = Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=600))
required: false
type: secret-input
url: https://platform.unstructured.io/app/account/api-keys
url: https://transform.unstructured.io/get-started
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants