Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ede8dbf
feat: critic agent
KERVIN-FARMER Jun 18, 2026
8652f51
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
6fa7a51
Fix: Type error deleted
KERVIN-FARMER Jun 18, 2026
5818ba6
Fix: Phase 3 completed
KERVIN-FARMER Jun 18, 2026
bb5a7bc
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
3bcf383
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
244c8c0
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
1ad1367
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
8bb3842
fix: bugs solved
KERVIN-FARMER Jun 18, 2026
7c665b0
Fix: Minor fixes in grammar and syntax
KERVIN-FARMER Jun 18, 2026
e16161d
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
1693147
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
850d3c8
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
4b6ddad
Update agent-strategies/self_refine_agent/strategies/self_refine.py
KERVIN-FARMER Jun 18, 2026
d36de97
Fix: Minor grammar mistakes
KERVIN-FARMER Jun 18, 2026
12bccfb
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jun 22, 2026
0041318
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jun 23, 2026
d7cadcd
Apply suggestion from @crazywoola
crazywoola Jun 30, 2026
72ae2b0
Apply suggestions from code review
crazywoola Jun 30, 2026
4394e3a
Apply suggestions from code review
crazywoola Jun 30, 2026
8ffd156
Apply suggestion from @crazywoola
crazywoola Jun 30, 2026
1906437
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jul 1, 2026
61b8401
Update README.md
KERVIN-FARMER Jul 1, 2026
59049b7
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jul 1, 2026
1abb10b
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jul 9, 2026
262a125
Merge branch 'main' into feat/Critic-agent
KERVIN-FARMER Jul 10, 2026
3e09faf
Merge branch 'main' into feat/Critic-agent
crazywoola Jul 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,5 +69,5 @@ Version format: MAJOR.MINOR.PATCH — each segment may be 2 digits (e.g. 10.11.2
Python venv aligned with `manifest.yaml`, `pyproject.toml`, and `uv.lock` (or `requirements.txt` for legacy plugins).
-->

- [ ] Local deployment — Dify version: <!-- e.g. 1.2.0 -->
- [ ] Local deployment — Dify version: 1.7.0
- [ ] SaaS (cloud.dify.ai)
4 changes: 4 additions & 0 deletions agent-strategies/self_refine_agent/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
INSTALL_METHOD=remote
REMOTE_INSTALL_HOST=debug-plugin.dify.dev
REMOTE_INSTALL_PORT=5003
REMOTE_INSTALL_KEY=your-plugin-key-here
104 changes: 104 additions & 0 deletions agent-strategies/self_refine_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Self-Refine Agent Strategy

A Dify agent strategy plugin that improves output quality through iterative self-critique and refinement.

## What It Does

Self-Refine Agent Strategy implements the [Self-Refine](https://arxiv.org/abs/2303.17651) (NeurIPS 2023) algorithm, enabling agents to automatically evaluate and improve their own outputs through multiple refinement cycles.

The strategy works by having the LLM act as its own critic:
1. **Execute** - Agent generates an initial response
2. **Evaluate** - LLM judges the output quality (0-100 score)
3. **Critique** - LLM identifies specific issues and improvement opportunities
4. **Refine** - Critique is injected into context for the next iteration
5. **Repeat** - Process continues until quality threshold is met or max refinements reached

This approach is particularly effective for tasks requiring high-quality outputs such as writing, analysis, code generation, and complex reasoning.

## Key Features

- **Automatic Quality Control** - LLM evaluates its own outputs without human intervention
- **Iterative Improvement** - Each refinement cycle builds on previous critiques
- **Transparent Process** - All intermediate steps (execution, evaluation, critique) are visible
- **Configurable** - Control maximum iterations and refinement cycles

## Usage

### Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model` | model-selector | - | LLM model to use |
| `tools` | array[tools] | - | Available tools for the agent |
| `instruction` | string | - | System instructions for the agent |
| `query` | string | - | User query to process |
| `context` | array[object] | - | Optional context information |
| `maximum_iterations` | number | 5 | Max iterations per agent execution |
| `max_refinements` | number | 2 | Max refinement cycles (0-5) |

### Configuration Example

When configuring the agent in Dify:

1. Select **Self-Refine Agent** as your strategy
2. Choose your preferred LLM model
3. Set `max_refinements` to control quality vs. speed tradeoff:
- `0` - No refinement (standard agent behavior)
- `1-2` - Balanced (recommended for most cases)
- `3-5` - High quality (best for critical outputs)

### Local Development

To test the plugin locally:

1. Create a `.env` file:
```bash
INSTALL_METHOD=remote
DIFY_PLUGIN_DAEMON_URL=http://localhost:5003
```

2. Ensure Dify is running locally (via Docker Compose)

3. Start the plugin:
```bash
python main.py
```

## How It Works

The refinement loop follows this pattern:

```
Initial Query → Agent Execution → Output
Evaluation (score)
[Score ≥ 80?] → Yes → Return Final Output
↓ No
Generate Critique
Inject Critique into Context
Re-execute Agent → Output
[Max refinements reached?] → Yes → Return Best Output
↓ No
Repeat Cycle
```

The agent automatically stops refining when:
- Output quality score reaches 80 or above
- Maximum refinement cycles are reached
- No significant improvements are detected

## Reference

Based on the paper:

Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., ... & Clark, P. (2023).
**Self-Refine: Iterative Refinement with Self-Feedback.**
NeurIPS 2023. [arXiv:2303.17651](https://arxiv.org/abs/2303.17651)

## License

MIT License
14 changes: 14 additions & 0 deletions agent-strategies/self_refine_agent/_assets/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions agent-strategies/self_refine_agent/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import sys

from dify_plugin import DifyPluginEnv, Plugin

plugin = Plugin(DifyPluginEnv(MAX_REQUEST_TIMEOUT=240, MAX_INVOCATION_TIMEOUT=900))

if __name__ == "__main__":
plugin.run()
35 changes: 35 additions & 0 deletions agent-strategies/self_refine_agent/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
version: 0.0.1
type: plugin
author: "KERVIN-FARMER"
name: "self_refine_agent"
label:
en_US: "Self-Refine Agent Strategy"
zh_Hans: "自我优化 Agent 策略"
created_at: "2025-06-18T00:00:00.000000000Z"
icon: icon.svg
description:
en_US: "Agent strategy with automatic self-critique and iterative refinement. Based on Self-Refine (NeurIPS 2023)."
zh_Hans: "支持自动自我批判和迭代优化的 Agent 策略。基于 Self-Refine (NeurIPS 2023)。"
tags:
- "agent"
resource:
memory: 1048576
permission:
tool:
enabled: true
model:
enabled: true
llm: true
plugins:
agent_strategies:
- "provider/self_refine.yaml"
meta:
version: 0.0.1
minimum_dify_version: "1.7.0"
arch:
- "amd64"
- "arm64"
runner:
language: "python"
version: "3.12"
entrypoint: "main"
65 changes: 65 additions & 0 deletions agent-strategies/self_refine_agent/prompt/templates.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
"""
Prompt templates for Self-Refine Agent Strategy

Based on Self-Refine: Iterative Refinement with Self-Feedback (NeurIPS 2023)
https://arxiv.org/abs/2303.17651
"""

# System prompt for initial execution
EXECUTION_SYSTEM_PROMPT = """You are a helpful AI assistant. Follow the user's instructions carefully and provide detailed, accurate responses.

{{instruction}}

Available tools:
{{tools}}

Use the tools when necessary to gather information and complete the task. Provide thorough and well-reasoned answers."""

# System prompt for execution with refinement context
REFINEMENT_EXECUTION_PROMPT = """You are a helpful AI assistant. Follow the user's instructions carefully and provide detailed, accurate responses.

{{instruction}}

Available tools:
{{tools}}

Use the tools when necessary to gather information and complete the task.

[SELF-REFINEMENT CONTEXT]
Your previous attempt had the following issues:
{{critique}}

Please address these issues in this attempt and provide an improved response."""
Comment thread
KERVIN-FARMER marked this conversation as resolved.

# Evaluation prompt for assessing output quality
EVALUATION_PROMPT = """You are an expert evaluator. Assess the quality of the following agent output for the given query.

Query: {{query}}

Agent Output:
{{output}}

Evaluate the output based on these criteria:
1. Accuracy: Is the information correct and factual?
2. Completeness: Does it fully address the query?
3. Clarity: Is it clear and well-structured?
4. Relevance: Does it stay focused on the query?

Respond in the following JSON format:
{
"is_satisfactory": true/false,
"issues": "Specific description of issues if not satisfactory, or empty string if satisfactory",
"score": 0-100
}

Only set is_satisfactory to false if there are significant issues that require correction."""

# Fallback critique when evaluation fails
FALLBACK_CRITIQUE = "Unable to evaluate output quality. Please review and improve the response for accuracy, completeness, and clarity."

SELF_REFINE_TEMPLATES = {
"execution_system": EXECUTION_SYSTEM_PROMPT,
"refinement_execution": REFINEMENT_EXECUTION_PROMPT,
"evaluation": EVALUATION_PROMPT,
"fallback_critique": FALLBACK_CRITIQUE
}
5 changes: 5 additions & 0 deletions agent-strategies/self_refine_agent/provider/self_refine.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from dify_plugin.interfaces.agent import AgentProvider


class SelfRefineAgentProvider(AgentProvider):
pass
15 changes: 15 additions & 0 deletions agent-strategies/self_refine_agent/provider/self_refine.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
identity:
author: KERVIN-FARMER
name: self_refine
label:
en_US: Self-Refine Agent
zh_Hans: 自我优化 Agent
description:
en_US: Self-refining agent with automatic critique and iterative improvement
zh_Hans: 支持自动批判和迭代改进的自我优化 Agent
icon: icon.svg
strategies:
- strategies/self_refine.yaml
extra:
python:
source: provider/self_refine.py
13 changes: 13 additions & 0 deletions agent-strategies/self_refine_agent/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[project]
name = "self_refine_agent"
version = "0.0.1"
description = "Self-Refine Agent Strategy for Dify - Automatic critique and iterative refinement"
readme = "README.md"
requires-python = ">=3.12"

dependencies = [
"dify_plugin>=0.9.0",
]

[dependency-groups]
dev = []
Loading
Loading