refactor(finance): Refactor FinanceCompositionEvaluator and Enable Independent Model Configuration

- Added a new OpenJudge-based `FinanceCompositionEvaluator` to replace the legacy implementation.
- Implemented domain-based routing to direct requests to the appropriate set of graders, supporting multiple fields such as stock analysis and industry research.
- Implemented an asynchronous pairwise evaluation interface that returns scores within the 0–1 range.
- Enabled independent configuration for `finance_llm`; if not explicitly configured, the general `openjudge_llm` model is reused.
- Cleaned up redundant imports and deprecated code within `DeepFinanceJudgeByOpenJudge`.
- Updated `deep_finance_openjudge_template.yaml` to include documentation for the `finance_llm` option.
- Refined the description of "evidence traceability" in `deep_finance.md`, renaming it to "Reference Logic Audit" and enhancing the details regarding the workflow and judgment criteria.

Author: TaoShuchang

tutorial/example_deep_finance/deep_finance.md

@@ -232,44 +232,38 @@ final_score = 0.5 × coverage + 0.5 × grounding            # 综合分数
 
 ------
 
-### 4) 证据溯源（EBTU - Evidence-Backed Trace Units）
+### 4) 引用逻辑审计（AUDIT - Citation Integrity Audit）
 
-**目标**：对报告中的每个「原子断言」做证据锚定审计——回答「每个数字、每个事实，能否追溯到工具返回的原始数据？」
+**目标**：审计 AI 研究报告中的每一个引用标记 `[n]` 是否严格符合「逻辑蕴含（Logical Entailment）」原则——回答「每个引用是否被原始证据严格支撑？」
 
-**核心理念：证据优先（Evidence-first）**。审计官必须先给出证据锚点（step + quote），再下裁决，严禁先下结论再找证据。
+**核心理念：证据优先（Evidence-first）**。审计官必须像法官判案一样，先罗列证据，再进行逻辑推导，最后下达判决，严禁先下结论再找证据。
 
-**审计流程**：
+**三步验证流程**：
 
-1. 从报告中提取所有原子断言（Trace Units），标记类型（numeric/temporal/event/comparison/causal 等）
-2. 标记硬度：`hard`（确定性事实） / `soft`（明确标注为推测/假设）
-3. 对每个断言在 Evidence 中寻找锚点（anchors），要求：
+1. **提取（Extract）**：锁定报告中由 `[n]` 支撑的陈述片段（Claim）
+2. **溯源（Trace）**：在 Reference 列表中找到 `[n]` 对应的原始文本，摘录核心证据句（Source Quote）
+3. **比对（Compare）**：分析 Claim 是否被 Source Quote 严格支撑
+   - Check: 数字/事实是否一致？
+   - Check: 语气是否一致（有没有把"可能"改成"确定"）？
+   - Check: 因果关系是否存在？
 
-- - 精确到 step 编号和原文引用（quote ≤ 120 字）
-  - 数字/日期必须能在 Evidence 原文中找到对应
+**判决标准（Verdict Criteria）**：
 
-1. 给出裁决（verdict）：
-
-| Verdict          | 含义                                      |
-| ---------------- | ----------------------------------------- |
-| `supported`      | 锚点直接支持断言                          |
-| `contradicted`   | 锚点与断言明确冲突                        |
-| `no_evidence`    | Evidence 中找不到支撑，且断言是确定性表述 |
-| `speculative_ok` | 断言明确为推测/假设，未伪装成事实         |
-| `unclear`        | Evidence 相关但不足以支持或反驳           |
-
-1. 标记问题类型（issue）：`entity_mismatch` / `time_mismatch` / `value_mismatch` / `scope_mismatch` / `logic_leap` / `over_precision` / `missing_anchor`
+| Verdict        | 含义                                                         |
+| -------------- | ------------------------------------------------------------ |
+| `Supported`    | 证据充分，逻辑闭环。允许合理的概括，但禁止添加细节           |
+| `Overstated`   | 夸大其词。证据只说了 A，报告却写成了 A+（如去掉限定词、强加因果） |
+| `Contradicted` | 事实冲突。报告内容与证据相反                                 |
+| `Hallucinated` | 无中生有。关键细节在证据中找不到，或引用编号不存在           |
+| `Irrelevant`   | 引用无效。证据内容真实，但与报告所述主题无关                 |
 
 **评分计算**（确定性打分，由 Python 代码计算，非 LLM 输出）：
 
 ```plain
-base = (supported - 1.4×contradicted - 0.9×no_evidence - 0.4×unclear) / hard_units
-misattrib_factor = max(0, 1 - 0.7 × misattrib_rate)     # 错误归因惩罚
-selection_factor = min(1, extracted_units / expected)    # 覆盖率因子
-cov_factor = 0.65 + 0.35 × digit_coverage               # 数字/日期覆盖
-score = base × misattrib_factor × selection_factor × cov_factor
+integrity_score = Supported数量 / 总引用数
 ```
 
-关键设计：LLM 只负责结构化输出（断言提取 + 锚点标注 + 裁决），分数完全由代码确定性计算，避免 LLM 自评分的不稳定性。
+关键设计：LLM 只负责结构化输出（Claim 提取 + 证据溯源 + 逻辑分析 + 判决），分数完全由代码确定性计算，避免 LLM 自评分的不稳定性。
 
 ------
 

tutorial/example_deep_finance/deep_finance_judge.py

@@ -8,34 +8,20 @@
 import time
 import logging
 from datetime import datetime
-from typing import Dict, Any, Optional, Tuple, List, Type
+from typing import Dict, Any, Optional, Tuple, List
 
 from ajet.task_judge.base_judge import BaseJudge
 from ajet.workflow import WorkflowOutput, WorkflowTask
 
 from openjudge.models.openai_chat_model import OpenAIChatModel
 from openjudge.runner.grading_runner import GraderConfig, GradingRunner
-from openjudge.graders.base_grader import BaseGrader
-from tutorial.example_deep_finance.judge import PresentationQualityGrader, GroundingGrader, AuditGrader, EBTUTraceabilityGrader
-
-# Finance Graders from OpenJudge cookbooks
-from cookbooks.finance_grader.stock_analysis.valuation_analysis import ValuationAnalysisGrader
-from cookbooks.finance_grader.stock_analysis.fundamental_analysis import FundamentalAnalysisGrader
-from cookbooks.finance_grader.stock_analysis.overall_logic import OverallLogicGrader
-from cookbooks.finance_grader.stock_analysis.stock_risk_analysis import StockRiskAnalysisGrader
-from cookbooks.finance_grader.macro_analysis.macro_analysis import MacroAnalysisGrader
-from cookbooks.finance_grader.macro_analysis.concept_explanation import ConceptExplanationGrader
-from cookbooks.finance_grader.industry_research.characteristics_analysis import CharacteristicsAnalysisGrader
-from cookbooks.finance_grader.industry_research.risk_analysis import RiskAnalysisGrader
-from cookbooks.finance_grader.industry_research.underlying_comparison import UnderlyingComparisonGrader
-from cookbooks.finance_grader.event_interpretation.event_analysis import EventAnalysisGrader
-from cookbooks.finance_grader.event_interpretation.event_identification import EventIdentificationGrader
-from cookbooks.finance_grader.stock_search.search_relevance import SearchRelevanceGrader
-from cookbooks.finance_grader.stock_search.search_integrity import SearchIntegrityGrader
-from cookbooks.finance_grader.stock_search.search_timeliness import SearchTimelinessGrader
-
-
-# OpenJudge imports
+from tutorial.example_deep_finance.judge import (
+    PresentationQualityGrader, 
+    GroundingGrader, 
+    AuditGrader, 
+    EBTUTraceabilityGrader,
+    FinanceCompositionEvaluator,
+)
 # =============================================================================
 # 全局辅助函数
 # =============================================================================
@@ -76,135 +62,6 @@ def load_reference_answers_from_file(file_path: str) -> Tuple[Dict[str, str], Di
         raise ValueError(f"Error loading reference answers: {e}")
 
 
-# =============================================================================
-# FinanceCompositionEvaluator - 基于 OpenJudge 的 Finance 评估器
-# =============================================================================
-
-class FinanceCompositionEvaluator:
-    """
-    基于 OpenJudge 的 Finance 组合评估器（替代 rm_gallery.FinanceComposition）
-    
-    功能：
-    - 根据 domain 路由到对应的 grader 集合
-    - 执行 pairwise 评估（比较 training answer 和 reference answer）
-    - 返回 0-1 范围的分数
-    
-    支持的 domain:
-    - stock_analysis: 股票分析
-    - industry_research: 行业研究  
-    - macro_analysis: 宏观分析
-    - event_interpretation: 事件解读
-    - stock_search: 股票搜索
-    """
-    
-    # Domain 到 Grader 类的映射（与 RM-Gallery 保持一致）
-    DOMAIN_GRADERS: Dict[str, List[Type[BaseGrader]]] = {
-        "stock_analysis": [
-            ValuationAnalysisGrader,
-            # FundamentalAnalysisGrader,
-            # OverallLogicGrader,
-            # StockRiskAnalysisGrader,
-        ],
-        "industry_research": [
-            CharacteristicsAnalysisGrader,
-            # RiskAnalysisGrader,
-            # UnderlyingComparisonGrader,
-        ],
-        "macro_analysis": [
-            MacroAnalysisGrader,
-            # ConceptExplanationGrader,
-        ],
-        "event_interpretation": [
-            EventAnalysisGrader,
-            # EventIdentificationGrader,
-        ],
-        "stock_search": [
-            SearchRelevanceGrader,
-            # SearchIntegrityGrader,
-            # SearchTimelinessGrader,
-        ],
-    }
-    
-    def __init__(self, model: OpenAIChatModel, params: Dict[str, Any] = None):
-        """
-        初始化 FinanceCompositionEvaluator
-        
-        Args:
-            model: OpenAIChatModel 实例
-            params: 额外参数（保留兼容性）
-        """
-        self.model = model
-        self.params = params or {}
-        self._grader_cache: Dict[str, List[BaseGrader]] = {}
-        
-    def _get_graders_for_domain(self, domain: str) -> List[BaseGrader]:
-        """
-        获取指定 domain 的 grader 实例列表（带缓存）
-        """
-        if domain not in self._grader_cache:
-            grader_classes = self.DOMAIN_GRADERS.get(domain, [])
-            self._grader_cache[domain] = [
-                grader_cls(model=self.model) for grader_cls in grader_classes
-            ]
-        return self._grader_cache[domain]
-    
-    async def aevaluate(self, query: str, current: str, reference: str, domain: str) -> float:
-        """
-        执行 pairwise 评估（异步版本，避免重复创建 event loop）
-        
-        Args:
-            query: 用户查询
-            current: 当前模型生成的回答 (training)
-            reference: 参考答案
-            domain: 任务领域（用于路由到对应 graders）
-            
-        Returns:
-            float: 0-1 范围的分数
-                - 1.0: current 优于 reference
-                - 0.0: reference 优于 current
-                - 0.5: 无法评估或出错
-        """
-        if not domain or domain not in self.DOMAIN_GRADERS:
-            print(f"⚠️ FinanceCompositionEvaluator: Unknown domain '{domain}', returning 0.5")
-            return 0.5
-            
-        graders = self._get_graders_for_domain(domain)
-        if not graders:
-            print(f"⚠️ FinanceCompositionEvaluator: No graders for domain '{domain}', returning 0.5")
-            return 0.5
-        
-        # 运行所有 graders
-        scores = []
-        for grader in graders:
-            try:
-                result = await grader.aevaluate(
-                    query=query,
-                    answer_1=current,    # training model output
-                    answer_2=reference,  # reference answer
-                )
-                
-                # 解析 GraderRank 结果
-                if hasattr(result, 'rank') and isinstance(result.rank, list):
-                    # rank = [1, 2] 表示 answer_1 (current) 更好 -> score = 1.0
-                    # rank = [2, 1] 表示 answer_2 (reference) 更好 -> score = 0.0
-                    if result.rank[0] == 1:
-                        scores.append(1.0)
-                    else:
-                        scores.append(0.0)
-                else:
-                    scores.append(0.5)  # 无法解析，返回中间值
-                    
-            except Exception as e:
-                grader_name = getattr(grader, 'name', grader.__class__.__name__)
-                print(f"⚠️ FinanceCompositionEvaluator: Grader {grader_name} failed: {e}")
-                scores.append(0.5)
-        
-        # 计算平均分数
-        if scores:
-            return sum(scores) / len(scores)
-        return 0.5
-
-
 # =============================================================================
 # DeepFinanceJudgeByOpenJudge 类
 # =============================================================================
@@ -287,6 +144,7 @@ def _init_finance_evaluator(self):
         初始化 FinanceCompositionEvaluator（仅当 finance_weight > 0 时）
         
         使用 OpenJudge 的 finance graders 替代原 rm_gallery 实现
+        支持独立的 finance_llm 配置，若未配置则复用 openjudge_llm
         """
         self._finance_enabled = (self.w.get("finance", 0) > 0)
         if self._finance_enabled:
@@ -302,15 +160,35 @@ def _create_finance_evaluator(self):
         """
         创建 FinanceCompositionEvaluator 实例（基于 OpenJudge）
         
-        复用已初始化的 OpenJudge model，无需单独配置
+        支持独立的 finance_llm 配置：
+        - 若 config.ajet.judge.finance_llm 有值，则使用独立的 model
+        - 若未配置或为空，则复用已初始化的 OpenJudge model
         """
         try:
-            # 复用 OpenJudge model（已在 _init_openjudge_model 中初始化）
+            # 检查是否配置了独立的 finance_llm
+            finance_llm_name = getattr(self.config.ajet.judge, "finance_llm", None)
+            
+            if finance_llm_name and finance_llm_name.strip():
+                # 使用独立的 finance model
+                finance_base_url = os.environ.get("FINANCE_BASE_URL") or os.environ.get("OPENJUDGE_BASE_URL")
+                finance_api_key = os.environ.get("FINANCE_API_KEY") or os.environ.get("OPENJUDGE_API_KEY")
+                
+                finance_model = OpenAIChatModel(
+                    model=finance_llm_name,
+                    base_url=finance_base_url,
+                    api_key=finance_api_key,
+                )
+                print(f"[Init FinanceCompositionEvaluator] Using dedicated finance model: {finance_llm_name}")
+            else:
+                # 复用 OpenJudge model（已在 _init_openjudge_model 中初始化）
+                finance_model = self.model
+                print(f"[Init FinanceCompositionEvaluator] Reusing OpenJudge model")
+            
             self.finance_evaluator = FinanceCompositionEvaluator(
-                model=self.model,
+                model=finance_model,
                 params={"is_parallel": True}
             )
-            print(f"[Init FinanceCompositionEvaluator] Using OpenJudge model, domains={list(FinanceCompositionEvaluator.DOMAIN_GRADERS.keys())}")
+            print(f"[Init FinanceCompositionEvaluator] domains={list(FinanceCompositionEvaluator.DOMAIN_GRADERS.keys())}")
         except Exception as e:
             print(f"✗ Failed to initialize FinanceCompositionEvaluator: {e}")
             import traceback

tutorial/example_deep_finance/judge/__init__.py

@@ -3,5 +3,12 @@
 from .presentation_quality.grader import PresentationQualityGrader
 from .audit.grader import AuditGrader
 from .ebtu.grader import EBTUTraceabilityGrader
+from .finance.grader import FinanceCompositionEvaluator
 
-__all__ = ["PresentationQualityGrader", "GroundingGrader", "AuditGrader", "EBTUTraceabilityGrader"]
+__all__ = [
+    "PresentationQualityGrader", 
+    "GroundingGrader", 
+    "AuditGrader", 
+    "EBTUTraceabilityGrader",
+    "FinanceCompositionEvaluator",
+]

tutorial/example_deep_finance/judge/finance/__init__.py

@@ -0,0 +1,4 @@
+"""Finance Composition Evaluator - 基于 OpenJudge 的 Finance 组合评估器"""
+from .grader import FinanceCompositionEvaluator
+
+__all__ = ["FinanceCompositionEvaluator"]