Release v1.3.3 hardening

- add output manifests, redaction, and truncation metadata

- harden remote workspace and explain output handling

- expand release gate, tests, and release documentation

Author: Hackerchen716

.github/workflows/publish.yml

@@ -35,52 +35,7 @@ jobs:
       - name: Run validation
         run: |
           python -m pip install --upgrade build twine pytest
-          python -m pytest -q
-          python -m compileall -q bla bla_cli.py setup.py tests
-          python bla_cli.py validate-rules --strict-metadata
-          python bla_cli.py ssh --help
-
-      - name: Build package
-        run: python -m build
-
-      - name: Check distribution metadata
-        run: |
-          python -m twine check dist/*
-          python - <<'PY'
-          from pathlib import Path
-          import tarfile
-          import zipfile
-          from bla.__version__ import __version__
-
-          dist = Path("dist")
-          wheels = sorted(dist.glob("*.whl"))
-          sdists = sorted(dist.glob("*.tar.gz"))
-          if len(wheels) != 1 or len(sdists) != 1:
-              raise SystemExit("Expected exactly one wheel and one source distribution")
-
-          with zipfile.ZipFile(wheels[0]) as archive:
-              names = archive.namelist()
-              if "bla/rules/web_attacks.yaml" not in names:
-                  raise SystemExit("Wheel is missing bla/rules/web_attacks.yaml")
-              if "bla/remote/ssh_workspace.py" not in names:
-                  raise SystemExit("Wheel is missing bla/remote/ssh_workspace.py")
-
-          with tarfile.open(sdists[0]) as archive:
-              names = archive.getnames()
-              if not any(name.endswith("/bla/rules/web_attacks.yaml") for name in names):
-                  raise SystemExit("Source distribution is missing bla/rules/web_attacks.yaml")
-              if not any(name.endswith("/bla/remote/ssh_workspace.py") for name in names):
-                  raise SystemExit("Source distribution is missing bla/remote/ssh_workspace.py")
-              required = [
-                  f"/docs/releases/v{__version__}.md",
-                  "/sample_logs/auth.log",
-                  "/sample_logs/windows_rdp_sample.xml",
-                  "/tests/fixtures/p0/hvv_chain.jsonl",
-              ]
-              for suffix in required:
-                  if not any(name.endswith(suffix) for name in names):
-                      raise SystemExit(f"Source distribution is missing {suffix}")
-          PY
+          python scripts/release_check.py --build
 
       - name: Smoke test built wheel
         run: |

MANIFEST.in

@@ -2,4 +2,5 @@ include README.md LICENSE pyproject.toml setup.py
 recursive-include bla/rules *.yaml
 recursive-include docs *.md *.png *.json
 recursive-include sample_logs *.log *.xml
+recursive-include scripts *.py
 recursive-include tests *.py *.json *.jsonl

README.md

@@ -44,21 +44,24 @@ BLA 的结果分成两类：给人看的应急判断，和给系统继续处理
 - **提取证据**：输出 IP、域名、URL、Hash、账户、进程、命令和可疑路径，方便封禁、狩猎和工单流转。
 - **交给系统**：同时生成 JSON、CSV 和 SARIF，便于二次分析、Excel 排查、CI 门禁和 Code Scanning。
 
-默认 `--out report/` 会落地 `index.html`、`report.json`、`events.csv`、`iocs.txt` 和 `report.sarif`，人能看，脚本也能继续处理。
+默认 `--out report/` 会落地 `index.html`、`report.json`、`events.csv`、`iocs.txt`、`report.sarif` 和 `manifest.json`，人能看，脚本也能继续处理。
 
-## 最新版本：v1.3.2
+## 最新版本：v1.3.3
 
-v1.3.2 是一次可信度与发布卫生更新，重点不是堆新功能，而是把解析、检测、终端输出、远程工作台和 README/发布流程一起收紧。
+v1.3.3 是一次产品打磨、性能和发布质量更新，重点是让报告更适合集成，让 P0 结构化日志解析更快，并补齐交付前可重复验证能力。
 
-| 方向 | v1.3.2 重点 |
+| 方向 | v1.3.3 重点 |
 | --- | --- |
-| 解析可信度 | P0 JSON object / JSONL 都能识别，解析失败会被统计；VPN 未知认证状态不再误判为登录成功 |
-| 检测可信度 | 暴力破解、密码喷洒和“爆破后成功登录”按时间窗口判断，避免跨天/月误聚合 |
-| RDP 专项 | `--rdp` 只保留 `LogonType=10` 且带远程来源 IP 的 Windows 4624/4625 登录事件 |
-| 输出安全 | 终端报告会清理日志中的控制序列，HTML/CSV/JSON/SARIF 继续保持离线输出 |
-| 发布卫生 | README、release checklist、打包清单和发布 workflow 同步校验版本、构建产物和安装烟测 |
+| 性能优化 | P0 日志优先使用 `log_type/source_type` 快速分流，减少每行大范围正则判断 |
+| 输出可读性 | JSON 报告增加顶层 `summary`，把风险、事件、告警、案件和级别分布放到固定入口 |
+| 交付追溯 | 标准报告目录新增 `manifest.json`，记录输入哈希、输出哈希、版本、限制和解析异常 |
+| 输出安全 | 分享型报告会清理终端控制序列并遮蔽常见 token、cookie、password 等敏感字段 |
+| 统计效率 | 解析统计不再为起止时间排序全量时间戳，降低大日志内存和 CPU 开销 |
+| 检测热路径 | 缓存 RFC1918 IP 判断，复用预编译模式，减少凭据检测里的重复字符串拼接 |
+| 规则维护 | `validate-rules` 增加自定义正则回溯风险提示，降低扩展规则拖慢分析的概率 |
+| 发布校验 | 发布脚本覆盖样例分析、报告包产物、版本面、规则校验、Remote Workspace help 和 benchmark |
 
-更多变更见 [v1.3.2 发布说明](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/releases/v1.3.2.md)，历史版本见 [docs/releases](https://github.com/Hackerchen716/blueteam-log-analyzer/tree/main/docs/releases)。
+更多变更见 [v1.3.3 发布说明](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/releases/v1.3.3.md)，历史版本见 [docs/releases](https://github.com/Hackerchen716/blueteam-log-analyzer/tree/main/docs/releases)。
 
 ## 核心能力
 
@@ -230,7 +233,8 @@ bla logs/ --out incident_report/
 # ├── report.json
 # ├── events.csv
 # ├── iocs.txt
-# └── report.sarif
+# ├── report.sarif
+# └── manifest.json
 
 # 生成 SARIF 报告（可上传到 GitHub Code Scanning）
 bla logs/ --sarif report.sarif
@@ -358,7 +362,7 @@ fi
 
 关键事件时间线、ATT&CK 技术映射、应急处置建议、Top 攻击源 IP：
 
-![BLA 终端报告示例](https://raw.githubusercontent.com/Hackerchen716/blueteam-log-analyzer/main/docs/screenshots/demo.png)
+![BLA 终端报告示例](https://raw.githubusercontent.com/Hackerchen716/blueteam-log-analyzer/main/docs/screenshots/terminal-report.png)
 
 ---
 
@@ -371,7 +375,7 @@ fi
 | SecRepo auth.log | 86,839 | 27,075 | 624 | 100/100（严重） | SSH 暴力破解、密码喷洒、Top IP/Top User、IOC 提取 |
 | SecRepo Web access.log | 2,928 | 236 | 2 | 100/100（严重） | 敏感路径探测、Web 访问日志解析、`cn-hvv` 画像、IOC 提取 |
 
-完整复现命令、数据来源和结果摘要见 [SecRepo 真实样本实测记录](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/secrepo-demo.md)。
+完整复现命令、数据来源和结果摘要见 [SecRepo 公开样本验证记录](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/secrepo-sample-validation.md)。
 
 ### SecRepo auth.log 实测总览
 
@@ -394,7 +398,7 @@ fi
 ```
 ╔══════════════════════════════════════════════════════════════════════════════╗
 ║         BlueTeam Log Analyzer (BLA)  -  Blue Team Incident Response          ║
-║                    Version 1.3.2  |  100% Offline  |  No AI                  ║
+║                    Version 1.3.3  |  100% Offline  |  No AI                  ║
 ╚══════════════════════════════════════════════════════════════════════════════╝
 
 📊 分析总览
@@ -458,12 +462,12 @@ python3 -m unittest discover -s tests -v
 - Remote Workspace 的 `bla FILE`、`--rdp`、`journalctl:` 和 `--exit-on` 行为
 - 大型日志可通过 `--max-alerts` 控制终端告警展示数量
 - 可通过 `--syslog-year` 固定 Linux syslog 无年份时间戳
-- `--out` 标准报告目录可一次生成 HTML/JSON/CSV/IOC/SARIF
+- `--out` 标准报告目录可一次生成 HTML/JSON/CSV/IOC/SARIF/manifest
 - 内置 YAML Web 规则与 `--rules` 自定义规则加载
 - 自动识别 Linux/Web 日志时可走逐行解析路径，避免大文件一次性读入内存
 
 更多可用于评估 BLA 的公开日志与靶场资源见 [测试资源推荐清单](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/testing-resources.md)。
-SecRepo 真实样本的完整复现实测见 [SecRepo 真实样本实测记录](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/secrepo-demo.md)。
+SecRepo 公开样本的完整复现实测见 [SecRepo 公开样本验证记录](https://github.com/Hackerchen716/blueteam-log-analyzer/blob/main/docs/secrepo-sample-validation.md)。
 
 ---
 
@@ -523,39 +527,48 @@ blueteam-log-analyzer/
 │   │   ├── registry.py     # DetectorRegistry / DetectorSpec
 │   │   ├── enrichment.py   # 统一字段富化
 │   │   └── correlation.py  # Incident 级跨源关联
-	│   ├── output/
+│   ├── output/
 │   │   ├── terminal.py     # 终端彩色输出（ANSI，支持 Windows 10+）
 │   │   ├── html_report.py  # HTML 报告生成（独立单文件）
 │   │   ├── json_report.py  # JSON 报告输出
 │   │   ├── csv_report.py   # CSV 事件导出
 │   │   ├── ioc_report.py   # IOC 清单导出
-	│   │   ├── sarif_report.py # SARIF 2.1.0 输出（接入 GitHub Code Scanning 等）
-	│   │   └── bundle.py       # --out 标准报告目录生成
-	│   ├── remote/
-	│   │   └── ssh_workspace.py# SSH 远程日志工作台
-	│   └── utils/
-	│       └── helpers.py      # 工具函数
+│   │   ├── sarif_report.py # SARIF 2.1.0 输出（接入 GitHub Code Scanning 等）
+│   │   ├── manifest.py     # 报告包交付清单与哈希记录
+│   │   └── bundle.py       # --out 标准报告目录生成
+│   ├── remote/
+│   │   └── ssh_workspace.py# SSH 远程日志工作台
+│   └── utils/
+│       └── helpers.py      # 工具函数
 ├── docs/
 │   ├── assets/             # README 与发布素材
 │   ├── architecture.md     # 可扩展内核设计说明
 │   ├── releases/           # 版本发布说明
 │   ├── screenshots/        # 界面截图
 │   ├── allowlist-example.json # 白名单示例
-│   ├── secrepo-demo.md     # SecRepo 真实样本实测记录
+│   ├── secrepo-sample-validation.md # SecRepo 公开样本验证记录
 │   └── testing-resources.md# 测试资源推荐清单
-	├── sample_logs/
-	│   ├── auth.log            # Linux SSH 暴力破解示例日志
-	│   ├── access.log          # Web 攻击示例日志（SQLi/XSS/LFI/扫描）
-	│   ├── remote_ssh_auth.log # Remote Workspace 烟测示例
-	│   ├── windows_rdp_sample.xml
-	│   └── windows_4688_sample.xml
-	├── tests/
-	│   ├── fixtures/p0/        # P0 结构化样本和 golden incident
-	│   └── test_regressions.py # 安全与解析回归测试
-	├── pyproject.toml          # PEP 517/621 构建配置
-	├── setup.py                # Python 包安装配置
-	├── MANIFEST.in             # sdist 文档/样本/测试清单
-	└── README.md
+├── sample_logs/
+│   ├── auth.log            # Linux SSH 暴力破解示例日志
+│   ├── access.log          # Web 攻击示例日志（SQLi/XSS/LFI/扫描）
+│   ├── remote_ssh_auth.log # Remote Workspace 烟测示例
+│   ├── windows_rdp_sample.xml
+│   └── windows_4688_sample.xml
+├── tests/
+│   ├── fixtures/p0/        # P0 结构化样本和 golden incident
+│   ├── _support.py         # 回归测试公共支撑
+│   ├── test_detection.py
+│   ├── test_outputs.py
+│   ├── test_p0_security.py
+│   ├── test_parsers.py
+│   ├── test_remote_workspace.py
+│   └── test_release_hygiene.py
+├── scripts/
+│   └── release_check.py    # 本地发布质量检查
+├── pyproject.toml          # PEP 517/621 构建配置
+├── setup.py                # Python 包安装配置
+├── MANIFEST.in             # sdist 文档/样本/测试清单
+└── README.md
 ```
 
 ---

bla/__init__.py

@@ -6,6 +6,6 @@
 平台: macOS / Linux / Windows
 """
 
-from .__version__ import __version__
+from .__version__ import __version__ as __version__
 
 __author__ = "Hackerchen716"

bla/__version__.py

@@ -1,3 +1,3 @@
 """Single source of truth for the BLA package version."""
 
-__version__ = "1.3.2"
+__version__ = "1.3.3"

bla/core/pipeline.py

@@ -5,7 +5,7 @@
 import os
 from concurrent.futures import ThreadPoolExecutor, as_completed
 from dataclasses import dataclass, field
-from typing import Callable, Iterable, List, Optional
+from typing import Any, Callable, Dict, Iterable, List, Optional
 
 from ..allowlist import apply_allowlist, load_allowlist
 from ..config import DEFAULT_THRESHOLDS, load_thresholds, load_thresholds_from_env, set_thresholds
@@ -190,6 +190,7 @@ def write_reports(
     parse_results: List[ParseResult],
     summary: AnalysisSummary,
     outputs: AnalysisOutputs,
+    manifest_context: Optional[Dict[str, Any]] = None,
 ) -> None:
     if outputs.html:
         generate_html_report(parse_results, summary, outputs.html)
@@ -202,7 +203,7 @@ def write_reports(
     if outputs.sarif:
         generate_sarif_report(parse_results, summary, outputs.sarif)
     if outputs.bundle_dir:
-        generate_report_bundle(parse_results, summary, outputs.bundle_dir)
+        generate_report_bundle(parse_results, summary, outputs.bundle_dir, manifest_context=manifest_context)
 
 
 def _configure_runtime(options: AnalysisOptions) -> None:

bla/detection/engine.py

@@ -21,6 +21,12 @@
 
 _CONFIDENCE_DOWNGRADE = {"high": "medium", "medium": "low", "low": "low"}
 _WINDOWS_ACCOUNT_CHAIN_WINDOW_SECONDS = 10 * 60
+_CREDENTIAL_TOOL_RE = re.compile(r'mimikatz|lsadump|sekurlsa|kerberos::ptt|privilege::debug|credential.?dump', re.I)
+_LSASS_RE = re.compile(r'lsass', re.I)
+_CREDENTIAL_DETAIL_KEYS = (
+    "command", "cmd", "commandline", "process", "processname", "image",
+    "alert", "threat", "message", "rule", "signature", "file", "path",
+)
 
 
 def _adjust_for_private_ip(ip: str, confidence: str, evidence: List[str]) -> str:
@@ -31,7 +37,7 @@ def _adjust_for_private_ip(ip: str, confidence: str, evidence: List[str]) -> str
     """
     if not ip or not is_private_ip(ip):
         if ip:
-            evidence.append(f"来源类型: 公网")
+            evidence.append("来源类型: 公网")
         return confidence
     evidence.append(f"来源类型: 内网/私有 IP（{ip}）")
     return _CONFIDENCE_DOWNGRADE.get(confidence, confidence)
@@ -396,9 +402,7 @@ def detect_defense_evasion(events: List[LogEvent]) -> List[DetectionAlert]:
 
 def detect_credential_access(events: List[LogEvent]) -> List[DetectionAlert]:
     alerts = []
-    mimi = [e for e in events if
-            re.search(r'mimikatz|lsadump|sekurlsa|kerberos::ptt|privilege::debug|credential.?dump',
-                      e.message + e.raw_line + " ".join(str(v) for v in e.details.values()), re.I)]
+    mimi = [e for e in events if _has_credential_dump_indicator(e)]
     if mimi:
         alerts.append(DetectionAlert(
             id="a"+gen_id("ca"), rule_id="CRED-001", rule_name="Mimikatz / 凭据转储工具",
@@ -410,7 +414,7 @@ def detect_credential_access(events: List[LogEvent]) -> List[DetectionAlert]:
             timestamp=max(e.timestamp for e in mimi), confidence="high",
         ))
     lsass = [e for e in events if "lsass-dump" in e.tags or
-             (re.search(r'lsass', e.message + e.raw_line, re.I) and "sysmon" in e.tags)]
+             (_LSASS_RE.search(e.message + e.raw_line) and "sysmon" in e.tags)]
     if lsass:
         alerts.append(DetectionAlert(
             id="a"+gen_id("ls"), rule_id="CRED-002", rule_name="LSASS 进程访问",
@@ -424,6 +428,17 @@ def detect_credential_access(events: List[LogEvent]) -> List[DetectionAlert]:
     return alerts
 
 
+def _has_credential_dump_indicator(event: LogEvent) -> bool:
+    if "lsass-dump" in event.tags or "credential-access" in event.tags:
+        return True
+    text = event.message + " " + event.raw_line
+    if _CREDENTIAL_TOOL_RE.search(text):
+        return True
+    details = event.details
+    detail_text = " ".join(str(details.get(key, "")) for key in _CREDENTIAL_DETAIL_KEYS)
+    return bool(detail_text and _CREDENTIAL_TOOL_RE.search(detail_text))
+
+
 def detect_suspicious_execution(events: List[LogEvent]) -> List[DetectionAlert]:
     alerts = []
     critical_ps = [e for e in events if e.category == "PowerShell" and e.level == ThreatLevel.CRITICAL]