Generated 2026-03-22 | StarUML project:
docs/UML/skill_seekers.mdj
Skill Seekers converts documentation from 17 source types into production-ready formats for 24+ AI platforms. The architecture follows a layered module design with 8 core modules and 5 utility modules.
Core Modules (upper area):
- CLICore -- Git-style command dispatcher, entry point for all
skill-seekerscommands - Scrapers -- 17 source-type extractors (web, GitHub, PDF, Word, EPUB, video, etc.)
- Adaptors -- Strategy+Factory pattern for 20+ output platforms (Claude, Gemini, OpenAI, RAG frameworks)
- Analysis -- C3.x codebase analysis pipeline (AST parsing, 10 GoF pattern detectors, guide builders)
- Enhancement -- AI-powered skill improvement via
AgentClient(API mode: Anthropic/Kimi/Gemini/OpenAI + LOCAL mode: Claude Code/Kimi/Codex/Copilot/OpenCode/custom, --enhance-level 0-3) - Packaging -- Package, upload, and install skills to AI agent directories
- MCP -- FastMCP server exposing 40 tools via stdio/HTTP transport (includes marketplace and config publishing)
- Sync -- Documentation change detection and re-scraping triggers
Utility Modules (lower area):
- Parsers -- CLI argument parsers (30+ SubcommandParser subclasses)
- Storage -- Cloud storage abstraction (S3, GCS, Azure)
- Embedding -- Multi-provider vector embedding generation
- Benchmark -- Performance measurement framework
- Utilities -- Shared helpers (LanguageDetector, RAGChunker, MarkdownCleaner, etc.)
Entry point: skill-seekers CLI. CLIDispatcher maps subcommands to modules via COMMAND_MODULES dict. CreateCommand auto-detects source type via SourceDetector.
18 scraper classes implementing IScraper. Each has a main() entry point. Notable: GitHubScraper (3-stream fetcher) + GitHubToSkillConverter (builder), UnifiedScraper (multi-source orchestrator).
SkillAdaptor ABC with 3 abstract methods: format_skill_md(), package(), upload(). Two-level hierarchy: direct subclasses (Claude, Gemini, OpenAI, Markdown, OpenCode, RAG adaptors) and OpenAICompatibleAdaptor intermediate (MiniMax, Kimi, DeepSeek, Qwen, OpenRouter, Together, Fireworks).
UnifiedCodebaseAnalyzer controller orchestrates: CodeAnalyzer (AST, 9 languages), PatternRecognizer (10 GoF detectors via BasePatternDetector), TestExampleExtractor, HowToGuideBuilder, ConfigExtractor, SignalFlowAnalyzer, DependencyAnalyzer, ArchitecturalPatternDetector.
Two enhancement hierarchies: AIEnhancer (API mode, multi-provider via AgentClient) and UnifiedEnhancer (C3.x pipeline enhancers). Each has specialized subclasses for patterns, test examples, guides, and configs. WorkflowEngine orchestrates multi-stage EnhancementWorkflow. The AgentClient (cli/agent_client.py) centralizes all AI invocations, supporting API mode (Anthropic, Moonshot/Kimi, Gemini, OpenAI) and LOCAL mode (Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents).
PackageSkill delegates to adaptors for format-specific packaging. UploadSkill handles platform API uploads. InstallSkill/InstallAgent install to AI agent directories. OpenCodeSkillSplitter handles large file splitting.
SkillSeekerMCPServer (FastMCP) with 40 tools in 10 categories. Supporting classes: SourceManager (config CRUD), AgentDetector (environment detection), GitConfigRepo (community configs), MarketplacePublisher (publish skills to marketplace repos), MarketplaceManager (marketplace registry CRUD), ConfigPublisher (push configs to registered source repos).
SyncMonitor controller schedules periodic checks via ChangeDetector (SHA-256 hashing, HTTP headers, content diffing). Notifier sends alerts when changes are found. Pydantic models: PageChange, ChangeReport, SyncConfig, SyncState.
SubcommandParser ABC with 27 subclasses -- one per CLI subcommand (Create, Scrape, GitHub, PDF, Word, EPUB, Video, Unified, Analyze, Enhance, Package, Upload, Jupyter, HTML, OpenAPI, AsciiDoc, Pptx, RSS, ManPage, Confluence, Notion, Chat, Config, Estimate, Install, Stream, Quality, SyncConfig).
BaseStorageAdaptor ABC with S3StorageAdaptor, GCSStorageAdaptor, AzureStorageAdaptor. StorageObject dataclass for file metadata.
EmbeddingGenerator (multi-provider: OpenAI, Sentence Transformers, Voyage AI). EmbeddingPipeline coordinates provider, caching, and cost tracking. EmbeddingProvider ABC with OpenAI and Local implementations.
BenchmarkRunner orchestrates Benchmark instances. BenchmarkResult collects timings/memory/metrics and produces BenchmarkReport. Supporting data types: Metric, TimingResult, MemoryUsage, ComparisonReport.
16 shared helper classes: LanguageDetector, MarkdownCleaner, RAGChunker, RateLimitHandler, ConfigManager, ConfigValidator, SkillQualityChecker, QualityAnalyzer, LlmsTxtDetector/Downloader/Parser, ConfigSplitter, ConflictDetector, IncrementalUpdater, MultiLanguageManager, StreamingIngester.
| Pattern | Where | Classes |
|---|---|---|
| Strategy + Factory | Adaptors | SkillAdaptor ABC + get_adaptor() factory + 20+ implementations |
| Strategy + Factory | Storage | BaseStorageAdaptor ABC + S3/GCS/Azure |
| Strategy + Factory | Embedding | EmbeddingProvider ABC + OpenAI/Local |
| Command | CLI | CLIDispatcher + COMMAND_MODULES lazy dispatch |
| Template Method | Pattern Detection | BasePatternDetector + 10 GoF detectors |
| Template Method | Parsers | SubcommandParser + 27 subclasses |
CreateCommand is a dispatcher, not a pipeline orchestrator. Flow: User → execute() → SourceDetector.detect(source) → validate_source() → _validate_arguments() → _route_to_scraper() → scraper.main(argv). The 5 phases (scrape, build_skill, enhance, package, upload) all happen inside each scraper's main() — CreateCommand only sees the exit code.
UnifiedScraper orchestrates GitHub scraping (3-stream fetch) then delegates to analyze_codebase(enhance_level) for C3.x analysis. Shows all 5 C3.x stages: PatternRecognizer (C3.1), TestExampleExtractor (C3.2), HowToGuideBuilder with examples from C3.2 (C3.3), ConfigExtractor (C3.4), and ArchitecturalPatternDetector (C3.5). Note: enhance_level is the sole AI control parameter — enhance_with_ai/ai_mode are internal to C3.x classes only.
Activity diagram showing source_detector.py decision tree in correct code order: file extension first (.json config, .pdf/.docx/.epub/.ipynb/.html/.pptx/etc) → video URL → os.path.isdir() (Codebase) → GitHub pattern (owner/repo or github.com URL) → http/https URL (Web) → bare domain inference → error.
MCP Client (Claude Code/Cursor) → FastMCPServer (stdio/HTTP) with two invocation paths: Path A (scraping tools) uses subprocess.run(["skill-seekers", ...]), Path B (packaging/config tools) uses direct Python imports (get_adaptor(), sync_config()). Both return TextContent → JSON-RPC.
--enhance-level decision flow with precise internal variable mapping: Level 0 sets ai_mode=none, skips all AI. Level >= 1 selects ai_mode=api (if any supported API key set: Anthropic, Moonshot/Kimi, Gemini, OpenAI) or ai_mode=local (via AgentClient with configurable agent: Claude Code, Kimi, Codex, Copilot, OpenCode, or custom), then SKILL.md enhancement happens post-build via enhance_command. Level >= 2 enables enhance_config=True, enhance_architecture=True inside analyze_codebase(). Level 3 adds enhance_patterns=True, enhance_tests=True.
Component diagram with corrected runtime dependencies. Key flows: CLI Core dispatches to Scrapers (via scraper.main(argv)) and to Adaptors (via package/upload commands). Scrapers call Codebase Analysis via analyze_codebase(enhance_level). Codebase Analysis uses C3.x Classes internally and Enhancement when level ≥ 2. MCP Server reaches Scrapers via subprocess and Adaptors via direct import. Scrapers optionally use Browser Renderer (Playwright) via render_page() when --browser flag is set for JavaScript SPA sites.
When --browser flag is set, DocScraper.scrape_page() delegates to BrowserRenderer.render_page(url) instead of requests.get(). The renderer auto-installs Chromium on first use, navigates with wait_until='networkidle' to let JavaScript execute, then returns the fully-rendered HTML. The rest of the pipeline (BeautifulSoup → extract_content() → save_page()) remains unchanged. Optional dependency: pip install "skill-seekers[browser]".
- StarUML project:
docs/UML/skill_seekers.mdj - Diagram exports:
docs/UML/exports/*.png - Source code:
src/skill_seekers/