feat: add pipeline validation error analysis (#75)

Author: mathislucka

README.md

@@ -122,18 +122,16 @@ Claude will ask for your permission before a tool call is executed. You can opt
 
 Unfortunately, you need to set the workspace and organization (through the API key) in the `claude_desktop_config.json`.
 There is no way to pass the API key dynamically to Claude in a secure way.
-The workspace could be passed into the tool call, it's an easy enhancement but I'd like to get feedback first.
+The workspace could be passed into the tool call, it's an easy enhancement, but I'd like to get feedback first.
 
 
 
 
 ## Further improvements ideas
 
-- remove hardcoded workspace
 - expose standard prompts via MCP e.g., for debugging, fixing pipelines, reading logs etc
 - fix the docker run command to clear cache
 - the ability to dump the conversation of improving the copilot
-- test with different models not just Claude Sonnet 3.7
 - test with different clients other than Claude Desktop app
 
 

TODO.md

@@ -0,0 +1,20 @@
+How do we get from collection of tools to Agents that work well for defined use cases?
+
+## Use Case Ideas
+- create a pipeline from scratch
+- edit an existing pipeline
+- let the agent create connections in an incomplete pipeline
+- resolve validation errors
+- fix a pipeline based on logs
+- optimize the prompt in a pipeline
+- optimize retrieval in a pipeline
+
+
+## Benchmark
+- create test cases
+- let agent run on test cases
+- score:
+  - pipeline validates
+  - pipeline matches expected solution
+  - pipeline yields search results
+- dig into common validation errors to find best cases to cover

data/processed/created_updated_deployed_events.csv

@@ -17,6 +17,14 @@ dependencies = [
 [project.scripts]
 deepset-mcp = "deepset_mcp.main:main"
 
+[project.optional-dependencies]
+analysis = [
+    "jupyterlab",
+    "pandas",
+    "matplotlib",
+    "seaborn"
+]
+
 [build-system]
 requires = ["hatchling"]
 build-backend = "hatchling.build"
@@ -37,6 +45,7 @@ lint = [
 types = [
     "mypy",
     "types-PyYAML",
+    "pandas-stubs",
 ]
 
 [tool.pytest.ini_options]