create the local pipe from the concrete discovered source/target operation

Author: vsilent

CHANGELOG.md

@@ -8,7 +8,8 @@ All notable changes to this project will be documented in this file.
 
 - **`stacker target [local|cloud|server]`** — switch deployment target mode; persists in `.stacker/active-target`
 - **Local pipe creation** — `stacker pipe create` works without a cloud deployment (`deployment_hash` is now optional, `is_local` flag on PipeInstance/PipeExecution)
-- **Local scanning** — `stacker pipe scan` discovers containers via `docker ps` in local mode
+- **Local scanning** — `stacker pipe scan` now performs local endpoint/resource discovery on matched containers instead of only listing Docker inventory
+- **Explicit scan modes** — `stacker pipe scan --containers [FILTER]` for local container discovery + probing and `stacker pipe scan --app <APP> [--container <NAME>]` for remote endpoint probing
 - **Local triggering** — `stacker pipe trigger` executes via `docker exec` / HTTP against local containers
 - **`stacker pipe deploy <id> --deployment <hash>`** — promote a local pipe to a remote deployment (clones config to new remote instance)
 - **`GET /api/v1/pipes/instances/local`** — list local pipe instances for the authenticated user
@@ -484,4 +485,4 @@ stacker init --with-ai  # no Ollama running → template fallback
 - Casbin reload is guarded to avoid blocking request handling and re-applies route matching after reload.
 
 ### Fixed
-- Status panel command updates query uses explicit bindings to avoid SQLx type inference errors.
+- Status panel command updates query uses explicit bindings to avoid SQLx type inference errors.

README.md

@@ -160,7 +160,9 @@ The end-user tool. No server required for local deploys.
 | `stacker agent configure-proxy` | Configure Nginx Proxy Manager via the agent |
 | `stacker agent history` | Show recent command execution history |
 | `stacker agent exec` | Execute a raw agent command with JSON parameters |
-| `stacker pipe scan <app>` | Discover API endpoints on a running container |
+| `stacker pipe scan` | Discover local endpoints/resources from running containers (when target is `local`) |
+| `stacker pipe scan --containers [filter]` | Discover local endpoints/resources for matching containers |
+| `stacker pipe scan --app <app>` | Probe a remote app for API endpoints |
 | `stacker pipe create <src> <tgt>` | Create a data pipe between two containers (interactive) |
 | `stacker pipe list` | List pipe instances for the current deployment |
 | `stacker pipe activate <id>` | Activate a pipe (start listening for triggers) |

docs/DAG_PIPES_PART1_CLI_GUIDE.md

@@ -41,11 +41,11 @@ stacker status
 ### Step 1 — Scan your services
 
 ```bash
-# See what APIs your website exposes
-stacker pipe scan website
+# Remote deployment: probe app endpoints
+stacker pipe scan --app website
 
 # See what APIs are available with sample data
-stacker pipe scan website --capture-samples
+stacker pipe scan --app website --capture-samples
 ```
 
 Output:
@@ -67,7 +67,7 @@ stacker pipe create website telegram
 ```
 
 The wizard will:
-1. Scan both apps for endpoints
+1. Scan both apps/containers for endpoints
 2. Let you pick source endpoint (POST /api/contact)
 3. Let you pick target endpoint (sendMessage)
 4. Auto-match fields (`name` → text, `email` → text)
@@ -205,7 +205,9 @@ stacker pipe trigger <slack-pipe-id> \
 
 | Command | What it does |
 |---------|-------------|
-| `stacker pipe scan <app>` | Discover what APIs an app exposes |
+| `stacker pipe scan` | Discover local Docker containers |
+| `stacker pipe scan --containers [filter]` | Discover local containers by name |
+| `stacker pipe scan --app <app>` | Discover what APIs a remote app exposes |
 | `stacker pipe create <source> <target>` | Create a pipe (interactive wizard) |
 | `stacker pipe list` | Show all pipes for your deployment |
 | `stacker pipe activate <id>` | Start the pipe (begin listening) |
@@ -274,15 +276,19 @@ stacker target
 # Output: Active target: local
 
 # All pipe commands now show [local] prefix
-stacker pipe scan website
+stacker pipe scan
 # [local] ✓ 3 containers discovered
+# [local] ✓ 7 endpoints/resources discovered
 ```
 
 ### Local Workflow
 
 ```bash
-# 1. Discover local containers (uses docker ps)
-stacker pipe scan website
+# 1. Discover local endpoints/resources from running containers
+stacker pipe scan
+
+# Optional: narrow to matching container names
+stacker pipe scan --containers website
 
 # 2. Create a pipe — no deployment hash needed
 stacker pipe create website telegram
@@ -314,7 +320,9 @@ stacker target         # show current
 
 | Command | Local Behavior |
 |---------|---------------|
-| `pipe scan` | Runs `docker ps` to discover containers |
+| `pipe scan` | Discovers local endpoints/resources from running containers |
+| `pipe scan --containers [filter]` | Filters matching containers, then probes their endpoints/resources |
+| `pipe scan --app <app>` | Not used locally — use container discovery instead |
 | `pipe create` | Creates pipe with `is_local=true`, no deployment hash |
 | `pipe list` | Shows your local pipes only |
 | `pipe trigger` | Executes via `docker exec` / HTTP |
@@ -323,6 +331,47 @@ stacker target         # show current
 | `pipe activate/deactivate` | Remote only (use after deploy) |
 | `pipe replay` | Remote only |
 
+### Scan Semantics
+
+- **Local target** → scan works with **containers**
+- **Remote target** → scan works with **apps**, optionally narrowed by `--container`
+
+```bash
+# Local
+stacker pipe scan
+stacker pipe scan --containers upload
+
+# Remote
+stacker pipe scan --app website
+stacker pipe scan --app website --container website-web-1
+```
+
+Legacy `stacker pipe scan <ARG>` still works during the transition:
+
+- in **local mode** it is treated as a container name filter
+- in **remote mode** it is treated as an app code
+
+When local scan succeeds, expect output like:
+
+```text
+[local] ✓ 1 container(s) discovered
+
+  Containers matched: 1
+    local-device-api-1  [syncopia] syncopia/device-api:local
+      addresses: 172.18.0.20:5050
+
+  App: device-api
+  Protocols detected: openapi, postgres
+
+  [openapi] http://172.18.0.20:5050/openapi.json
+       GET /devices
+           fields: [id, name]
+
+  Resources:
+    [postgres] postgres://172.18.0.10:5432/app (local-postgres-1)
+      table public.devices  -- CDC candidate
+```
+
 ---
 
 ## What's Next?

docs/PIPING.md

@@ -28,27 +28,36 @@ Data piping connects containerized apps in a deployment, routing data from one s
 
 1. **CLI** (`stacker pipe`) - user-facing commands
 2. **Server** (`/api/v1/pipes`) - REST API, validation, persistence
-3. **Agent** (status-panel) - runs on the deployment, probes containers, executes pipe triggers
+3. **Agent** (status-panel) - runs on the deployment, probes app/container endpoints, executes pipe triggers
 
-> **Local mode**: When `stacker target local` is active, scan uses `docker ps` and trigger uses `docker exec` — no agent required. Pipes are stored with `is_local=true` and no `deployment_hash`.
+> **Local mode**: When `stacker target local` is active, scan starts with Docker discovery (`docker ps` + `docker inspect`) and then probes matched containers for endpoints/resources locally — no remote agent required. Pipes are stored with `is_local=true` and no `deployment_hash`.
+
+> **Scan semantics**: local scan is **container-first**; remote scan is **app-first** with optional `--container` narrowing.
 
 ## Quick Start
 
 ### 1. Scan for connectable endpoints
 
 ```bash
-# Discover what APIs wordpress exposes
-stacker pipe scan wordpress
+# Local target: discover endpoints/resources from running containers
+stacker pipe scan
+
+# Filter local containers by name, then probe them
+stacker pipe scan --containers wordpress
 
-# With sample response capture (shows actual data)
-stacker pipe scan wordpress --capture-samples
+# Remote target: probe a deployed app for endpoints
+stacker pipe scan --app wordpress --capture-samples
 
 # Scan specific protocols
-stacker pipe scan wordpress --protocols openapi,html_forms,rest
+stacker pipe scan --app wordpress --protocols openapi,html_forms,rest
 ```
 
 Output:
 ```
+  Containers matched: 1
+    local-wordpress-1 [blog] wordpress:latest
+      addresses: 172.18.0.8:80
+
   App: wordpress
   Protocols detected: openapi
 
@@ -205,8 +214,8 @@ When `--capture-samples` is enabled during scanning, the agent:
 
 ```bash
 # 1. Scan both services
-stacker pipe scan wordpress --capture-samples
-stacker pipe scan mailchimp --capture-samples
+stacker pipe scan --app wordpress --capture-samples
+stacker pipe scan --app mailchimp --capture-samples
 
 # 2. Create the pipe
 stacker pipe create wordpress mailchimp
@@ -290,7 +299,7 @@ All endpoints require authentication. Pipe instance access is verified through d
 
 | Command | Direction | Description |
 |---------|-----------|-------------|
-| `probe_endpoints` | Server -> Agent | Discover API endpoints on a container |
+| `probe_endpoints` | Server -> Agent | Discover API endpoints for an app, optionally narrowed to a container |
 | `activate_pipe` | Server -> Agent | Start webhook listener or poll scheduler |
 | `deactivate_pipe` | Server -> Agent | Stop listener/scheduler |
 | `trigger_pipe` | Server -> Agent | One-shot pipe execution |

src/bin/agent_executor.rs

@@ -19,7 +19,9 @@ use tokio::signal;
 use tokio::sync::Notify;
 use tracing::{error, info};
 
-use stacker::models::agent_protocol::{routing, RetryPolicy, StepCommand, StepResultMsg, StepStatus};
+use stacker::models::agent_protocol::{
+    routing, RetryPolicy, StepCommand, StepResultMsg, StepStatus,
+};
 use stacker::services::resilience_engine::{
     execute_with_resilience, CircuitBreakerConfig, InMemoryCircuitBreaker,
 };
@@ -28,7 +30,11 @@ use stacker::services::resilience_engine::{
 #[command(name = "agent-executor", about = "Pipe step executor agent")]
 struct Args {
     /// AMQP connection URL
-    #[arg(long, env = "AMQP_URL", default_value = "amqp://guest:guest@localhost:5672")]
+    #[arg(
+        long,
+        env = "AMQP_URL",
+        default_value = "amqp://guest:guest@localhost:5672"
+    )]
     amqp_url: String,
 
     /// Deployment hash to scope this executor to