chore: sync CLI changes from langchainplus [copybara]#187
Closed
Ramon Nogueira (ramon-langchain) wants to merge 9 commits into
Closed
chore: sync CLI changes from langchainplus [copybara]#187Ramon Nogueira (ramon-langchain) wants to merge 9 commits into
Ramon Nogueira (ramon-langchain) wants to merge 9 commits into
Conversation
…000] [copybara] (#27446) Stop excluding `.github/**` from the Copybara workflows so the vendored SDKs' GitHub Actions are actually imported (and managed in both directions). - Removed the `.github/**` exclusion from both the **vendor** (`origin_files`) and **export** (`destination_files`) globs for the cli, go, and terraform SDKs. This monorepo now imports each SDK's `.github/` and is authoritative over it on export. - Dropped the now-dead cli `core.replace` (RSA-key obfuscation): the OSS repo already stores the obfuscated split form, so the transform was a guaranteed no-op and broke `vendor_cli`. - Re-vendored all three SDKs from their OSS `main`. Beyond the new `.github/workflows`, this pulls in accumulated forward-sync drift (verified as real upstream commits, not local edits) — e.g. go's `datasetplaygroundexperiment.go` deletion from langsmith-go `#105`, and terraform's `docs/`/`examples/`/`templates/`. `[copybara]` is in the title because the change touches `sdks/go/`, which `check_sdk_copybara.yml` gates. ## Self-Hosted Release Note none ## Test Plan - [x] none GitOrigin-RevId: db2713ebcf6b634274bb91055dae87f50e29dfdf
## What `langsmith api <shorthand>` (a path with no leading `/`) hardcoded an `/api/v1/` prefix in both `resolveEndpoint` and `LookupEndpoint`. Any surface not under `/api/v1` — `/v2/*` (all sandboxes, 18 endpoints), `/commits`, `/me`, `/orgs`, etc. — was rewritten to `/api/v1/v2/...` and silently misrouted to the Python backend, returning a confusing `404`. This resolves shorthands against the cached OpenAPI spec: prefer `/api/v1/<path>`, then an exact `/<path>`, then a unique spec path whose trailing segments match (so `sandboxes/boxes` → `/v2/sandboxes/boxes`), falling back to `/api/v1/<path>` when the spec isn't cached. Absolute paths (`/v2/...`) and full URLs are unchanged. Resolution is **cache-only** — it reads the spec the browse commands (`api ls`/`api info`) already populate and never fetches, so there's no network round-trip added to the request hot path. ## Test Plan - [x] `go build ./...` + `go test ./internal/cmd/api/` pass - [x] New table tests in `resolve_test.go` cover spec-backed shorthand resolution, query-string preservation, fallback, and absolute-path passthrough - [x] Verified against dev: `langsmith api sandboxes/boxes` now resolves to `/v2/sandboxes/boxes` (200), previously a `uvicorn` 404 GitOrigin-RevId: 1409b98a30e526cee5f2161d8f62ffebc690a60e
…copybara] (#28676) ## Description Imports the latest `main` from the upstream OSS repos via the Copybara vendor workflows: - `sdks/go` ← `langchain-ai/langsmith-go` (`./scripts/copybara copy.bara.sky vendor_go --folder-dir sdks/go`) - `sdks/cli` ← `langchain-ai/langsmith-cli` (`./scripts/copybara copy.bara.sky vendor_cli --folder-dir sdks/cli`) Also adds a reversible `copy.bara.sky` transform (`CLI_GO_MOD_REPLACE`) that injects `replace github.com/langchain-ai/langsmith-go => ../go` into the vendored CLI's `go.mod` on `vendor_cli`, so the CLI always builds against the in-repo `sdks/go` instead of the published module — the two vendors can no longer drift out of sync. The transform is reversed on `export_cli` so upstream `langsmith-cli` never carries the local path. No hand edits to vendored files. ## Test Plan - [x] `go build ./...` passes in `sdks/go` and `sdks/cli` - [x] `sdks/cli` builds against the in-repo `sdks/go` via the injected `replace` directive GitOrigin-RevId: c0b787a61bc1020db37497bdf78bd6857d673b2f
## Description This PR changes only the Go CLI command: langsmith evaluator upload <file> --replace - Updates `langsmith evaluator upload --replace` to patch the existing code evaluator rule instead of deleting it before creating the replacement. References [LSO-3365](https://linear.app/langchain/issue/LSO-3365/cli-evaluator-upload-replace-can-delete-the-existing-evaluator-before) ## Test Plan - [x] Unit tests Co-authored-by: Cursor <cursoragent@cursor.com> GitOrigin-RevId: 6c4d578278fbc632a6658c83b578c1d051b477ae
## Summary - Patch the minimum fixed versions for repository-owned critical/high Dependabot findings across Go, Python, and frontend dependency manifests. - Align `smith-clio`'s LangChain/LangGraph constraints so the patched LangChain release can resolve. - Regenerate affected lockfiles and Go checksums. No matching open Linear tickets were found. ## Validation - `uv lock --check` passed for the root and `smith-clio` environments. - Frozen pnpm lockfile checks passed for `smith-frontend-perf` and `smith-polly`. - `git diff --check` passed. - Go tests could not run locally: the installed toolchain is Go 1.26.2 while cached artifacts were built with Go 1.25.6. `smith-go` dependency fetching was also blocked by Socket Firewall TLS verification for `golang.org/x/image@v0.41.0`. ## Test Plan - [x] Confirm Go tests and dependency checks pass in CI with the repository-supported toolchain. - [x] Confirm repository-owned critical/high alerts clear after merge. [no screenshot] GitOrigin-RevId: 3e790a2ba2b17429ec4a72bec08765a7882f4410
## Description Bumps langsmith-go module requirements to v0.19.0 without importing the vendored generated SDK. This keeps the PR focused on Go module metadata only. ## Test Plan - [x] `go test ./...` in `cmd/langdev`, `sdks/cli`, and `sandbox-runner-controller`. Made by [Open SWE](https://openswe.vercel.app/agents/a9a13dbd-e06f-88b4-7dcc-42401154c3fd) --------- Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com> GitOrigin-RevId: f3178c2e6eedd22e1dd81d72bb767a759f50545a
## Description - Adds `langsmith evaluator create-llm` to create structured LLM-as-judge evaluator rules from the CLI (project or dataset target). - Supports inline JSON prompt + schema files, Prompt Hub refs (`--hub-ref`), and a serialized model config file (`--model-config`). - Adds `--trace-filter` to `evaluator upload` for parity with the API and `create-llm`. <img width="2168" height="583" alt="Screenshot 2026-06-30 at 2 15 41 PM" src="https://github.com/user-attachments/assets/e7372607-a2f7-428a-83e0-37929a07795b" /> References [LSE-2496](https://linear.app/langchain/issue/LSE-2496/feature-request-create-llm-as-judge-structured-evaluators-via-the-cli) ### Example Add model API keys to workspace **Secrets** first. Copy `structured.model` from an existing UI evaluator (or `GET /runs/rules`) into `model.json`. ```bash langsmith evaluator create-llm \ --name relevance \ --project my-app \ --prompt prompt.json \ --schema schema.json \ --model-config model.json \ --variable-mapping '{"input":"input.question","output":"output.answer"}' \ --trace-filter 'eq(is_root, true)' \ --sampling-rate 0.05 ``` ## Test Plan - [x] Added unit tests for prompt file parsing, model-config requirement, project/dataset targeting, and create-llm command registration - [x] Created an LLM evaluator against a test project locally and verified it appears in the UI --------- Co-authored-by: Cursor <cursoragent@cursor.com> GitOrigin-RevId: 41b9fb8ed04acb65761c7bab9f1b60868950564c
…30615) ## Description Syncs the vendored Go SDK from `langchain-ai/langsmith-go` main using the repository's Copybara workflow, bringing it to v0.20.1. ## Test Plan - [x] `go -C sdks/go test ./...` - [x] Verify changed Go files pass `gofmt -l` Made by [Open SWE](https://openswe.vercel.app/agents/80e7d2a9-2816-8835-06d6-935f9fdfb4a4) ## References - Slack thread: https://langchain.slack.com/archives/C0B3HS27NTH/p1784219110832059?thread_ts=1784219110.832059&cid=C0B3HS27NTH --------- Co-authored-by: open-swe[bot] <open-swe@users.noreply.github.com> GitOrigin-RevId: 24e8b80797548cc311562992249982b2087de6ce
…1219]" (#30694) Reverts langchain-ai/langchainplus#30615 GitOrigin-RevId: e5f86b6b5935a2db90f13584577f83c717df5efd
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
4 tasks
Member
Author
|
Superseded by #188, which ports the two genuinely-missing changes (LLM-judge — Claude |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated by
copybara export_clifromlangchain-ai/langchainplus(the monorepo wheresdks/cliwas vendored). This surfaces every CLI-touching commit made in langchainplus since the last export baseline (12bf33a, 2026-06-08).Because Copybara makes this repo match the langchainplus vendored copy, the diff both adds langchainplus-only work and reverts independent upstream work that was committed here directly and never vendored back into langchainplus.
Genuinely missing upstream (the reason for this sync):
internal/cmd/evaluator_llm.go+create-llmregistration inevaluator.go— LLM-as-judge evaluator creation (langchainplus #28904)Would be lost if merged (upstream-only, must be preserved):
internal/cmd/install_method.go+install_method_test.go(deleted here) — the self-update / install-method feature, never present in langchainplusupdate.go,internal/client/client.go,internal/cmdutil/resolve.go,internal/extract/extract.go,helpers.go,scripts/install.sh, andgo.mod/go.sumdependency bumpsRecommended path
Cherry-pick only the additive pieces (#28904, #30192) onto current
mainrather than merging this branch, then cut a release. This PR exists so the complete langchainplus-vs-upstream delta is reviewable in one place.