Skip to content

feat(store): bitrix_portals store + migration 000058 + CLI (split 2/3 from #1057)#1060

Open
tech-synity wants to merge 9 commits into
nextlevelbuilder:devfrom
tech-synity:feat/bitrix24-store-migration
Open

feat(store): bitrix_portals store + migration 000058 + CLI (split 2/3 from #1057)#1060
tech-synity wants to merge 9 commits into
nextlevelbuilder:devfrom
tech-synity:feat/bitrix24-store-migration

Conversation

@tech-synity
Copy link
Copy Markdown

Summary

PR 2/3 split from #1057. Persistence layer for Bitrix24 portals only — channel implementation lands in PR 3.

Stacked on PR 1 (#1059). Diff vs `dev` will show PR 1's content too until PR 1 merges.

Commits (2)

  1. `feat(store): add bitrix_portals store + migration 000058` — interface + PG/SQLite impls + migration files + version bumps
  2. `feat(cmd): add goclaw bitrix-portal create for seeding portal rows` — CLI command for tenant admins to provision portals from credentials

Scope

  • Migration `000058_bitrix_portals` (PostgreSQL) + SQLite v26→v27 — table for tenant-scoped Bitrix24 portal OAuth state
  • `internal/store/bitrix_portal_store.go` — store interface
  • `internal/store/pg/bitrix_portals.go` — PG implementation
  • `internal/store/sqlitestore/bitrix_portals.go` (+ schema patch) — SQLite implementation for Lite/desktop edition
  • `internal/upgrade/version.go` — `RequiredSchemaVersion` 57 → 58
  • `cmd/bitrix_portal.go` — `goclaw bitrix-portal create` CLI

Why split

Maintainer review on #1057 flagged MEGA-PR risk. This isolates the persistence change so it can be reviewed against the schema/migration policy independently from channel logic. No code in this PR consumes the store — that's PR 3's responsibility.

Security

  • All credentials (`client_id`/`client_secret`) and state (access/refresh tokens, member_id, app_token, registered_bots, media folders) stored as AES-256-GCM ciphertext via `internal/crypto/aes.go`.
  • Empty encryption key falls back to plaintext with warn-log per existing crypto contract.
  • `tenant_id` FK + `UNIQUE(tenant_id, name)` enforces tenant isolation at the schema level.
  • Webhook lookup index on `domain` enables router resolution before channel instance loads.

Test plan

  • `go build .` (PG) passes
  • `go build -tags sqliteonly ./internal/store/sqlitestore/` passes
  • SQLite store unit test (`bitrix_portals_test.go`) bundled
  • PG integration test — runs on maintainer CI (requires pgvector pg18 fixture)

Coordination

tech-synity and others added 9 commits April 28, 2026 17:58
The bare pattern `pkg-helper` on line 6 matched any file or directory
named pkg-helper anywhere in the tree — including the tracked source
directory `cmd/pkg-helper/`. Git tolerated this because already-tracked
files override ignore rules, but uploaders that apply .gitignore patterns
naively to the filesystem (notably `railway up`) stripped the source
directory from the build context, breaking `go build ./cmd/pkg-helper`
in Docker with `stat /src/cmd/pkg-helper: directory not found`.

Anchor both binary-artifact patterns (`/openclaw-go`, `/pkg-helper`) to
the repo root so they only match the compiled output, not the source.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Working tree kept surfacing `goclaw.exe` / `goclaw-local.exe` (Windows
dev builds) and ad-hoc `tmp-reset-bot/` probe scripts as untracked.
A single `git add .` would commit them — both are useless outside the
author's machine and the debug probes import `internal/` packages from
outside `cmd/` (ugly but legal), so they should never ride into Git.

Add to .gitignore (anchored like the existing `/openclaw-go` entry, so
matching doesn't accidentally touch source dirs — same concern that
broke Docker builds in 2de99e26):

  /goclaw.exe
  /goclaw-local.exe
  /tmp-reset-bot/
  /tmp-probe*/

Also add .railwayignore file — new; Railway copies the working tree
verbatim and already ignored tests/node_modules/etc. Add `tmp-*` glob
so ad-hoc probe dirs don't bloat image layers on `railway up`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Routine dependency refresh. v2.12 pulls in a new indirect dep
(git.sr.ht/~jackmordaunt/go-toast/v2 v2.0.3 — used by Wails'
notification path) and otherwise contains only maintenance fixes
from upstream.

Verified: `go build -tags sqliteonly ./internal/... ./cmd/` green —
desktop edition's Wails bindings still compile. No goclaw code calls
the notification API directly, so the new indirect is latent unless
Wails decides to pull it into the main path later.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… in strict mode

OpenAI strict mode rejected `use_skill` calls with:
  invalid_function_parameters: 'additionalProperties' is required to be
  supplied and to be false
  at path ('properties', 'params', 'type', '0')

Root cause: applyStrictMode early-returned when a node was
`{"type":"object"}` with no nested `properties` key, leaving
additionalProperties unset. Then makeNullable widened the type to
`["object","null"]`, and the strict validator checks each branch
independently — the "object" variant was missing additionalProperties
so the whole schema failed validation.

Fix: when typ=="object", set additionalProperties:false even if the
node has no inner properties. This covers the common "bag of
free-form params" shape tool authors write as:

  params: { type: "object", description: "..." }

Trade-off worth documenting: strict mode + bare object ⇒ OpenAI will
reject any call that actually populates those params. If a tool truly
needs a free-form dict of arbitrary keys, strict mode is the wrong
fit — either declare the known keys under `properties`, or opt that
tool out of strict. Left a comment on the branch spelling this out.

Regression test added: TestApplyStrictMode_BareObjectProperty asserts
both `additionalProperties:false` AND the `["object","null"]` type
union survive the transform.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously, builds without the `embedui` tag left "/" unhandled,
so http.ServeMux returned a bare 404 when an operator opened the
deployed URL in a browser. That's needlessly opaque — it looks
like the service is broken even though the gateway is healthy.

Register a small JSON index handler for "/" in the no-UI branch.
It returns service status + protocol version + a list of real
endpoints for an exact "/" match, and a JSON 404 for any other
unmatched path (http.ServeMux routes "/" as a catch-all).

Embedui builds are unchanged — the webui.Handler() still owns "/"
and takes precedence.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lets admins attach server-specific quirks (e.g. "no trailing semicolons
in code args") to MCP tool descriptions without modifying the upstream
MCP server or redeploying goclaw. Hints are stored in the existing
mcp_servers.settings JSONB (no migration) following the same pattern as
require_user_credentials.

Backend:
- Add mcp.ToolHints type + ParseToolHints() helper alongside requireUserCreds
- Add BridgeTool.WithHints(global, toolHint) setter that appends a suffix
  to Description() so the LLM sees hints as part of the tool schema
- Thread hints through connectServer / connectViaPool / loop_mcp_user
- 12 new unit tests for ParseToolHints + WithHints (+ edge cases)

Frontend:
- New "AI Agent Hints (Optional)" section in MCP form dialog: a global
  textarea + a key-value list for per-tool hints
- Extend KeyValueEditor with valueAs="textarea" prop (reusable)
- Zod schema + TS types + i18n (en/vi/zh) updated

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…cake parity)

Creating a Bitrix24 channel_instance failed with "invalid channel_type"
even though Phase 03 wired the factory and the UI added Bitrix24 to the
picker. The write path hits two parallel whitelist switches — one for
WS RPC in internal/gateway/methods and one for HTTP in internal/http —
and both missed bitrix24. The WS switch also silently missed facebook
and pancake, so anything driven through the WS API was rejecting
channels the HTTP API already accepted.

- Add bitrix24 to both switches.
- Add facebook and pancake to the WS switch so the two functions stop
  drifting. CHANNEL_TYPES in the UI constants file was already offering
  both.
- Annotate each switch with a keep-in-sync comment pointing at its twin
  and at ui/web/src/constants/channels.ts. The previous drift was silent
  (untyped string list across three separate files); the comment at
  least makes the next channel addition noisy enough to catch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the persistence layer for Bitrix24 portal OAuth state, decoupled
from the channel implementation so the channel package (PR 3) can land
incrementally:

- Migration 000058_bitrix_portals (PostgreSQL) — tenant-scoped portal
  rows with AES-GCM encrypted credentials and refresh state.
- internal/store/bitrix_portal_store.go — store interface (BitrixPortalStore).
- internal/store/pg/bitrix_portals.go — PostgreSQL implementation.
- internal/store/sqlitestore/bitrix_portals.go + schema patch — SQLite
  implementation for the desktop edition (lite). Schema v26 → v27.
- internal/upgrade/version.go — RequiredSchemaVersion 57 → 58.
- factory + stores.go wiring so consumers can resolve BitrixPortals from
  the standard StoreFactory.

Storage uses internal/crypto/aes.go (AES-256-GCM) for both credentials
(client_id + client_secret) and state (access/refresh tokens, member_id,
app_token, registered_bots, media folders). Empty encryption key falls
back to plaintext with a warn log per existing crypto contract.

Indexes:
  - UNIQUE (tenant_id, name) — one portal name per tenant.
  - (domain) — lookup by incoming webhook domain.

This is PR 2/3 split from nextlevelbuilder#1057. Stacked on PR 1 (nextlevelbuilder#1059).
Phase 03 wired the Bitrix24 OAuth install handler at `/bitrix24/install`
but shipped no RPC/UI path for creating the `bitrix_portals` row that
handler looks up. Operators currently have to shell into Postgres to
register a portal, which blocks the UI-driven setup flow that the
channel picker now offers.

This adds a direct-DB CLI command mirroring the `cmd/migrate.go`
pattern (resolveDSN → sql.Open(pgx, dsn)):

- `bitrix-portal create --tenant-id --name --domain --client-id
  --client-secret` writes one row via `PGBitrixPortalStore.Create`, so
  credentials are AES-256-GCM encrypted under `GOCLAW_ENCRYPTION_KEY`
  just like runtime writes. When the key is unset we log a visible
  WARNING — the pg store falls back to plaintext and we do not want
  operators to discover that silently.

- `bitrix-portal list [--tenant-id]` prints id/tenant/name/domain.
  Credentials are deliberately never printed; rows whose creds failed
  to decrypt surface as `(creds:empty)` so a corrupt row is obvious.

- `normalizeBitrixDomain` strips scheme + trailing slash so
  `https://tamgiac.bitrix24.com/` and `tamgiac.bitrix24.com` both land
  as the bare host Bitrix24's OAuth callback compares against.

After `create`, the printed hint guides the operator to the install
URL: `https://<public_url>/bitrix24/install?state=<tenant>:<name>`,
which populates the row's `state` column with the OAuth token.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant