feat: consolidate tracer, fees & CRM into the midaz v4 monorepo#2159
Open
fredcamaral wants to merge 456 commits into
Open
feat: consolidate tracer, fees & CRM into the midaz v4 monorepo#2159fredcamaral wants to merge 456 commits into
fredcamaral wants to merge 456 commits into
Conversation
…B resolver T1: buildTracerReserver fails fast when MULTI_TENANT_ENABLED && TRACER_BASE_URL is set but no M2M auth provider is wired (none exists yet) — refuses to ship unauthenticated, tenant-less reserve calls on the transaction hot path. F1: inject the fees Mongo manager into TransactionHandler so the fee seam resolves the tenant fee DB (test in transaction_fee_tenant_test.go).
…ient seams T4: InjectHTTPContext in do() so all five reservation ops continue the ledger trace instead of starting orphaned roots. T5/T6: remove the orphaned circuit-breaker seam (test-only, never wired). T7: remove dead WithHTTPClient option.
…gauges, replay metric R1: drop dead CountBackupQueue + lying doc. R2: drop write-only serviceName field/param. R3/R4: parse each backup record once per cycle (track oldest-TTL in the dispatch pass). R5: span attr + counter for the overdraft replay-divergence branch. R6: hoist redis.NewScript to package vars (no per-call SHA1 on the hot path).
…stration tests K1: CreateHolderWithID treats a document-association collision on the deterministic id as idempotent (re-fetch by id, errors.As+Code), so a doc owned by another holder no longer bricks the run; resolves K4. K2: single-tenant Run injects the onboarding PG conn into ctx before RunTenant. K3: MT loop tests via injectable seams. K5: holderReaderAdapter.Exists table test. K6/K7: bind constant.ExternalAccountType. K8: nil-tenant guard.
N1/N5: span attr key app.request.alias_id -> app.request.instrument_id (8 sites). N2: alias-named locals + span-error/log messages -> instrument. N4: span op-names (incl. Count's misnomer); aliases_<org> collection name kept (renaming orphans prod data). N3: single time.Now() on create. N6: fixed times in tests.
… faults E5: ErrMidazQueryFailed 500->503, ErrMissingSegmentContext 400->500 (server-misconfig, not a 4xx). E9: ErrTransactionReservationUnavailable(0178) 422->503 (retryable dependency outage). E3/E4: comma-ok settings assertions (no panic). E6: drop dead HTTPError branch (type kept as Swagger anchor). E8: declaration-order contract test + doc. BREAKING CHANGE: three error codes change HTTP status. Done inside the v4 breaking-change window; documented in docs/plans/2026-06-07-v4-error-status-migration-notes.md and locked by mainline_error_contract_test.go.
…ords persist Q1: the unmarshal_failure poison record is by definition non-JSON; a JSONB NOT NULL column rejected it (SQLSTATE 22P02), so it could never quarantine and looped in Redis forever. Column is opaque/never queried as JSON -> BYTEA. Migration 000034 is net-new on this branch (edited in place). Adds a testcontainer Insert test with a non-JSON payload.
…e2e comments S1/S2/S3: assert the HolderID value-copy (a mis-wire to another *string field would have compiled and passed). S4: delete comments referencing an e2e mirror struct that does not exist.
… ctx-aware sleep B2/B3: retry.go uses commons/backoff.WaitContext(ctx) — interruptible, abandons on cancel (safe redelivery); SleepFunc removed. B1 (third rail): delete BackoffCalculator, route all call sites through commons/backoff.ExponentialWithJitter + min(cap). B4/B5/E7: delete dead deprecated wrappers + orphaned shim re-exports. Behavior change: full-jitter [0,exp) replaces the old [exp,2*exp] — retry sleeps are smaller on average (canonical AWS full-jitter).
…anager guard D1/D7: PG/Mongo schema validation agree on empty-field tables (+ parity test); drop dead Mongo matchedTables. D2/D3: fix stale TPL-/Fetcher comments. D4: delete dead sentinel aliases. D5: ValidateSchema matches GetDataSourceSchema not-found wrap. D6: shared validateTenantID predicate. D8: BASE-TABLE introspection drift-lock test. D9: nil-manager panic guard at construction.
Audited 68 Gate-8 findings (54 REAL, 10 PARTIALLY_VALID, 4 FALSE_POSITIVE) -> 38 distinct work units across 4 tiers, with per-finding verdict, chosen fix, and status.
… OpenAPI annotations Where the response status was semantically wrong, fix the code, not just the docs. State-machine and uniqueness conflicts are 409 Conflict, not 422 Unprocessable Entity. Done in the v4 breaking window. BREAKING CHANGE: 7 sentinels now return HTTP 409 instead of 422: 0087 ErrTransactionIDHasAlreadyParentTransaction 0088 ErrTransactionIDIsAlreadyARevert 0093 ErrBalancesCantBeDeleted 0099 ErrCommitTransactionNotPending 0199 ErrPackageRange 0219 ErrBillingRouteOverlap 0016 ErrBalanceRemainingDeletion Kept at 422 (payload-content invariants, not state conflicts): 0089, 0090, 0150, 0182. - pkg/errors.go: UnprocessableOperationError -> EntityConflictError in errorMap - contract + unit tests: assertion flips to 409 - handlers: @failure annotations (commit/cancel/revert/packages now carry both 409 and 422; delete-balance/delete-asset/billing-package -> 409) - regen: ledger/tracer/reporter specs + postman mirrors - OpenAPI annotation completeness across ~19 handlers (descriptions, 401/403, X-Request-Id, UUID-format params) Note: EntityConflictError.Error() returns only Message (no code prefix), unlike UnprocessableOperationError.Error(); the code survives on the typed .Code field and the wire `code` JSON.
…all components Phase 5 (schema deepening) + the M13 @name-sweep completion, folded. Schema deepening: - tracer pkg/model: Rule/Limit/Scope/AuditEvent/TransactionValidation/ ValidationResponse/EvaluationResult get swagger:model+@name+@description, field examples, enum hints; Rule.Expression carries a CEL example; the swaggerignore that hid EvaluationResult's promoted fields is removed. - fees: 10 billing structs annotated; Calculation.Type enums [percentage,flat]; FeeCalculate/FeeEstimate.transaction documented (renders as TransactionInput). - reporter: CreateReportInput.Filters nested-shape description; Pagination.Items typed; metricsResponse/errorMetrics/notificationItem/notificationResponse exported (MetricsResponse/ErrorMetrics/NotificationItem/NotificationResponse) with @name+@description+examples; test mirrors repointed to the exported types. - CRM: Holder/Instrument deepened to the Account standard; LegalPerson's mislabeled @description corrected. @name sweep (finish M13 for tracer + ledger tail): - tracer pkg/model enum types (Decision/RuleStatus/LimitStatus/LimitType/ ActorType/AuditEventType/ResourceType/TransactionType) inline-typed on every serialized field; context sub-structs, Actor, and the http/in handler DTOs (Create/Update*Input, List*Response, Reserve*, *ActionResponse, ValidationSummary) get closing-brace @name directives. - ledger: pkg/net/http.Pagination -> @name CursorPagination (distinct from the page/total Pagination); mmodel LedgerSettings/TracerSettings/AccountingValidation named. Result: zero dotted/package-qualified schema keys across all three specs (only lib-commons pkg.HTTPError remains). Regenerated ledger/tracer/reporter api/* + postman/specs/*. Non-breaking: json wire shapes unchanged, only schema names/descriptions/examples added. ledger security 111/111, parity green.
…(0486) ErrCommitTransactionNotPending (0099) was returned from three semantically distinct conditions in the commit/cancel/revert state handlers: - status != PENDING (commit) -> genuine state conflict - status != APPROVED (revert) -> genuine state conflict - Redis SetNX lock NOT acquired -> concurrency, NOT a status problem The lock-contention path returned "Invalid Transaction Status / the status does not allow the requested action", which misleads the client: the status may be perfectly valid (PENDING) — a concurrent commit just holds the lock. The honest signal there is "retry shortly", not "fix your request". New sentinel ErrPendingTransactionLocked (0486), mapped to EntityConflictError (409, unchanged status), with a retry-worded message. Only the SetNX-not-acquired site (transaction_state_handlers.go) switches to it; the two genuine state-conflict sites keep 0099. BREAKING CHANGE: the transaction lock-contention response now carries error code 0486 instead of 0099 (HTTP status stays 409). Clients matching on 0099 for that path must handle 0486. Done in the v4 breaking window.
The alias->instrument rename already moved the span attributes to
app.request.instrument_id; this finishes the cleanup:
- drop the dead "alias_id" entry from constant.UUIDPathParameters (no route
registers it; the instrument handlers read the "instrument_id" path key)
- rename leftover aliasID locals/params to instrumentID across the instrument
handlers, CRM services, and Mongo adapters (they all denote the instrument
UUID, never an account alias)
No wire impact: span attribute keys were already instrument_id; this only
removes stale config and corrects identifier names.
…O (L8)
The POST /estimates response embedded FeeCalculate — the mutable fee-engine
carrier built on the transaction *input* shape (TransactionInput), carrying the
deprecated `route` field. Introduce an output-only FeeEstimateResult +
FeeAdjustedTransaction projection and serialize that instead. The engine keeps
FeeCalculate as its in-place-mutated carrier; the wire no longer sees it.
BREAKING CHANGE: the `feesApplied` object in POST /v1/organizations/{id}/estimates
now serializes as FeeEstimateResult/FeeAdjustedTransaction instead of FeeCalculate,
and the deprecated `route` field is dropped from the fee-estimate transaction
(use `routeId`). Computed fee values and HTTP status are unchanged.
…ggle Replace the ledger→tracer reservation call's unauthenticated/M2M-token seam with a gRPC+mTLS transport, kept behind a TRACER_TRANSPORT=grpc|rest toggle (default rest for safe rollout). Single-tenant; trusted x-tenant-id propagation lands in Phase 2. - proto/reservation/v1: reservation contract mirroring the Go wire types field-for-field (amount as decimal-as-string), buf codegen via pinned `go run`, make proto/proto-check, check-proto CI job. - tracer: gRPC ReservationService server delegating to the same use-case core the REST handler uses (no logic fork); mTLS on both transports via a single shared buildSeamTLSConfig (RequireAndVerifyClientCert). - ledger: gRPC client behind the TracerReserver port + transport selection; client-side mTLS via buildSeamClientTLSConfig; boot-guard reworked from the MT-refusal block to an mTLS-material assertion. Removed the dead M2MTokenProvider/Bearer seam, preserving the x-tenant-id injection hook. Cert material loads through lib-commons certificate.Manager for rotation. Fixes a latent last-WithTransportCredentials-wins bug that would have dialed plaintext under grpc+mtls. mesh mode dials plaintext for sidecar-terminated mTLS.
…tion seam Phase 2 of the ledger↔tracer seam: multi-tenant reservations over the mTLS/mesh-verified transport. The ledger forwards the request's tenant id to the tracer on every reservation RPC; the tracer resolves the per-tenant PostgreSQL pool from that TRUSTED key — the verified mTLS peer is the identity, so the forwarded x-tenant-id needs no JWT claim behind it. Ledger (client side): - REST: populate the injectTenant hook to set X-Tenant-Id from tmcore.GetTenantIDContext; omit the header entirely in single-tenant mode. - gRPC: a single client interceptor appends the tenant to outgoing metadata for all five reservation RPCs (no per-method copy-paste). The metadata key is derived from the REST header (strings.ToLower(TenantHeader)) so the wire key cannot drift across transports. - The tenant value is never logged on either transport. Tracer (server side): - A shared seamtenant.Resolver binds the per-tenant pool via the lib-commons tenant manager (tmcore.ContextWithPG/ContextWithTenantID), used by both a REST middleware and a gRPC interceptor wired ONLY onto the reservation surface. User-facing routes keep their JWT-claim tenant path: the shared JWT middleware is skipped on /v1/reservations* (Fiber Group.Use matches by prefix, so a separate group would not isolate it) — without the skip the mTLS-authenticated seam would be 401'd for lacking a Bearer token. - Fail-clean invariant: under multi-tenant mode a missing/empty/invalid tenant key returns ErrReservationTenantRequired (gRPC InvalidArgument / REST 400) and NEVER falls back to a default pool. Single-tenant mode is a no-op. - MetadataKey derives from HeaderName via strings.ToLower, mirroring the ledger. Boot-guard unchanged: it discriminates solely on transport security (mtls + missing material fails fast), never on tenancy. New sentinel: ErrReservationTenantRequired (0487), mapped as a ValidationError. Tests: resolver branches (DI pool seam), gRPC interceptor + REST middleware fail-clean cases, wire-key equality locks on both transports, and an integration test proving tenant A's reservation is invisible to tenant B over a real loopback mTLS+gRPC handshake with distinct per-tenant pools.
gRPC is the seam's production transport; REST is retained as a fallback, selectable with TRACER_TRANSPORT=rest. The empty default now resolves to gRPC instead of REST. The tracer's REST surface on :4020 is an operations/configuration surface (rules, limits, validations — operator-facing, internal), so there is no direct end-customer access that a uniform mTLS posture would demote; gRPC on its own seam port is the clean production target. Operator note: a deploy that wires the tracer (TRACER_BASE_URL set) without overriding TRACER_TRANSPORT now speaks gRPC, so the tracer must expose its gRPC seam via TRACER_GRPC_PORT and, under TRACER_TLS_MODE=mtls, both ends need cert material. Set TRACER_TRANSPORT=rest to keep the previous behavior. gRPC+mTLS has not yet been soaked end-to-end in a live cluster; the default is set to the target transport ahead of that soak by deliberate decision.
Add docs/architecture/ledger-tracer-reporter-topology.md — the single canonical reference for how the ledger, tracer, and reporter products relate at deploy time. Helm/k8s manifests stay infra-owned and out of this repo by design. Covers the seven topology contracts, each grounded in code/config (or labeled as an infra-owned recommendation where no repo artifact expresses it): product segregation matrix, co-scheduling (separate Deployments + soft podAffinity, rejecting sidecar/same-pod with the per-tenant PG-pool fan-out reason), independent scaling (hot-path reserve vs post-commit confirm/release), graceful absence (TRACER_BASE_URL nil-reserver + failPosture), the mTLS model (mtls|mesh, cert rotation, trusted x-tenant-id rationale), transport/ports (gRPC default + REST fallback, TRACER_GRPC_PORT), and the reporter async RabbitMQ consumer. Includes a documented gap: neither .env.example yet surfaces the new seam env vars (they exist only as Go struct tags) — flagged for the Phase 5 docs sweep.
components/crm was a ledger feature living under the components/ product-boundary namespace. Move it to components/ledger/internal/crm (git mv, history preserved) so the namespace means "separately-buildable product" again. Pure move: import paths only, zero behavior change. CI analysis filters roll CRM up under components/ledger; living docs deferred to the P5 doc sweep.
Phase 5 (final) of the ledger<->tracer seam + CRM consolidation campaign.
Two thrusts across 11 living docs:
1. Stale-path sweep: components/crm -> components/ledger/internal/crm
across llms.txt, llms-full.txt, STRUCTURE.md, AGENTS.md, CLAUDE.md,
docs/PROJECT_RULES.md (surgical path strings only), and
docs/standards/{telemetry,error-handling}.md. Dated docs/plans/* left
untouched.
2. Surface the reservation-seam env vars (mirroring config.go doc-comment
semantics): the 7 CLIENT vars in components/ledger/.env.example
(TRACER_BASE_URL opt-in, TRACER_TIMEOUT_MS, TRACER_TRANSPORT grpc|rest,
TRACER_TLS_MODE mtls|mesh, TRACER_TLS_{CERT,KEY,CA}_FILE) and the 5
SERVER vars in components/tracer/.env.example (TRACER_GRPC_PORT distinct
from :4020, TRACER_TLS_MODE, TRACER_TLS_{CERT,KEY}_FILE,
TRACER_TLS_CLIENT_CA_FILE verifying the ledger client cert).
README.md + llms-full.txt + llms.txt gain seam-reality prose (gRPC-default
transport toggle, mutual-TLS-or-mesh identity, trusted x-tenant-id
propagation) cross-linked to docs/architecture/ledger-tracer-reporter-topology.md.
Apply standard Go formatting (gofmt and goimports) across the ledger, tracer, and reporter components to maintain codebase consistency. Specific changes: - Sort imports alphabetically in various tests and configuration files. - Group constant declarations in reservation_server_test.go. - Remove an obsolete nolint:bodyclose comment in the tracer client. - Fix struct tag alignment spacing in the reporter model. - Format multi-line struct definitions in integration tests. These changes are necessary to adhere to strict Go styling guidelines, improving overall code readability and easing automated linting checks.
Refactor component Makefiles (infra, ledger, reporter, tracer) to use shared templates from the mk/ directory to keep build logic DRY. Remove duplicate or destructive targets (e.g., migrate-down-all, update-deps) and unneeded component-local makefile_colors.mk files. Add root CI gate targets (make ci, make ci-tests) to standardize pipeline workflow. Update shell scripts (check-logs.sh, run-tests.sh) to dynamically discover Go components via globbing, removing hardcoded references to deprecated directories (onboarding, transaction). Add executable permissions to utility scripts. Pin tooling versions (golangci-lint, gosec, migrate) to ensure deterministic CI builds.
… API fix(reporter): map env names to valid zap environments to pass validation
Add `@redocly/cli` step in the doc generation scripts to join the isolated OpenAPI specs from ledger, tracer, and reporter into a single consolidated definition (`midaz.openapi.json` and `.yaml`). This improves API discoverability and provides a unified view of the Midaz platform. Additionally: - Add a custom `redocly.yaml` ruleset to lint the consolidated spec. - Add Swagger annotation for the `Items` field in the `Pagination` model to clarify its usage and override behavior. - Update the Postman collection to consistently enable and correctly order the `Authorization` header across all operations.
Epic 4.2 — idempotent CreateHolder/CreateInstrument:
- Narrow CRM-local IdempotencyRepo port over the shared transaction Redis
infrastructure (SetNX/Get/Set); the transaction-typed idempotency
use-case methods and the money path stay untouched.
- Standard X-Idempotency-Key header; CRM-namespaced keys
(idempotency:crm:{holder,instrument}:...); an empty key falls back to a
request-body SHA-256 hash. A replay returns the original entity with
X-Idempotency-Replayed:true; an in-flight slot returns 409.
- Claim/replay/store orchestrated in the handler layer via a shared
helper; use-case signatures unchanged. A nil repo disables the feature
as a passthrough.
Epic 4.3 — instrument-create referential validation:
- CreateInstrument verifies the body-supplied ledgerId/accountId exist
within the request organization before persisting, via a narrow
LedgerAccountReader port implemented over the ledger query use case
(CRM never imports the query package).
- Missing references return 422 UnprocessableOperation (new sentinels
0488/0489), not the query layer's 404; malformed UUIDs return 400.
contextcheck reports one violation per analysis pass, so the eight //nolint:contextcheck directives in internal/bootstrap — each suppressing a documented false positive, since the boot context must not be threaded into long-lived routes, workers, circuit breakers, and scrape closures — left seven appearing unused to nolintlint at any given build-graph state, failing the lint gate non-deterministically. Exclude contextcheck for the bootstrap package (it has no true positives to offer in a composition root) and drop the now-redundant directives.
…d emit events for create/update/delete actions ✨
…tity and emit events for create/update/delete actions ✨
…date streaming configuration ✨
…ce event emission logic 🔨
feat(fees): add lib-streaming events to fees flow
Reconcile the 35 incoming develop commits with the monorepo-consolidation branch (PR #2159). Resolved 31 conflicts + adapted develop features to the consolidated v4 architecture: - go.mod: kept v4 module, unioned deps (added redpanda/kadm from develop, kept higher tink-go v2.7.0 / franz-go v1.21.3); go mod tidy. - Removed the dead standalone components/crm/* (folded into ledger) and the dead swaggo api artifacts (branch is Huma-generated); kept deletions. - CRM streaming: adapted develop's alias.* events to the branch's renamed instrument.* domain (instrument_{created,updated,deleted,related_party_deleted}), kept holder.* events, wired EmitImportant helpers into the consolidated CRM services, registered the 7 CRM defs in the catalog, injected the emitter via DI. - block/unblock transactions: migrated develop's inline-Fiber routes to the branch's Huma dual-registration pattern (contract + runtime), regenerated specs. - v3 -> v4 import fixup across all incoming develop files. - Fixed a pre-existing branch build break in streaming_integration_test.go (fees billing-package event constructors now take the 9-arg signature). Validation: go build/vet (incl. -tags=integration), lint (0 issues), unit (14113 tests, 0 failures), integration (all packages green). Gate 8 review (streaming + code + nil-safety): PASS, 0 issues. E2E deferred as a tracked follow-up (end-to-end suite is pre-consolidation topology; separate task once it is updated for CRM-on-:3002).
Add StreamingEnabled and SASL config fields with safe off-by-default values, plus a self-contained BuildStreamingEmitter mirroring the ledger bootstrap. tracerEventDefinitions() is empty for now, and three NoopEmitter fallbacks (disabled, empty brokers, empty catalog) keep the disabled path a no-op with zero behavior change. Documents the streaming block in .env.example. No events are emitted yet. X-Lerian-Ref: 0x1
Build the lib-streaming emitter once in bootstrap and hold it on Service, and register a conditional SIGTERM producer-drain runnable (registered only when streaming is enabled and a close hook exists; close-once and nil-safe). Also make the CloudEvents source config-driven via STREAMING_CLOUDEVENTS_SOURCE, defaulting to lerian.midaz.tracer. No events are emitted yet; the disabled path stays a NoopEmitter with zero behavior change. X-Lerian-Ref: 0x1
Add a nil-safe Streaming (libStreaming.Emitter) field to all 12 rule and limit command structs and set it post-construction from the shared bootstrap emitter via initRuleService and initLimitService. Constructors are unchanged, so the field defaults to nil and existing tests are unaffected. No events are emitted yet; this only establishes the injection seam for a later emission epic. X-Lerian-Ref: 0x1
Add the six Rule event wire contracts (rule.created/updated/activated/deactivated/drafted/deleted) plus a shared RuleScopePayload, each following the 4-part events pattern (Definition, Payload typed independently of the domain, constructor, ToEmitRequest). The payload fence keeps name, description, expression, and the compiled program off the wire, locked by JSONShape tests that populate those fields on purpose. Events are not registered in the catalog or emitted yet. X-Lerian-Ref: 0x1
Populate tracerEventDefinitions() with the six Rule lifecycle event definitions (the single registration point feeding both the catalog and routes) and add a bijection test locking definitions to routes with no ghost topics. When streaming is enabled without brokers, degrade to a NoopEmitter (Warn) instead of failing startup; other load-config errors still propagate. Replace the obsolete empty-catalog test locks with coverage assertions. X-Lerian-Ref: 0x1
Emit the six rule.* lib-streaming events (created, updated, activated, deactivated, drafted, deleted) after the transaction commits, via private emit<Event>Event helpers routed through EmitImportant (IMPORTANT posture: nil-safe, never fails the request; idempotent no-op transitions emit nothing). Add a clock to DeleteRuleService so rule.deleted sources its timestamp deterministically. X-Lerian-Ref: 0x1
Add the six Limit event wire contracts (limit.created/updated/activated/deactivated/drafted/deleted) following the 4-part events pattern, reusing the shared scope payload and adding a TimeOfDay formatter. The fence keeps name, description, and the monetary maxAmount off the wire (JSONShape tests assert their absence). Register all six in tracerEventDefinitions() and extend the catalog/routes bijection test to lock all twelve rule and limit events. X-Lerian-Ref: 0x1
Emit the six limit.* lib-streaming events (created, updated, activated, deactivated, drafted, deleted) after the transaction commits, via private emit<Event>Event helpers routed through EmitImportant (IMPORTANT posture: nil-safe, never fails the request; idempotent and no-change paths emit nothing). The delete path reuses the command's existing clock for the deleted timestamp. Completes tracer emission for all twelve rule and limit events. X-Lerian-Ref: 0x1
Add a one-shot midaz-redpanda-init service to the infra compose that pre-provisions the full platform topic set (ledger/CRM/fees from midazEventDefinitions() + the 12 tracer rule/limit topics from tracerEventDefinitions()) idempotently. Add a build-tagged (integration) TestStreamingSmoke that emits all 12 tracer lifecycle events through the real emitter, consumes them back from Redpanda, and asserts the CloudEvents headers (ce-type/ce-subject/ce-tenantid) plus the payload fence. The consume budget counts only this run's tenant so the smoke is re-runnable against a reused broker. Promote franz-go/pkg/kadm to a direct dependency. X-Lerian-Ref: 0x1
X-Lerian-Ref: 0x1
Group the internal events import separately, guard BuildStreamingEmitter with an early ctx.Err() check before broker-bound work, trim call-graph narration from the limit delete doc comments, and drop the CRM architecture comparison from the event catalog doc. X-Lerian-Ref: 0x1
Absorve os 3 chore(deps) de origin/develop (2026-07-03): - lib-auth v2.8.1->v2.9.0 (security hotfix: isolamento authz cross-product) - lib-streaming v1.6.2->v1.7.0 (aditivo; franz-go v1.21.3) - clear .lerianstudiolibignore pins Resolucao: go.mod/go.sum via nosso superset + tidy (mantem tink-go v2.7.0 do CRM KMS, kmsg sobre kadm); pkg/streaming/tenant.go mantem o nosso (superset com guard ctx==nil). build ./... + go test ./pkg/streaming/... verdes. Claude-Session: https://claude.ai/code/session_018JBQfQo2BrEhswe3npMqv8
Resolves the two GoFiber MEDIUM findings blocking the required 'validate / Security' check on PR #2159 (both pre-existing, not introduced by the develop re-sync): - CVE-2026-44332 (username enumeration via BasicAuth timing oracle): bump gofiber/fiber/v3 v3.2.0 -> v3.3.0 (indirect, via humafiber). Real upstream patch. - CVE-2026-45045 (X-Real-IP spoofing via Header.Add in proxy BalancerForward): affects gofiber/fiber/v2 v2.52.13, the HTTP runtime. No upstream fix exists (v2.52.13 is the latest v2; fix only in v3). midaz does not use fiber's proxy/BalancerForward middleware, so the vulnerable path is unreachable. Suppressed in .trivyignore with a documented, dated (exp:2027-01-05) justification. Verified: go build ./... green; trivy fs confirms 44332 gone and 45045 suppressed (0 findings) with the ignore file, still flagged without it. Claude-Session: https://claude.ai/code/session_018JBQfQo2BrEhswe3npMqv8
feat(tracer): emit rule and limit lifecycle events via lib-streaming
Direct pushes to develop bypass the PR huma golden-file gate. This push-triggered workflow regenerates the ledger and tracer Huma OAS 3.1 dumps from code, verifies the cross-plane spec locks, and pushes a [skip ci] fix-up commit when a direct push left the committed specs stale. Auth: MIDAZ push-bot App token for the protected-branch push; private modules via MANAGE_TOKEN; commit GPG-signed as the CI CD user. Claude-Session: https://claude.ai/code/session_013kJ9BuhVsVJXVBivdKLNt2
The tier-2 direct-push fix-up regenerated only the two components/*/api Huma dumps, but the required check-docs gate diffs those dumps AND their consolidated postman/specs hub (drift_check: git diff --exit-code -- 'components/*/api' postman/specs). A direct push changing a ledger/tracer route drifts both, yet the workflow committed only the dumps — leaving postman/specs stale, so it did NOT reproduce what the gate catches. Reuse `make generate-docs` (the same generate-docs.sh the gate runs under CHECK_DOCS_REGEN=1) in place of two hand-rolled `go test -update` steps, so the workflow regenerates the full gate-enforced set and divergence is impossible by construction. Expand the fix-up pathspec to the gate's verbatim 'components/*/api' postman/specs. Toolchain stays Go + Node/npm (no swag, no Docker); header comment corrected to match. Claude-Session: https://claude.ai/code/session_013kJ9BuhVsVJXVBivdKLNt2
…aths Wave-H security-review Mediums, supervisor fixes: - The App token was minted as the first step with the App's full installation permission set, live on the runner while the regen steps execute third-party Go/Node dependency code. Now minted immediately before the push step and scoped with permission-contents: write. - on.push.paths omitted pkg/**, so a dump-affecting change flowing only through shared pkg/ code (RFC 9457 Error schema) never triggered the fix-up — the same reason release.yml declares pkg/ as a shared_path. Claude-Session: https://claude.ai/code/session_013kJ9BuhVsVJXVBivdKLNt2
Second re-sync of the consolidation branch with develop (76 commits since merge-base 1029ff0). develop's improvements adopted on top of the consolidated structure; the branch's spirit (folded CRM/fees/tracer, v4 module, unified ledger binary) preserved. Resolution decisions: - Service Discovery (new develop subsystem): adopted. libsd + pkg/servicediscovery wired into the ledger bootstrap; our tracer teardown grafted into develop's refactored launcherApps() helper. CRM-standalone SD wiring dropped (CRM is folded, sharing the ledger process identity). - balance.changed streaming, external-account create, asset-delete external cascade: taken from develop. - CRM PII-safe span attributes: adopted develop's repositoryInputAttributes() helper (DRY, tested) over our inline booleans; instrument index_tracker package fixed from the alias-path merge artifact. - Standalone components/crm service: kept DELETED; develop's index-once fix landed via the folded internal/crm path. - v3->v4 module sweep across all .go (develop is still on v3). - go.mod: lib-commons v5.9.0, lib-streaming v1.8.0, lib-service-discovery v0.6.0, toolchain go1.26.5; golangci gate kept at our v2.12.2 + Go 1.26.5. - Structured logging (T6) kept over develop's fmt.Sprintf-in-logger; overdraft "not enabled" log kept at Debug (T7, per-transaction path) with develop's extracted env-name constant. - composition Type now ACCEPTS external, mirroring develop's CreateAccountInput (enables creating distinct external accounts for the br-spb rail). Dropped the invalidstrings=external block and the now-orphaned validateInvalidStrings validator (validator + registration + translation), aligning with develop. Gates: go build ./... clean; make lint 0 issues; make test-unit 14366 pass / 6 skip / 0 fail. Claude-Session: https://claude.ai/code/session_01Y5b5NhVjTez11DHGCEfJVk
Unblocks PR #2159 CI: - Tracer Dockerfile builder was golang:1.26.4-alpine, so the shipped binary stamped stdlib 1.26.4 and Trivy flagged CVE-2026-39822 (HIGH, os.Root symlink-following directory traversal), fixed in Go 1.26.5. Ledger already builds on 1.26.5; align tracer to match. - lib-commons/v5 v5.9.0 -> v5.10.0 to satisfy the Lerian Library Version Check gate. Build (all packages) and unit tests green. Claude-Session: https://claude.ai/code/session_01TD7yfr219zoX4LQAVtbAPd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Consolidates tracer, plugin-fees, and CRM into the midaz v4 monorepo — unified
components/ledgerbinary (onboarding + transaction + CRM + fees on :3002) plus co-locatedcomponents/tracer(:4020). Single rootgo.mod(github.com/LerianStudio/midaz/v4), nogo.work.Supersedes the closed #2156 / #2154. The load-bearing difference from those PRs:
⭐ The reporter is NO LONGER part of this consolidation
Earlier attempts folded the reporter in too. That decision was reversed — the reporter is a separately-sellable product coupled to the ledger only over the wire (RabbitMQ + API), not in-process, so it belongs in its own repo. It has been extracted back to
LerianStudio/reporter(PR #696) and removed from this monorepo (626 files).A direct consequence: the private
fetcher/pkg/enginedependency is gone from midaz'sgo.mod— restoring a clean source-available build for external clones (the ledger core never imported it, but a single root module made it a build-time dep for everyone).Scope
pkg/fee, shared typespkg/feeshared, use casesinternal/services/fees; applied at thetransaction_create.gofee seam.components/ledger/internal/crm(package tree, nocmd//internal/); routes register under themidazauthz namespace — the tenant-manager policy migration is the X1 release gate (docs/auth/RBAC-NAMESPACES.md,docs/runbooks/v4-x1-rbac-migration-and-rollback.md).components/tracer; ledger↔tracer seam over gRPC + mTLS (docs/architecture/ledger-tracer-topology.md).Verification
go build ./...+go vet ./...→ exit 0 (post-extraction).https://claude.ai/code/session_01RzpM5cJt1wAqEZQ1mHL63n