feat(pivot): fact pivot engine for dimensional XBRL rendering#7
Merged
Conversation
Rebuild the render path as a fact pivot engine. A section is an XBRL hypercube; every fact is a point in aspect space (concept, period, entity, unit, dimensions), a PivotConfig assigns each aspect a role (row / column / slicer), and cells are matched by full aspect signature instead of the collision-prone (element, period.end) key. Closes the dimensional-rendering gaps: segment/component breakdowns no longer overwrite the consolidated total, each cell owns its own period/unit metadata (no borrowing from a merged column), the over-aggressive period filter is replaced by a conservative metadata-safe prune, non-USD units render with their own symbol, and monetary values honor a per-section scale (with per-share amounts never rescaled). - pivot.ts: buildPivot / buildPivots / defaultPivotConfig / reportSections; hybrid presentation walk (abstract headers pre-order, concrete subtotals post-order) so totals render last for both us-gaap and rs-gaap shapes - model: DimensionQualifier on Fact, NumericKind, UnitInfo.symbol, pivot types - sec adapter: fetch FACT_HAS_DIMENSION (fast-path query); map dimensions, numericKind (item_type/is_shares), and unit currency symbol; drop the has_dimensions:false filters and filterToDominantPeriods - format: unit- and scale-aware formatValue; currencySymbolFor; numericKindOf - components: StatementTable renders the pivot (slicer bars, member columns with multi-level headers, [Roll Up] abstract rows, scale caption); FactInspector is an aspect panel reading each cell's own fact
A section's factset can hold facts from other tables — e.g. segment/geography revenue tagged to the income-statement role. defaultPivotConfig pivoted every axis any fact carried, turning the income statement into an empty segment x geography grid with stray Nature-of-Expense / Equity-Components slicers. Restrict the pivotable (and slicer) axes to those whose axis element appears in the section's presentation network — a statement owns only the axes its hypercube declares. Matched by local name so a company-namespaced axis lines up regardless of prefix. Facts carrying an undeclared axis are excluded as another table's breakdown; the equity statement (which does declare StatementEquityComponentsAxis) still pivots its components.
…e sections Abstract section headers (OperatingExpensesAbstract, EarningsPerShareAbstract, IncomeStatementAbstract, ...) carry no facts, so the facts query never registered them. The projection then fell back to a synthetic element with abstract=false, treated them as concrete, and emitted them post-order as empty "—" rows *below* their section instead of as headers above it. Carry element name + is_abstract on the association query and register any presentation-only element (fact-bearing elements are already registered and take precedence). Abstract labels drop the XBRL "Abstract" suffix (OperatingExpensesAbstract -> "Operating Expenses").
…ction) A dimensional statement's factset carries incidental context dates that are not reporting periods — a lone transaction date, a prior-year opening balance, a stub period — which leaked in as near-empty columns. The SEC/Arelle renderer selects reporting periods by type + recency (most-recent instants aligned to the fiscal year-end for a balance sheet; day-count-classified durations for income). Density approximates that here: a real comparative repeats most line items and clears the bar, while an incidental date is sparse. Raise the prune from the too-lax >=2 floor to >= max(2, 20% of the densest period). Confirmed against MARIMED's FY2024 balance sheet: 2024-12-31 (50) and 2023-12-31 (49) kept; 2022-12-31 (7) and 2023-03-09 (1) dropped. Metadata-safe.
…nk empty cells Two rendering improvements for dimensional statements: 1. Default dimensions to *rows* (nested under the concept) instead of columns. Statements are tall, not wide, so member breakdowns stack more legibly: a concept total row heads its indented member sub-rows (member-only labels, the concept row supplies the name). `pivotDimensionsOn(config, placement)` flips every dimension axis between rows and columns, and ReportView exposes it as a "Dimensions: as rows / as columns" toggle (shown only when the report has dimensions). buildPivots gains a per-section configFor transform to drive the toggle across the whole report. 2. Distinguish "no fact" from "a null/zero fact". A cell with no fact for its coordinate now renders blank; a fact that is genuinely null (Commitments & Contingencies) or $0 renders an em-dash. Previously both showed a dash. The domain-total combo keys by the element alone (its own total row, same as the undimensioned case); only member sub-rows carry the dimensional signature.
In dims-as-rows mode a concept's total row heads its indented member breakdown
and supplies the concept name. A concept reported only per-member — no
consolidated total, common in detail disclosures (a debt schedule's face
amount, interest rate, maturity per instrument) — had no heading, so its
members rendered as orphaned member-only labels ("CREM Loan · Mortgages" with
no indication of what is being measured). Emit a valueless concept heading
above the member sub-rows in that case.
With dimensions on rows, list the member breakdown first and the concept's own (memberless) total below and outdented, so the total reads as the sum of its members above it — the standard components-then-total layout. Flip the row domain-total ordering from first to last.
Render the rows/columns toggle inside StatementTable, right below the slicer bar, instead of at the top of ReportView. It shows only for dimensional sections; ReportView drives it via dimensionPlacement + onDimensionPlacementChange props (report-level state, so it stays consistent across a multi-statement report).
A percentItemType fact stores a decimal fraction (0.10 = 10%), so it must be scaled ×100 before the % suffix — otherwise a 10% interest rate rendered as 0.1% and a −107% effective tax rate rendered as (1.1%). formatValue now scales percent facts ×100. Also split pureItemType from percentItemType: a pure fact is a bare dimensionless ratio (2.5 debt-to-equity), not a percentage — it renders as a plain decimal with no % suffix. New NumericKind 'pure'; deriveNumericKind maps the two item types apart.
Sibling "DEBT, X (Details)" networks share one hypercube factset (every debt instrument), distinguished only by which members each network's presentation declares. Gating by axis alone let every instrument leak into every one of these disclosures — the DuQuoin State Bank section showed BNE Delaware, CREM Loan, etc. Also gate by member: when a section's presentation lists explicit members, a fact whose member (on any axis) is not declared belongs to a sibling network and is out of scope. Applied consistently in defaultPivotConfig (so an axis with a single in-scope member becomes a slicer, not a spurious breakdown) and in buildPivot (the fact filter).
Member ordering looked up the presentation index by the full member qname, but a company member reaches the renderer as a bare local name (mrmd:ProductSalesRetailMember → ProductSalesRetailMember), missing the prefixed presentation node and falling back to alphabetical (Other, Retail, Wholesale). Key the presentation index by local name — like the scope gating already does — and look members up the same way, so rows follow the filer's order (Retail, Wholesale, Other). us-gaap members were unaffected (prefixes already matched).
The member-combo union (and cross-product across axes) yields (period × member) columns that never carry a fact — e.g. an "Extinguishment Of Debt Payment Occurrence" member column empty for every other line item. Drop any column whose coordinate no fact occupies, mirroring the row behavior where only member rows that have facts are emitted.
XBRL durationItemType facts (debt maturities, lease terms) arrive as ISO-8601 strings in the text path. humanizeDuration parses them (P1Y6M → "1 year 6 months", P30D → "30 days") and returns null for non-durations, so a prose fact that happens to start with "P" (a company name) is untouched. StatementTable renders the humanized term right-aligned with the numbers — it reads as a value, not prose — and FactInspector humanizes its Value field too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebuilds the render path as a fact pivot engine. A report section is treated as an XBRL hypercube: every fact is a point in aspect space (concept · period · entity · unit · dimensions), a
PivotConfigassigns each aspect a role (row / column / slicer), and cells are matched by their full aspect signature rather than the old(element, period.end)key. This closes the dimensional-rendering gaps that made segment/equity/detail disclosures render incoherently, and adds a rows↔columns toggle, correct percentage/currency/duration formatting, and presentation-scoped member handling. Validated live against real SEC filings (NVDA, MARIMED) through the holon-viewer.Changes
New engine —
src/pivot.ts(766 lines)buildPivot/buildPivots/defaultPivotConfig/reportSections/pivotDimensionsOn. Cells are an exact-signature lookup on(element, rowCombo, periodEnd, colCombo)— no last-write-wins collision; every cell owns its own fact, so metadata (period/unit) is exact.pivotDimensionsOn; statements are tall, not wide.Model —
src/model.tsDimensionQualifieronFact;NumericKind(monetary/perShare/shares/percent/pure/integer/other);UnitInfo.symbol; the full pivot model (AspectKey,PivotConfig,PivotTable,PivotRow,ResolvedScale, …).SEC adapter —
src/adapters/sec.tsFACT_HAS_DIMENSION(fast-path query) and maps axis/member →DimensionQualifier; mapsitem_type/is_shares→numericKind; unit measure → currency symbol.has_dimensions:falsefilters and the oldfilterToDominantPeriods.Formatting —
src/format.tsformatValueis unit- and scale-aware (fixes non-USD$);percentscaled ×100 (a 10% rate no longer shows 0.1%; a −107% tax rate no longer shows −1.1%);pureratios kept as plain decimals; per-share/shares never rescaled.humanizeDuration(P10Y→ "10 years");currencySymbolFor;numericKindOf.Components
StatementTablerenders the pivot: slicer bars, multi-level period×member headers,[Roll Up]abstract rows, blank vs em-dash cells (no fact → blank; null/$0 fact → dash), a "Dimensions: as rows / as columns" toggle below the slicers, blank first-column header, an 18rem concept-column floor for wide-table horizontal scroll.FactInspectoris a Pesseract/Workiva-style aspect panel reading each cell's own fact (concept · period · entity · dimensions · unit · decimals) + live footing.ReportViewwiresbuildPivots+ the toggle, keyed by row key.project.tstrimmed to the shared helpers (presentationOrder,calcSubtotals,footCheckover aPivotTable,sliceReportSection).Tests
test/pivot.test.ts(new, 751 lines) — collision, member scoping, ordering, incomplete periods, empty columns, headings, rows/columns toggle.test/format.test.ts(new) — percent ×100, pure-as-decimal, non-USD symbol, scale, per-share, duration humanization.test/{project,sec,render}.test.tsmigrated from the legacybuildStatementsto the pivot engine.Testing
npm run test:all— green (this session): Prettier check, ESLint,tsc --noEmit, 55 Vitest tests pass, tsup ESM + DTS build succeeds. Also exercised end-to-end against the live SEC repository viarobosystems-holon-viewer(local-linked build) across NVDA and MARIMED filings — equity, income, balance sheet, segment, and multi-axis debt/mezzanine detail disclosures.Notes / Follow-ups
Element → Labelis multi-tenant-ambiguous, so there is no cheap report-scoped label lookup. The fix is upstream in the SEC ingestion pipeline (resolve + denormalize the label onto the report-scoped fact/association) — spec'd separately; the adapter change here is additive when it lands.STATEMENT TABLE/LINE ITEMS), real-SEC-section CI fixtures, and jsonld dimension-emission hooks (inert until the backend emits dimensions on the holon).