Skip to content

Commit c238852

Browse files
alex-rawlings-yycalex-rawlings-yyc
authored
Add USJ book parsing pipeline and interlinearizer display WebView (#30)
Implements an end-to-end interlinearizer feature backed by a new USJ parsing pipeline. Key additions: **Parsers & Data Model** - `usjBookExtractor` and `bookTokenizer` transform Platform.Bible USJ books into a Book/Segment/Token interlinear model - Correct handling of chapter/verse traversal, surrogate pairs, combining marks, astral-plane characters, and word-final apostrophe glottal markers - Heading/title/spacing para markers are skipped during verse text accumulation - `stableStringify` with deterministic code-point sorting powers `fnv1a32`-based segment hashing (8-char padded output) - `parseStrictNumber()` with strict range/index validation (whole numbers only) for PT9 XML parser **WebView** - Replaces placeholder WebView with `SegmentView` rendering tokenized word and punctuation chips - `BookChapterControl` verse picker wired to scroll-group `scrRef`; renders full chapters at a time - `TabToolbar` added to WebView - Error UI displays labelled heading + `<pre>` block on tokenization failure; errors logged via `papi` logger in `useEffect` - `useMemo` deps narrowed to primitives; `projectId` and `scrRef.book` included **Tests & Mocks** - 100% parser and WebView coverage; coverage collection expanded to all source files - Corrected mock signatures for `useLocalizedStrings` (tuple) and `useRecentScriptureRefs` (object) - Jest mocks for `lucide-react` and `platform-bible-react` unblock ESM imports - Non-vacuous `every()` assertions; cross-segment token ID uniqueness; stable-ref object identity checks
1 parent 3340605 commit c238852

23 files changed

Lines changed: 1839 additions & 247 deletions

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ The general file structure for an extension is as follows:
9595
- `src/` contains the source code for the extension
9696
- `src/main.ts` is the main entry file for the extension (registers commands and wires interlinear XML)
9797
- `src/types/interlinearizer.d.ts` is this extension's types file that defines how other extensions can use this extension through the `papi`. It is copied into the build folder
98-
- `src/parsers/interlinearXmlParser.ts` parses interlinear XML into structured data (uses fast-xml-parser). The PT9 XML schema and parsed output are documented in `src/parsers/pt9-xml.md`
98+
- `src/parsers/pt9/` contains parser and schema for parsing PT9 interlinear XML into structured data
9999
- `*.web-view.tsx` files will be treated as React WebViews
100100
- `*.web-view.scss` files provide styles for WebViews
101101
- `*.web-view.html` files are a conventional way to provide HTML WebViews (no special functionality)

‎__mocks__/interlinearXmlContent.ts‎

Lines changed: 0 additions & 9 deletions
This file was deleted.

‎__mocks__/papi-backend.ts‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -51,14 +51,6 @@ module.exports = {
5151
__esModule: true,
5252
default: defaultExport,
5353
logger: mockLogger,
54-
__mockRegisterWebViewProvider: mockRegisterWebViewProvider,
55-
__mockRegisterCommand: mockRegisterCommand,
56-
__mockOpenWebView: mockOpenWebView,
57-
__mockSelectProject: mockSelectProject,
58-
__mockGetOpenWebViewDefinition: mockGetOpenWebViewDefinition,
59-
__mockOnDidOpenWebView: mockOnDidOpenWebView,
60-
__mockOnDidCloseWebView: mockOnDidCloseWebView,
61-
__mockLogger: mockLogger,
6254
};
6355

6456
/** Marks this file as a module so top-level const/let are module-scoped; avoids TS "redeclare" when both papi-backend and papi-frontend mocks are in the project (they are used mutually exclusively by Jest). */

‎__mocks__/papi-frontend-react.ts‎

Lines changed: 50 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,70 @@
11
/**
2-
* @file Jest mock for @papi/frontend/react. Provides stub implementations of various PAPI React hooks so
2+
* @file Jest mock for @papi/frontend/react. Provides stub implementations of PAPI React hooks so
33
* WebView/frontend components can be unit-tested without the real Platform API.
44
*/
55

66
/**
7-
* useData('providerName') returns an object whose keys are data type names and values are hooks.
8-
* Mock: any property returns a function that returns [undefined, setter, false].
7+
* Mock for `useProjectData`. Returns a `Proxy` whose property accesses each yield a function
8+
* returning `[undefined, jest.fn(), false]`, matching the real hook's `[data, setter, isLoading]`
9+
* tuple without requiring a live data provider.
910
*/
10-
const createUseDataLikeHook = () =>
11-
jest.fn(() =>
12-
new Proxy(
13-
{},
14-
{
15-
get: () => () => [undefined, jest.fn(), false],
16-
},
17-
),
18-
);
11+
const useProjectData = jest.fn(() =>
12+
new Proxy(
13+
{},
14+
{
15+
get: () => () => [undefined, jest.fn(), false],
16+
},
17+
),
18+
);
1919

20-
const useDataProvider = jest.fn().mockReturnValue(undefined);
21-
const useData = createUseDataLikeHook();
22-
const useScrollGroupScrRef = jest.fn().mockReturnValue([undefined, jest.fn()]);
23-
const useSetting = jest.fn().mockImplementation((_key: string, defaultState: unknown) => [defaultState, jest.fn()]);
24-
const useProjectData = createUseDataLikeHook();
25-
const useProjectDataProvider = jest.fn().mockReturnValue(undefined);
20+
/**
21+
* Mock for `useProjectSetting`. Returns `[defaultState, setSetting, resetSetting, isLoading]`,
22+
* passing `defaultState` through unchanged so callers receive a predictable initial value.
23+
*
24+
* @param _projectDataProviderSource - Ignored project data provider source.
25+
* @param _key - Ignored setting key.
26+
* @param defaultState - Value surfaced as the current setting state.
27+
* @returns Tuple of `[defaultState, jest.fn(), jest.fn(), false]`.
28+
*/
2629
const useProjectSetting = jest
2730
.fn()
28-
.mockImplementation((_projectInterface: string, _projectIdOrPdp: unknown, _key: string, defaultState: unknown) => [
31+
.mockImplementation((_projectDataProviderSource: unknown, _key: string, defaultState: unknown) => [
2932
defaultState,
3033
jest.fn(),
34+
jest.fn(),
35+
false,
3136
]);
32-
const useDialogCallback = jest.fn().mockReturnValue(jest.fn());
33-
const useDataProviderMulti = jest.fn().mockReturnValue([]);
34-
/** Returns a map of localization key -> key (so tests get a string for each key). */
35-
const useLocalizedStrings = jest.fn().mockImplementation((keys: string[]) =>
36-
Array.isArray(keys) ? keys.reduce<Record<string, string>>((acc, k) => ({ ...acc, [k]: k }), {}) : {},
37-
);
38-
const useWebViewController = jest.fn().mockReturnValue(undefined);
39-
const useRecentScriptureRefs = jest.fn().mockReturnValue([]);
37+
38+
/**
39+
* Mock for `useLocalizedStrings`. Maps each requested key to itself so tests receive a
40+
* predictable `Record<string, string>` without a real localization service.
41+
*
42+
* @param keys - BCP47-style string keys to resolve.
43+
* @returns Tuple of `[record, isLoading]` where every key maps to itself and `isLoading` is
44+
* `false`.
45+
*/
46+
const useLocalizedStrings = jest.fn().mockImplementation((keys: string[]) => [
47+
Array.isArray(keys) ? keys.reduce<Record<string, string>>((acc, k) => { acc[k] = k; return acc; }, {}) : {},
48+
false,
49+
]);
50+
51+
/**
52+
* Mock for `useRecentScriptureRefs`. Returns an empty history and a no-op `addRecentScriptureRef`
53+
* so components that display recent references render without errors.
54+
*
55+
* @returns Object with `recentScriptureRefs` (empty array) and `addRecentScriptureRef` (jest spy).
56+
*/
57+
const useRecentScriptureRefs = jest
58+
.fn()
59+
.mockImplementation(() => ({ recentScriptureRefs: [], addRecentScriptureRef: jest.fn() }));
4060

4161
module.exports = {
4262
__esModule: true,
43-
useDataProvider,
44-
useData,
45-
useScrollGroupScrRef,
46-
useSetting,
4763
useProjectData,
48-
useProjectDataProvider,
4964
useProjectSetting,
50-
useDialogCallback,
51-
useDataProviderMulti,
5265
useLocalizedStrings,
53-
useWebViewController,
5466
useRecentScriptureRefs,
55-
__mockUseDataProvider: useDataProvider,
56-
__mockUseData: useData,
57-
__mockUseLocalizedStrings: useLocalizedStrings,
58-
__mockUseSetting: useSetting,
59-
__mockUseProjectData: useProjectData,
60-
__mockUseProjectDataProvider: useProjectDataProvider,
61-
__mockUseProjectSetting: useProjectSetting,
62-
__mockUseWebViewController: useWebViewController,
6367
};
68+
69+
/** Marks this file as a module so top-level const/let are module-scoped. */
70+
export {};

‎__mocks__/papi-frontend.ts‎

Lines changed: 3 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
/**
2-
* @file Jest mock for @papi/frontend. Provides papi, logger, network, projectDataProviders, and other
3-
* renderer API stubs so WebView/frontend code can be unit-tested without loading the real
4-
* Platform API.
2+
* @file Jest mock for @papi/frontend. Provides a logger stub so WebView/frontend code can be
3+
* unit-tested without loading the real Platform API.
54
*/
65

76
const mockLogger = {
@@ -11,50 +10,10 @@ const mockLogger = {
1110
warn: jest.fn(),
1211
};
1312

14-
const mockNetwork = {
15-
request: jest.fn(),
16-
subscribe: jest.fn().mockReturnValue({ dispose: jest.fn() }),
17-
};
18-
19-
const mockProjectDataProviders = {
20-
get: jest.fn().mockResolvedValue(undefined),
21-
register: jest.fn().mockResolvedValue({ dispose: jest.fn() }),
22-
};
23-
24-
const mockWebViews = {
25-
getWebView: jest.fn(),
26-
openWebView: jest.fn().mockResolvedValue(undefined),
27-
};
28-
29-
/** Default papi object shape used in renderer/WebViews. Only commonly used services are stubbed. */
30-
const papi = {
31-
logger: mockLogger,
32-
network: mockNetwork,
33-
projectDataProviders: mockProjectDataProviders,
34-
webViews: mockWebViews,
35-
react: {}, // Re-export of @papi/frontend/react; tests usually import that module directly.
36-
};
37-
38-
const defaultExport = {
39-
...papi,
40-
__mockLogger: mockLogger,
41-
__mockNetwork: mockNetwork,
42-
__mockProjectDataProviders: mockProjectDataProviders,
43-
__mockWebViews: mockWebViews,
44-
};
45-
4613
module.exports = {
4714
__esModule: true,
48-
default: defaultExport,
4915
logger: mockLogger,
50-
network: mockNetwork,
51-
projectDataProviders: mockProjectDataProviders,
52-
webViews: mockWebViews,
53-
__mockLogger: mockLogger,
54-
__mockNetwork: mockNetwork,
55-
__mockProjectDataProviders: mockProjectDataProviders,
56-
__mockWebViews: mockWebViews,
5716
};
5817

59-
/** Marks this file as a module so top-level const/let are module-scoped; avoids TS "redeclare" when both papi-backend and papi-frontend mocks are in the project (they are used mutually exclusively by Jest). */
18+
/** Marks this file as a module so top-level const/let are module-scoped. */
6019
export {};

‎__mocks__/platform-bible-react.tsx‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
/**
2+
* @file Jest mock for platform-bible-react. The real package ships ESM which Jest cannot parse
3+
* without extra transform configuration. This stub provides the subset used by extension
4+
* components: `BookChapterControl`, `BOOK_CHAPTER_CONTROL_STRING_KEYS`, `TabToolbar`, and
5+
* `ScrollGroupSelector`.
6+
*/
7+
8+
import type { ReactElement, ReactNode } from 'react';
9+
10+
interface SerializedVerseRef {
11+
book: string;
12+
chapterNum: number;
13+
verseNum: number;
14+
verse?: string;
15+
versificationStr?: string;
16+
}
17+
18+
export const BOOK_CHAPTER_CONTROL_STRING_KEYS: string[] = [];
19+
20+
export function TabToolbar({
21+
startAreaChildren,
22+
endAreaChildren,
23+
}: Readonly<{
24+
className?: string;
25+
startAreaChildren?: ReactNode;
26+
endAreaChildren?: ReactNode;
27+
onSelectProjectMenuItem?: () => void;
28+
onSelectViewInfoMenuItem?: () => void;
29+
}>): ReactElement {
30+
return (
31+
<div data-testid="tab-toolbar">
32+
<div data-testid="tab-toolbar-start">{startAreaChildren}</div>
33+
<div data-testid="tab-toolbar-end">{endAreaChildren}</div>
34+
</div>
35+
);
36+
}
37+
38+
export function ScrollGroupSelector({
39+
availableScrollGroupIds,
40+
scrollGroupId,
41+
onChangeScrollGroupId,
42+
}: Readonly<{
43+
availableScrollGroupIds?: (number | undefined)[];
44+
scrollGroupId?: number;
45+
onChangeScrollGroupId?: (id: number | undefined) => void;
46+
}>): ReactElement {
47+
return (
48+
<select
49+
data-testid="scroll-group-selector"
50+
value={scrollGroupId ?? ''}
51+
onChange={(e) => onChangeScrollGroupId?.(e.target.value === '' ? undefined : Number(e.target.value))}
52+
>
53+
<option value="">—</option>
54+
{availableScrollGroupIds?.map((id) => (
55+
<option key={id ?? 'undefined'} value={id ?? ''}>
56+
{id ?? '—'}
57+
</option>
58+
))}
59+
</select>
60+
);
61+
}
62+
63+
export function BookChapterControl({
64+
scrRef,
65+
handleSubmit,
66+
onAddRecentSearch,
67+
}: Readonly<{
68+
scrRef: SerializedVerseRef;
69+
handleSubmit: (ref: SerializedVerseRef) => void;
70+
className?: string;
71+
localizedStrings?: Record<string, string>;
72+
recentSearches?: SerializedVerseRef[];
73+
onAddRecentSearch?: (scrRef: SerializedVerseRef) => void;
74+
id?: string;
75+
}>): ReactElement {
76+
return (
77+
<div data-testid="book-chapter-control">
78+
{scrRef.book} {scrRef.chapterNum}:{scrRef.verseNum}
79+
<button type="button" onClick={() => {handleSubmit(scrRef); onAddRecentSearch?.(scrRef);}}>
80+
Submit reference
81+
</button>
82+
</div>
83+
);
84+
}

‎__mocks__/platform-bible-utils.ts‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ class UnsubscriberAsyncList {
3636
(unsubscriber as { dispose: UnsubscriberFn }).dispose.bind(unsubscriber)
3737
);
3838
} else if (typeof unsubscriber === 'function') {
39-
this.unsubscribers.add(unsubscriber as UnsubscriberFn);
39+
this.unsubscribers.add(unsubscriber);
4040
}
4141
});
4242
}
@@ -53,13 +53,16 @@ class UnsubscriberAsyncList {
5353
}
5454
}
5555

56-
/** Minimal PlatformError shape matching the real platform-bible-utils type. */
56+
/**
57+
* Minimal PlatformError shape matching the real platform-bible-utils type. Uses `platformErrorVersion`
58+
* as the discriminant — the same field the real `isPlatformError` checks.
59+
*/
5760
interface PlatformError {
5861
message: string;
59-
isPlatformError: true;
62+
platformErrorVersion: number;
6063
}
6164

6265
const isPlatformError = (value: unknown): value is PlatformError =>
63-
typeof value === 'object' && value !== null && (value as PlatformError).isPlatformError === true;
66+
typeof value === 'object' && value !== null && 'platformErrorVersion' in (value);
6467

6568
export { UnsubscriberAsyncList, isPlatformError };

‎cspell.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,13 @@
2222
"eflomal",
2323
"electronmon",
2424
"endregion",
25+
"eten",
2526
"finalizer",
2627
"Fragmenter",
2728
"guids",
2829
"hopkinson",
2930
"iframes",
31+
"imte",
3032
"interlinearization",
3133
"interlinearizer",
3234
"localstorage",
@@ -44,6 +46,8 @@
4446
"pdps",
4547
"plusplus",
4648
"proxied",
49+
"Punct",
50+
"recalc",
4751
"reinitializing",
4852
"reserialized",
4953
"scriptio",

‎jest.config.ts‎

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,10 @@ const config: Config = {
2323
*/
2424
collectCoverage: false,
2525

26-
/**
27-
* Collect coverage from parsers, main entry (main.ts), and WebView UI
28-
* (interlinearizer.web-view.tsx). Excludes test files, type declarations, and build artifacts.
29-
*/
26+
/** Collect coverage from all source files. Excludes type declarations and test files. */
3027
collectCoverageFrom: [
31-
'src/parsers/**/*.ts',
32-
'src/main.ts',
33-
'src/**/*.web-view.tsx',
34-
'!src/parsers/**/*.d.ts',
28+
'src/**/*.{ts,tsx}',
29+
'!src/**/*.d.ts',
3530
'!src/**/__tests__/**',
3631
'!src/**/*.test.{ts,tsx}',
3732
'!src/**/*.spec.{ts,tsx}',
@@ -92,12 +87,12 @@ const config: Config = {
9287
'^@papi/frontend/react$': '<rootDir>/__mocks__/papi-frontend-react.ts',
9388
/** Mock so test-helpers get UnsubscriberAsyncList without loading ESM deps. */
9489
'^platform-bible-utils$': '<rootDir>/__mocks__/platform-bible-utils.ts',
90+
/** Mock ESM deps that Jest cannot parse. */
91+
'^platform-bible-react$': '<rootDir>/__mocks__/platform-bible-react.tsx',
9592
/** Resolve webpack ?inline imports. */
9693
'^(.+)\\.web-view\\?inline$': '<rootDir>/__mocks__/web-view-inline.ts',
9794
/** Resolve webpack ?inline imports: SCSS content. */
9895
'^(.+)\\.(scss|sass|css)\\?inline$': '<rootDir>/__mocks__/styleInlineMock.ts',
99-
/** Resolve webpack ?raw import for test XML in web-view. */
100-
'^(.+)/Interlinear_en_MAT\\.xml\\?raw$': '<rootDir>/__mocks__/interlinearXmlContent.ts',
10196
},
10297

10398
/** Exclude dist from module resolution to avoid Haste naming collision with root package.json. */

0 commit comments

Comments
 (0)