Skip to content

Commit 1dc017c

Browse files
committed
chore: type fix, review changes
1 parent 62fc0c3 commit 1dc017c

6 files changed

Lines changed: 58 additions & 51 deletions

File tree

docs/docs/04-typescript-api/02-computer-vision/PoseEstimationModule.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ detections[0].THUMB_TIP; // { x, y }
9696

9797
The `.pte` binary must expose a `forward` method (or per-input-size methods such as `forward_384`, `forward_512`, `forward_640` for multi-resolution models) with the following interface:
9898

99-
**Input:** one `float32` tensor of shape `[1, 3, H, W]` — a single RGB image, values in `[0, 1]` after optional per-channel normalization `(pixel − mean) / std`. H and W are read from the model's declared input shape at load time. The mean and std vectors are supplied via `preprocessorConfig.normMean` and `preprocessorConfig.normStd` on the [`PoseEstimationConfig`](../../06-api-reference/interfaces/PoseEstimationConfig.md) you pass to `fromCustomModel`; if omitted, the runtime feeds the resized image without normalization.
99+
**Input:** one `float32` tensor of shape `[1, 3, H, W]` — a single RGB image, values in `[0, 1]` after optional per-channel normalization `(pixel − mean) / std`. H and W are read from the model's declared input shape at load time. The mean and std vectors are supplied via `preprocessorConfig.normMean` and `preprocessorConfig.normStd` on the [`PoseEstimationConfig`](../../06-api-reference/type-aliases/PoseEstimationConfig.md) you pass to `fromCustomModel`; if omitted, the runtime feeds the resized image without normalization.
100100

101101
**Outputs:** exactly three `float32` tensors, in this order:
102102

packages/react-native-executorch/common/rnexecutorch/tests/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,5 +69,8 @@ To add new test you need to:
6969
LIBS opencv_deps
7070
)
7171
```
72-
* Lastly, add the test executable name to the run_tests script along with all the needed URL and assets.
72+
* In `run_tests.sh`:
73+
* Add the test executable name to `TEST_EXECUTABLES`.
74+
* Add any models/files the test downloads at runtime to `MODELS` (filename + URL), **and** register every downloaded file the test loads in the `models_for_test()` case statement. The runner pushes only the files listed there from `$MODELS_DIR` to the device for that test, runs it, and removes them afterwards — anything missing won't be on the device when the test runs. Tests with no model dependencies don't need an entry.
75+
* Repo-bundled fixtures (small images, audio, etc.) go in `TEST_ASSETS` instead. Those are pushed once up front and stay on the device; do not list them in `models_for_test()`.
7376

packages/react-native-executorch/common/rnexecutorch/tests/run_tests.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,7 @@ models_for_test() {
209209
LLMTests) echo "smolLm2_135M_8da4w.pte smollm_tokenizer.json lfm2_5_vl_quantized_xnnpack_v2.pte lfm2_vl_tokenizer.json lfm2_vl_tokenizer_config.json test_image.jpg" ;;
210210
TextToImageTests) echo "t2i_tokenizer.json t2i_encoder.pte t2i_unet.pte t2i_decoder.pte" ;;
211211
InstanceSegmentationTests) echo "yolo26n-seg.pte segmentation_image.jpg" ;;
212+
PoseEstimationTests) echo "yolo26n-pose.pte" ;;
212213
SemanticSegmentationTests) echo "deeplabV3_xnnpack_fp32.pte test_image.jpg" ;;
213214
OCRTests | VerticalOCRTests) echo "xnnpack_craft_quantized.pte xnnpack_crnn_english.pte" ;;
214215
*) echo "" ;;

packages/react-native-executorch/src/constants/poseEstimation.ts

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -3,22 +3,22 @@
33
* Use for type-safe keypoint access: `keypoints[CocoKeypoint.NOSE]`
44
* @category Types
55
*/
6-
export const CocoKeypoint = {
7-
NOSE: 0,
8-
LEFT_EYE: 1,
9-
RIGHT_EYE: 2,
10-
LEFT_EAR: 3,
11-
RIGHT_EAR: 4,
12-
LEFT_SHOULDER: 5,
13-
RIGHT_SHOULDER: 6,
14-
LEFT_ELBOW: 7,
15-
RIGHT_ELBOW: 8,
16-
LEFT_WRIST: 9,
17-
RIGHT_WRIST: 10,
18-
LEFT_HIP: 11,
19-
RIGHT_HIP: 12,
20-
LEFT_KNEE: 13,
21-
RIGHT_KNEE: 14,
22-
LEFT_ANKLE: 15,
23-
RIGHT_ANKLE: 16,
24-
} as const;
6+
export enum CocoKeypoint {
7+
NOSE = 0,
8+
LEFT_EYE = 1,
9+
RIGHT_EYE = 2,
10+
LEFT_EAR = 3,
11+
RIGHT_EAR = 4,
12+
LEFT_SHOULDER = 5,
13+
RIGHT_SHOULDER = 6,
14+
LEFT_ELBOW = 7,
15+
RIGHT_ELBOW = 8,
16+
LEFT_WRIST = 9,
17+
RIGHT_WRIST = 10,
18+
LEFT_HIP = 11,
19+
RIGHT_HIP = 12,
20+
LEFT_KNEE = 13,
21+
RIGHT_KNEE = 14,
22+
LEFT_ANKLE = 15,
23+
RIGHT_ANKLE = 16,
24+
}

packages/react-native-executorch/src/modules/computer_vision/PoseEstimationModule.ts

Lines changed: 26 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
1-
import { Frame, PixelData, ResourceSource } from '../../types/common';
1+
import {
2+
Frame,
3+
LabelEnum,
4+
PixelData,
5+
ResourceSource,
6+
} from '../../types/common';
27
import {
38
Keypoint,
49
PersonKeypoints,
@@ -7,7 +12,6 @@ import {
712
PoseEstimationModelSources,
813
PoseEstimationModelName,
914
PoseEstimationConfig,
10-
KeypointEnum,
1115
} from '../../types/poseEstimation';
1216
import { RnExecutorchErrorCode } from '../../errors/ErrorCodes';
1317
import { RnExecutorchError } from '../../errors/errorUtils';
@@ -29,13 +33,13 @@ const ModelConfigs = {
2933
'yolo26n-pose': YOLO_POSE_CONFIG,
3034
} as const satisfies Record<
3135
PoseEstimationModelName,
32-
PoseEstimationConfig<KeypointEnum>
36+
PoseEstimationConfig<LabelEnum>
3337
>;
3438

3539
type ModelConfigsType = typeof ModelConfigs;
3640

3741
/**
38-
* Resolves the {@link KeypointEnum} for a given built-in pose estimation model name.
42+
* Resolves the {@link LabelEnum} for a given built-in pose estimation model name.
3943
* @typeParam M - A built-in model name from {@link PoseEstimationModelName}.
4044
* @category Types
4145
*/
@@ -45,10 +49,10 @@ export type PoseEstimationKeypoints<M extends PoseEstimationModelName> =
4549
type ModelNameOf<C extends PoseEstimationModelSources> = C['modelName'];
4650

4751
/** @internal */
48-
type ResolveKeypoints<T extends PoseEstimationModelName | KeypointEnum> =
52+
type ResolveKeypoints<T extends PoseEstimationModelName | LabelEnum> =
4953
ResolveConfigOrType<T, ModelConfigsType, 'keypointMap'>;
5054

51-
function mapPersonKeypoints<K extends KeypointEnum>(
55+
function mapPersonKeypoints<K extends LabelEnum>(
5256
raw: Keypoint[][],
5357
entries: [string, number][],
5458
maxIndex: number
@@ -71,26 +75,33 @@ function mapPersonKeypoints<K extends KeypointEnum>(
7175
/**
7276
* Pose estimation module for detecting human body keypoints.
7377
* @typeParam T - Either a built-in model name (e.g. `'yolo26n-pose'`)
74-
* or a custom {@link KeypointEnum} keypoint map.
78+
* or a custom {@link LabelEnum} keypoint map.
7579
* @category Typescript API
7680
*/
7781
export class PoseEstimationModule<
78-
T extends PoseEstimationModelName | KeypointEnum,
82+
T extends PoseEstimationModelName | LabelEnum,
7983
> extends VisionModule<PoseDetections<ResolveKeypoints<T>>> {
8084
private readonly keypointMap: ResolveKeypoints<T>;
81-
private readonly modelConfig: PoseEstimationConfig<KeypointEnum>;
85+
private readonly modelConfig: PoseEstimationConfig<LabelEnum>;
86+
// Numeric TS enums double-list
87+
// their keys at runtime (value → name); we keep only the (name, index) pairs
88+
private readonly keypointEntries: [string, number][];
8289
private readonly maxKeypointIndex: number;
8390

8491
private constructor(
8592
keypointMap: ResolveKeypoints<T>,
86-
modelConfig: PoseEstimationConfig<KeypointEnum>,
93+
modelConfig: PoseEstimationConfig<LabelEnum>,
8794
nativeModule: unknown
8895
) {
8996
super();
9097
this.keypointMap = keypointMap;
9198
this.modelConfig = modelConfig;
9299
this.nativeModule = nativeModule;
93-
this.maxKeypointIndex = Math.max(...Object.values(keypointMap));
100+
this.keypointEntries = [];
101+
for (const [name, value] of Object.entries(keypointMap)) {
102+
if (typeof value === 'number') this.keypointEntries.push([name, value]);
103+
}
104+
this.maxKeypointIndex = Math.max(...this.keypointEntries.map(([, v]) => v));
94105
}
95106

96107
/**
@@ -106,7 +117,7 @@ export class PoseEstimationModule<
106117
const { modelSource } = namedSources;
107118
const modelConfig = ModelConfigs[
108119
namedSources.modelName
109-
] as PoseEstimationConfig<KeypointEnum>;
120+
] as PoseEstimationConfig<LabelEnum>;
110121
const { keypointMap, preprocessorConfig } = modelConfig;
111122
const normMean = preprocessorConfig?.normMean ?? [];
112123
const normStd = preprocessorConfig?.normStd ?? [];
@@ -133,7 +144,7 @@ export class PoseEstimationModule<
133144
* @param onDownloadProgress - Optional callback to monitor download progress (0-1).
134145
* @returns A Promise resolving to a `PoseEstimationModule` instance typed to the provided keypoint map.
135146
*/
136-
static async fromCustomModel<K extends KeypointEnum>(
147+
static async fromCustomModel<K extends LabelEnum>(
137148
modelSource: ResourceSource,
138149
config: PoseEstimationConfig<K>,
139150
onDownloadProgress: (progress: number) => void = () => {}
@@ -195,7 +206,7 @@ export class PoseEstimationModule<
195206
this.modelConfig.defaultKeypointThreshold ?? 0.5;
196207
const defaultInputSize = this.modelConfig.defaultInputSize;
197208
const availableInputSizes = this.modelConfig.availableInputSizes;
198-
const keypointEntries = Object.entries(this.keypointMap);
209+
const keypointEntries = this.keypointEntries;
199210
const maxKeypointIndex = this.maxKeypointIndex;
200211
return (
201212
frame: Frame,
@@ -308,10 +319,9 @@ export class PoseEstimationModule<
308319
methodName
309320
);
310321

311-
const entries = Object.entries(this.keypointMap);
312322
return mapPersonKeypoints<ResolveKeypoints<T>>(
313323
raw,
314-
entries,
324+
this.keypointEntries,
315325
this.maxKeypointIndex
316326
);
317327
}

packages/react-native-executorch/src/types/poseEstimation.ts

Lines changed: 7 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,9 @@
1-
import { Frame, PixelData, ResourceSource } from './common';
1+
import { Frame, LabelEnum, PixelData, ResourceSource } from './common';
22
import { CocoKeypoint } from '../constants/poseEstimation';
33
import { RnExecutorchError } from '../errors/errorUtils';
44

55
export { CocoKeypoint };
66

7-
/**
8-
* A keypoint enum maps keypoint names to their indices.
9-
* Similar to LabelEnum but specifically for pose keypoints.
10-
* @category Types
11-
*/
12-
export type KeypointEnum = Readonly<Record<string, number>>;
13-
147
/**
158
* A single keypoint with x, y coordinates
169
* @category Types
@@ -22,30 +15,30 @@ export interface Keypoint {
2215

2316
/**
2417
* Keypoints for a single detected person, keyed by name from the keypoint map.
25-
* @typeParam K - The {@link KeypointEnum} for this model.
18+
* @typeParam K - The {@link LabelEnum} for this model.
2619
* @category Types
2720
* @example
2821
* ```ts
2922
* person.NOSE; // { x, y }
3023
* ```
3124
*/
32-
export type PersonKeypoints<K extends KeypointEnum = typeof CocoKeypoint> = {
25+
export type PersonKeypoints<K extends LabelEnum = typeof CocoKeypoint> = {
3326
readonly [Name in keyof K]: Keypoint;
3427
};
3528

3629
/**
3730
* Pose estimation result containing all detected people.
3831
* @category Types
3932
*/
40-
export type PoseDetections<K extends KeypointEnum = typeof CocoKeypoint> =
33+
export type PoseDetections<K extends LabelEnum = typeof CocoKeypoint> =
4134
PersonKeypoints<K>[];
4235

4336
/**
4437
* Configuration for pose estimation model behavior.
4538
* @category Types
4639
* @typeParam K - The keypoint enum type for this model.
4740
*/
48-
export type PoseEstimationConfig<K extends KeypointEnum> = {
41+
export type PoseEstimationConfig<K extends LabelEnum> = {
4942
keypointMap: K;
5043
preprocessorConfig?: {
5144
normMean?: readonly [number, number, number];
@@ -112,10 +105,10 @@ export interface PoseEstimationOptions {
112105

113106
/**
114107
* Return type of usePoseEstimation hook.
115-
* @typeParam K - The {@link KeypointEnum} representing the model's keypoint schema.
108+
* @typeParam K - The {@link LabelEnum} representing the model's keypoint schema.
116109
* @category Types
117110
*/
118-
export interface PoseEstimationType<K extends KeypointEnum> {
111+
export interface PoseEstimationType<K extends LabelEnum> {
119112
/**
120113
* Contains the error object if the model failed to load or encountered a runtime error.
121114
*/

0 commit comments

Comments
 (0)