diff --git a/docs/docs/computer-vision/useObjectDetection.md b/docs/docs/computer-vision/useObjectDetection.md index c03cce0a51..86ad57b443 100644 --- a/docs/docs/computer-vision/useObjectDetection.md +++ b/docs/docs/computer-vision/useObjectDetection.md @@ -60,13 +60,13 @@ For more information on that topic, you can check out the [Loading models](https The hook returns an object with the following properties: -| Field | Type | Description | -| ------------------ | ----------------------------------------- | ---------------------------------------------------------------------------------------- | -| `forward` | `(input: string) => Promise` | A function that accepts an image (url, b64) and returns an array of `Detection` objects. | -| `error` | string | null | Contains the error message if the model loading failed. | -| `isGenerating` | `boolean` | Indicates whether the model is currently processing an inference. | -| `isReady` | `boolean` | Indicates whether the model has successfully loaded and is ready for inference. | -| `downloadProgress` | `number` | Represents the download progress as a value between 0 and 1. | +| Field | Type | Description | +| ------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `forward` | `(input: string, detectionThreshold: number = 0.7) => Promise` | A function that accepts an image (url, b64) and returns an array of `Detection` objects. `detectionThreshold` can be supplied to alter the sensitivity of the detection. | +| `error` | string | null | Contains the error message if the model loading failed. | +| `isGenerating` | `boolean` | Indicates whether the model is currently processing an inference. | +| `isReady` | `boolean` | Indicates whether the model has successfully loaded and is ready for inference. | +| `downloadProgress` | `number` | Represents the download progress as a value between 0 and 1. | ## Running the model diff --git a/docs/docs/typescript-api/ClassificationModule.md b/docs/docs/typescript-api/ClassificationModule.md index 795437c4f8..7ede208188 100644 --- a/docs/docs/typescript-api/ClassificationModule.md +++ b/docs/docs/typescript-api/ClassificationModule.md @@ -14,20 +14,22 @@ import { const imageUri = 'path/to/image.png'; +const module = new ClassificationModule(); + // Loading the model -await ClassificationModule.load(EFFICIENTNET_V2_S); +await module.load(EFFICIENTNET_V2_S); // Running the model -const classesWithProbabilities = await ClassificationModule.forward(imageUri); +const classesWithProbabilities = await module.forward(imageUri); ``` ### Methods -| Method | Type | Description | -| -------------------- | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | -| `load` | `(modelSource: ResourceSource): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. | -| `forward` | `(input: string): Promise<{ [category: string]: number }>` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. | -| `onDownloadProgress` | `(callback: (downloadProgress: number) => void): any` | Subscribe to the download progress event. | +| Method | Type | Description | +| --------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `load` | `(modelSource: ResourceSource, onDownloadProgressCallback: (_: number) => void () => {}): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. To track the download progress, supply a callback function `onDownloadProgressCallback`. | +| `forward` | `(input: string): Promise<{ [category: string]: number }>` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. | +| `delete` | `(): void` | Release the memory held by the module. Calling `forward` afterwards is invalid. |
Type definitions @@ -40,8 +42,12 @@ type ResourceSource = string | number | object; ## Loading the model -To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. +To load the model, create a new instance of the module and use the `load` method on it. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. ## Running the model -To run the model, you can use the `forward` method. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or an object containing categories with their probabilities. +To run the model, you can use the `forward` method on the module object. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or an object containing categories with their probabilities. + +## Managing memory + +The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method `delete()` on the module object you will no longer use, and want to remove from the memory. Note that you cannot use `forward` after `delete` unless you load the module again. diff --git a/docs/docs/typescript-api/ImageSegmentationModule.md b/docs/docs/typescript-api/ImageSegmentationModule.md index 058ea017c0..f9d2708875 100644 --- a/docs/docs/typescript-api/ImageSegmentationModule.md +++ b/docs/docs/typescript-api/ImageSegmentationModule.md @@ -14,20 +14,22 @@ import { const imageUri = 'path/to/image.png'; +const module = new ImageSegmentationModule(); + // Loading the model -await ImageSegmentationModule.load(DEEPLAB_V3_RESNET50); +await module.load(DEEPLAB_V3_RESNET50); // Running the model -const outputDict = await ImageSegmentationModule.forward(imageUri); +const outputDict = await module.forward(imageUri); ``` ### Methods -| Method | Type | Description | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `load` | `(modelSource: ResourceSource): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. | -| `forward` | `(input: string, classesOfInterest?: DeeplabLabel[], resize?: boolean) => Promise<{[key in DeeplabLabel]?: number[]}>` | Executes the model's forward pass, where :
\* `input` can be a fetchable resource or a Base64-encoded string.
\* `classesOfInterest` is an optional list of `DeeplabLabel` used to indicate additional arrays of probabilities to output (see section "Running the model"). The default is an empty list.
\* `resize` is an optional boolean to indicate whether the output should be resized to the original image dimensions, or left in the size of the model (see section "Running the model"). The default is `false`.

The return is a dictionary containing:
\* for the key `DeeplabLabel.ARGMAX` an array of integers corresponding to the most probable class for each pixel
\* an array of floats for each class from `classesOfInterest` corresponding to the probabilities for this class. | -| `onDownloadProgress` | `(callback: (downloadProgress: number) => void): any` | Subscribe to the download progress event. | +| Method | Type | Description | +| --------- | ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `load` | `(modelSource: ResourceSource, onDownloadProgressCallback: (_: number) => void () => {}): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. To track the download progress, supply a callback function `onDownloadProgressCallback`. | +| `forward` | `(input: string, classesOfInterest?: DeeplabLabel[], resize?: boolean) => Promise<{[key in DeeplabLabel]?: number[]}>` | Executes the model's forward pass, where :
\* `input` can be a fetchable resource or a Base64-encoded string.
\* `classesOfInterest` is an optional list of `DeeplabLabel` used to indicate additional arrays of probabilities to output (see section "Running the model"). The default is an empty list.
\* `resize` is an optional boolean to indicate whether the output should be resized to the original image dimensions, or left in the size of the model (see section "Running the model"). The default is `false`.

The return is a dictionary containing:
\* for the key `DeeplabLabel.ARGMAX` an array of integers corresponding to the most probable class for each pixel
\* an array of floats for each class from `classesOfInterest` corresponding to the probabilities for this class. | +| `delete` | `(): void` | Release the memory held by the module. Calling `forward` afterwards is invalid. |
Type definitions @@ -40,11 +42,11 @@ type ResourceSource = string | number | object; ## Loading the model -To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. +To load the model, create a new instance of the module and use the `load` method on it. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. ## Running the model -To run the model, you can use the `forward` method. It accepts three arguments: a required image, an optional list of classes, and an optional flag whether to resize the output to the original dimensions. +To run the model, you can use the `forward` method on the module object. It accepts three arguments: a required image, an optional list of classes, and an optional flag whether to resize the output to the original dimensions. - The image can be a remote URL, a local file URI, or a base64-encoded image. - The `classesOfInterest` list contains classes for which to output the full results. By default the list is empty, and only the most probable classes are returned (essentially an arg max for each pixel). Look at `DeeplabLabel` enum for possible classes. @@ -58,3 +60,7 @@ Setting `resize` to true will make `forward` slower. - For the key `DeeplabLabel.ARGMAX` the array contains for each pixel an integer corresponding to the class with the highest probability. - For every other key from `DeeplabLabel`, if the label was included in `classesOfInterest` the dictionary will contain an array of floats corresponding to the probability of this class for every pixel. + +## Managing memory + +The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method `delete()` on the module object you will no longer use, and want to remove from the memory. Note that you cannot use `forward` after `delete` unless you load the module again. diff --git a/docs/docs/typescript-api/ObjectDetectionModule.md b/docs/docs/typescript-api/ObjectDetectionModule.md index eca5296412..2afba4587f 100644 --- a/docs/docs/typescript-api/ObjectDetectionModule.md +++ b/docs/docs/typescript-api/ObjectDetectionModule.md @@ -14,20 +14,22 @@ import { const imageUri = 'path/to/image.png'; +const module = new ObjectDetectionModule(); + // Loading the model -await ObjectDetectionModule.load(SSDLITE_320_MOBILENET_V3_LARGE); +await module.load(SSDLITE_320_MOBILENET_V3_LARGE); // Running the model -const detections = await ObjectDetectionModule.forward(imageUri); +const detections = await module.forward(imageUri); ``` ### Methods -| Method | Type | Description | -| -------------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | -| `load` | `(modelSource: ResourceSource): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. | -| `forward` | `(input: string): Promise` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. | -| `onDownloadProgress` | `(callback: (downloadProgress: number) => void): any` | Subscribe to the download progress event. | +| Method | Type | Description | +| --------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `load` | `(modelSource: ResourceSource, onDownloadProgressCallback: (_: number) => void () => {}): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. To track the download progress, supply a callback function `onDownloadProgressCallback`. | +| `forward` | `(input: string, detectionThreshold: number = 0.7): Promise` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. `detectionThreshold` can be supplied to alter the sensitivity of the detection. | +| `delete` | `(): void` | Release the memory held by the module. Calling `forward` afterwards is invalid. |
Type definitions @@ -53,8 +55,12 @@ interface Detection { ## Loading the model -To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. +To load the model, create a new instance of the module and use the `load` method on it. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. ## Running the model -To run the model, you can use the `forward` method. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or an array of `Detection` objects. Each object contains coordinates of the bounding box, the label of the detected object, and the confidence score. +To run the model, you can use the `forward` method on the module object. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or an array of `Detection` objects. Each object contains coordinates of the bounding box, the label of the detected object, and the confidence score. + +## Managing memory + +The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method `delete()` on the module object you will no longer use, and want to remove from the memory. Note that you cannot use `forward` after `delete` unless you load the module again. diff --git a/docs/docs/typescript-api/StyleTransferModule.md b/docs/docs/typescript-api/StyleTransferModule.md index 88a7207d4a..ed308cd833 100644 --- a/docs/docs/typescript-api/StyleTransferModule.md +++ b/docs/docs/typescript-api/StyleTransferModule.md @@ -14,20 +14,22 @@ import { const imageUri = 'path/to/image.png'; +const module = new StyleTransferModule(); + // Loading the model -await StyleTransferModule.load(STYLE_TRANSFER_CANDY); +await module.load(STYLE_TRANSFER_CANDY); // Running the model -const generatedImageUrl = await StyleTransferModule.forward(imageUri); +const generatedImageUrl = await module.forward(imageUri); ``` ### Methods -| Method | Type | Description | -| -------------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | -| `load` | `(modelSource: ResourceSource): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. | -| `forward` | `(input: string): Promise` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. | -| `onDownloadProgress` | `(callback: (downloadProgress: number) => void): any` | Subscribe to the download progress event. | +| Method | Type | Description | +| --------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `load` | `(modelSource: ResourceSource, onDownloadProgressCallback: (_: number) => void () => {}): Promise` | Loads the model, where `modelSource` is a string that specifies the location of the model binary. To track the download progress, supply a callback function `onDownloadProgressCallback`. | +| `forward` | `(input: string): Promise` | Executes the model's forward pass, where `input` can be a fetchable resource or a Base64-encoded string. | +| `delete` | `(): void` | Release the memory held by the module. Calling `forward` afterwards is invalid. |
Type definitions @@ -40,8 +42,12 @@ type ResourceSource = string | number | object; ## Loading the model -To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. +To load the model, create a new instance of the module and use the `load` method on it. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. ## Running the model -To run the model, you can use the `forward` method. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or a URL to generated image. +To run the model, you can use the `forward` method on the module object. It accepts one argument, which is the image. The image can be a remote URL, a local file URI, or a base64-encoded image. The method returns a promise, which can resolve either to an error or a URL to generated image. + +## Managing memory + +The module is a regular JavaScript object, and as such its lifespan will be managed by the garbage collector. In most cases this should be enough, and you should not worry about freeing the memory of the module yourself, but in some cases you may want to release the memory occupied by the module before the garbage collector steps in. In this case use the method `delete()` on the module object you will no longer use, and want to remove from the memory. Note that you cannot use `forward` after `delete` unless you load the module again.