diff --git a/components/Sidebar/index.jsx b/components/Sidebar/index.jsx index 5f4a06e..0d88837 100644 --- a/components/Sidebar/index.jsx +++ b/components/Sidebar/index.jsx @@ -11,7 +11,7 @@ const PrefetchLink = props => ; */ export default ({ metadata }) => ( macOS developers may find it more convenient to install CMake using [Homebrew](https://brew.sh). With Homebrew installed, CMake can be installed using a `brew install cmake` command. + +You can verify your CMake installation with the command: + +```bash +cmake --version +``` + +As a Node native module developer, you may find it convenient to install CMake.js as a global command line tool: + +```bash +npm install cmake-js -g +``` + +You can verify your CMake.js installation with the command: + +```bash +cmake-js --version +``` + +## package.json + +Your `package.json` file needs to have a couple of entries for your native module to work with CMake.js. + +Since your native module needs to be compiled using CMake.js on installation, the `scripts` property of your `package.json` file needs an `install` entry to make this happen: + +```json + "scripts": { + "install": "cmake-js compile" + } +``` + +It is unlikely that the users of your native module will have CMake.js installed as a global command line tool. Therefore, your project needs to declare a development dependency on CMake.js. This can be accomplished by entering this command: + +```bash +npm install cmake-js --save-dev +``` + +An alternative is to manually add the development dependency to your `package.json` file: + +```json + "devDependencies": { + "cmake-js": "^6.0.0" + } +``` + +An example of this approach is [available here](https://github.com/nodejs/node-addon-examples/blob/main/src/8-tooling/build_with_cmake/node-addon-api/package.json). + +## CMakeLists.txt + +Native modules built on CMake.js have a `CMakeLists.txt` that describes how the module is to be built. The file serves the same purpose as the `binding.gyp` for projects that use `node-gyp`. + +In addition to the entries required for any CMake build, additional entries are required when building native modules. + +### CMake.js + +Here are the lines required for all native modules built using CMake.js: + +```cpp +project(napi-cmake-build-example) +include_directories(${CMAKE_JS_INC}) +file(GLOB SOURCE_FILES "hello.cc") +add_library(${PROJECT_NAME} SHARED ${SOURCE_FILES} ${CMAKE_JS_SRC}) +set_target_properties(${PROJECT_NAME} PROPERTIES PREFIX "" SUFFIX ".node") +target_link_libraries(${PROJECT_NAME} ${CMAKE_JS_LIB}) +``` + +### NAPI_VERSION + +When building a native module based on Node-API, it is important to declare the minimum Node-API version against which your module is designed to work. For CMake.js, this is accomplished by adding a line like this to the `CMakeLists.txt` file: + +```cpp +# define NAPI_VERSION +add_definitions(-DNAPI_VERSION=3) +``` + +> In the absence of other requirements, Node-API version 3 is a good choice as this is the Node-API version active when Node-API left experimental status. + +### node-addon-api + +Additional configuration values are required for Node-API modules based on `node-addon-api`. + +`node-addon-api` requires C++11. These configuration lines at the top of the `CMakeLists.txt` file specify this requirement: + +```cpp +cmake_minimum_required(VERSION 3.9) +cmake_policy(SET CMP0042 NEW) +set (CMAKE_CXX_STANDARD 11) +``` + +Modules based on `node-addon-api` include additional header files that are not part of Node itself. These lines instruct CMake.js where to find these files: + +```cpp +# Include Node-API wrappers +execute_process(COMMAND node -p "require('node-addon-api').include" + WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} + OUTPUT_VARIABLE NODE_ADDON_API_DIR + ) +string(REPLACE "\n" "" NODE_ADDON_API_DIR ${NODE_ADDON_API_DIR}) +string(REPLACE "\"" "" NODE_ADDON_API_DIR ${NODE_ADDON_API_DIR}) +target_include_directories(${PROJECT_NAME} PRIVATE ${NODE_ADDON_API_DIR}) +``` + +An example of this approach is [available here](https://github.com/nodejs/node-addon-examples/blob/main/src/8-tooling/build_with_cmake/node-addon-api/CMakeLists.txt). diff --git a/pages/napi/build-tools/index.md b/pages/napi/build-tools/index.md new file mode 100644 index 0000000..6b30f6d --- /dev/null +++ b/pages/napi/build-tools/index.md @@ -0,0 +1,22 @@ +--- +authors: avivkeller +--- + +# Build Tools + +Building a native Node.js addon requires compiling C/C++ source code into a binary `.node` file that Node.js can load at runtime. This section covers the build tools available to you and the trade-offs between them. + +## Compiling on installation vs. distributing pre-built binaries + +There are two broad strategies for shipping a native addon: + +**Compile on the user's machine** - When a user runs `npm install`, the build tool compiles your C/C++ source on their system. This is simple to set up but requires every user to have a working C/C++ toolchain installed. + +**Distribute pre-built binaries** - You compile binaries for each supported platform and architecture ahead of time and upload them somewhere users can download. Users who download a matching binary skip the compilation step entirely; others fall back to compiling locally. + +## Build tools covered in this section + +- [node-gyp](/learn/napi/build-tools/node-gyp.md) - the default build tool bundled with npm; uses Google's GYP format and is nearly universally supported in the Node ecosystem +- [CMake.js](/learn/napi/build-tools/cmake-js.md) - a CMake-based alternative, well-suited for projects that already use CMake +- [node-pre-gyp](/learn/napi/build-tools/node-pre-gyp.md) - a layer on top of node-gyp for distributing pre-built binaries via Amazon S3 +- [prebuild](/learn/napi/build-tools/prebuild.md) - an alternative pre-build tool that publishes binaries as GitHub Releases diff --git a/pages/napi/build-tools/node-gyp.md b/pages/napi/build-tools/node-gyp.md new file mode 100644 index 0000000..cc9622d --- /dev/null +++ b/pages/napi/build-tools/node-gyp.md @@ -0,0 +1,30 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# node-gyp + +[node-gyp](https://github.com/nodejs/node-gyp) is the standard build tool for native Node.js addons and is used by the vast majority of packages in the npm ecosystem. It is actively maintained by the Node.js team. Most of the examples on this site use node-gyp to build binaries. + +node-gyp is based on Google's [GYP](https://gyp.gsrc.io/) build tool. GYP provides a single cross-platform configuration format for C/C++ builds. Although Google archived the upstream GYP repository, node-gyp continues to receive active development and maintenance independently. + +> node-gyp requires **Python 3.6 or later**. Python 2 is not supported. The full list of requirements for each platform can be found in the [node-gyp installation docs](https://github.com/nodejs/node-gyp#installation). + +node-gyp is included with npm; when npm sees `"gypfile": true` in `package.json`, it invokes node-gyp automatically during `npm install`. You can also install and use it directly: + +```bash +npm install -g node-gyp +``` + +For developers who find node-gyp too constraining, [CMake.js](/learn/napi/build-tools/cmake-js.md) is a good alternative. + +### Pros + +- Included with npm - no separate global install required for consumers. +- Nearly universally used in the Node.js ecosystem, with broad documentation and community knowledge. +- Supports Windows, macOS, and Linux from a single `binding.gyp` configuration file. + +### Cons + +- The underlying GYP format is no longer actively developed by Google. +- Some developers find GYP's configuration syntax verbose or difficult to debug. diff --git a/pages/napi/build-tools/node-pre-gyp.md b/pages/napi/build-tools/node-pre-gyp.md new file mode 100644 index 0000000..d848c79 --- /dev/null +++ b/pages/napi/build-tools/node-pre-gyp.md @@ -0,0 +1,157 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# node-pre-gyp + +One of the limitations of native addons is that they must be compiled for each target platform and architecture. Without pre-built binaries, every user who installs your package must have a working C/C++ toolchain on their machine. + +[node-pre-gyp](https://github.com/mapbox/node-pre-gyp) solves this by letting you build binaries ahead of time, upload them to a remote location, and have users download the right binary at install time - falling back to compiling from source only if a matching binary is not available. + +> Note that Node-API support was added to node-pre-gyp in version 0.8.0. + +> [prebuild](/learn/napi/build-tools/prebuild.md) is an alternative tool that addresses the same problem. + +This page describes the changes required to a Node-API addon to support node-pre-gyp. + +## Amazon S3 + +By default, node-pre-gyp uploads binaries to [Amazon S3](https://aws.amazon.com/s3/). + +> The [node-pre-gyp-github](https://github.com/bchr02/node-pre-gyp-github) module adds support for publishing to GitHub Releases instead. + +### Amazon S3 Requirements + +Before uploading you need: + +1. An Amazon Web Services account. +2. An IAM user or role with permission to upload to S3. +3. An [S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html) to host the binaries. + +### AWS Credentials + +Never store credentials in your repository. node-pre-gyp supports two common approaches for providing credentials during development: + +1. A `~/.node_pre_gyprc` file: + + ```json + { + "accessKeyId": "xxx", + "secretAccessKey": "xxx" + } + ``` + +2. Environment variables: + + ```bash + export node_pre_gyp_accessKeyId=xxx + export node_pre_gyp_secretAccessKey=xxx + ``` + +For CI environments, prefer IAM roles or short-lived credentials rather than long-lived access keys. See the [node-pre-gyp credentials documentation](https://github.com/mapbox/node-pre-gyp#3-configure-aws-credentials) for additional options. + +## package.json + +### The `dependencies` and `devDependencies` properties + +The package is now published under the `@mapbox` scope. Use `aws-sdk` as a dev dependency for the upload step. + +```json +"dependencies": { + "@mapbox/node-pre-gyp": "^1.0.0" +}, +"devDependencies": { + "aws-sdk": "^2.0.0" +} +``` + +### The `scripts` property + +The `install` script should invoke node-pre-gyp with `--fallback-to-build` so that users who don't have a pre-built binary available can still compile locally: + +```json +"scripts": { + "install": "node-pre-gyp install --fallback-to-build" +} +``` + +### The `binary` property + +The `binary` property tells node-pre-gyp which Node-API versions your addon supports and where to find/upload binaries: + +```json +"binary": { + "module_name": "your_module", + "module_path": "./lib/binding/napi-v{napi_build_version}", + "remote_path": "./{module_name}/v{version}/{configuration}/", + "package_name": "{platform}-{arch}-napi-v{napi_build_version}.tar.gz", + "host": "https://your_bucket.s3.us-west-1.amazonaws.com", + "napi_versions": [3] +} +``` + +Set `module_name` to a valid C identifier. The `napi_versions` array lists which Node-API versions to build for; `3` is a reasonable minimum for most addons. + +See the [node-pre-gyp docs](https://github.com/mapbox/node-pre-gyp#1-add-new-entries-to-your-packagejson) for a complete reference, including [Node-API considerations](https://github.com/mapbox/node-pre-gyp#n-api-considerations). + +## binding.gyp + +### New target + +Add a post-build target to copy the compiled binary to the path specified by `module_path`: + +```json +{ + "target_name": "action_after_build", + "type": "none", + "dependencies": ["<(module_name)"], + "copies": [ + { + "files": ["<(PRODUCT_DIR)/<(module_name).node"], + "destination": "<(module_path)" + } + ] +} +``` + +### NAPI_VERSION + +Include the Node-API version in the first target's `defines` so the header files configure themselves correctly: + +```json +"defines": [ + "NAPI_VERSION=<(napi_build_version)" +] +``` + +## JavaScript updates + +JavaScript code that loads the native binary must dynamically resolve the path to the correct `.node` file: + +```cjs +const binary = require('@mapbox/node-pre-gyp'); +const path = require('path'); +const bindingPath = binary.find( + path.resolve(path.join(__dirname, './package.json')) +); +const binding = require(bindingPath); +``` + +## Build + +Once everything is in place, build from source: + +```bash +npm install --build-from-source +``` + +## Package and publish + +```bash +./node_modules/.bin/node-pre-gyp package +./node_modules/.bin/node-pre-gyp publish +``` + +## CI and automated builds + +Use [GitHub Actions](https://docs.github.com/en/actions) to build, test, and publish binaries for multiple platforms and architectures. A typical workflow matrix covers `ubuntu-latest`, `macos-latest`, and `windows-latest`, plus any architecture variants you need (e.g. `x64`, `arm64`). See the node-pre-gyp repository for [example workflow configurations](https://github.com/mapbox/node-pre-gyp). diff --git a/pages/napi/build-tools/prebuild.md b/pages/napi/build-tools/prebuild.md new file mode 100644 index 0000000..7d43c07 --- /dev/null +++ b/pages/napi/build-tools/prebuild.md @@ -0,0 +1,141 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# prebuild + +One of the limitations of native addons is that they must be compiled for each target platform and architecture. Without pre-built binaries, every user who installs your package must have a working C/C++ toolchain on their machine. + +[prebuild](https://github.com/prebuild/prebuild) solves this by letting you compile binaries ahead of time and publish them as [GitHub Releases](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases). When a user installs your package, `prebuild-install` downloads the right pre-built binary for their platform and Node-API version - falling back to compiling from source only if no matching binary is found. + +> Node-API support was added to prebuild in version 8.1.0. + +> [node-pre-gyp](/learn/napi/build-tools/node-pre-gyp.md) is an alternative tool that uses Amazon S3 instead of GitHub Releases. + +## prebuild and prebuild-install + +The tooling is split into two packages: + +- [prebuild](https://github.com/prebuild/prebuild) - used by you (the addon developer) to build and upload binaries. +- [prebuild-install](https://github.com/prebuild/prebuild-install) - used by your consumers to download the pre-built binary at install time. + +## Installing + +Install `prebuild` as a dev dependency so it is available during your CI publish workflow: + +```bash +npm install prebuild --save-dev +``` + +## GitHub personal access token + +Uploading a release asset to GitHub requires a personal access token. + +1. Go to [GitHub Settings → Developer settings → Personal access tokens](https://github.com/settings/tokens). +2. Click **Generate new token** and choose **Fine-grained token** (recommended) or classic. + - For a fine-grained token: grant **Read and write** access to **Contents** on the target repository. + - For a classic token: select the `public_repo` scope (or `repo` for private repositories). +3. Copy the generated token - it is only shown once. +4. Set it as an environment variable: + + ```bash + export GITHUB_TOKEN= + ``` + +In CI, store the token as a repository secret and expose it to the workflow via `env`. + +## package.json + +### The `repository` property + +Because prebuild uploads to GitHub, you must declare the repository: + +```json +"repository": { + "type": "git", + "url": "https://github.com/yourorg/your-napi-addon.git" +} +``` + +### The `dependencies` property + +Consumers need `prebuild-install` to download pre-built binaries: + +```json +"dependencies": { + "prebuild-install": "^7.0.0" +} +``` + +### The `scripts` property + +```json +"scripts": { + "install": "prebuild-install --runtime napi || node-gyp rebuild", + "rebuild": "node-gyp rebuild", + "prebuild": "prebuild --runtime napi --all --strip --verbose", + "upload": "prebuild --runtime napi --upload ${GITHUB_TOKEN}" +} +``` + +The `--runtime napi` flag is required for Node-API builds. The `--all` flag builds for every supported Node-API version declared in the `binary` property. Without `--all` or explicit `--target` flags, prebuild builds only for the Node-API version of the current Node.js process. + +### The `binary` property + +Declare which Node-API versions your addon supports: + +```json +"binary": { + "napi_versions": [3] +} +``` + +`3` is a reasonable default - it was the Node-API version when the API left experimental status, and binaries built against it run on all later versions. + +## NAPI_VERSION + +The `NAPI_VERSION` preprocessor value must be defined so the Node-API headers configure themselves for the correct version. How you set it depends on your build tool. + +### node-gyp + +In `binding.gyp`: + +```json +"defines": [ + "NAPI_VERSION=<(napi_build_version)" +] +``` + +### cmake-js + +In `CMakeLists.txt`: + +```cpp +add_compile_definitions(NAPI_VERSION=${napi_build_version}) +``` + +## Building and uploading + +Build and package your binaries: + +```bash +npm run prebuild +``` + +When you are ready to publish a release, upload to GitHub: + +```bash +npm run upload +``` + +This requires `GITHUB_TOKEN` to be set in the environment. In CI, trigger the upload step only after a version tag is pushed. + +## Life for your users + +When a user runs `npm install`, the `install` script in your `package.json` runs: + +```bash +prebuild-install --runtime napi || node-gyp rebuild +``` + +`prebuild-install` fetches the GitHub Releases page for your repository, finds a binary that matches the user's platform, architecture, and Node-API version, and installs it. If no match is found, `node-gyp rebuild` compiles from source as a fallback. diff --git a/pages/napi/getting-started/index.md b/pages/napi/getting-started/index.md new file mode 100644 index 0000000..156469d --- /dev/null +++ b/pages/napi/getting-started/index.md @@ -0,0 +1,25 @@ +--- +authors: avivkeller +--- + +# Getting Started with Node-API + +This section walks you through everything you need to begin building native Node.js addons with Node-API. It is designed for developers who are new to native addon development and want a guided path from zero to a working module. + +## What you will learn + +- [Prerequisites](/learn/napi/getting-started/prerequisites.md) - the C/C++ and JavaScript knowledge you need before diving in +- [The tools you'll need](/learn/napi/getting-started/tools.md) - how to install Node, npm, Git, and the C/C++ toolchain on Windows, macOS, and Linux +- [Anatomy of a Node-API project](/learn/napi/getting-started/project-structure.md) - the common directory layout, `package.json` entries, `binding.gyp`, and the JavaScript wrapper layer shared by every `node-addon-api` project +- [Your first project](/learn/napi/getting-started/your-first-project.md) - building and running your first Node-API module +- [ObjectWrap](/learn/napi/getting-started/objectwrap.md) - exposing C++ objects as JavaScript objects using `node-addon-api`'s `ObjectWrap` class +- [Migrating from NAN](/learn/napi/getting-started/migration.md) - converting an existing NAN-based addon to Node-API + +## Which API level should I use? + +Node-API operates at two levels: + +- **C API** - built directly into Node.js and documented on the [Node.js API pages](https://nodejs.org/api/n-api.html). Gives you maximum control and no extra dependencies. +- **C++ wrapper (`node-addon-api`)** - an npm package that wraps the C API in a friendlier C++ object model. Recommended for most projects because it reduces boilerplate significantly while preserving full ABI stability. + +The examples in this section use `node-addon-api`. diff --git a/pages/napi/getting-started/migration.md b/pages/napi/getting-started/migration.md new file mode 100644 index 0000000..00a3c2a --- /dev/null +++ b/pages/napi/getting-started/migration.md @@ -0,0 +1,169 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Migrating to Node-API + +The objective of this tutorial is to give you a good idea of the steps necessary and the tools available to migrate an existing [NAN](https://github.com/nodejs/nan) Node native add-on module to [Node-API](https://nodejs.org/api/n-api.html) using the [node-addon-api](https://github.com/nodejs/node-addon-api) package. + +This tutorial uses the conversion tool supplied with Node-API to give you a head start on the migration. However, the conversion tool will only get you so far. Further manual cleanup is still required as described below. + +To keep things somewhat constrained, this tutorial uses [node-microtime](https://github.com/wadey/node-microtime) which is a simple NAN-based native add-on. This add-on makes system calls to determine the current time to microsecond resolution if supported by the operating system. + +Before you start, make sure you've got all the necessary [prerequisites](/learn/napi/getting-started/prerequisites.md) and [tools](/learn/napi/getting-started/tools.md) installed. + +> Node-API is stable in all currently supported Node.js releases. Use an [Active LTS or Maintenance LTS release](https://nodejs.org/en/about/releases/). You can check the version of Node.js you are running with `node -v`. + +## Clone `node-microtime` + +As a first step, clone the [node-microtime](https://github.com/wadey/node-microtime) GitHub repository to your system: + +```bash +git clone https://github.com/wadey/node-microtime.git +``` + +Before we make our modifications, it's a good idea to first build and test `node-microtime` to help verify that the necessary development tools have been correctly installed and configured. + +> Since `node-microtime` has already migrated to `node-addon-api`, you have to move to tag `v2.1.9` in order to follow the tutorial. + +```bash +cd node-microtime +git checkout tags/v2.1.9 +npm install +npm test +``` + +The `npm install` command invokes the build process and `npm test` runs the code. You may see compiler warnings that do not affect the ability to run the code. When successfully built and run, you should see output that looks something like this: + +``` +microtime.now() = 1526334357974754 +microtime.nowDouble() = 1526334357.976626 +microtime.nowStruct() = [ 1526334357, 976748 ] + +Guessing clock resolution... +Clock resolution observed: 1us +``` + +## Run the conversion tool + +Once the basic operation of the code has been verified, the next step is to run the [Node-API Conversion Tool](https://github.com/nodejs/node-addon-api/blob/main/doc/conversion-tool.md). Be aware that the conversion tool _replaces files in place_. Never run the conversion tool on the only copy of your project. And obviously, you want to run it only _once_. + +```bash +npm install --save node-addon-api +node ./node_modules/node-addon-api/tools/conversion.js ./ +``` + +For this small project, the conversion tool runs very quickly. At this point, the conversion tool has modified the following project files: + +- `binding.gyp` +- `package.json` +- `src/microtime.cc` + +Go ahead and rebuild the converted code: + +```bash +npm install +``` + +As you'll see, there are one or more compile errors that need to be addressed. There can be quite a few at times, but nothing insurmountable. + +## Cleanup + +The conversion tool cannot anticipate every coding situation. So there will typically be issues that need to be addressed manually. Below are the issues you're likely to encounter with this project. The best approach is to address each issue, one at a time, and attempt the `npm install` after addressing each issue until there are no more errors. + +### Cannot find module 'nan' + +This error, and its counterpart where `napi.h` cannot be found, is due to code missing in the `binding.gyp` file. For this project, you'll see this code in the binding.gyp: + +```json +'include_dirs' : [ ' Node-API is stable in all currently supported Node.js releases. Use an [Active LTS or Maintenance LTS release](https://nodejs.org/en/about/releases/) for the best experience. You can check the version of Node.js you are running with `node -v`. + +## Directory layout + +``` +. +├── binding.gyp # tells node-gyp how to compile the C/C++ source +├── build/ # compiled output (generated) +├── lib/ +│ └── binding.js # JavaScript layer that loads the compiled binary +├── node_modules/ +├── src/ +│ └── *.cc / *.h # your C/C++ implementation +├── test/ +│ └── *.js # test code +├── package.json +└── package-lock.json +``` + +## package.json + +Two entries in `package.json` are specific to native addons. + +### `node-addon-api` dependency + +```json +"dependencies": { + "node-addon-api": "^8.0.0" +} +``` + +[`node-addon-api`](https://github.com/nodejs/node-addon-api) adds a C++ wrapper to the C API built into Node.js. It makes creating and manipulating JavaScript objects in C++ straightforward and is useful even when the underlying library you are wrapping is written in C. + +### `"gypfile": true` + +```json +"gypfile": true +``` + +This tells npm that the package requires a native compilation step. When npm sees this entry it automatically invokes its bundled copy of `node-gyp`, which reads `binding.gyp` to build the binary. + +## binding.gyp + +`binding.gyp` is a [GYP](https://gyp.gsrc.io/) file that describes how to compile and link your C/C++ code. It must be named exactly `binding.gyp`. + +[GYP](https://gyp.gsrc.io/) (Generate Your Projects) lets you write a single build description that works on Windows, macOS, and Linux. [`node-gyp`](https://github.com/nodejs/node-gyp) reads this file and produces the platform-appropriate build files (MSVC project on Windows, Makefile on Linux, Xcode project on macOS), then invokes the compiler. + +A minimal `binding.gyp` for a `node-addon-api` project looks like this: + +```json +{ + "targets": [ + { + "target_name": "my_addon", + "sources": ["src/my_addon.cc"], + "include_dirs": [" If you install packages globally with npm and encounter `Error: EACCES: permission denied`, use [`nvm`](https://github.com/nvm-sh/nvm) to manage your Node.js installation. With nvm, global installs go into your home directory and never require `sudo`. diff --git a/pages/napi/getting-started/tools.md b/pages/napi/getting-started/tools.md new file mode 100644 index 0000000..67544a4 --- /dev/null +++ b/pages/napi/getting-started/tools.md @@ -0,0 +1,116 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Tools + +## Node.js + +[Node.js](https://nodejs.org/) is the runtime that executes JavaScript on your machine. It bundles the [V8 JavaScript engine](https://developers.google.com/v8/) together with a set of built-in modules that let you run JavaScript outside of a browser. + +Download the appropriate installer for your platform from the [Node.js downloads page](https://nodejs.org/en/download/). The **LTS (Long Term Support)** release is the most stable choice and is recommended for addon development. The installer includes npm. + +> Node-API is stable in all currently supported Node.js releases. Any Active LTS or Maintenance LTS release will work. + +## npm + +[npm](https://www.npmjs.com) is the package manager for Node.js. It is installed alongside Node.js. For most Node-API developers, the goal is to publish an npm package that wraps a C/C++ library and makes it available to JavaScript users. + +npm is included with Node.js. You can keep it up to date with: + +```bash +npm install -g npm@latest +``` + +## Git + +[Git](https://git-scm.com) is not strictly required for Node-API work, but it is used throughout the ecosystem. Most example repositories and dependency installations rely on it. + +## C/C++ compiler and Python + +In addition to Node and npm, you need a C/C++ compiler toolchain and Python (required by [node-gyp](/learn/napi/build-tools/node-gyp.md). + +### Windows + +The recommended approach is to install [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2022) with the **"Desktop development with C++"** workload selected. This provides the MSVC compiler, Windows SDK, and the build infrastructure node-gyp requires. + +Alternatively, you can install via [winget](https://learn.microsoft.com/en-us/windows/package-manager/winget/): + +```powershell +winget install Microsoft.VisualStudio.2022.BuildTools +``` + +During or after installation, open Visual Studio Installer and ensure the **"Desktop development with C++"** workload is checked. + +Python 3 is also required. Install it from [python.org](https://www.python.org/downloads/windows/) or via winget: + +```powershell +winget install Python.Python.3 +``` + +> Run PowerShell or `cmd.exe` as **Administrator** when installing global tools. + +### macOS + +Install Apple's command-line developer tools. If you haven't installed Xcode already, the quickest route is: + +```bash +xcode-select --install +``` + +If that fails, install the full [Xcode IDE](https://developer.apple.com/xcode/) from the Mac App Store, which includes the necessary compiler toolchain. + +Python 3 is **not** bundled with modern macOS. Install it using [Homebrew](https://brew.sh): + +```bash +brew install python3 +``` + +Or download an installer from [python.org](https://www.python.org/downloads/macos/). + +### Linux + +On most Linux distributions the required C/C++ toolchain and Python are either pre-installed or easily added. For Debian/Ubuntu-based systems: + +```bash +sudo apt-get update +sudo apt-get install -y build-essential python3 +``` + +For other distributions, refer to your package manager's documentation or the [LLVM installation guide](https://llvm.org/docs/GettingStarted.html). + +## Verifying your tools + +After installation, verify each tool is on your PATH. + +### macOS and Linux + +```bash +node --version +npm --version +python3 --version +git --version +cc --version +make --version +``` + +### Windows (PowerShell) + +```powershell +node --version +npm --version +python --version +git --version +``` + +To confirm the MSVC compiler is available, open a **Developer Command Prompt** (installed with Visual Studio Build Tools) and run: + +```bash +cl +``` + +## Other tools + +You'll need a shell - Terminal on macOS/Linux, PowerShell or Windows Terminal on Windows. + +A capable code editor is strongly recommended. [Visual Studio Code](https://code.visualstudio.com) has excellent C/C++ support via the [C/C++ extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools) and integrates well with node-gyp builds. [CLion](https://www.jetbrains.com/clion/) is another popular choice for C++ development. diff --git a/pages/napi/getting-started/your-first-project.md b/pages/napi/getting-started/your-first-project.md new file mode 100644 index 0000000..b4c1ee5 --- /dev/null +++ b/pages/napi/getting-started/your-first-project.md @@ -0,0 +1,65 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Your First Project + +Before you start, make sure you've got all the necessary [prerequisites](/learn/napi/getting-started/prerequisites.md) and [tools](/learn/napi/getting-started/tools.md) installed, and read [Anatomy of a Node-API project](/learn/napi/getting-started/project-structure.md) to understand the common layout and configuration files shared by all `node-addon-api` projects. + +This tutorial uses `node-addon-api` at the C++ wrapper level. + +## Creating a project + +The quickest way to get a working project is to copy the Hello World example from the [node-addon-examples](https://github.com/nodejs/node-addon-examples) repository: + +```bash +git clone https://github.com/nodejs/node-addon-examples.git +cp -r node-addon-examples/src/1-getting-started/a-first-project/node-addon-api hello-world +cd hello-world +npm install +``` + +Alternatively, set up the project manually: + +```bash +mkdir hello-world +cd hello-world +npm init -y +npm install node-addon-api +``` + +Then create the source files described below. Once the project is set up, verify everything works: + +```bash +npm test +``` + +## src/hello_world.cc + +[`hello_world.cc`](https://github.com/nodejs/node-addon-examples/blob/main/src/1-getting-started/a-first-project/node-addon-api/src/hello_world.cc) is perhaps the simplest useful Node-API file you can write. + +The file defines a C++ `Method` function that takes a single `Napi::CallbackInfo&` argument. This `info` object provides access to the JavaScript environment, including any arguments passed in from JavaScript. + +> `info` behaves like an array of JavaScript arguments. + +`Method` uses `info` to obtain a `Napi::Env`, then creates and returns a `Napi::String` with the value `"world"`. + +The `Init` function registers the single export from this module: the name `"HelloWorld"` maps to the `Method` function. + +The `NODE_API_MODULE` macro at the bottom ensures `Init` is called when the module is loaded. + +## lib/binding.js + +[`binding.js`](https://github.com/nodejs/node-addon-examples/blob/main/src/1-getting-started/a-first-project/node-addon-api/lib/binding.js) loads the compiled binary and re-exports its contents. The sole export from the binary is the `HelloWorld` function. + +## test/test_binding.js + +[`test_binding.js`](https://github.com/nodejs/node-addon-examples/blob/main/src/1-getting-started/a-first-project/node-addon-api/test/test_binding.js) uses `require` to load the `HelloWorld` function from `binding.js`. The `testBasic` function calls it and verifies the result. + +## Conclusion + +This project demonstrates a minimal Node-API module that exports a single function. Some things to try next: + +- Run `test_binding.js` in your debugger. Step through the code and observe what visibility you have into the JavaScript object created by the C++ code. +- Modify `test_binding.js` to require the compiled binary directly instead of going through `binding.js`. Step through in the debugger and note the difference. +- Modify `hello_world.cc` to read arguments passed from JavaScript. The [`node-addon-api` examples](https://github.com/nodejs/node-addon-api#examples) are a good reference. diff --git a/pages/napi/index.md b/pages/napi/index.md new file mode 100644 index 0000000..1bb4120 --- /dev/null +++ b/pages/napi/index.md @@ -0,0 +1,38 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Node-API + +[Node-API](https://nodejs.org/api/n-api.html) is a stable C API built into Node.js that lets C/C++ code create, read, and manipulate JavaScript values as if they were created by JavaScript itself. It was introduced experimentally in Node.js 8.0.0 and became stable (no longer behind a flag) in the Node.js 8/10 timeframe. + +Because Node-API is part of Node.js itself, it requires no additional installation. + +## Why Node-API instead of NAN? + +Before Node-API, the dominant approach for native addon development was [Native Abstractions for Node.js (NAN)](https://github.com/nodejs/nan). NAN works by calling directly into the [V8](https://developers.google.com/v8/) API, which changes with every major V8 release. This means NAN-based addons often need to be recompiled - and sometimes updated - when Node.js ships a new V8 version. + +Node-API abstracts the JavaScript engine behind a stable interface. This gives you two important guarantees: + +1. **Backward compatibility** - a module built today will run on all future versions of Node.js without any recompilation or code changes. +2. **ABI stability** - because the API is ABI-stable, the compiled `.node` binary itself continues to work across Node.js releases. + +## Common use cases + +### Wrapping an existing C/C++ library + +The most common reason to write a native addon is to expose an existing C/C++ library to JavaScript developers. Node-API lets you maintain your C/C++ code independently and keep the JavaScript bindings in sync with minimal effort. + +### Accessing OS resources + +Some applications - such as those built with [Electron](https://electronjs.org) or [NW.js](https://nwjs.io) - need access to system APIs that Node.js does not expose. Node-API provides the hooks to reach those resources. + +### Computationally intensive work + +For compute-heavy tasks, writing the hot path in C or C++ and calling it from JavaScript via Node-API can yield significant performance gains. Unlike JIT-compiled JavaScript, the compiled C/C++ binary is available to Node.js immediately without a warm-up phase. + +## `node-addon-api` + +An important companion to Node-API is the npm package [`node-addon-api`](https://www.npmjs.com/package/node-addon-api). It wraps the C API in an idiomatic C++ object model, reducing boilerplate and making common patterns - like wrapping C++ objects as JavaScript objects - much more ergonomic. It retains the full ABI-stability guarantee of Node-API. + +Most of the examples on this site use `node-addon-api`. diff --git a/pages/napi/special-topics/asyncworker.md b/pages/napi/special-topics/asyncworker.md new file mode 100644 index 0000000..e7c52d3 --- /dev/null +++ b/pages/napi/special-topics/asyncworker.md @@ -0,0 +1,68 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# AsyncWorker + +You may have a project in which you have a piece of long-running C/C++ code that you want to run in the background instead of on Node's main event loop. Node-API's [`AsyncWorker`](https://github.com/nodejs/node-addon-api/blob/main/doc/async_worker.md) class is designed specifically for this case. + +As a programmer, your job is essentially to subclass `AsyncWorker` and to implement the `Execute` method. You'll also probably implement a wrapper function to make using your `AsyncWorker` easier. + +In this example, we're going to create a `SimpleAsyncWorker` class that subclasses `AsyncWorker`. The worker will take an integer value indicating the length of time it is to "work." When the worker completes, it will return a text string indicating how long it worked. In one case, the worker will indicate an error instead. + +## SimpleAsyncWorker + +[**SimpleAsyncWorker.h**](https://github.com/nodejs/node-addon-examples/blob/main/src/5-async-work/napi-asyncworker-example/node-addon-api/src/SimpleAsyncWorker.h) is the C++ header for `SimpleAsyncWorker`. It declares a constructor that takes as an argument the length of time (in seconds) the `SimpleAsyncWorker` is to run. A private data member is declared to hold this value. + +The header also declares two methods, `Execute` and `OnOK`, which override methods declared by `AsyncWorker`, and are described in more detail below. + +[**SimpleAsyncWorker.cc**](https://github.com/nodejs/node-addon-examples/blob/main/src/5-async-work/napi-asyncworker-example/node-addon-api/src/SimpleAsyncWorker.cc) is the C++ implementation. The constructor takes two arguments. `callback` is the JavaScript function that gets called when the `Execute` method returns. `callback` gets called whether there was an error or not. The second constructor argument, `runTime`, is an integer value indicating how long (in seconds) the worker is to run. + +Node will run the code of the `Execute` method in a thread separate from the thread running Node's main event loop. The `Execute` method _has no access_ to any part of the Node-API environment. For this reason, the `runTime` input value was stored by the constructor as a private data member. + +In this implementation, the `Execute` method simply waits the number of seconds specified earlier by `runTime`. This is where the long running code goes in a real implementation. To demonstrate how error handling works, this `Execute` method declares an error when requested to run 4 seconds. + +The `OnOK` method is called after the `Execute` method returns unless the `Execute` method calls `SetError` or in the case where C++ exceptions are enabled and an exception is thrown. In the case of an error, the `OnError` method is called instead of `OnOK`. The default `OnError` implementation simply calls the `AsyncWorker` callback function with the error as the only argument. + +In this implementation, the `OnOK` method formulates a string value and passes it as the _second_ argument to the `callback` function specified in the constructor. The first argument passed to the `callback` function is a JavaScript `null` value. The reason for this is that a single callback function is called whether an error occurs or not. The default `OnError` method, which `SimpleAsyncWorker` does not override, passes the error as the first argument to the callback. This will become more clear in the next section. + +Note that unlike `Execute`, the `OnOK` and `OnError` methods _do_ have access to the Node-API environment. + +## RunSimpleAsyncWorker + +We need a C++ function that instantiates `SimpleAsyncWorker` objects and requests them to be queued. This function needs to be registered with Node-API so that it is accessible from the JavaScript code. [**RunSimpleAsyncWorker.cc**](https://github.com/nodejs/node-addon-examples/blob/main/src/5-async-work/napi-asyncworker-example/node-addon-api/src/RunSimpleAsyncWorker.cc) provides this wrapper. + +The `runSimpleAsyncWorker` function, which is accessible from JavaScript, takes two arguments which are passed through the `info` argument. The first argument, which is passed as `info[0]`, is the `runTime` and the second argument is the JavaScript callback function which gets called when the `Execute` method returns. + +The code then instantiates a `SimpleAsyncWorker` object and requests that it be queued for possible execution on the next tick. Unless the `SimpleAsyncWorker` object is queued, its `Execute` method will never be called. + +Once the `SimpleAsyncWorker` object is queued, `runSimpleAsyncWorker` formulates a text string and returns it to the caller. + +## Running in JavaScript + +[**Test.js**](https://github.com/nodejs/node-addon-examples/blob/main/src/5-async-work/napi-asyncworker-example/node-addon-api/test/Test.js) is a simple JavaScript program that shows how to run `SimpleAsyncWorker` instances. It calls `runSimpleAsyncWorker` three times, each with a different `runTime` parameter. Each call specifies `AsyncWorkerCompletion` as the callback function. + +The `AsyncWorkerCompletion` function is coded to handle the cases where the `Execute` method reports an error and when it does not. It simply logs to the console when it's called. + +Here's what the output looks like when the JavaScript successfully runs: + +``` +runSimpleAsyncWorker returned 'SimpleAsyncWorker for 2 seconds queued.'. +runSimpleAsyncWorker returned 'SimpleAsyncWorker for 4 seconds queued.'. +runSimpleAsyncWorker returned 'SimpleAsyncWorker for 8 seconds queued.'. +SimpleAsyncWorker returned 'SimpleAsyncWorker returning after 'working' 2 seconds.'. +SimpleAsyncWorker returned an error: [Error: Oops! Failed after 'working' 4 seconds.] +SimpleAsyncWorker returned 'SimpleAsyncWorker returning after 'working' 8 seconds.'. +``` + +As expected, each call to `runSimpleAsyncWorker` immediately returns. The `AsyncWorkerCompletion` function gets called when each `SimpleAsyncWorker` completes. + +## Caveats + +- It is _absolutely essential_ that the `Execute` method makes no Node-API calls. This means that the `Execute` method has _no access_ to any input values passed by the JavaScript code. + + Typically, the `AsyncWorker` class constructor will collect the information it needs from the JavaScript objects and store _copies_ of that information as data members. The results of the `Execute` method can then be turned back into JavaScript objects in the `OnOK` method. + +- The Node process is aware of all running `Execute` methods and will not terminate until all running `Execute` methods have returned. + +- An AsyncWorker can be safely terminated with a call to `AsyncWorker::Cancel` from the main thread. diff --git a/pages/napi/special-topics/context-awareness.md b/pages/napi/special-topics/context-awareness.md new file mode 100644 index 0000000..3d6805d --- /dev/null +++ b/pages/napi/special-topics/context-awareness.md @@ -0,0 +1,280 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Context Awareness + +Node.js has historically run as a single-threaded process. This all changed with the introduction of [Worker Threads](https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads) in Node 10. Worker Threads add a JavaScript-friendly concurrency abstraction that native add-on developers need to be aware of. What this means practically is that your native add-on may be loaded and unloaded more than once and its code may be executed concurrently in multiple threads. There are specific steps you must take to ensure your native add-on code runs correctly. + +The Worker Thread model specifies that each Worker runs completely independently of each other and communicate to the parent Worker using a MessagePort object supplied by the parent. This makes the Worker Threads essentially isolated from one another. The same is true for your native add-on. + +Each Worker Thread operates within its own environment which is also referred to as a context. The context is available to each Node-API function as an [`napi_env`](https://nodejs.org/api/n-api.html#napi_env) value. + +## Multiple loading and unloading + +If your native add-on requires persistent memory, allocating this memory in static global space is a recipe for disaster. Instead, it is _essential_ that this memory is allocated each time within the context in which the native add-on is initialized. This memory is typically allocated in your native add-on's `Init` method. But in some cases it can also be allocated as your native add-on is running. + +In addition to the multiple loading described above, your native add-on is also subject to automatic unloading by the JavaScript runtime engine's garbage collector when your native add-on is no longer in use. To prevent memory leaks, any memory your native add-on has allocated _must_ be freed when you native add-on is unloaded. + +The next sections describe two different techniques you can use to allocate and free persistent memory associated with your native add-on. The techniques may be used individually or together in your native add-on. + +## Instance data + +Node-API gives you the ability to associate a single piece of memory your native add-on allocates with the context under which it is running. This technique is called "instance data" and is useful when your native add-on allocates a single piece of data when its loaded. + +The `napi_set_instance_data` allows your native add-on to associate a single piece of allocated memory with the context under which your native add-on is loaded. The `napi_get_instance_data` can then be called anywhere in you native add-on to retrieve the location of the memory that was allocated. + +You specify a finalizer callback in your `napi_set_instance_data` call. The finalizer callback gets called when your native add-on is released from memory and is where you should release the memory associated with this context. + +### Resources + +[Environment Life Cycle APIs](https://nodejs.org/api/n-api.html#environment-life-cycle-apis) Node.js documentation for `napi_set_instance_data` and `napi_get_instance_data`. + +### Example + +In this example, a number of Worker Threads are created. Each Worker Thread creates an `AddonData` struct that is tied to the Worker Thread's context using a call to `napi_set_instance_data`. Over time, the value held in the struct is incremented and decremented using a computationally expensive operation. + +In time, the Worker Threads complete their operations at which time the allocated struct is freed in the `DeleteAddonData` function. + +#### binding.c + +```c +#include +#include +#include + +#define NAPI_EXPERIMENTAL +#include + +// Structure containing information needed for as long as the addon exists. It +// replaces the use of global static data with per-addon-instance data by +// associating an instance of this structure with each instance of this addon +// during addon initialization. The instance of this structure is then passed to +// each binding the addon provides. Thus, the data stored in an instance of this +// structure is available to each binding, just as global static data would be. +typedef struct { + double value; +} AddonData; + +// This is the actual, useful work performed: increment or decrement the value +// stored per addon instance after passing it through a CPU-consuming but +// otherwise useless calculation. +static int ModifyAddonData(AddonData* data, double offset) { + // Expensively increment or decrement the value. + data->value = tan(atan(exp(log(sqrt(data->value * data->value))))) + offset; + + // Round the value to the nearest integer. + data->value = + (double)(((int)data->value) + + (data->value - ((double)(int)data->value) > 0.5 ? 1 : 0)); + + // Return the value as an integer. + return (int)(data->value); +} + +// This is boilerplate. The instance of the `AddonData` structure created during +// addon initialization must be destroyed when the addon is unloaded. This +// function will be called when the addon's `exports` object is garbage collected. +static void DeleteAddonData(napi_env env, void* data, void* hint) { + // Avoid unused parameter warnings. + (void) env; + (void) hint; + + // Free the per-addon-instance data. + free(data); +} + +// This is also boilerplate. It creates and initializes an instance of the +// `AddonData` structure and ties its lifecycle to that of the addon instance's +// `exports` object. This means that the data will be available to this instance +// of the addon for as long as the JavaScript engine keeps it alive. +static AddonData* CreateAddonData(napi_env env, napi_value exports) { + AddonData* result = malloc(sizeof(*result)); + result->value = 0.0; + assert(napi_set_instance_data(env, result, DeleteAddonData, NULL) == napi_ok); + return result; +} + +// This function is called from JavaScript. It uses an expensive operation to +// increment the value stored inside the `AddonData` structure by one. +static napi_value Increment(napi_env env, napi_callback_info info) { + // Retrieve the per-addon-instance data. + AddonData* addon_data = NULL; + assert(napi_get_instance_data(env, ((void**)&addon_data)) == napi_ok); + + // Increment the per-addon-instance value and create a new JavaScript integer + // from it. + napi_value result; + assert(napi_create_int32(env, + ModifyAddonData(addon_data, 1.0), + &result) == napi_ok); + + // Return the JavaScript integer back to JavaScript. + return result; +} + +// This function is called from JavaScript. It uses an expensive operation to +// decrement the value stored inside the `AddonData` structure by one. +static napi_value Decrement(napi_env env, napi_callback_info info) { + // Retrieve the per-addon-instance data. + AddonData* addon_data = NULL; + assert(napi_get_instance_data(env, ((void**)&addon_data)) == napi_ok); + + // Decrement the per-addon-instance value and create a new JavaScript integer + // from it. + napi_value result; + assert(napi_create_int32(env, + ModifyAddonData(addon_data, -1.0), + &result) == napi_ok); + + // Return the JavaScript integer back to JavaScript. + return result; +} + +// Initialize the addon in such a way that it may be initialized multiple times +// per process. The function body following this macro is provided the value +// `env` which has type `napi_env` and the value `exports` which has type +// `napi_value` and which refers to a JavaScript object that ultimately contains +// the functions this addon wishes to expose. At the end, it must return a +// `napi_value`. It may return `exports`, or it may create a new `napi_value` +// and return that instead. +NAPI_MODULE_INIT(/*env, exports*/) { + // Create a new instance of the per-instance-data that will be associated with + // the instance of the addon being initialized here and that will be destroyed + // along with the instance of the addon. + AddonData* addon_data = CreateAddonData(env, exports); + + // Declare the bindings this addon provides. The data created above is given + // as the last initializer parameter, and will be given to the binding when it + // is called. + napi_property_descriptor bindings[] = { + {"increment", NULL, Increment, NULL, NULL, NULL, napi_enumerable, addon_data}, + {"decrement", NULL, Decrement, NULL, NULL, NULL, napi_enumerable, addon_data} + }; + + // Expose the two bindings declared above to JavaScript. + assert(napi_define_properties(env, + exports, + sizeof(bindings) / sizeof(bindings[0]), + bindings) == napi_ok); + + // Return the `exports` object provided. It now has two new properties, which + // are the functions we wish to expose to JavaScript. + return exports; +} +``` + +#### index.js + +```cjs +// Example illustrating the case where a native addon is loaded multiple times. +// This entire file is executed twice, concurrently - once on the main thread, +// and once on a thread launched from the main thread. + +// We load the worker threads module, which allows us to launch multiple Node.js +// environments, each in its own thread. +const { Worker, isMainThread } = require('worker_threads'); + +// We load the native addon. +const addon = require('bindings')('multiple_load'); + +// The iteration count can be tweaked to ensure that the output from the two +// threads is interleaved. Too few iterations and the output of one thread +// follows the output of the other, not really illustrating the concurrency. +const iterations = 1000; + +// This function is an idle loop that performs a random walk from 0 by calling +// into the native addon to either increment or decrement the initial value. +function useAddon(addon, prefix, iterations) { + if (iterations >= 0) { + if (Math.random() < 0.5) { + console.log(prefix + ': new value (decremented): ' + addon.decrement()); + } else { + console.log(prefix + ': new value (incremented): ' + addon.increment()); + } + setImmediate(() => useAddon(addon, prefix, --iterations)); + } +} + +if (isMainThread) { + // On the main thread, we launch a worker and wait for it to come online. Then + // we start the loop. + new Worker(__filename).on('online', () => + useAddon(addon, 'Main thread', iterations) + ); +} else { + // On the secondary thread we immediately start the loop. + useAddon(addon, 'Worker thread', iterations); +} +``` + +## Cleanup hooks + +Your native add-on can receive one or more notifications from the Node.js runtime engine when the context in which your native-add-on has been running is being destroyed. This gives your native add-on the opportunity to release any allocated memory before the context is destroyed by the Node.js runtime engine. + +The advantage of this technique is that your native add-on can allocate multiple pieces of memory to be associated with the context under which your native add-on is running. This can be useful if you need to allocate multiple memory buffers from different pieces of code as your native add-on is running. + +The drawback is that if you need to access these allocated buffer you are responsible for keeping track of the pointers yourself within the context your native add-on is running. Depending upon the architecture of your native add-on, this may or may not be an issue. + +### Resources + +[Cleanup on exit of the current Node.js instance](https://nodejs.org/api/n-api.html#cleanup-on-exit-of-the-current-nodejs-instance) Node.js documentation for `napi_add_env_cleanup_hook` and `napi_remove_env_cleanup_hook`. + +### Example + +Because keeping track of the allocated buffers is dependent upon the architecture of the native add-on, this is a trivial example showing how the buffers can be allocated and released. + +#### binding.c + +```c +#include +#include +#include "node_api.h" + +namespace { + +void CleanupHook (void* arg) { + printf("cleanup(%d)\n", *static_cast(arg)); + free(arg); +} + +napi_value Init(napi_env env, napi_value exports) { + for (int i = 1; i < 5; i++) { + int* value = (int*)malloc(sizeof(*value)); + *value = i; + napi_add_env_cleanup_hook(env, CleanupHook, value); + } + return exports; +} + +} // anonymous namespace + +NAPI_MODULE(NODE_GYP_MODULE_NAME, Init) +``` + +#### index.js + +```cjs +'use strict'; + +// We load the native addon. +const addon = require('bindings')('multiple_load'); +const assert = require('assert'); +const child_process = require('child_process'); + +assert.ok(addon); + +if (process.argv[2] === 'child') { + const childAddon = require('bindings')('multiple_load'); + assert.ok(childAddon); + process.exit(0); +} + +const child = child_process.fork(__filename, ['child'], { + stdio: 'inherit', +}); + +child.on('exit', code => { + assert.strictEqual(code, 0); +}); +``` diff --git a/pages/napi/special-topics/index.md b/pages/napi/special-topics/index.md new file mode 100644 index 0000000..b27234d --- /dev/null +++ b/pages/napi/special-topics/index.md @@ -0,0 +1,23 @@ +--- +authors: avivkeller +--- + +# Special Topics + +This section covers advanced Node-API patterns that you will encounter once your native addon grows beyond a simple synchronous function call. + +## Topics covered + +- [Object and function references](/learn/napi/special-topics/object-function-refs.md) - using `ObjectReference` and `FunctionReference` to keep JavaScript objects alive across call boundaries so the garbage collector does not reclaim them prematurely +- [AsyncWorker](/learn/napi/special-topics/asyncworker.md) - running long-running C/C++ operations on a background thread using `node-addon-api`'s `AsyncWorker` class, keeping Node's event loop unblocked +- [Thread-safe functions](/learn/napi/special-topics/thread-safe-functions.md) - calling back into JavaScript from native threads that are not the main Node.js thread, using the thread-safe function API +- [Context awareness](/learn/napi/special-topics/context-awareness.md) - writing addons that load and unload correctly in the presence of [Worker Threads](https://nodejs.org/api/worker_threads.html), including instance data and cleanup hooks + +## When do these topics apply? + +| Topic | When you need it | +| -------------------------- | ----------------------------------------------------------------------------------- | +| Object/function references | Storing a JS callback or object in a C++ data member that outlives the current call | +| AsyncWorker | Offloading CPU-bound or blocking I/O work off the main thread | +| Thread-safe functions | Invoking JS callbacks from native threads you manage yourself | +| Context awareness | Your addon is used in Worker Thread environments or loaded/unloaded multiple times | diff --git a/pages/napi/special-topics/object-function-refs.md b/pages/napi/special-topics/object-function-refs.md new file mode 100644 index 0000000..a59c75d --- /dev/null +++ b/pages/napi/special-topics/object-function-refs.md @@ -0,0 +1,62 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Object and Function References + +JavaScript implements a dynamic memory model. When objects are no longer reachable, they become eligible for reclamation by the garbage collector running in the background. + +There are situations when you need to ensure that objects created by your Node-API code remain allocated. In this case you need to explicitly create a reference to them. This is the purpose of the `ObjectReference` and `FunctionReference` classes. + +## Persistent Reference + +Object and function references can be instantiated as either `Weak` or `Persistent`. + +A `Persistent` reference initializes the internal reference count to one which prevents reclamation of the object's memory by the garbage collector. The referenced object will remain in memory for the life of the `Persistent` reference. + +Using a `Persistent` reference makes sense in the case where the duration of the reference is known ahead of time. The internal reference count is decremented when the `Persistent` reference is deleted. This will make the referenced object eligible for deletion if the internal reference count goes to zero. + +The most common case for using a `Persistent` reference is when you create a JavaScript class in your Node-API code and you need to insure its constructor remains allocated by the JavaScript runtime engine. + +## Weak Reference + +For more complex implementations where multiple AsyncWorkers rely on the referenced object, it may make better sense to use a `Weak` reference which initializes the internal reference count to zero. The reference count can then be maintained using the `Reference` `Ref` and `Unref` methods. + +The most common use case for a `Weak` reference is when your Node-API code needs to monitor when a JavaScript object you've created in your Node-API code is released by the garbage collector. + +## ObjectReference + +The `ObjectReference` class inherits from the `Reference` class. The value it adds is a collection of `Get` and `Set` methods that manipulate the referenced object's properties. + +## FunctionReference + +Like the `ObjectReference`, the `FunctionReference` class inherits from the `Reference` class. While the `ObjectReference` class adds the `Get` and `Set` methods, the `FunctionReference` class adds a set of `Call` methods that implement calls to the function. + +## Example + +This example code shows how to use the `FunctionReference` class. + +### src/native-addon.h + +[**native-addon.h**](https://github.com/nodejs/node-addon-examples/blob/main/src/4-references-and-handle-scope/function-reference-demo/node-addon-api/src/native-addon.h) declares the `NativeAddon` C++ class, which has two data members populated in the constructor: + +- `Napi::FunctionReference jsFnRef` +- `Napi::Function jsFn` + +### src/native-addon.cc + +[**native-addon.cc**](https://github.com/nodejs/node-addon-examples/blob/main/src/4-references-and-handle-scope/function-reference-demo/node-addon-api/src/native-addon.cc) contains the `NativeAddon` implementation. The constructor, which is called from JavaScript, takes two function arguments. The first argument is stored as a `Napi::FunctionReference` and the second is stored as a `Napi::Function`. + +> _There is a deliberate error in this code._ + +The second function is stored in the `Napi::Function jsFn` data member. **This is an error** because the lifetime of the second argument is limited to the lifetime of the constructor. The value of the `jsFn` data member will be invalid after the constructor returns. The first argument is stored in the `Napi::FunctionReference jsFnRef`. Because of the use of the `Napi::FunctionReference`, the value of `jsFnRef` will remain valid after the constructor returns. + +The `NativeAddon` class implements two methods which can be called from JavaScript: `TryCallByStoredReference` and `TryCallByStoredFunction`. Notice that the `Call` method is used the same way for both the `jsFnRef` and `jsFn` data members. + +### src/binding.cc + +[**binding.cc**](https://github.com/nodejs/node-addon-examples/blob/main/src/4-references-and-handle-scope/function-reference-demo/node-addon-api/src/binding.cc) is a standard binding file that registers the `NativeAddon` class with Node-API. + +### index.js + +[**index.js**](https://github.com/nodejs/node-addon-examples/blob/main/src/4-references-and-handle-scope/function-reference-demo/node-addon-api/index.js) shows the use of the `NativeAddon` class from JavaScript. Note that the call to the native `tryCallByStoredFunction` method fails because the data member on which it relies is not valid. diff --git a/pages/napi/special-topics/thread-safe-functions.md b/pages/napi/special-topics/thread-safe-functions.md new file mode 100644 index 0000000..c805466 --- /dev/null +++ b/pages/napi/special-topics/thread-safe-functions.md @@ -0,0 +1,86 @@ +--- +authors: gabrielschulhof, NickNaso, jschlight, mhdawson, KevinEady, avivkeller +--- + +# Thread-Safe Functions + +JavaScript functions can normally only be called from a native addon's main thread. If an addon creates additional threads, then node-addon-api functions that require a `Napi::Env`, `Napi::Value`, or `Napi::Reference` must not be called from those threads. + +When an addon has additional threads and JavaScript functions need to be invoked based on the processing completed by those threads, those threads must communicate with the addon's main thread so that the main thread can invoke the JavaScript function on their behalf. The thread-safe function APIs provide an easy way to do this. + +A thread-safe function is created on the main thread via [ThreadSafeFunction::New](https://github.com/nodejs/node-addon-api/blob/main/doc/threadsafe_function.md#new): + +```cpp +New(napi_env env, + const Function& callback, + const Object& resource, + ResourceString resourceName, + size_t maxQueueSize, + size_t initialThreadCount, + ContextType* context, + Finalizer finalizeCallback, + FinalizerDataType* data); +``` + +A thread-safe function encapsulates: + +- Message queue: Requests to run the JavaScript function are placed in a queue, processed asynchronously by the main thread. The amount of entries allowed in the queue before returning a "queue full" error on `NonBlockingCall()` is controlled via the `maxQueueSize` parameter (specify `0` for unlimited queue) +- JavaScript function: Callback to run (`callback` parameter). This function is either (a) automatically ran with no arguments when called via the no-argument `[Non]BlockingCall()` overloads, or (b) passed as an argument to the callback function provided in the `[Non]BlockingCall(DataType* data, Callback callback)` overloads. +- Context: Optional, arbitrary data (`context` parameter) to associate with the thread-safe function. +- Finalizer: Optional callback (`finalizeCallback` parameter) to run at destruction of the thread-safe function, when all threads have finished using it. +- Finalizer data: Optional data (`data` parameter) to provide to the finalizer callback. + +## Calling the Thread-Safe Function + +Threads may call into JavaScript via [`[Non]BlockingCall`](https://github.com/nodejs/node-addon-api/blob/main/doc/threadsafe_function.md#blockingcall--nonblockingcall). This will add an entry to the underlying thread-safe function's queue, to be handled asynchronously on the main thread during its processing of the event loop. + +## Thread Management + +Multiple threads can utilize the thread-safe function simultaneously. The thread-safe function manages its lifecycle through counting the number of threads actively utilizing it. This number starts at the initial thread count parameter in `New()`, increased via `Acquire()`, and decreased via `Release()`. Once the number of active threads reaches zero, the thread-safe function is destroyed, running the finalizer callback on the main thread if provided. + +Here are two general approaches to using thread-safe functions within applications: + +### Known Number of Threads + +If the amount of threads is known at thread-safe function creation, set the `initial_thread_count` parameter to this number in the call to `New()`. Each thread will have its own access to the thread-safe function until it calls `Release()`. Once all threads have made a call to `Release()`, the thread-safe function is destroyed. + +### Creating Threads + +Another common use-case is to dynamically create and destroy threads based on various logic at run-time. One way to handle this scenario is to expose several native JavaScript functions that interact with the thread-safe function APIs by: + +1. Creating a thread-safe function via `New()` with initial thread count of `1`. +2. Calling `Acquire()` and creating a new native thread. The new thread can now use `[Non]BlockingCall()`. +3. Initiating cleanup/destruction, for example by ... + +- calling `Abort()` and have each thread either call `[Non]BlockingCall()` or `Release()` +- using custom logic with other thread-safe APIs to ensure that all threads call `Release()` in order to decrease the active thread count to `0`. + +## Example + +This example exposes a single function that creates a thread-safe function and a native thread. The function returns a promise that resolves after the native thread calls into JavaScript ten times. The example consists of three source files: [**binding.gyp**](https://github.com/nodejs/node-addon-examples/blob/main/src/6-threadsafe-function/thread_safe_function_counting/node-addon-api/binding.gyp) configures the build, [**addon.cc**](https://github.com/nodejs/node-addon-examples/blob/main/src/6-threadsafe-function/thread_safe_function_counting/node-addon-api/addon.cc) implements the native module, and [**addon.js**](https://github.com/nodejs/node-addon-examples/blob/main/src/6-threadsafe-function/thread_safe_function_counting/node-addon-api/addon.js) exercises it from JavaScript. + +Running `addon.js` produces output similar to: + +``` +2019-11-25T22:14:56.175Z 0 +2019-11-25T22:14:56.380Z 1 +2019-11-25T22:14:56.582Z 2 +2019-11-25T22:14:56.787Z 3 +2019-11-25T22:14:56.987Z 4 +2019-11-25T22:14:57.187Z 5 +2019-11-25T22:14:57.388Z 6 +2019-11-25T22:14:57.591Z 7 +2019-11-25T22:14:57.796Z 8 +2019-11-25T22:14:58.001Z 9 +true +``` + +## Frequently Asked Questions + +### Q: My application isn't exiting correctly. It just hangs. + +By default, Node will wait until a thread-safe function is finalized before cleaning up and exiting. See [Thread Management](#Thread-Management). This behavior can be changed via a call to `Unref()`, permitting Node to clean up without waiting for the thread count to reach zero. A call to `Ref()` will return the threadsafe function to the previous exit behavior, requiring it to be `Release()`ed and/or `Abort()`ed by all threads utilizing it. + +### Q: If a thread receives `napi_closing` from a call to `[Non]BlockingCall()`, does it still need to call `Release()`? + +No. A return value of `napi_closing` should signify to the thread that the thread-safe function can no longer be utilized. This _includes_ the call to `Release()`. diff --git a/site.json b/site.json index 3283fea..7c28233 100644 --- a/site.json +++ b/site.json @@ -381,6 +381,27 @@ "label": "Collecting code coverage in Node.js" } ] + }, + { + "groupName": "Node-API", + "items": [ + { + "link": "/learn/napi", + "label": "What is Node-API?" + }, + { + "link": "/learn/napi/getting-started", + "label": "Getting Started" + }, + { + "link": "/learn/napi/build-tools", + "label": "Build Tools" + }, + { + "link": "/learn/napi/special-topics", + "label": "Special Topics" + } + ] } ] }