Dosc about napi

Author: adespawn

docs/src/internal/napi.md

@@ -0,0 +1,61 @@
+# Napi (and napi-rs)
+
+## Distinction
+
+Node-API and napi-rs are two different things. When encountering just "napi" in the documentation (or PRs/issues),
+what is meant in most cases is Node-API (although there may be some exceptions, especially in old docs).
+However, when you encounter napi in the code (mostly Rust code), this refers to napi-rs.
+
+In napi-rs there are different kinds of interfaces. Some of the interfaces napi-rs exposes are just 1-to-1 mappings from Node-API (mostly `napi::sys`).
+Then there are low-cost abstractions over Node-API, like `(To/From)NapiValue` traits and `napi::Env`.
+Then there are high-cost interfaces. Some of those add overhead that we would need to handle on our own (like **sync** functions: functions with the `#[napi]` macro).
+Others add high-cost overhead that we can avoid by using lower-abstraction-level approaches to achieve sometimes very significant performance improvements.
+This often comes at the cost of some limitations, however those limitations rarely pose a problem for our use cases.
+
+## Our interfaces
+
+The main benefit of high-cost napi-rs interfaces is that they provide abstractions that very significantly reduce required verbosity.
+This section briefly describes the available internal interfaces and their use cases:
+
+### To Napi Value
+
+When you want to convert a structured object from Rust to JS, use helpers defined in [to_napi_obj.rs](../../../src/utils/to_napi_obj.rs).
+Helpers defined there allow you to convert:
+
+- Rust structs and enums with values in variants to JS
+- Rust maps (String -> Any) into JS objects
+
+Details on how to use them are present in the code docs for that file.
+Those helpers are a specialized replacement for [napi-rs classes](https://napi.rs/docs/concepts/class).
+They provide better performance with a reasonable interface.
+However, in performance-critical sections, it's best to return an [array of values](https://github.com/scylladb/nodejs-rs-driver/blob/2e04a1701e4cfa039bac35c96d8b84b3a5f93867/src/paging.rs#L53-L73)
+and then construct the desired object [on the JS side](https://github.com/scylladb/nodejs-rs-driver/blob/f4d6a68641bec162f066641b599eb31ec6efc0fa/lib/client.js#L354-L377).
+
+### From Napi Value
+
+Similar to the previous section, we have helpers for converting JS objects to Rust structs. To see more, go to [from_napi_obj.rs](../../../src/utils/from_napi_obj.rs).
+This macro provides a much cleaner interface than [napi-rs classes](https://napi.rs/docs/concepts/class).
+Compared to napi-rs classes, it allows you to convert arbitrary JS objects with the desired structure, rather than only objects created through the napi-rs layer.
+
+When using structs from this macro, all fields are converted from JS to Rust. This means that when you have a struct with a high number of fields but use only a few of them,
+you will still pay the cost of converting all unused fields.
+
+### Casyn (async bridge)
+
+This is an open improvement that attempts to cut the very high CPU cost of synchronization in async functions.
+You can see the benchmarks for this feature in [#414](https://github.com/scylladb/nodejs-rs-driver/pull/414).
+TODO: finish this section once #414 is merged.
+
+### JSResults
+
+We use a custom JSResult for results that may end with an error. This replaces napi-rs Results, as those do not allow for custom errors.
+We need custom errors since the errors returned by the Rust driver contain additional information. For now we just pass the error name.
+We then need to use JSResult over regular Result since we cannot implement the ToNapiValue trait on a regular Result.
+This complicates the code slightly but is the solution we found with the lowest verbosity.
+
+## The napi problem
+
+The napi-rs library is not perfect. We have already [found and fixed](https://github.com/napi-rs/napi-rs/issues?q=author%3Aadespawn) multiple bugs in the library.
+The library is developed with heavy LLM usage. The quality and readability of the code in some places is less than ideal.
+However, this is still the best option we have. To reduce the impact of problems with the napi-rs code, we avoid using high-abstraction parts of napi-rs,
+and where possible switch to self-written, specialized abstractions.