other changes

Author: petrelharp

c/tskit/core.c

@@ -214,6 +214,7 @@ tsk_json_struct_metadata_get_blob(char *metadata, tsk_size_t metadata_length,
 out:
     return ret;
 }
+
 static const char *
 tsk_strerror_internal(int err)
 {

docs/metadata.md

@@ -527,7 +527,7 @@ of `B`, `H`, `I`, `L` or `Q` which have the same meaning as in the numeric
 types above. `L` is the default. As an example:
 
 ```
-{"type": "array", {"items": {"type":"number", "binaryFormat":"h"}}, "arrayLengthFormat":"B"}
+{"type": "array", "items": {"type":"number", "binaryFormat":"h"}, "arrayLengthFormat":"B"}
 ```
 
 Will result in an array of 2 byte integers, prepended by a single-byte array-length.
@@ -555,6 +555,73 @@ As a special case under the `struct` codec, the top-level type of metadata can b
 union of `object` and `null`. Set `"type": ["object", "null"]`. Properties should
 be defined as normal, and will be ignored if the metadata is `None`.
 
+(sec_metadata_codecs_jsonstruct)=
+
+### `json+struct`
+
+An additional codec provides the ability to store *both* JSON and binary-encoded data.
+This is provided for the case where we want to store some arbitrary metadata
+(as JSON) along with a relatively large amount of data (as binary, for efficiency).
+For instance, we might want to record a raster map of the sampled area
+along with a few pieces of generic information (e.g., the name of the area).
+
+The metadata schema for "json+struct" metadata basically just specifies both
+a JSON metadata schema and a struct metadata schema.
+Each entry in the metadata is encoded with either the JSON or the struct codec.
+Here is a simple example:
+
+```{code-cell}
+schema = {
+    "codec": "json+struct",
+    "json": {
+        "type": "object",
+        "properties": {
+            "label": {"type": "string"},
+            "id": {"type": "number"},
+        },
+        "required": ["label"],
+    },
+    "struct": {
+        "type": "object",
+        "properties": {
+            "values": {
+                "type": "array",
+                "arrayLengthFormat": "B",
+                "items": {"type": "number", "binaryFormat": "i"},
+            }, 
+        },
+    },
+}
+ms = tskit.MetadataSchema(schema)
+row = {"label": "alpha", "id": 7, "values": [5, 10, 2, 12]}
+encoded = ms.validate_and_encode_row(row)
+print("Encoded:", encoded)
+print("Decoded:", ms.decode_row(encoded))
+```
+
+This encodes two things in JSON: a label and an ID number,
+and then an array of integers in binary (using the ``struct`` codec).
+If the array of integers is large, this could result in
+much better performance.
+
+
+#### Binary representation
+
+The underlying structure of the JSON+struct codec is as follows.
+(If you're not writing out data in this format,
+you don't need to worry about this.)
+(1) some magic bytes;
+(2) a version number;
+(3) the length of the JSON in bytes;
+(4) the length of the binary (struct) data in bytes;
+(5) the JSON data;
+(6) zero-ed "padding" bytes to bring the start of the binary section
+ into 8-byte alignment;
+(7) the binary data.
+The structure of the binary data is specified using the "struct" portion
+of the metadata schema.
+
+
 (sec_metadata_schema_examples)=
 
 ## Schema examples