Use and effects of @JsonIdentityInfo for recursive json schema definitions #311
Replies: 13 comments 57 replies
-
|
Without knowing full details/intents I could be wrong, but I think use of Also quick note on "in practice": that's clumsy way of saying "this is how Jackson implements it" (but implying other implementations could produce output that is not necessarily ordered this way). |
Beta Was this translation helpful? Give feedback.
-
So could you summarize the actual problem? Given the use case of representing object graphs efficiently in json messages (streaming), that will work across impl language/libraries, what approach do you think would would work best?
I suppose I don't know what an impl of this would look like...in Jackson specifically, but more generally in other lang/json libs as well. |
Beta Was this translation helpful? Give feedback.
-
The problem/my concern is that Jackson may not be able to support use of Json References as the Object identifiers (via @JsonIdentityInfo and functionality that supports it). As far as I understand these definitions.
I think that can be ignored, it's more about what |
Beta Was this translation helpful? Give feedback.
-
Ok. Actually my use case (mcp protocol addition) doesn't require full json references as object identifiers support. Let me detail a little: I'm proposing adding a new primitive called Group to the mcp specification To experiment with this, I've implemented this in the mcp java sdk, which uses Jackson. Here's the java/Jackson code for the new Group type. Note the Group 'parent' attribute. This is a recursive attribute that allows hierarchies of groups. Using the @JsonIdentityInfo annotation and the "@id" works perfectly for hierarchical Groups...the first one in json serialization is complete and the subsequent ones are referenced as 1, 2, 3 (integers) as per the @JsonIdentifyInfo. If JsonIdentityInfo is not present, then each Group instance is serialized as a copy...including all the other attribute values (description, name, title). As you can imagine with the recursive Group parent definition this can be very inefficient. All I'm looking to do is to figure out a strategy that will interoperate with the other languages/json libs for the Group type only. There are no other recursive references in the mcp spec, so any solution (across languages) could be applied just for the Group type (e.g. custom serialization for that type only). My initial thought was to try to use the json ref spec/standards....as I thought this would be more likely to already exist in mature json libs like Jackson. But I'm not married to that strategy...it could very well be something custom to the Group type that is just implemented the same in all langs/libraries. Does that ease your concerns at all? |
Beta Was this translation helpful? Give feedback.
-
|
Ok so my hesitation wrt json-ref is solely based on whether Jackson's so Object Id to use can only be based on Java Object being serialized, and not on serialization state (like location/path of Object being serialized). So it needs to be sort of "pure" Object Id. This can be considered an unfortunate limitation in Does this make more sense? |
Beta Was this translation helpful? Give feedback.
-
Yes it does, thank you. I understand it's not your problem to fix, but I would like to find some answer for recursive object definitions (e.g. Group) that could work across languages/json libs without copying/re-serializing every object (Group) in the graph multiple times and without a gigantic effort for every json library. What about a new annotation (rather than @JsonIdentityInfo) that is specifically designed for using object refs? (e.g. @JsonObjectRefInfo)? It could reuse existing classes/code as much as possible, and introduce new classes/API where required. Another approach could be do implement the equivalent of @JsonIdentityInfo in each language/json library. I'm somewhat hesitant to do this, however, because it's not standardized at the json spec level afaik. |
Beta Was this translation helpful? Give feedback.
-
|
@cowtowncoder I've just discovered the JavaScript Object Graph project: https://github.com/jsog/jsog Interestingly, it has a Jackson plugin implementation https://github.com/jsog/jsog-jackson that uses the @JsonIdentityInfo generator property to customize the generator. I'm looking over this code myself, but wanted to make you aware of it's existence too. Looks like it was produced a while ago, so I'm hoping it still functions as described with recent Jackson. |
Beta Was this translation helpful? Give feedback.
-
|
I vaguely recall JSOG. It got things almost right; too bad And yes, |
Beta Was this translation helpful? Give feedback.
-
|
First, re: jsog. I've been looking at the json schema spec, and the various impls in jsog (as well as impl of JsonIdentityInfo). My conclusion: All that's needed here to get cross/language standardization (which is required for my use case...i.e. MCP is multi-language), is to standardize the name and type of the dynamically set/read object id. To this end, I've already opened this discussion on the jsog repo. And this discussion/proposal on the json schema specification repo. Your involvement in the standardization effort would be most welcome. Second issue: I would like to continue with providing documentation on JsonIdentityInfo. What I'm looking for from you on that is what form would you like it to take? example?, tutorial?, docs on existing tests? etc. thanks. |
Beta Was this translation helpful? Give feedback.
-
|
Documentation would be very helpful, although the structure to add to is missing. First obvious place is Javadocs for But beyond that I am actually not sure. Open to suggestions. Most obvious place beyond that is https://github.com/FasterXML/jackson-docs/ although I haven't really maintained it. But could definitely give access. Other relevant Jackson repos are:
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Tatu. Over the holidays, I had some q and a with folks involved with json schema specification (i.e. JRef and hyper jump) https://github.com/hyperjump-io/browser The JRef spec is based upon Json pointers and adds a json reference type (and mime type for document), that is then serialized to (e.g.): { "$ref": "#/path/to/object/in/current/doc" }. I learned that the complete JRef spec (and hyperjump browser as example) extends the JSON pointer syntax to include an arbitrary uri...e.g. { "$ref": "https://foo.bar/path/to/doc#/path/to/object/in/doc" }. As you can see, however, if the uri fragment only is used (i.e '#'...with the json pointer syntax), that the current document is assumed. After working with Jason a bit, he's written a 'fragment (#)/current doc-supporting only' typescript implementation of serialization and validation. Given his ts impl, I've created a python impl and have now got it working for the python impl of the MCP sdk (pydantic as json ser/parse lib)...specifically for serializing object graphs without sending multiple copies of referenced objects (as JsonIdentityInfo does). One utility of this...like JsonIdentityInfo, is to serialize object graphs without having to send multiple copies of object data when a reference will do. Because of the pointer specification...$ref name and specified value syntax...it works same in all languages, and without any additional json-schema-specific standardization. I was thinking of now creating Jackson-based Java impl and contributing it to Jackson, but thought that it might be better to discuss with you before doing so. Now that I've done it in Python/pydantic...it's very short...I believe it would be easy to produce in Java or most other languages, really. I have the typescript/javascript impls, along with the python impl and can provide them to you. I'm working out with Jason how he wants to distribute this 'reduced/fragment-only' impls of JRef (not full URI support...which obviously implies client->server (http/https) communication for references). Since this functionality is all about interoperability, it makes most sense (IMHO) to make freely available for all (specification + impl). |
Beta Was this translation helpful? Give feedback.
-
|
Right, I understood there's "full" reference that allows cross-document references. Those would not be supportable by Jackson. As to Jackson 2.x vs 3.x: I would definitely recommend focus on 3.x as 2.x is moving into more of a maintenance mode. |
Beta Was this translation helpful? Give feedback.
-
Hi @cowtowncoder. We've now got impls of the jref 'local' serializers in java, typescript and python. Typescript was the original impl, and they all do the same thing (RFC 6901). We would like to consider contributing the java impl (or some form of it) to Jackson. This seems more appropriate to me as a custom serializer (e.g.) than as a new usage of JsonIdentityInfo mechanisms, but I will defer to your judgment about mechanisms. |
Beta Was this translation helpful? Give feedback.
-
|
To be honest: with a quick look at Java impl, I am not quite sure how that would fit within Jackson handling. It seems like its own stand-alone resolved of really simplified Object graphs, not meant as an embeddable component. |
Beta Was this translation helpful? Give feedback.
-
I'm happy to rewrite it as an 'embeddable component' if that can be defined more clearly (e.g. custom serializer/validators?). All it does is a depth-first traversal of the object graph via recursion, which is what the python and typescript impls do also...as they are intended to be used more broadly than for a single library. Naturally, they will have to be customized for each library's component/extensibility architecture. |
Beta Was this translation helpful? Give feedback.
-
|
Jackson does all Object graph traversal (introspection) on its own, and that cannot really be replaced. I'd almost suggest that If it doen't fit within |
Beta Was this translation helpful? Give feedback.
-
|
On Tue, Apr 14, 2026, 4:19 PM Tatu Saloranta ***@***.***> wrote:
Jackson does all Object graph traversal (introspection) on its own, and
that cannot really be replaced.
I'm not suggesting replacement, Im suggesting enhancement...perhaps with
new options to enable/support json ptrs or not.
If support for json ptrs (rfc 6901) were standardized....across languages
and libs...wouldn't that warrant a Jackson enhancement to support?
I don't understand the hesitation.
I don't really have a recipe to give unfortunately.
…
I'd almost suggest that If it doen't fit within @JsonIdentityInfo
framework, I am not sure it belongs inside Jackson at all. Perhaps it
should be an external companion library or something.
—
Reply to this email directly, view it on GitHub
<#311 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABHB5BZUAP2BXC4MQLR6SD4V3BQ3AVCNFSM6AAAAACJ557PEWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMNJWGI3TOMA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Ok, FWTW, most relevant pieces behind
Couple of notes:
Expanding on (3)... Reading (resolving of) references seems relatively straight-forward. But question there is... how would it be used? Resulting So maybe that is a dead-end too. What I am now starting to suspect, in fact, is that if this feature was to be done, maybe it would instead require yet another approach, in which some other intermediate structure would be used, in which References would need to be first-class entities... Ok. This was more stream-of-consciousness style writing than I was planning to do. |
Beta Was this translation helpful? Give feedback.
-
|
One other question: example shown on https://github.com/OpenMCPTools/jref_java is actually DAG, not Cyclic graph. |
Beta Was this translation helpful? Give feedback.
-
Yes, cycles are supported like JsonIdentityInfo. |
Beta Was this translation helpful? Give feedback.
-
Why not consider a whole other approach: e.g. adding this (standard) capability at another level in Jackson...e.g. an ObjectMapper enhancement or custom Serializer? I don't have any arguments with JsonIdentityInfo, but if it's not a good fit as implemented then perhaps something else is warranted. |
Beta Was this translation helpful? Give feedback.
-
|
You cannot just add custom serializer(s) (and of course matching deserializer(s)) for such cross-cutting concern, unless support is also via some custom type or types (one(s) like But if not, then something else needs to be designed. What that would be, I do not know at this point. |
Beta Was this translation helpful? Give feedback.
-
|
On Fri, Apr 24, 2026, 10:33 AM Tatu Saloranta ***@***.***> wrote:
You cannot just add custom serializer(s) (and of course matching
deserializer(s)) for such cross-cutting concern, unless support is also via
some custom type or types (one(s) like JsonNode). Ideally this would
integrate with other functionality.
But if not, then something else needs to be designed. What that would be,
I do not know at this point.
I find it difficult to believe that Jackson can't support this without a
rewrite.
ObjectMapper?.
|
Beta Was this translation helpful? Give feedback.
-
|
You are completely free to prove me wrong here :) The challenge is that handling of Object references is a cross-cutting feature that affects very low-level and fundamental handling. It does not sit at level where it could be abstracted out, or layered nicely. Jackson is very modular and parts (like value/type serializers, deserializers) are designed to interoperate, based on logical abstraction of token streams/sequences. Everything builds on that. State kept during reading and writing is kept if not strictly minimal at least very limited; and in case of Object identity that is only tracked for POJO types that indicate that is needed (usually via Conversely, if it was to be added "outside" of existing handler paths, then we'd lose about everything Jackson offers from property introspection to actual mapping between JSON (and other formats) and Java types (JDK, POJOs), having to re-implement that. This is not something I think is worth the effort. So the likely path would be something in-between the extremes; maybe pre- and/or post-processing steps; perhaps using But although I have tried thinking through options nothing so far seems quite workable. |
Beta Was this translation helpful? Give feedback.
-
I might try to ask my AI code agent to give it a try after giving Jackson source as context.
I don't understand why the graph traversal could not be done twice rather than replacing the low-level handling (I also don't see understand why the traversal can't be augmented without breaking everything in Jackson). Yes it would have a runtime cost to traverse multiple times, but if optional and for right use cases (e.g. multiple refs to large/media objects) the traversal cost could be far less than the bandwidth/transmission cost of multiple copies.
I'm not asking to bolt it in. I'm just surprised that with all of Jackson's extensibility that there is apparently no way to extend it to support a long-existing json standard.
I would also like to see it fit into the JsonIdentityInfo extensibility mechanism. Perhaps being clearer about what 'quite intrusive/API changes' would mean (code references please).
Something I don't understand: The method here will take an Object instance...with field references to other objects and return a Map<String,Object> where the top level subject's fields are map keys and the values are...for the first reference only...Objects. All internal references after first one will be "$ref": "#/some/path"....i.e. String, String but the first reference will be untouched. Couldn't ObjectMapper/extension optionally call the jref serializer code before doing core serialization calling? All of <= 1 references will be untouched by jref, and so serialized by Jackson as normal, with the > 1 refs being serialized by Jackson as String->String. Then the reverse order of invoke for ObjectMapper-based deserialization/validation. Yes it's somewhat wasteful on ser/des but again...the use cases/transmission performance may be worth it to Jackson consumers. |
Beta Was this translation helpful? Give feedback.
-
Because Jackson does not buffer Object graph; it is traversed just once both for output (ser) and input (deser). You keep on using "why dont... just..." without trying to understand things to see why. I feel I am wasting my time here, going in circles. Good luck with your investigation. I am done here for now. |
Beta Was this translation helpful? Give feedback.
-
|
> Because Jackson does not buffer Object graph; it is traversed just once both for output (ser) and input (deser).
I get that. After 30+ years of reading java code, I know that that even well-established, bug-free, heavily used, high-quality code can sometimes be enhanced...without necessarily breaking backward compatibility.
With due respect, that's because your explanations (here) are not detailed enough. I would appreciate someone with the appropriate knowledge of Jackson source (assumed to be you) pointing directly to the code that implements this (in this case the core serialization/de-serialization traversal). Tatu: I am also a (very) long time maintainer of a heavily-used java codebase. I know what that means for your time and effort. I'm trying to help you/Jackson/the Java dev community by contributing to Jackson wrt language and library interoperability and performance/scalability. I don't want to take over stewardship of your project, I don't want to contribute to Jackson long term, I don't want payment, or even thanks. What I'm looking for is some sharing of expertise (and associated code navigation) so that I can not spend 6 months navigating Jackson's impl of everything, rather than giving up on Jackson and finding another project(s) to contribute json ptr support. |
Beta Was this translation helpful? Give feedback.
-
FYI: google/gson#1443 Seems like maybe ObjectMapper might be a better place to start? |
Beta Was this translation helpful? Give feedback.
-
|
Created a new branch of jackson-databind here. Here's the diff I added a new MapperFeature.USE_JREF, very simple changes to ObjectMapper.writeValueAsString and readValue(String,Class) added the current jref_java class, and created a simple JsonJRefTest. There are two test methods in JsonJRefTest: testBuildAndResolveRefs which only uses JRef.buildRefs and JRef.resolveRefs (no ObjectMapper) and testObjectMapperJRef which obviously does use the enhanced JRef for buildRefs (called prior to jackson serialization), and resolveRefs (called after jackson deserialization). Here's the output of the testBuildAndResolveTests (JRef only): ---testBuildAndResolveRefs
message=Message[items=[Human[name=wendy, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}], Human[name=rick, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}]]]
jrefSerResult={items=[{name=wendy, parent={name=sam, parent=null, props={s1=1}}, props={p={$ref=#/items/0/parent}, q=r}}, {name=rick, parent={$ref=#/items/0/parent}, props={$ref=#/items/0/props}}]}
resolveRefResult={items=[{name=wendy, parent={name=sam, parent=null, props={s1=1}}, props={p={name=sam, parent=null, props={s1=1}}, q=r}}, {name=rick, parent={name=sam, parent=null, props={s1=1}}, props={p={name=sam, parent=null, props={s1=1}}, q=r}}]}The Human and Mess types exist in the original message, the jrefSerResult creates/substitutes the {"$ref":"#/path/to/ref"} references...after the first one, and the resolveRefResult resolves the references. Obviously the jref resolve produces Map<String,Object> types rather than the original types, since there's no type information in the json. Resolve refs does it's job. Now for the jref enhanced object mapper output. ---testObjectMapperJRef
message=Message[items=[Human[name=wendy, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}], Human[name=rick, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}]]]
serializedMessage={
"items" : [ {
"name" : "wendy",
"parent" : {
"name" : "sam",
"parent" : null,
"props" : {
"s1" : 1
}
},
"props" : {
"p" : {
"$ref" : "#/items/0/parent"
},
"q" : "r"
}
}, {
"name" : "rick",
"parent" : {
"$ref" : "#/items/0/parent"
},
"props" : {
"$ref" : "#/items/0/props"
}
} ]
}
deserializedMessage=Message[items=[Human[name=wendy, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}], Human[name=rick, parent=Human[name=null, parent=null, props=null], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}]]]The message is same as jref only...just the printed out object graph. The serializedMessage also looks right to me...as all > 1 internal references are replaced with (e.g.): { "$ref" : "#/items/0/parent" } via the call to JRef.buildRefs(result). The deserializedMessage (after Jackson deserialize + JRef.resolveRefs called) looks mostly right deserializedMessage=Message[items=[Human[name=wendy, parent=Human[name=sam, parent=null, props={s1=1}], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}], Human[name=rick, parent=Human[name=null, parent=null, props=null], props={p=Human[name=sam, parent=null, props={s1=1}], q=r}]]]All the references were resolved properly (to sam) except for the rick.parent reference: Human[name=rick, parent=Human[name=null, parent=null, props=null]The human name, parent, and props are null...i.e. the JRef resolve did not result in pointing to Human[name=sam...] This is in contrast to the JRef-only test...where the resolveRefs is done fine (even though the type of the resulting reference is Map). I looked in the debugger and found 
Just before calling JRef.resolveRefs, the Object result from _readMapAndClose(...) has the rick.parent reference set to object of type Human, but with all fields set to null. The string input to readValue(String,Class) has this as it's rick.parent json value: "parent" : {
"$ref" : "#/items/0/parent"
},As you can see from the serializedMessage value. But unlike the other references in the readValue input string, rick.parent somehow was set during Jackson deserialization to a pojo of Human type, but with the fields set to null. This is puzzling to me as I can't explain how the parent json ptr got somehow resolved to an empty Human instance during deserialization when none of the others had this (they come out of jackson deserialization as a LinkedHashMap with entry.key="$ref" and value="#/whatever". Does this make deserialization behavior make sense to you? |
Beta Was this translation helpful? Give feedback.
-
Right, I don't think serialization is problematic here; nor arrays (since their ordering is strictly retained). But Objects' property order may be changed by intermediate processing (simple scripts that read-modify-write f.ex) so that document order may change to include refs from earlier (in physical document order) to later sub-trees, even if serialization did not produce such document.
Yes. I don't think any of existing handlers would quite work based on our discussions. But at some point you may need to trace actual call patterns, delegation, from As to alternative of pre-/post-processing, maybe using |
Beta Was this translation helpful? Give feedback.
-
|
First, thanks for the comment and pointers.
Right. In retrospect I think that's why I have been inclined to focus on the object graph/mapper layer, as opposed to the serialization layer underneath, as there are helpful constraints imposed by the object graph layer. For example, 'local only' object graph means that all references (graph links) essentially have to be in memory/resolved...both before serialization and after de-serialization. These constraints are at least potentially helpful for simplifying the problem. My take anyway.
Thanks. I will do some code examination and Eclipse debugger stepping. What tests or examples would you suggest I use for running a debugger? Are there tests for BeanSerializerModifier specifically? Also...are there specific tracing options in that code that I should/could enable?
My inclination is to investigate post-processing first at the ObjectMapper layer because of the available constraints as discussed. Then some JsonNode/intermediate representation unless you think that's more potentially fruitful. |
Beta Was this translation helpful? Give feedback.
-
|
I've been learning the ValueDeserializerModifier and I've run into a problem I don't understand. First here's the src for a new test case Here's the ValueDeserializationModifier and Here's the new deserializer. As you can see I've implemented createContextual, so that all the properties/fields can be intercepted as well. My problem is that when the target model has a self-reference...e.g. in the Human parent the current implementation of createContextual gets into an infinite loop...i.e. createContextual gets called for the Human type (for first time as element of items array property), if delegate.createContextual is called it iterates through the properties for Human and when it encounters parent....it repeats and gets into a loop. eventually having stack overflow. Is there something I should be doing in createContextual to properly deal with this case? (a parent self-reference)? Or is there some option I need to handle that case properly with the ValueDeserializationModifier? |
Beta Was this translation helpful? Give feedback.
-
|
Handling of certain aspects is done in Another thing that could be useful (but not necessary) is using existing |
Beta Was this translation helpful? Give feedback.
-
|
DelegatingDeserializer is perfect...thanks! |
Beta Was this translation helpful? Give feedback.
-
|
I've added the following test method @Test
void testDeserializationOnly() throws Exception {
String serMessage = "{\r\n"
+ " \"items\" : [ {\r\n"
+ " \"name\" : \"wendy\",\r\n"
+ " \"parent\" : {\r\n"
+ " \"name\" : \"sam\",\r\n"
+ " \"parent\" : null,\r\n"
+ " \"props\" : {\r\n"
+ " \"s1\" : 1\r\n"
+ " }\r\n"
+ " },\r\n"
+ " \"props\" : {\r\n"
+ " \"p\" : {\r\n"
+ " \"$ref\" : \"#/items/0/parent\"\r\n"
+ " },\r\n"
+ " \"q\" : \"r\"\r\n"
+ " }\r\n"
+ " }, {\r\n"
+ " \"name\" : \"rick\",\r\n"
+ " \"parent\" : {\r\n"
+ " \"$ref\" : \"#/items/0/parent\"\r\n"
+ " },\r\n"
+ " \"props\" : {\r\n"
+ " \"$ref\" : \"#/items/0/props\"\r\n"
+ " }\r\n"
+ " } ]\r\n"
+ "}";
// Use MAPPER / no JRef
Object value = MAPPER.readValue(serMessage, Message.class);
System.out.println(value);
}
}The MAPPER is a plain ol ObjectMapper...i.e. not the JRef-enhanced one. In the string input this is the value for rick.parent "parent" : {
"$ref" : "#/items/0/parent"
},After deserialization the above rick.parent value is: Human[name=null, parent=null, props=null] I presume what's happening is that the deserialization is expecting type Human because that's the type of the parent property in the enclosing object. Is this where mapper.readTree or something else would come in? Perhaps that you are right that such standardization just can't be a feature of Jackson json serialization. Apologies for my persistence. |
Beta Was this translation helpful? Give feedback.
-
|
Right, yes, deserializers are constructed along with type hierarchy so JSON value matching |
Beta Was this translation helpful? Give feedback.
-
|
I've been looking at the tools.jackson.databind.objectid.JSOGDeserialize622Test and have landed upon the following strategy. Please let me know if this can't work: Like the test, create a new type called JRef (analogous to JSOGRef in test code). static class JRef
{
@JsonProperty(REF_KEY)
public String ref;
public JRef() { }
public JRef(String val) {
ref = val;
}
@Override
public String toString() { return "[JRef="+ref+"]"; }
@Override
public int hashCode() {
return ref.hashCode();
}
@Override
public boolean equals(Object other) {
return (other instanceof JRef)
&& ((JRef) other).ref == this.ref;
}
}Then create a deserializer for this type static class JRefDeserializer extends ValueDeserializer<JRef>
{
@Override
public JRef deserialize(JsonParser p, DeserializationContext ctxt)
{
JsonNode node = ctxt.readTree(p);
if (node.isString()) {
return new JRef(node.asString());
}
JsonNode n = node.get(REF_KEY);
if (n == null) {
ctxt.reportInputMismatch(JRef.class, "Could not find key '"+REF_KEY
+"' from ("+node.getClass().getName()+"): "+node);
}
return new JRef(n.asString());
}
}and call addDeserializers in JRef module with this type static class JRefDeserializerResolver implements Deserializers {
@Override
public ValueDeserializer<?> findBeanDeserializer(JavaType type,
DeserializationConfig config, BeanDescription.Supplier beanDescRef)
{
if (!JRef.class.isAssignableFrom(type.getRawClass())) {
return null;
}
return new JRefDeserializer();
}
@Override
public boolean hasDeserializerFor(DeserializationConfig config,
Class<?> valueType) {
return false;
}
}I notice, however, that the JRefDeserializerResolver is never consulted upon readValue, apparently because the type is never referred to statically in the java classes, but rather is by default handled by the Map serializer (because jreferences have form { "$ref": string } Is there some way to hook into the runtime default deserializer lookup....so that the maps with a single entry with key == "$ref" can be deserialized into JRef? AFAICT the JsonIdentityInfo annotation...which triggers using references is all done statically/contextually based upon the return types rather than based upon the actual value in the input stream (defaulting to map). Or perhaps I should be looking to create some reference type instead of JRef during deserialization so that the reference can be resolved using existing reference handling logic (e.g. to handle forward references)? Thanksinadvance. |
Beta Was this translation helpful? Give feedback.
-
|
I'm thinking of taking the strategy of modifying BeanDeserializer and all it's property's ValueDeserializers on construction/contextualization so that they all can be referred to during deserialization via a Map with one key='$ref' and value of path string...e.g. '{ "$ref": "#/path/to/local/instance" }'. The idea is to have every BeanDeserializer be dynamically setup as if it was defined statically with a JsonIdentityInfo declaration. I've been debugging through the JsonIdentityInfo annotation processor to get hints about how/when this is all setup. If there are tests or other code that can/could help with that understanding I would appreciate pointers. |
Beta Was this translation helpful? Give feedback.
-
|
I'm going to abandon reusing JsonIdentityInfo-like dynamic configuration to reuse existing deserializers+valueInstantiators for deserializing jref objects. Id creation/definition/setup and resolution doesn't seem well-suited for the jref case...where there is no explicit object id, but rather just a reference to a local object. |
Beta Was this translation helpful? Give feedback.
-
|
Quick note: instead of directly registering deserializers capable of resolving jrefs, you probably want/need But you do want/need to wrap EVERY deserialize this way since target type can be anything. |
Beta Was this translation helpful? Give feedback.
-
|
Yes...thanks kindly. I've realized this and created JRefValueDeserializer wrapping/delegating instance for every target type. I'm trying to figure out if if there is some way to intercept deserialization to find reference type objects...e.g. : { "$ref": "#/path" }. If found, I'm thinking of deferring resolving the ref until after root object has been deserialized (probably with new Deffered types and additional/non objectId post-serialization resolve) as then local JRef references should be resolvable. But if jref not found in stream then normal deserialization should be done. Is there some way to do this in StdDeserializer super class? Or does it have to be done for each type of StdDeserializer in separate way/overriding separate methods? And also...is there some mechanism that exists for dealing with the JsonParser state...i.e. if "$ref" object field not found...continue on with normal deserialization? There is something called a TokenBuffer that I'm using/creating to detect the JRef instances in JsonParser with BeanValueDeserializer right now. But if "$ref" not found, how can I pass to normal deserialization? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Unfortunately In Jref case it may be easiest to check if current value is JSON Object (current token is But none of above comes out-of-the-box. |
Beta Was this translation helpful? Give feedback.
-
|
I've made some progress on my jref_experiment_1 branch. I've created JRefValueInstantiator and JRefValueDeserializer classes in tools.jackson.databind package. Also in this package are JRef.java (currently just a copy of the JRef prototype impl...some of which will/can be used in JRefResolver.resolve), JRefResolver, and a small addition to ObjectMapper._readMapAndClose after the top level result has been returned, to attempt to resolve any unresolved JRefResolver instances (placed in ctxt attribute named jrefs (list of JRefResolver instances by processing in JRefValueDeserializer.deserialize. Here's the JRefValueDeserializer.deserialize impl that does this. For a reason I don't yet understand, the JRefValueDeserializer in the JRefResolver.resolve method returns a null ValueInstantiator in response to JRefValueDeserializer.getValueInstantiator(), which I was thinking would be the way to get the 'target' object of appropriate type for the resolve(Object root) method implementation. There's a new test: tools.jackson.databind.deser.bean.JRefBeanDeserializerTest that uses/tests the JRefValueDeserializer by using a ValueDeserializerModifier to create delegate/wrapper JRefValueDeserializer (and JRefValueInstantiators) in the JRefModule. This test runs/parses/sets everything that's not a jref. There are two jrefs that are found/added to list of JRefResolvers to the context. See the JRefResolver.resolve method for the business that tries to use the JRefValueDeserializer.getValueInstantiator(). Obviously I don't understand something about the ValueDeserializer -> ValueInstantiator setup and usage for creating/setting object values. Any guidance would be appreciated. Perhaps I should be setting up the JRefResolver with some sort of SettableBeanProperty rather than a JRefValueDeserializer for the eventual resolve(root) call. |
Beta Was this translation helpful? Give feedback.
-
|
Signs of life! New branch json_deserialization See JRef* classes in tools.jackson.databind package JRefModule a SimpleModule that implements ValueDeserializerModifier to create JRefValueDeserializer instances to wrap all value deserializers. JRefValueDeserializer.deserialize method uses the ctxt.readTree and inspects the JsonNode for { "$ref", "#/path/to/object" } (JRefs) and if found returns a JRefResolver instance with the deserialization context and path. Currently, I've only handled the returned JRefResolver instance in the BeanDeserializer MethodProperty here. What this does is call JRefResolver.setSetter(MethodProperty,Object) to later be used in resolve(Object root). Obviously this only is appropriate for BeanDeserializer/MethodProperty, and so the JRefResolver structure has to be generalized so that it can do the appropriate value setting for all types of ValueDeserializers (maps, lists, objects, etc). But for now the JRefBeanSerializerTest code input string only has jrefs (2 of them) that are both associated with bean types/BeanValueDeserializers. The MethodProperty impl calls the JRefResolver.setSetter and then adds the JRefResolver to a list of JRefResolver attached to the BeanDeserializationContext instance as an attribute named "jrefs". Using a context attribute may not be what you want to do...I just used it as a way to associate JRefResolver instances with a single context. There may be a better/cleaner way to do so (e.g. adding list of jref resolvers to context). After the root object value is returned by ObjectMapper._readRootValue each JRefResolver's resolve(Object root) method is called, and the JRefResolver.resolve(root) uses the jref local path and some JRef code to lookup the root object graph and get the referenced object. Finally the JRefResolve.resolve(root) calls the methodProperty.set method to actually set the bean property to the referenced object value. This only works with bean properties, and JRefResolver needs to be able to handle JRefResolver instances (created by JRefValueDeserializer) when returned in all other contexts. I'm not sure how best this is to be done...e.g. by adding methods to StdSerializer and subclasses or generalizing setSetter (e.g. providing a function). There also might be some way to reuse what's already present in Jackson to handle all the different types of valuedeserializers in a cleaner way. |
Beta Was this translation helpful? Give feedback.
-
|
@cowtowncoder as per the previous posting/second paragraph, I'm now trying to understand how best to setup the JRefResolver with an appropriate function that can be called during JRefResolver.resolve(root) so that the resolve uses the ctxt, path to lookup on root the first object, which according to jref local contract has a deserialized version of the jref referenced object of any java type. What is currently provided by JRefResolver.setSetter is (first arg) the MethodProperty and (second arg) the target object...i.e. that will have the target method property set called with the value found during JRefResolver lookup on root. I'm trying to generalize this so that it will work on any type of ValueDeserializer/StdValueDeserializer instead of just BeanDeserializer as the impl does now. SettableBeanProperty is superclass of MethodProperty. Would it be possible to dynamically construct an appropriate SettableBeanProperty subclass...e.g for a MapDeserializer that simply implements set() calls/sets via target.put(name,value). Maybe a new protected methods on StdDeserializer for handling JRefs/JRefResolvers by a) setting the path, deserialization ctxt upon construction in JRefValueDeserializer.deserialize/2. Does this (new subtypes of SettableBeanProperty seem like a good way to go to handle all types of ValueDeserializers like Map, List, Object...anything that can a jref can refer to and later resolved? in addition to BeanValueDeserializers? Or is there a better way of calling a setter on any deserialized object? |
Beta Was this translation helpful? Give feedback.
-
|
I'm not sure I understand need for Adding new JRef-specific |
Beta Was this translation helpful? Give feedback.
-
|
Ahhh. I forgot that |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. I think I've got a much easier and more general set of classes now (pushed this evening) on this branch. In this package See the new abstract superclass JRefSetter and subclass JRefSetterFunction (which takes a Function<Object,Object> callback function for actually setting the instance to the value retrieved from traversing the root when it's available. The first example of using the JRefSetterFunction is in MethodProperty (for BeanDeserialization). JRefResolver resolver = new JRefResolver((JRefPath) value, new JRefSetterFunction((v) -> {
try {
_setter.get().invokeExact(instance, v);
} catch (Throwable e) {
_throwAsJacksonE(p, e, v);
}
return instance;
}));Note that the try/catch block uses precisely the same code (below) that is run in this context if an actual instance of expected type (rather than JRefPath) is returned from the JRefValueDeserializaer. The idea here is that it should be possible to modify all the other deserializer types similarly...and use an appropriate Function block to do the target instance setting....during JRefResolver.resolve...after all the non-JRef objects have been deserializaed rather than during the main deserialization. Using callback functions to do the actual setting, I don't think it will be necessary to create SettableBeanProperty subclasses....rather it should be possible to use the Function callback for most JRefSetter situations. And if a custom setter is needed for some reason, it can be created as a peer class to JRefSetterFunction. |
Beta Was this translation helpful? Give feedback.
-
|
@cowtowncoder Question: In this first usage of the JRefPath, there is logic before this code that sets the value using deserializeWithType rather than deserialize. value = _valueDeserializer.deserializeWithType(p, ctxt, _valueTypeDeserializer);My question: Currently JRefValueSerializer only overrides the deserialize/2 method. Should it also similarly override deserializeWithType?...this would allow JRefPaths to be detected and returned from deserializeWithType also, but I'm not sure if support in deserializeWithType support would be desired or not. What I'm planning to do now is to add JRefPath deserialization (JRefResolver creation) support to the other StdValueDeserializer sub classes and add minimal String input test cases to the JRefBeanDeserializationTest class to test each newly supporting type. |
Beta Was this translation helpful? Give feedback.
-
|
Update: I simplified the JRef handling considerably by adding a new functional interface: SetterFunction and removing two JRefSetter* classes. Also added ValueDeserializer.setWithJRef method that can be used by all types of ValueDeserializers. addWithJRef is now used in MapDeserializer, CollectionDeserializer, and MethodProperty/BeanDeserializaer. Added tests to JRefBeanDeserializerTest class for CollectionDeserializer. I will be adding appropriate use of setWithJRef in other Deserializer types...with associated tests added to JRefBeanDeserializerTests. If you see fault with what I'm doing...or I'm not taking advantage of mechanism that can be reused for jref resolution...please let me know. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm a little new to the Jackson API, although very experienced in Java.
I'm working on a proposal for adding Groups to the MCP protocol. The mcp-java-sdk uses Jackson (2.19.something I think).
In the Groups proposal I've suggested introducing a nested Group structure, by adding a 'parent' Group attribute. This provides for arbitrary hierarchy of groups.
With normal serialization/deserialization of json with this schema the Group.parent recursive/nested structure could produce a lot of duplication of groups (for example if the same nested hierarchy of Groups was assigned to multiple Tools).
I've heard that the @JsonIdentityInfo annotation can be used for this situation, to replace all but the first given Group instance with a json reference, so that on deserialization the Group tree structure is reconstituted by clients.
I've read the docs for @JsonIdentityInfo and it says this:
...In practice this is done by serializing the first instance as full object and object identity, and other references to the object as reference values....
My question is: Is adding @JsonIdentityInfo to the Group type...e.g. here all that's necessary to get this 'in practice' behavior? (i.e. serialize to json the first reference to a given Group, and send a json reference for all other references?). Or are there other options required for getting this in practice behavior (and/or changing/customizing that behavior)?
Beta Was this translation helpful? Give feedback.
All reactions