Support for auxiliary resources and manifest (#67)

[damooo]

Summary

This PR addresses the issue #67, and issue #86 by introducing support for auxiliary resources. And thus makes changes to logical resource organization, operations, etc.

The storage system organizes resources into two distinct layers: primary and auxiliary. The two layers are complementary. The primary layer organizes resources spatially and hierarchically using containment relations, while the auxiliary layer attaches supplementary resources to each node in that hierarchy using auxiliary relations.

The auxiliary resource layer is flat. Each primary resource owns a local set of auxiliary resources. An auxiliary resource is an LWS resource that is used to provide additional information or functionality related to a unique principal primary resource. The lifetime of an auxiliary resource is bound to the lifetime of its principal resource. There MUST be a link from aux resource to it's principal resource with rel value principal, and a link from principal to it's each auxiliary resource with specialized rel value for each auxiliary relation called auxiliary rel. This link also have auxiliary=true param set. auxiliary rels are functional. linkset is such an auxiliary rel, and linkset resources are auxiliary resources.

Most importantly, though server can manage (like linkset), or interpret (like acl) the auxiliary resources, the client too has freedom to create auxiliary resources with any aux-rel subject to functional constraint. Thus auxiliary resources are free axis , available for client as a tool for modelling auxiliarity interoperably across servers. For example, an app can choose to persist rights-info, or data-provenanace, etc using custom auxiliary relations and content media-types. Servers need not to pre-configured to support each relation specifically.

Auxiliary resources can be discovered from linkset by filtering links with parameter auxiliary=true.

Container Manifest resource.

Each container resource has a manifest auxiliary resource. The manifest describes basic resource metadata like type, etc. and provide contained member listing for containers. This manifest resource has the media-type lws+json. Manifest persists as a separate auxiliary resource. Thus also freeing containers to have arbitrary hypermedia representations.

Operations:

The Create operation is changed to support creation of auxiliary members as well. At the time of creation clients provide membership link, linking it to target via either up or principal rel corresponding to containment membership or auxiliary membership. The target resource is called as patron in both cases.

This is the gist. It is a significant change. But I believe it as obvious generalization, instead of forceful feature. Request to review the model first. The grammar, etc. can be updated if the direction find some resonance. Feel free to say no.

Thanks a lot for your time and consideration.

CC: @pchampin , @acoburn

---

Preview | Diff

[kaefer3000]

I see that you copied those words from the current LWS-core, so nothing to complain about in your PR. Maybe a separate discussion is to be had what the storage system is and what is its root? The file system where the resource representations are physically located? On Windows, that is C:\?

[damooo]

I updated the proposal for following changes:

1. Now only containers have container manifest resource. Listing it's contained members.
2. Auxiliary links from a principal to auxiliary have auxiliary=true param set, distinguishing them from other links. This is enough for auxiliary discovery through linkset. No need to have them listed again in a general manifest resource.

This makes only containers having manifest, and tidy ups auxiliary resource linking. This is final proposal. Request to re-review.

@kaefer3000 @pchampin

[pchampin]

Thanks a lot @damooo .
There are part of this PR (additional terminology, clear definition of auxiliary resources) that I like very much and, IMO, could easily reach consensus in the WG.
Unfortunately, there are other parts (the shift of membership metadata into an auxiliary resource) which are much more controversial.

I think it would be valuable to split this PR into several steps, so that at least the (least controversial) first ones could be merged more easily. I'm aware that this requires a lot of work (in addition to what you already did!)...

[damooo]

@pchampin Thanks for review, and sorry for late reply.

Can you specify which parts can be in this PR? I essentially care about having support for auxiliary resources. Container representation issue I am going to close.

Specifically:

Is it acceptable to WG to allow clients to create auxiliary resources with arbitrary aux relation? Is the proposed generalization of POST with ability to choose auxiliary membership Okay?

Or does WG wants to make them purely server managed (Like auto creating all server supported aux resources at the time of creation of principal resource, etc.)? But this case will be problematic, as clients cannot have custom aux relations. And for some auxiliary relations, creation of the aux resource MUST be deliberate decision of client (For eg: WAC inheritence algo makes decisions based on existence, and not just content of acl resources in inheritense chain. Thus acl creation MUST be at the explicit intent of client, and cannot be auto decision of the server)

[gibsonf1]

Thanks a lot @damooo . There are part of this PR (additional terminology, clear definition of auxiliary resources) that I like very much and, IMO, could easily reach consensus in the WG. Unfortunately, there are other parts (the shift of membership metadata into an auxiliary resource) which are much more controversial.

I think it would be valuable to split this PR into several steps, so that at least the (least controversial) first ones could be merged more easily. I'm aware that this requires a lot of work (in addition to what you already did!)...

For implementations where all resources are effectively data (with files accessed one step away via that data) such as TwinPod, it makes no sense at all to have another data resource about the data resource - all the data would be in the one resource. I think its fine to talk about different representations of the resource, but any kind of spec that is implementation specific (such as an implementation that has to have multiple files to handle things) and requiring additional resources that would make no sense in a given implementation, I don't think should be in the spec.

[damooo]

@gibsonf1
Thanks for review.

I noticed that, in many of your comments you conflate the notion of information resource / document with the subject resource in the content. That may be your constrained design to have information resources be rdf resources describing themselves alone. That need not be the case.

For example, an information resource /a/b can have subject resource wikidata:z1234, or multiple others. It can be a nano publication that describe many other subjects. In rdf world relations between the documents, and subjects are many-to-many. Some may constrain them.

In any case, the documents need not be rdf-sources either. They can be html, json, xml, images, videos,... They need to have separate description documents.

And it is the standard practice to have acls, provenanace, changelog, etc. of any info resource to be separate. For, they are not part of it's content.

[gibsonf1]

@gibsonf1 Thanks for review.

I noticed that, in many of your comments you conflate the notion of information resource / document with the subject resource in the content. That may be your constrained design to have information resources be rdf resources describing themselves alone. That need not be the case.

For example, an information resource /a/b can have subject resource wikidata:z1234, or multiple others. It can be a nano publication that describe many other subjects. In rdf world relations between the documents, and subjects are many-to-many. Some may constrain them.

In any case, the documents need not be rdf-sources either. They can be html, json, xml, images, videos,... They need to have separate description documents.

And it is the standard practice to have acls, provenanace, changelog, etc. of any info resource to be separate. For, they are not part of it's content.

I'm not suggesting that it is not possible to have an infinite number of resources about other resources. My point is that for the spec, in the case of implementations that are data first only such as TwinPod, it would make no sense to have another data resource about the data resource when that all is in one resource. Having multiple representations of that same resource is fine. So my point is that to keep the spec implementation details neutral, make this about an auxiliary representation, rather than an auxiliary resource, as then all implementations can comply rather than only specific file based implementations.

[pchampin]

@damooo what characterizes an auxiliary resource, IMO, is that its lifetime is bound by that of another resource; this is obviously the case for Linkset or ACL resource. In those two examples, their content is (at least partially, for Linkesets) client-managed. Whether clients should be allowed to create new auxiliary resources is an open question, AFAICT.

@gibsonf1 I don't think the discussion about auxiliary resources is in any way implementation-specific. The notion of auxiliary resource (not "representation") already exists in Solid. Are you suggesting we should depart from it?

[gibsonf1]

@gibsonf1 I don't think the discussion about auxiliary resources is in any way implementation-specific. The notion of auxiliary resource (not "representation") already exists in Solid. Are you suggesting we should depart from it?

Yes, I am suggesting we depart from any implementation ideas in the spec such that the spec does not assume how an implementation does things, such as the need for multiple resources for a more file based system, when a data first system would not want to do this. So if we use "representation" instead of "resource", then the problem is solved for both file based and data first systems where the implementation can decide on how to create the representation from their internal resources.