write_package function is way to opinionated

[ipimpat]

The current implementation of the write_package function in frictionless-r is too restrictive, as it couples the writing of the descriptor file and the resources. This design limits flexibility and adds unnecessary friction to workflows requiring customized output.

Limitations:

1. Hardcoded Descriptor File Name:
   write_package always writes the descriptor file as datapackage.json, without allowing file name customization.

2. No YAML Support:
   Descriptors can only be written in JSON format, personally I prefer YAML over JSON for descriptor files.

3. Resource Path Restrictions:
   Resources are always written to the base directory, with no way to specify custom paths for individual resources.

Proposed Solution:

Refactor the functionality by splitting it into two distinct functions:

1. write_package() - A function responsible only for writing the descriptor file in either JSON or YAML.
2. write_resource() - Export a new function dedicated to writing individual resources.

[peterdesmet]

Hi, thanks for these suggestions. Answers below.

1. Hardcoded Descriptor File Name:
   write_package always writes the descriptor file as datapackage.json, without allowing file name customization.

2. No YAML Support:
   Descriptors can only be written in JSON format, personally I prefer YAML over JSON for descriptor files.

Correct, that is because frictionless-r aims to follow the Data Package specification where it can, which states:

When a data package is available as a physical or virtual directory containing files, a descriptor MUST be named datapackage.json and it MUST be placed on the top-level of the file structure (relative to any other resources provided as part of the data package).

So while read_package() can read other filenames and YAML, we stick to the specification when creating packages. Note that conversion from JSON to YAML (or another filename) can easily be done outside frictionless-r.

3. Resource Path Restrictions:
   Resources are always written to the base directory, with no way to specify custom paths for individual resources.

True. This is in part to guarantee that datapackage.json is the top-level file (relative to the resources). While it would be possible to have subdirectories with the resources, we didn't implement that as part of write_package() to keep it simple.

Note that it is always possible to directly provide the path of a CSV file to add_resource() (rather than a dataframe). This gives you the freedom to use subdirectories. The function will return an error if you provide to absolute paths (./path) or higher relative paths (../../path), since those are not allowed by the spec.

Some of the granularity/customization that you are looking for in write_resource() is thus implemented at the level of add_resource(). A singular write_package() helps users create Data Packages that are compliant with the spec.

[ipimpat]

I appreciate the response, but I strongly disagree with these points.

You mention that frictionless-r enforces datapackage.json because of the specification. However, the same specification explicitly states:

A file containing a Data Package descriptor MAY have another name rather than datapackage.json as an internal part of some project or system if supported by corresponding implementations.

This means there is no strict requirement that the descriptor must always be named datapackage.json, only that it should be externally published under that name. The Frictionless ecosystem should enable, not restrict, users who want to manage descriptors under different names internally. There is no technical or compliance reason to prevent an option for specifying the file name in write_package(), as long as users understand the best practice for external publication.

Similarly, YAML is supported just fine by the Frictionless python package. If read_package() already supports YAML, then there is no reason why write_package() shouldn't as well. Forcing users to perform an additional conversion outside of frictionless-r adds unnecessary friction to their workflow.

I also disagree with the claim that enforcing a strict flat directory structure is necessary for compliance. The specification does not prohibit storing resources in subdirectories - it only states that the descriptor should be the top-level relative to resources. Nothing in the spec prevents write_package() from supporting subdirectories, as long as resource paths are properly referenced.

Saying that add_resource() allows specifying paths does not solve the problem, because users still have to work around write_package()'s limitations. If the package format allows subdirectories (which it does), then the tooling should support it instead of making users jump through extra steps.

The current write_package() implementation is unnecessarily restrictive and does not align with the flexibility allowed by the Data Package specification itself. There is no reason why frictionless-r should take a more rigid stance than the spec requires. Allowing configurable descriptor names, YAML support, and resource paths would enhance usability without breaking compliance.

I'd be happy to work on a merge request to address some of these issues and introduce more flexibility while keeping everything compliant with the specification. Let me know if this would be welcome, because these restrictions will make it very hard for R users in our organization to adopt the Data Package standard. We have a large number of R users, and the Python Frictionless package does not impose these kinds of restrictions or assumptions.