fix DataProcess.input_data to match DataDescription.source_data

[tmchartrand]

we may be able to start aligning these representations without breaking anything - input_data is a DataAsset object with a s3 url, while source_data is just an asset name.
maybe we want to allow either url or name in the DataAsset as a non-breaking change and move in practice to name?
we should also check more thoroughly on input_data usage - did the upgrader drop any input_data represented in 1.0? does the transfer service use this field?

[tmchartrand]

@dbirman so i just remembered to look into this more: the input_data field is not used at all (except one i created) in v2, and input_location in v1 is used in many pipeline processes but no analysis.
The usages in the v1 pipelines are in HCR, exaSPIM, SmartSPIM, and multiplane-ophys, and are currently getting dropped by the upgrader.

[tmchartrand]

@dougollerenshaw the point of this issue was to look into partial fixes to the situation that wouldn't require breaking changes, hence the v2.x - did you intend to push it back?

[dougollerenshaw]

I think I assigned this a schema 3.0 milestone label during one of our board reviews at Dan's suggestion. I think he assumed this was a breaking change. @dbirman can you comment?

[dbirman]

I think that asset_name is fine here, since asset_name can be resolved to the S3 location by going into that asset. If DataAsset is unused we should remove it probably.

For the upgrader I guess you want pipeline process input_location -> Code.input_data?

[tmchartrand]

So the plan would be to flatten DataProcess.input_data into a list of asset name strings, which is breaking but we know nobody is using it currently? Possibly also with a validator that screens out common mistakes like s3 urls or local paths as invalid names.

I think you could make the upgrader cover all the possible use cases by checking the input_location against the "asset name" pattern and moving the cases that don't match into DataProcess.code.parameters.input_location. Ideally also try to match to "s3://{bucket}/{name}/*" and fill the name from that as a fallback.