Datastore-specific defaults

[jakedt]

Right now in the pkg/cmd/datastore package we have a set of defaults that apply equally to all datastores. Sometimes this is not a single default that works for all datastores. For example, spanner is limited to 100 usersets in the query splittler based on the structural limitations of the datastore. All other implementations can split at ~1k usersets before they start running into query size constraints.

It would make sense to allow some options to let the datastore pick its own default. We should audit our datastore flags to figure out other cases where this may make sense, and then update the pkg/cmd/datastore package accordingly.

[fearlessfe]

how to do this?

set the default SplitQueryCount to 0, and the in newPostgresDatastore or newSpannerDatastore ... func,
check whethe the SplitQueryCount is 0. if true, set the default value, else use the SplitQueryCount

[miparnisari]

PR #2867 partially addresses this for the Watch API options. It adds a DefaultsWatchOptions() method on the Datastore interface so each implementation declares its own defaults for CheckpointInterval, WatchBufferLength, etc. and removes the per-datastore custom flags from kg/cmd/datastore/datastore.go. The merge precedence lives in pkg/datalayer/watchoptions.go.

The same pattern could be extended to other flags by adding similar Defaults*() methods to the Datastore interface and having the DataLayer object call that method.

(The downside of that PR is that anyone calling datastore.Watch directly instead of datalayer.Watch will have to be careful to pass the correct value for each setting. datastore.Watch now assumes they are valid.)