Skip to content

Commit 15de10c

Browse files
authored
Merge pull request #1610 from fsprojects/copilot/sub-pr-1609
Docs: Update CsvProvider and JsonProvider for DateOnly/TimeOnly inference
2 parents 5d99718 + b14073a commit 15de10c

2 files changed

Lines changed: 41 additions & 5 deletions

File tree

docs/library/CsvProvider.fsx

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,8 @@ The following sample calls the `Load` method with an URL that points to a live C
8989
let msft = Stocks.Load(__SOURCE_DIRECTORY__ + "/../data/MSFT.csv").Cache()
9090

9191
// Look at the most recent row. Note the 'Date' property
92-
// is of type 'DateTime' and 'Open' has a type 'decimal'
92+
// is of type 'DateOnly' (on .NET 6+) or 'DateTime' (on older targets)
93+
// and 'Open' has a type 'decimal'
9394
let firstRow = msft.Rows |> Seq.head
9495
let lastDate = firstRow.Date
9596
let lastOpen = firstRow.Open
@@ -107,8 +108,9 @@ collection of rows. We iterate over the rows using a `for` loop. As you can see
107108
to the columns in the CSV file.
108109
109110
As you can see, the type provider also infers types of individual rows. The `Date`
110-
property is inferred to be a `DateTime` (because the values in the sample file can all
111-
be parsed as dates) while HLOC prices are inferred as `decimal`.
111+
property is inferred to be a `DateOnly` on .NET 6 and later (because the values in the sample
112+
file are date-only strings without a time component), or `DateTime` on older targets. HLOC
113+
prices are inferred as `decimal`.
112114
113115
## Using units of measure
114116
@@ -269,8 +271,13 @@ it by specifying the `InferRows` static parameter of `CsvProvider`. If you speci
269271
Columns with only `0`, `1`, `Yes`, `No`, `True`, or `False` will be set to `bool`. Columns with numerical values
270272
will be set to either `int`, `int64`, `decimal`, or `float`, in that order of preference.
271273
272-
If a value is missing in any row, by default the CSV type provider will infer a nullable (for `int` and `int64`) or an optional
273-
(for `bool`, `DateTime` and `Guid`). When a `decimal` would be inferred but there are missing values, we will infer a
274+
On .NET 6 and later, columns whose values are all date-only strings (without a time component, e.g. `2023-01-15`)
275+
are automatically inferred as `DateOnly`. If a column mixes `DateOnly` and `DateTime` values, it is unified to `DateTime`.
276+
Note that `TimeOnly` is **not** auto-inferred because it is ambiguous with `TimeSpan`; use an explicit schema annotation
277+
(see the list of valid types below) to get a `TimeOnly` column.
278+
279+
If a value is missing in any row, by default the CSV type provider will infer a nullable (for `int`, `int64`, and `DateOnly`) or an optional
280+
(for `bool`, `DateTime`, `DateTimeOffset`, and `Guid`). When a `decimal` would be inferred but there are missing values, we will infer a
274281
`float` instead, and use `Double.NaN` to represent those missing values. The `string` type is already inherently nullable,
275282
so by default, we won't generate a `string option`. If you prefer to use optionals in all cases, you can set the static parameter
276283
`PreferOptionals` to `true`. In that case, you'll never get an empty string or a `Double.NaN` and will always get a `None` instead.
@@ -303,6 +310,12 @@ specify the units of measure. This will override both `AssumeMissingValues` and
303310
* `guid`
304311
* `guid?`
305312
* `guid option`
313+
* `dateonly` (.NET 6+ only)
314+
* `dateonly?` (.NET 6+ only)
315+
* `dateonly option` (.NET 6+ only)
316+
* `timeonly` (.NET 6+ only)
317+
* `timeonly?` (.NET 6+ only)
318+
* `timeonly option` (.NET 6+ only)
306319
* `string`
307320
* `string option`.
308321
@@ -373,6 +386,22 @@ for row in titanic2.Rows |> Seq.truncate 10 do
373386
374387
You can even mix and match the two syntaxes like this `Schema="int64,DidSurvive,PClass->Passenger Class=string"`
375388
389+
### DateOnly and TimeOnly (on .NET 6+)
390+
391+
On .NET 6 and later, date-only strings are automatically inferred as `DateOnly`. For example, a column
392+
like `EventDate` containing values such as `2023-06-01` will be given the type `DateOnly`.
393+
394+
You can also explicitly request a `DateOnly` or `TimeOnly` column using schema annotations:
395+
396+
[lang=text]
397+
EventDate,Duration (timeonly?)
398+
2023-06-01,08:30:00
399+
2023-06-02,
400+
401+
In the example above, `EventDate` is auto-inferred as `DateOnly` and `Duration` is explicitly
402+
annotated as `timeonly?` (a nullable `TimeOnly`). Note that `TimeOnly` is never auto-inferred
403+
(to avoid confusion with `TimeSpan`), so the schema annotation is required to get a `TimeOnly` column.
404+
376405
## Transforming CSV files
377406
378407
In addition to reading, `CsvProvider` also has support for transforming the row collection of CSV files. The operations

docs/library/JsonProvider.fsx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,13 @@ type-safe access to the values, but not in the original order (if order matters,
119119
you can use the `mixed.JsonValue` property to get the underlying `JsonValue` and
120120
process it dynamically as described in [the documentation for `JsonValue`](JsonValue.html).
121121
122+
### Inferring date types
123+
124+
String values in JSON that look like dates are inferred as `DateTime` or `DateTimeOffset`.
125+
On .NET 6 and later, strings that represent a date without a time component (e.g. `"2023-01-15"`)
126+
are inferred as `DateOnly`. Full datetime strings (e.g. `"2023-01-15T10:30:00"`) continue to be
127+
inferred as `DateTime`.
128+
122129
### Inferring record types
123130
124131
Now let's look at a sample JSON document that contains a list of records. The

0 commit comments

Comments
 (0)