Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 34 additions & 5 deletions docs/library/CsvProvider.fsx
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,8 @@ The following sample calls the `Load` method with an URL that points to a live C
let msft = Stocks.Load(__SOURCE_DIRECTORY__ + "/../data/MSFT.csv").Cache()

// Look at the most recent row. Note the 'Date' property
// is of type 'DateTime' and 'Open' has a type 'decimal'
// is of type 'DateOnly' (on .NET 6+) or 'DateTime' (on older targets)
// and 'Open' has a type 'decimal'
let firstRow = msft.Rows |> Seq.head
let lastDate = firstRow.Date
let lastOpen = firstRow.Open
Expand All @@ -107,8 +108,9 @@ collection of rows. We iterate over the rows using a `for` loop. As you can see
to the columns in the CSV file.

As you can see, the type provider also infers types of individual rows. The `Date`
property is inferred to be a `DateTime` (because the values in the sample file can all
be parsed as dates) while HLOC prices are inferred as `decimal`.
property is inferred to be a `DateOnly` on .NET 6 and later (because the values in the sample
file are date-only strings without a time component), or `DateTime` on older targets. HLOC
prices are inferred as `decimal`.

## Using units of measure

Expand Down Expand Up @@ -269,8 +271,13 @@ it by specifying the `InferRows` static parameter of `CsvProvider`. If you speci
Columns with only `0`, `1`, `Yes`, `No`, `True`, or `False` will be set to `bool`. Columns with numerical values
will be set to either `int`, `int64`, `decimal`, or `float`, in that order of preference.

If a value is missing in any row, by default the CSV type provider will infer a nullable (for `int` and `int64`) or an optional
(for `bool`, `DateTime` and `Guid`). When a `decimal` would be inferred but there are missing values, we will infer a
On .NET 6 and later, columns whose values are all date-only strings (without a time component, e.g. `2023-01-15`)
are automatically inferred as `DateOnly`. If a column mixes `DateOnly` and `DateTime` values, it is unified to `DateTime`.
Note that `TimeOnly` is **not** auto-inferred because it is ambiguous with `TimeSpan`; use an explicit schema annotation
(see the list of valid types below) to get a `TimeOnly` column.

If a value is missing in any row, by default the CSV type provider will infer a nullable (for `int`, `int64`, and `DateOnly`) or an optional
(for `bool`, `DateTime`, `DateTimeOffset`, and `Guid`). When a `decimal` would be inferred but there are missing values, we will infer a
`float` instead, and use `Double.NaN` to represent those missing values. The `string` type is already inherently nullable,
so by default, we won't generate a `string option`. If you prefer to use optionals in all cases, you can set the static parameter
`PreferOptionals` to `true`. In that case, you'll never get an empty string or a `Double.NaN` and will always get a `None` instead.
Expand Down Expand Up @@ -303,6 +310,12 @@ specify the units of measure. This will override both `AssumeMissingValues` and
* `guid`
* `guid?`
* `guid option`
* `dateonly` (.NET 6+ only)
* `dateonly?` (.NET 6+ only)
* `dateonly option` (.NET 6+ only)
* `timeonly` (.NET 6+ only)
* `timeonly?` (.NET 6+ only)
* `timeonly option` (.NET 6+ only)
* `string`
* `string option`.

Expand Down Expand Up @@ -373,6 +386,22 @@ for row in titanic2.Rows |> Seq.truncate 10 do

You can even mix and match the two syntaxes like this `Schema="int64,DidSurvive,PClass->Passenger Class=string"`

### DateOnly and TimeOnly (on .NET 6+)

On .NET 6 and later, date-only strings are automatically inferred as `DateOnly`. For example, a column
like `EventDate` containing values such as `2023-06-01` will be given the type `DateOnly`.

You can also explicitly request a `DateOnly` or `TimeOnly` column using schema annotations:

[lang=text]
EventDate,Duration (timeonly?)
2023-06-01,08:30:00
2023-06-02,

In the example above, `EventDate` is auto-inferred as `DateOnly` and `Duration` is explicitly
annotated as `timeonly?` (a nullable `TimeOnly`). Note that `TimeOnly` is never auto-inferred
(to avoid confusion with `TimeSpan`), so the schema annotation is required to get a `TimeOnly` column.

## Transforming CSV files

In addition to reading, `CsvProvider` also has support for transforming the row collection of CSV files. The operations
Expand Down
7 changes: 7 additions & 0 deletions docs/library/JsonProvider.fsx
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,13 @@ type-safe access to the values, but not in the original order (if order matters,
you can use the `mixed.JsonValue` property to get the underlying `JsonValue` and
process it dynamically as described in [the documentation for `JsonValue`](JsonValue.html).

### Inferring date types

String values in JSON that look like dates are inferred as `DateTime` or `DateTimeOffset`.
On .NET 6 and later, strings that represent a date without a time component (e.g. `"2023-01-15"`)
are inferred as `DateOnly`. Full datetime strings (e.g. `"2023-01-15T10:30:00"`) continue to be
inferred as `DateTime`.

### Inferring record types

Now let's look at a sample JSON document that contains a list of records. The
Expand Down