Skip to content

Commit 49d4888

Browse files
github-actions[bot]CopilotdsymeRepo Assist
authored
[Repo Assist] Add PreferDateTimeOffset parameter to CsvProvider, JsonProvider, and XmlProvider (#1668)
* Add PreferDateTimeOffset parameter to CsvProvider, JsonProvider, XmlProvider Adds a new PreferDateTimeOffset static parameter (default: false) to CsvProvider, JsonProvider, and XmlProvider. When set to true, date-time values that would normally be inferred as DateTime are instead inferred as DateTimeOffset (using the local UTC offset). Values that already parse with an explicit timezone offset are already inferred as DateTimeOffset regardless of this parameter. This addresses the long-standing request in #1100 (configure default DateTimeKind for dates without timezone) and #1072 (control over timezone in CSV date writing). Closes #1100 Closes #1072 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger CI checks * Add PreferDateTimeOffset documentation to CsvProvider, JsonProvider, XmlProvider, and TypeInference docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger CI checks --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Don Syme <dsyme@users.noreply.github.com> Co-authored-by: Repo Assist <repo-assist@github.com>
1 parent e9bc521 commit 49d4888

10 files changed

Lines changed: 192 additions & 39 deletions

File tree

RELEASE_NOTES.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
## 8.1.0-beta
44

5+
- Add `PreferDateTimeOffset` parameter to `CsvProvider`, `JsonProvider`, and `XmlProvider`: when true, date-time values without an explicit timezone offset are inferred as `DateTimeOffset` (using local offset) instead of `DateTime` (closes #1100, #1072)
56
- Make `Http.AppendQueryToUrl` public (closes #1325)
67
- Add `PreferOptionals` parameter to `JsonProvider` and `XmlProvider` (defaults to `true` to match existing behavior; set to `false` to use empty string or `NaN` for missing values, like the CsvProvider default) (closes #649)
78

docs/library/CsvProvider.fsx

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,8 +90,8 @@ let msft = Stocks.Load(__SOURCE_DIRECTORY__ + "/../data/MSFT.csv").Cache()
9090

9191
// Look at the most recent row. Note the 'Date' property
9292
// is of type 'DateTime' by default. Set PreferDateOnly = true
93-
// to use 'DateOnly' on .NET 6+
94-
// and 'Open' has a type 'decimal'
93+
// to use 'DateOnly' on .NET 6+, or PreferDateTimeOffset = true
94+
// to use 'DateTimeOffset'. 'Open' has a type 'decimal'.
9595
let firstRow = msft.Rows |> Seq.head
9696
let lastDate = firstRow.Date
9797
let lastOpen = firstRow.Open
@@ -111,6 +111,7 @@ to the columns in the CSV file.
111111
As you can see, the type provider also infers types of individual rows. The `Date`
112112
property is inferred as `DateTime` by default. When you set `PreferDateOnly = true`
113113
on .NET 6 and later, date-only strings (without a time component) are inferred as `DateOnly`.
114+
Set `PreferDateTimeOffset = true` to infer date-time values as `DateTimeOffset`.
114115
HLOC prices are inferred as `decimal`.
115116
116117
## Using units of measure
@@ -276,6 +277,10 @@ On .NET 6 and later, when you set `PreferDateOnly = true`, columns whose values
276277
are inferred as `DateOnly`. Time-only strings are inferred as `TimeOnly`. If a column mixes `DateOnly` and `DateTime` values, it is unified to `DateTime`.
277278
By default (`PreferDateOnly = false`), all date values are inferred as `DateTime` for backward compatibility.
278279
280+
Set `PreferDateTimeOffset = true` to infer all date-time columns (that would otherwise be `DateTime`) as `DateTimeOffset` instead.
281+
Values that already carry an explicit timezone offset (e.g. `2023-06-15T10:30:00+05:30`) are always inferred as `DateTimeOffset` regardless of this flag.
282+
`PreferDateTimeOffset` and `PreferDateOnly` are independent: `DateOnly` columns stay as `DateOnly` even when `PreferDateTimeOffset=true`.
283+
279284
If a value is missing in any row, by default the CSV type provider will infer a nullable (for `int`, `int64`, and `DateOnly`) or an optional
280285
(for `bool`, `DateTime`, `DateTimeOffset`, and `Guid`). When a `decimal` would be inferred but there are missing values, we will infer a
281286
`float` instead, and use `Double.NaN` to represent those missing values. The `string` type is already inherently nullable,
@@ -403,6 +408,13 @@ You can also explicitly request a `DateOnly` or `TimeOnly` column using schema a
403408
In the example above, `EventDate` is explicitly annotated as `dateonly` and `Duration` is explicitly
404409
annotated as `timeonly?` (a nullable `TimeOnly`).
405410
411+
### DateTimeOffset
412+
413+
Set `PreferDateTimeOffset = true` to infer all date-time columns (that would otherwise be `DateTime`) as
414+
`DateTimeOffset` instead. This is useful when you need to preserve or work with timezone-aware values.
415+
Values that already carry an explicit timezone offset in the sample data (e.g. `2023-06-15T10:30:00+05:30`)
416+
are always inferred as `DateTimeOffset` regardless of this flag.
417+
406418
## Transforming CSV files
407419
408420
In addition to reading, `CsvProvider` also has support for transforming the row collection of CSV files. The operations

docs/library/JsonProvider.fsx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,10 @@ On .NET 6 and later, when you set `PreferDateOnly = true`, strings that represen
126126
are inferred as `DateOnly`, and time-only strings are inferred as `TimeOnly`. By default (`PreferDateOnly = false`),
127127
all dates are inferred as `DateTime` for backward compatibility.
128128
129+
Set `PreferDateTimeOffset = true` to infer all date-time values (that would otherwise be `DateTime`) as `DateTimeOffset`.
130+
Values that already contain an explicit timezone offset (e.g. `"2023-06-15T10:30:00+05:30"`) are always inferred as
131+
`DateTimeOffset` regardless of this flag.
132+
129133
### Inferring record types
130134
131135
Now let's look at a sample JSON document that contains a list of records. The

docs/library/TypeInference.fsx

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,14 +83,21 @@ The providers recognise date and time strings in standard ISO 8601 formats:
8383
| Inferred Type | When Used | Example Value |
8484
|---|---|---|
8585
| `DateTime` | Date + time strings (default) | `"2023-06-15T12:00:00"` |
86-
| `DateTimeOffset` | Date + time + timezone offset | `"2023-06-15T12:00:00+02:00"` |
86+
| `DateTimeOffset` | Date + time + timezone offset (always) | `"2023-06-15T12:00:00+02:00"` |
87+
| `DateTimeOffset` | Any date + time string when `PreferDateTimeOffset=true` | `"2023-06-15T12:00:00"` |
8788
| `DateOnly` (.NET 6+) | Date-only strings when `PreferDateOnly=true` | `"2023-06-15"` |
8889
| `TimeOnly` (.NET 6+) | Time-only strings when `PreferDateOnly=true` | `"12:00:00"` |
8990
9091
By default (`PreferDateOnly = false`), date-only strings such as `"2023-06-15"` are
9192
inferred as `DateTime` for backward compatibility. Set `PreferDateOnly = true` on
9293
.NET 6 and later to infer them as `DateOnly` instead.
9394
95+
Set `PreferDateTimeOffset = true` to infer all date-time values (that would otherwise be
96+
`DateTime`) as `DateTimeOffset` instead. Values that already carry an explicit timezone
97+
offset (e.g. `"2023-06-15T12:00:00+02:00"`) are always inferred as `DateTimeOffset`
98+
regardless of this flag. `PreferDateTimeOffset` and `PreferDateOnly` are independent:
99+
`DateOnly` values stay as `DateOnly` even when `PreferDateTimeOffset=true`.
100+
94101
If a column mixes `DateOnly` and `DateTime` values, they are unified to `DateTime`.
95102
96103
## Missing Values and Optionals
@@ -270,6 +277,7 @@ The following static parameters let you override the default inference behaviour
270277
| `InferRows` | CSV | Number of rows to use for type inference (default 1000; 0 = all rows) |
271278
| `SampleIsList` | JSON, XML | Treat the top-level array as a list of sample objects, not a single sample |
272279
| `PreferDateOnly` | CSV, JSON, XML | Infer date-only strings as `DateOnly` on .NET 6+ (default `false`) |
280+
| `PreferDateTimeOffset` | CSV, JSON, XML | Infer all date-time values as `DateTimeOffset` instead of `DateTime` (default `false`) |
273281
| `InferenceMode` | JSON, XML | Enable inline schema annotations (`ValuesAndInlineSchemasHints` or `ValuesAndInlineSchemasOverrides`) |
274282
| `Schema` | CSV | Override column names and/or types directly |
275283

docs/library/XmlProvider.fsx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,20 @@ attributes in our sample, so it is inferred as `string`), and then it recursivel
330330
the content of all `<div>` elements. If the element does not contain nested elements,
331331
then we print the `Value` (inner text).
332332
333+
## Inferring date types
334+
335+
Element and attribute values that look like dates are inferred as `DateTime` or `DateTimeOffset`.
336+
Values that already carry an explicit timezone offset (e.g. `"2023-06-15T12:00:00+02:00"`) are always
337+
inferred as `DateTimeOffset`.
338+
339+
On .NET 6 and later, when you set `PreferDateOnly = true`, values that represent a date without a time
340+
component (e.g. `"2023-01-15"`) are inferred as `DateOnly`, and time-only values as `TimeOnly`.
341+
By default (`PreferDateOnly = false`), all date values are inferred as `DateTime` for backward compatibility.
342+
343+
Set `PreferDateTimeOffset = true` to infer all date-time values (that would otherwise be `DateTime`) as
344+
`DateTimeOffset`. This is useful when you need timezone-aware values. `PreferDateTimeOffset` and
345+
`PreferDateOnly` are independent: `DateOnly` values stay as `DateOnly` even when `PreferDateTimeOffset=true`.
346+
333347
## Loading Directly from a File or URL
334348
335349
In many cases, we might want to define schema using a local sample file, but then directly

src/FSharp.Data.DesignTime/Csv/CsvProvider.fs

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ type public CsvProvider(cfg: TypeProviderConfig) as this =
5959
let strictBooleans = args.[17] :?> bool
6060
let useOriginalNames = args.[18] :?> bool
6161
let preferFloats = args.[19] :?> bool
62+
let preferDateTimeOffset = args.[20] :?> bool
6263

6364
// This provider already has a schema mechanism, so let's disable inline schemas.
6465
let inferenceMode = InferenceMode'.ValuesOnly
@@ -120,15 +121,22 @@ type public CsvProvider(cfg: TypeProviderConfig) as this =
120121
strictBooleans,
121122
preferFloats
122123
)
124+
125+
let fields =
123126
#if NET6_0_OR_GREATER
124-
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
125-
fields
126-
else
127-
fields |> List.map StructuralInference.downgradeNet6PrimitiveProperty
127+
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
128+
fields
129+
else
130+
fields |> List.map StructuralInference.downgradeNet6PrimitiveProperty
128131
#else
129-
fields
132+
fields
130133
#endif
131134

135+
if preferDateTimeOffset then
136+
fields |> List.map StructuralInference.upgradeToDateTimeOffsetPrimitiveProperty
137+
else
138+
fields
139+
132140
use _holder = IO.logTime "TypeGeneration" sample
133141

134142
let csvType, csvErasedType, rowType, stringArrayToRow, rowToStringArray =
@@ -244,7 +252,8 @@ type public CsvProvider(cfg: TypeProviderConfig) as this =
244252
ProvidedStaticParameter("PreferDateOnly", typeof<bool>, parameterDefaultValue = false)
245253
ProvidedStaticParameter("StrictBooleans", typeof<bool>, parameterDefaultValue = false)
246254
ProvidedStaticParameter("UseOriginalNames", typeof<bool>, parameterDefaultValue = false)
247-
ProvidedStaticParameter("PreferFloats", typeof<bool>, parameterDefaultValue = false) ]
255+
ProvidedStaticParameter("PreferFloats", typeof<bool>, parameterDefaultValue = false)
256+
ProvidedStaticParameter("PreferDateTimeOffset", typeof<bool>, parameterDefaultValue = false) ]
248257

249258
let helpText =
250259
"""<summary>Typed representation of a CSV file.</summary>
@@ -270,7 +279,8 @@ type public CsvProvider(cfg: TypeProviderConfig) as this =
270279
(e.g. 'MyCompany.MyAssembly, resource_name.csv'). This is useful when exposing types generated by the type provider.</param>
271280
<param name='PreferDateOnly'>When true on .NET 6+, date-only strings are inferred as DateOnly and time-only strings as TimeOnly. Defaults to false for backward compatibility.</param>
272281
<param name='StrictBooleans'>When true, only <c>true</c> and <c>false</c> (case-insensitive) are inferred as boolean. Values such as <c>0</c>, <c>1</c>, <c>yes</c>, and <c>no</c> are treated as integers or strings respectively. Defaults to false.</param>
273-
<param name='UseOriginalNames'>When true, CSV column header names are used as-is for generated property names instead of being normalized (e.g. capitalizing the first letter). Defaults to false.</param>"""
282+
<param name='UseOriginalNames'>When true, CSV column header names are used as-is for generated property names instead of being normalized (e.g. capitalizing the first letter). Defaults to false.</param>
283+
<param name='PreferDateTimeOffset'>When true, date-time strings without an explicit timezone offset are inferred as DateTimeOffset (using the local offset) instead of DateTime. Defaults to false.</param>"""
274284

275285
do csvProvTy.AddXmlDoc helpText
276286
do csvProvTy.DefineStaticParameters(parameters, buildTypes)

src/FSharp.Data.DesignTime/Json/JsonProvider.fs

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ type public JsonProvider(cfg: TypeProviderConfig) as this =
6161
let useOriginalNames = args.[12] :?> bool
6262
let omitNullFields = args.[13] :?> bool
6363
let preferOptionals = args.[14] :?> bool
64+
let preferDateTimeOffset = args.[15] :?> bool
6465

6566
let inferenceMode =
6667
InferenceMode'.FromPublicApi(inferenceMode, inferTypesFromValues)
@@ -109,15 +110,22 @@ type public JsonProvider(cfg: TypeProviderConfig) as this =
109110
""
110111
sampleJson)
111112
|> Array.fold (StructuralInference.subtypeInfered (not preferOptionals)) InferedType.Top
113+
114+
let rawInfered =
112115
#if NET6_0_OR_GREATER
113-
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
114-
rawInfered
115-
else
116-
StructuralInference.downgradeNet6Types rawInfered
116+
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
117+
rawInfered
118+
else
119+
StructuralInference.downgradeNet6Types rawInfered
117120
#else
118-
rawInfered
121+
rawInfered
119122
#endif
120123

124+
if preferDateTimeOffset then
125+
StructuralInference.upgradeToDateTimeOffset rawInfered
126+
else
127+
rawInfered
128+
121129
use _holder = IO.logTime "TypeGeneration" (if schema <> "" then schema else sample)
122130

123131
let ctx =
@@ -178,7 +186,8 @@ type public JsonProvider(cfg: TypeProviderConfig) as this =
178186
ProvidedStaticParameter("PreferDateOnly", typeof<bool>, parameterDefaultValue = false)
179187
ProvidedStaticParameter("UseOriginalNames", typeof<bool>, parameterDefaultValue = false)
180188
ProvidedStaticParameter("OmitNullFields", typeof<bool>, parameterDefaultValue = false)
181-
ProvidedStaticParameter("PreferOptionals", typeof<bool>, parameterDefaultValue = true) ]
189+
ProvidedStaticParameter("PreferOptionals", typeof<bool>, parameterDefaultValue = true)
190+
ProvidedStaticParameter("PreferDateTimeOffset", typeof<bool>, parameterDefaultValue = false) ]
182191

183192
let helpText =
184193
"""<summary>Typed representation of a JSON document.</summary>
@@ -205,7 +214,8 @@ type public JsonProvider(cfg: TypeProviderConfig) as this =
205214
<param name='PreferDateOnly'>When true on .NET 6+, date-only strings (e.g. "2023-01-15") are inferred as DateOnly and time-only strings as TimeOnly. Defaults to false for backward compatibility.</param>
206215
<param name='UseOriginalNames'>When true, JSON property names are used as-is for generated property names instead of being normalized to PascalCase. Defaults to false.</param>
207216
<param name='OmitNullFields'>When true, optional fields with value None are omitted from the generated JSON rather than serialized as null. Defaults to false.</param>
208-
<param name='PreferOptionals'>When set to true (default), inference will use the option type for missing or null values. When false, inference will prefer to use empty string or double.NaN for missing values where possible, matching the default CsvProvider behavior.</param>"""
217+
<param name='PreferOptionals'>When set to true (default), inference will use the option type for missing or null values. When false, inference will prefer to use empty string or double.NaN for missing values where possible, matching the default CsvProvider behavior.</param>
218+
<param name='PreferDateTimeOffset'>When true, date-time strings without an explicit timezone offset are inferred as DateTimeOffset (using the local offset) instead of DateTime. Defaults to false.</param>"""
209219

210220
do jsonProvTy.AddXmlDoc helpText
211221
do jsonProvTy.DefineStaticParameters(parameters, buildTypes)

src/FSharp.Data.DesignTime/Xml/XmlProvider.fs

Lines changed: 29 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ type public XmlProvider(cfg: TypeProviderConfig) as this =
5454
let dtdProcessing = args.[11] :?> string
5555
let useOriginalNames = args.[12] :?> bool
5656
let preferOptionals = args.[13] :?> bool
57+
let preferDateTimeOffset = args.[14] :?> bool
5758

5859
let inferenceMode =
5960
InferenceMode'.FromPublicApi(inferenceMode, inferTypesFromValues)
@@ -80,15 +81,22 @@ type public XmlProvider(cfg: TypeProviderConfig) as this =
8081

8182
let t =
8283
schemaSet |> XsdParsing.getElements |> List.ofSeq |> XsdInference.inferElements
84+
85+
let t =
8386
#if NET6_0_OR_GREATER
84-
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
85-
t
86-
else
87-
StructuralInference.downgradeNet6Types t
87+
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
88+
t
89+
else
90+
StructuralInference.downgradeNet6Types t
8891
#else
89-
t
92+
t
9093
#endif
9194

95+
if preferDateTimeOffset then
96+
StructuralInference.upgradeToDateTimeOffset t
97+
else
98+
t
99+
92100
use _holder = IO.logTime "TypeGeneration" sample
93101

94102
let ctx =
@@ -137,15 +145,22 @@ type public XmlProvider(cfg: TypeProviderConfig) as this =
137145
(not preferOptionals)
138146
globalInference
139147
|> Array.fold (StructuralInference.subtypeInfered (not preferOptionals)) InferedType.Top
148+
149+
let t =
140150
#if NET6_0_OR_GREATER
141-
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
142-
t
143-
else
144-
StructuralInference.downgradeNet6Types t
151+
if preferDateOnly && ProviderHelpers.runtimeSupportsNet6Types cfg.RuntimeAssembly then
152+
t
153+
else
154+
StructuralInference.downgradeNet6Types t
145155
#else
146-
t
156+
t
147157
#endif
148158

159+
if preferDateTimeOffset then
160+
StructuralInference.upgradeToDateTimeOffset t
161+
else
162+
t
163+
149164
use _holder = IO.logTime "TypeGeneration" sample
150165

151166
let ctx =
@@ -205,7 +220,8 @@ type public XmlProvider(cfg: TypeProviderConfig) as this =
205220
ProvidedStaticParameter("PreferDateOnly", typeof<bool>, parameterDefaultValue = false)
206221
ProvidedStaticParameter("DtdProcessing", typeof<string>, parameterDefaultValue = "Ignore")
207222
ProvidedStaticParameter("UseOriginalNames", typeof<bool>, parameterDefaultValue = false)
208-
ProvidedStaticParameter("PreferOptionals", typeof<bool>, parameterDefaultValue = true) ]
223+
ProvidedStaticParameter("PreferOptionals", typeof<bool>, parameterDefaultValue = true)
224+
ProvidedStaticParameter("PreferDateTimeOffset", typeof<bool>, parameterDefaultValue = false) ]
209225

210226
let helpText =
211227
"""<summary>Typed representation of a XML file.</summary>
@@ -232,7 +248,8 @@ type public XmlProvider(cfg: TypeProviderConfig) as this =
232248
<param name='PreferDateOnly'>When true on .NET 6+, date-only strings are inferred as DateOnly and time-only strings as TimeOnly. Defaults to false for backward compatibility.</param>
233249
<param name='DtdProcessing'>Controls how DTD declarations in the XML are handled. Accepted values: "Ignore" (default, silently skips DTD processing, safe for most cases), "Prohibit" (throws on any DTD declaration), "Parse" (enables full DTD processing including entity expansion, use with caution).</param>
234250
<param name='UseOriginalNames'>When true, XML element and attribute names are used as-is for generated property names instead of being normalized to PascalCase. Defaults to false.</param>
235-
<param name='PreferOptionals'>When set to true (default), inference will use the option type for missing or absent values. When false, inference will prefer to use empty string or double.NaN for missing values where possible, matching the default CsvProvider behavior.</param>"""
251+
<param name='PreferOptionals'>When set to true (default), inference will use the option type for missing or absent values. When false, inference will prefer to use empty string or double.NaN for missing values where possible, matching the default CsvProvider behavior.</param>
252+
<param name='PreferDateTimeOffset'>When true, date-time strings without an explicit timezone offset are inferred as DateTimeOffset (using the local offset) instead of DateTime. Defaults to false.</param>"""
236253

237254

238255
do xmlProvTy.AddXmlDoc helpText

0 commit comments

Comments
 (0)