You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<td>Both Python Series and column expression should be supported as inputs, with the output reflecting the users input. Use a <ahref="https://docs.python.org/3/library/typing.html#typing.TypeVar">TypeVar</a> rather than directly using union types to make type checking easier.
35
+
<td>Both Python Series and column expression should be supported as inputs,
36
+
with the output reflecting the users input. Use a <a
rather than directly using union types to make type checking easier.
34
39
<p>
35
40
<strong>Special considerations for Series inputs:</strong>
36
41
<p>
37
-
If an input and output are both a Series with the same number of rows, make sure the output Series is implicitly (row identity) alignable with the original input. In other words, don't generate a table expression.
42
+
If an input and output are both a Series with the same number of rows, make sure
43
+
the output Series is implicitly (row identity) alignable with the original
44
+
input. In other words, don't generate a table expression.
38
45
<p>
39
-
If there are multiple Series inputs, they should be implicitly aligned if possible so as not to generate unnecessary table expressions.
46
+
If there are multiple Series inputs, they should be implicitly aligned if
47
+
possible so as not to generate unnecessary table expressions.
40
48
</td>
41
49
<td>Most scalar <ahref="https://docs.cloud.google.com/bigquery/docs/reference/standard-sql/functions-all">functions </a>accept one or more column expressions as input.
42
50
</td>
@@ -49,7 +57,10 @@ If there are multiple Series inputs, they should be implicitly aligned if possib
<td>Theoretically, we could try to get the type system to help the user disambiguate between this case and the "Column expression" case, but I think that's more trouble than it it's worth with regards to the expectations of Python users.
60
+
<td>Theoretically, we could try to get the type system to help the user
61
+
disambiguate between this case and the "Column expression" case, but I think
62
+
that's more trouble than it it's worth with regards to the expectations of
63
+
Python users.
53
64
</td>
54
65
<td><ul>
55
66
@@ -62,28 +73,38 @@ If there are multiple Series inputs, they should be implicitly aligned if possib
All columns are included as normal columns in the input table expression, including named index columns. If column names aren't unique or contain characters not compatible with BigQuery flexible column names, raise an error.
76
+
All columns are included as normal columns in the input table expression,
77
+
including named index columns. If column names aren't unique or contain
78
+
characters not compatible with BigQuery flexible column names, raise an error.
66
79
<p>
67
80
Outputs are unordered and unindexed to allow for cleaner mapping with SQL.
68
81
</td>
69
-
<td>Most APIs that take a table expression as input, also output a table expression with the same number of rows and passing through all unused columns. \
70
-
\
71
-
This should be used to pass through any index or ordering columns (as well as all other columns, if that's the SQL behavior), to allow for easy joining with the original input DataFrame.
82
+
<td>Most APIs that take a table expression as input, also output a table
83
+
expression with the same number of rows and passing through all unused
84
+
columns.
85
+
86
+
<p>This should be used to pass through any index or ordering columns (as well
87
+
as all other columns, if that's the SQL behavior), to allow for easy joining
88
+
with the original input DataFrame.
72
89
</td>
73
90
<td>Same number of rows as the input, so we should preserve index and ordering:<ul>
Different number of rows in output, so no need to preserve index or ordering. Default index / ordering should be specified with the Session's configuration:<ul>
95
+
Different number of rows in output, so no need to preserve index or ordering.
96
+
Default index / ordering should be specified with the Session's
@@ -174,7 +195,7 @@ If the associated table expression is input as a DataFrame, validate that these
174
195
<tr>
175
196
<td>Literal values
176
197
</td>
177
-
<td>corresponding literal Python value (e.g. int, float, string)
198
+
<td>corresponding literal Python value (e.g. int, float, string)
178
199
</td>
179
200
<td>For cases where scalar values are also supported, it should be safe to start with this and then expand to support expressions without a breaking change, as is done in <ahref="https://github.com/googleapis/google-cloud-python/pull/16606">https://github.com/googleapis/google-cloud-python/pull/16606</a>.
180
201
</td>
@@ -197,49 +218,42 @@ Would need some sort of bigframes deferred expression that can be tied to a tabl
197
218
</tr>
198
219
</table>
199
220
200
-
201
-
202
221
## Python policies
203
222
204
-
205
223
### Naming
206
224
207
225
Take the SQL function name, keyword name (used as a function name in Python), or argument name and transform them to lower_snake_case to reflect Python conventions.
208
226
209
-
210
227
### Internal expressions
211
228
212
-
Prefer creating deferred BigFrames expression objects where feasible. For example, all scalar outputting functions should return a `bigframes.pandas.Series` or `bigframes.core.col.Expression` that wraps a `bigframes.core.expression.Expression`.
229
+
Prefer creating deferred BigFrames expression objects where feasible. For
230
+
example, all scalar outputting functions should return a
231
+
`bigframes.pandas.Series` or `bigframes.core.col.Expression` that wraps a
232
+
`bigframes.core.expression.Expression`.
213
233
214
-
Prefer returning a `bigframes.pandas.DataFrame` that wraps a `bigframes.bigframes.core`
215
-
216
-
217
-
```
218
-
.bigframe_node.BigFrameNode. See from_bq_data_source in bigframes.core
219
-
```
220
-
221
-
222
-
`.array_value.ArrayValue`, as an example.
223
-
224
-
Exceptions to this are cases where the output schema is likely to evolve or differ in ways that are difficult to model, such as the `ML.PREDICT` SQL function, where output columns differ based on the model type and support for model types are frequently added to BigQuery. In these exceptional cases, the generated query should run immediately and the returned value should wrap the results.
234
+
Prefer returning a `bigframes.pandas.DataFrame` that wraps a
235
+
`bigframes.bigframes.core.bigframe_node.BigFrameNode`. See `from_bq_data_source` in
236
+
`bigframes.core.array_value.ArrayValue`, as an example.
225
237
238
+
Exceptions to this are cases where the output schema is likely to evolve or
239
+
differ in ways that are difficult to model, such as the `ML.PREDICT` SQL
240
+
function, where output columns differ based on the model type and support for
241
+
model types are frequently added to BigQuery. In these exceptional cases, the
242
+
generated query should run immediately and the returned value should wrap the
243
+
results.
226
244
227
245
### Argument syntax details
228
246
229
247
Arguments in Python can be one of:
230
248
249
+
* Positional
250
+
* Supported by `*args` in Python, but not recommended. Positional arguments in SQL should map to named positional or keyword arguments in Python.
251
+
* Positional or keyword
252
+
* Required positional arguments should be positional, just like they are in SQL.
253
+
* Keyword-only
254
+
* All other arguments should be keyword-only. Use `, * ,` Python syntax to achieve this.
231
255
232
-
233
-
* Positional
234
-
* Supported by `*args` in Python, but not recommended. Positional arguments in SQL should map to named positional or keyword arguments in Python.
235
-
* Positional or keyword
236
-
* Required positional arguments should be positional, just like they are in SQL.
237
-
* Keyword-only
238
-
* All other arguments should be keyword-only. Use `, * ,` Python syntax to achieve this.
239
-
240
-
For optional parameters, use an optional sentinel (see: https://stackoverflow.com/a/76606310/101923) and omit the value from the generated SQL if the user doesn't explicitly provide one. This ensures that an explicit NULL / None value can be passed in.
241
-
242
-
256
+
For optional parameters, use an optional sentinel (see: <https://stackoverflow.com/a/76606310/101923>) and omit the value from the generated SQL if the user doesn't explicitly provide one. This ensures that an explicit NULL / None value can be passed in.
0 commit comments