Skip to content

Commit c49c3db

Browse files
authored
Make fetchdf notebook magic a cell magic (#989)
* Change fetchdf magic to cell_magic * Update notebook reference * Add %context magic to notebook reference * Wordsmith
1 parent 4bbc8a1 commit c49c3db

File tree

2 files changed

+47
-11
lines changed

2 files changed

+47
-11
lines changed

docs/reference/notebook.md

Lines changed: 37 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,50 @@
11
# Notebook
22

3-
SQLMesh supports Jupyter and Databricks Notebooks. Magics are loaded automatically and use the variable `context` to locate a SQLMesh project.
3+
SQLMesh supports Jupyter and Databricks Notebooks. Magics are loaded automatically when `sqlmesh` or one of its modules is imported.
4+
5+
## SQLMesh project setup
6+
7+
Notebooks locate a SQLMesh project by setting a `context` with either the Python API or a notebook magic.
8+
9+
Set the context with the Python `Context` function as follows:
410

511
```python
612
from sqlmesh import Context
713

814
context = Context(paths="path_to_sqlmesh_project")
915
```
1016

11-
To create a fresh project in Databricks using the example project under Get Started, use the following.
17+
Alternatively, set the context with a notebook magic:
18+
19+
``` python
20+
import sqlmesh
21+
22+
%context path_to_sqlmesh_project
23+
```
24+
25+
### Quick start project
26+
27+
If desired, you can create the [quick start example project](../quick_start.md) with the `init_example_project` function:
1228

1329
```python
1430
from sqlmesh.cli.example_project import init_example_project
1531

16-
init_example_project("local_dbfs_path")
32+
init_example_project("path_to_project_directory")
33+
```
34+
35+
### Databricks notebooks
36+
To use your Databricks cluster, update the `default_connection` and `test_connection` in the `config.yaml` file.
37+
38+
See the [Execution Engines](../integrations/engines.md#databricks) page for information on configuring a Databricks connection.
39+
40+
## context
41+
```
42+
%context paths
43+
44+
positional arguments:
45+
paths Path(s) to one ore more directories containing SQLMesh
46+
projects
1747
```
18-
Update the `default_connection` and `test_connection` in the config.yaml file to use your Databricks cluster. For Databricks connection configuration, see the [Execution Engines](https://sqlmesh.readthedocs.io/en/stable/integrations/engines/) page.
1948

2049
## plan
2150
```
@@ -105,12 +134,13 @@ options:
105134

106135
## fetchdf
107136
```
108-
%fetchdf [df_var]
137+
%%fetchdf [df_var]
109138
110-
Fetches a dataframe from sql, optionally storing it in a variable.
139+
Fetch a dataframe with a cell's SQL query, optionally storing it in a variable.
111140
112141
positional arguments:
113-
df_var An optional variable name to store the resulting dataframe.
142+
df_var An optional variable name to store the resulting
143+
dataframe.
114144
```
115145

116146
## test

sqlmesh/magics.py

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,13 @@
44
from collections import defaultdict
55

66
from IPython.core.display import HTML, display
7-
from IPython.core.magic import Magics, line_cell_magic, line_magic, magics_class
7+
from IPython.core.magic import (
8+
Magics,
9+
cell_magic,
10+
line_cell_magic,
11+
line_magic,
12+
magics_class,
13+
)
814
from IPython.core.magic_arguments import argument, magic_arguments, parse_argstring
915

1016
from sqlmesh.core import constants as c
@@ -32,7 +38,7 @@ def display(self) -> t.Callable:
3238
from sqlmesh import runtime_env
3339

3440
if runtime_env.is_databricks:
35-
# Use Databrick's special display instead of the normal IPython display
41+
# Use Databricks' special display instead of the normal IPython display
3642
return self._shell.user_ns["display"]
3743
return display
3844

@@ -271,7 +277,7 @@ def plan(self, line: str) -> None:
271277
)
272278
@argument("--start", "-s", type=str, help="Start date to evaluate.")
273279
@argument("--end", "-e", type=str, help="End date to evaluate.")
274-
@argument("--skip-janitor", action="store_true", help="Skip the jantitor task.")
280+
@argument("--skip-janitor", action="store_true", help="Skip the janitor task.")
275281
@line_magic
276282
def run_dag(self, line: str) -> None:
277283
"""Evaluate the DAG of models using the built-in scheduler."""
@@ -349,7 +355,7 @@ def render(self, line: str) -> None:
349355
type=str,
350356
help="An optional variable name to store the resulting dataframe.",
351357
)
352-
@line_cell_magic
358+
@cell_magic
353359
def fetchdf(self, line: str, sql: str) -> None:
354360
"""Fetches a dataframe from sql, optionally storing it in a variable."""
355361
args = parse_argstring(self.fetchdf, line)

0 commit comments

Comments
 (0)