Skip to content

Commit ab6bc17

Browse files
mesmith027crerichatreyspsungchun12
authored
Observability doc drafts (#3515)
Co-authored-by: Chris Rericha <tristipher@gmail.com> Co-authored-by: Trey Spiller <treyspiller@gmail.com> Co-authored-by: Sung Won Chung <sungwonchung3@gmail.com>
1 parent 17a2e4f commit ab6bc17

36 files changed

Lines changed: 326 additions & 11 deletions

docs/cloud/features/model_freshness.md

Lines changed: 0 additions & 3 deletions
This file was deleted.
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Development Environment
2+
3+
Tobiko Cloud extends the SQLMesh CLI to advance your development workflow. Instead of relying on a static terminal output isolated to your local machine when running `tcloud sqlmesh plan dev`, Tobiko Cloud tracks development history automatically displayed in a rich user interface. We want mental load at a minimum so you can focus on your most important work.
4+
5+
At its core, this transforms development from a single-player to a multi-player experience. Instead of sharing screenshots and scrolling through terminal history, all you have to do now is share a link to your work.
6+
7+
### When you might use this
8+
9+
**Team Collaboration**
10+
11+
Developers can easily see who is working on specific models, prevent workflow conflicts, and avoid duplicate efforts.
12+
<May revert to: The platform helps foster team collaboration by providing clear visibility into team activities. Developers can easily see who is working on specific models, prevent workflow conflicts, and avoid duplicate efforts. This creates a truly collaborative development environment where the team can work together seamlessly, supporting a dynamic, multiplayer-like development experience.>
13+
14+
**Performance Tracking**
15+
16+
You can monitor changes over time, review recent activities including successes and failures, and gain detailed insights into specific plan execution outcomes. You get a better sense of trends.
17+
18+
<TODO: add a screenshot with multiple plans and a glance at if things are running long or short.>
19+
20+
**Simplified Communication and Team Alignment**
21+
22+
Eliminate friction in sharing complex development context through manual pull requests or direct messages. These URLs serve as comprehensive summaries, displaying last run times, data intervals for incremental models, and detailed change information such as metadata modifications and model removals.
23+
24+
<Add gif of url opening in a new tab to feel how fast it should be.>
25+
26+
## Using the Environments Tab
27+
The Environments page shows an overview of all the environments that exist in your project (both yours and any your teammates have created).
28+
29+
![tcloud environment page](./development_environment/environments.png)
30+
31+
The page's table includes a link to each environment's page, along with the environment's creation date, the date it was last updated, and the date it will expire if not updated again. Clicking an environment's name from the main environments page takes you to its individual page.
32+
33+
![tcloud development environment](./development_environment/tcloud_development_environment.png)
34+
35+
## Individual Environment page
36+
The page begins with an at-a-glance summary of the most recent plan applied to the environment.
37+
38+
![tcloud environment page layout](./development_environment/tcloud_dev_env_labelled.png)
39+
40+
1. Its completion status and time of the last plan applied
41+
2. The latest time interval backfilled by the plan
42+
3. Count of models present in the environment
43+
4. An interactive visualization that summarizes the differences between the environment's models and the `prod` environment's models
44+
- The count of directly modified models is represented in blue
45+
- The count of added models is green
46+
- The count of removed models is red
47+
48+
??? "ProTip:"
49+
50+
If a stakeholder or else anyone on your team is looking to understand an environment you own and are working on, you can share the link with them and they will be able to access and see all of the information about your environment.
51+
52+
It's a great place to start to have open conversations about what was recently added, removed or changed in an environment!
53+
54+
55+
## Differences from Prod section
56+
57+
Development environments are used to prepare and test changes before deploying them to `prod`, with separate tabs for each type of change (directly modified, indirectly modified, metadata-only changes, added, removed). Below is a screen shot from a environement version that shows all these tab options.
58+
59+
![Prod Differences section with all options](./development_environment/dev_env_comprehensive.png)
60+
61+
In the summary, each model's name is a link to [its model page](./model.md). This links to the information about the version of the model used in _this environment_ not the overall prod model. This means that you can get insight into what your working on in dev instead of the "stale" version in prod ("stale" relative to your work).
62+
63+
## Plan history information
64+
65+
The plan applications chart is a calendar visualization of all plans that have been applied to the environment in the previous 2 weeks.
66+
67+
![Plan History Information](./development_environment/plan_history.png)
68+
69+
The chart represents days on its `x-axis` (each column is a day with the corresponding date across the top) and the time of day on its `y-axis` (each day begins at the top and ends at the bottom).
70+
71+
Each day displays zero or more horizantal bars representing `plan` duration. If no `plans` occurred on a day, no bars will be displayed. If multiple `plans` occurred on the same day, their horizantal bars will be stacked.
72+
73+
The chart uses color to convey the staus of a `plan` at a glance. Green is completed, grey is in progress, red is failed.
74+
75+
Hovering over a bar reveals summary information about the `plan`, including its completion status, start time, end time, total duration, and change summary. The summary includes a link to [the `plan`'s page](./plan.md).
63.7 KB
Loading
219 KB
Loading
106 KB
Loading
344 KB
Loading
357 KB
Loading
File renamed without changes.
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Models
2+
3+
From the main environments list, you can access individual models to explore their comprehensive observability features and detailed summary information. This centralized view provides quick access to critical model metrics and performance data.
4+
5+
Click the environment you want to explore from the environments list
6+
7+
![Tobiko Cloud environment page](./model/tcloud_environments.png)
8+
9+
Navigate to the Models section and click "Explore" to view available models
10+
11+
![Tobiko Cloud environment page explore models link](./model/tcloud_environment_explore-models.png)
12+
13+
Browse through the model list and select your desired model to access its detailed information
14+
15+
![Tobiko Cloud environment models list](./model/tcloud_model_list.png)
16+
17+
Each model presents a comprehensive summary overview that includes several key components and metrics for monitoring and analysis.
18+
19+
The following detailed information outlines the different sections:
20+
21+
![Tobiko Cloud model status and metadata](./model/tcloud_model_status-metadata.png)
22+
23+
- Current status: Provides visual representations of model health through freshness indicators and detailed daily execution graphs
24+
25+
- Model details: Features comprehensive tabs that display summary statistics, complete source code documentation, and interactive model lineage visualizations
26+
27+
![Tobiko Cloud model version history](./model/tcloud_model_2.png)
28+
29+
- Version history: Delivers a comprehensive chronological view of all model versions, with detailed information including:
30+
- Precise timestamp of version promotion
31+
- Clear indication of change impact (breaking or non-breaking modifications)
32+
- Direct access to the complete implementation plan code
33+
- Loaded intervals: these periods represent the time spans between consecutive cron job executions, from the start of one cron job to the end of the next cron job. These intervals are crucial for understanding the boundaries of data processing cycles
34+
- the table displays which specific model version was responsible for generating and processing data during each distinct cron interval, enabling precise tracking of version-specific outputs
35+
- helps track forward-only model changes by maintaining a clear chronological record of modifications, ensuring data consistency and preventing retroactive alterations
36+
- provides comprehensive visibility into completed data processing operations, allowing users to monitor progress and verify successful execution of scheduled tasks
37+
- Recent activity: Maintains a detailed log of version executions and comprehensive version audits
283 KB
Loading

0 commit comments

Comments
 (0)