Skip to content

Iceberg Integration documentation#5918

Draft
MeelahMe wants to merge 62 commits intomasterfrom
inceberg
Draft

Iceberg Integration documentation#5918
MeelahMe wants to merge 62 commits intomasterfrom
inceberg

Conversation

@MeelahMe
Copy link
Copy Markdown
Contributor

@MeelahMe MeelahMe commented Mar 19, 2025

Description

To introduce documentation for the new Iceberg integration with Snowflake for InfluxDB Cloud Dedicated. To create documentation that allows users to export time series data snapshots into Apache Iceberg format and query them directly from Snowflake—without the need for complex ETL processes.

To-do

  • Feature overview and benefits
  • Create steps to request and configure the integration via InfluxData Sales and Support
  • Explain details on how the InfluxData team sets up the Snowflake external stage and catalog
  • Share a export process overview, including current limitations (e.g., customer-triggered exports not yet supported)
  • Explain guidance for querying Iceberg tables in Snowflake
  • Talk about considerations and known limitations

Why This Matters

This documentation gives users the information they need to request access and understand the setup and usage of the integration clearly and confidently.

Copy link
Copy Markdown
Contributor

@jstirnaman jstirnaman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this looks good. My suggestions are mostly for following our style and "tightening up". We should add examples where noted ASAP, but it shouldn't necessarily hold up publishing when the feature is ready.

@MeelahMe
Copy link
Copy Markdown
Contributor Author

Thank you for the suggestions and help! I should probably go slower and triple check that I am using the Google developers documentation style guide. @jstirnaman

@MeelahMe MeelahMe added Feature-WIP Feature is still a work in progress and is not yet merged. and removed InfluxDB Cloud Dedicated InfluxDB Cloud Serverless labels Mar 19, 2025
@jstirnaman
Copy link
Copy Markdown
Contributor

jstirnaman commented Mar 20, 2025

Thank you for the suggestions and help! I should probably go slower and triple check that I am using the Google developers documentation style guide. @jstirnaman

Ideally, Vale should be catching violations as well, but I think we're missing some config for the newer directories.
You can tell CoPilot to always use it and suggest fixes. I also give it an alias, like

If I say "gdd", it means Google Developer Documentation style.
Use gdd and condense the selection...

Copy link
Copy Markdown
Contributor

@jstirnaman jstirnaman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more minor suggestions

Comment on lines +153 to +161
#### Example: Enable Iceberg feature and export a snapshot

```sh
# Enable Iceberg feature
$ influxctl enable-iceberg

# Export a snapshot
$ influxctl export --namespace foo --table bar
```
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This commands should be used in the process above, not influxdb_iox

Copy link
Copy Markdown
Collaborator

@ritwika314 ritwika314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments, we should save the material for when more self-support features are added. Thank you!

@MeelahMe MeelahMe changed the title WIP: Iceberg Integration documentation Iceberg Integration documentation Mar 26, 2025
Copy link
Copy Markdown
Contributor

@jstirnaman jstirnaman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to the notes here, I'm pushing an update to your branch--I'll PR so you can see the diff.

@MeelahMe MeelahMe marked this pull request as ready for review March 31, 2025 18:35
@jstirnaman jstirnaman marked this pull request as draft April 2, 2025 20:06
MeelahMe and others added 4 commits May 27, 2025 09:41
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
Co-authored-by: Jason Stirnaman <jstirnaman@influxdata.com>
MeelahMe and others added 27 commits May 27, 2025 09:41
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
- Add directory and frontmatter
- Revise content to be Cloud Dedicated specific, omitting implementation and config detail that will be handled by InfluxData Support
- Fix code samples
- Organize, clean up headings

- TODO: Comment out influxctl and API code if necessary (until features are completed)
- TODO: Verify details with PM
- TODO: Move content as noted in the iceberg-for-snowflake.md comment
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
- Merge fixes
- Provide instruction for checking snapshot status
- Cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat Feature-WIP Feature is still a work in progress and is not yet merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants