Getting Started

Versatile Data Kit (VDK) is a data framework that enables Data Engineers to

🧑‍💻 develop,
▶️ run,
📊 and manage data workloads, aka data jobs

What Problem Does Versatile Data Kit Solve?

• Ingest data from different sources.
• Use Python/SQL and VDK templates to transform data.
• Package, version, and deploy data applications while dealing with credentials, retries, and reconnects.
• Provide built-in monitoring and smart notification capabilities.
• Track code and data modifications for quicker troubleshooting and version rollback.

See our introduction blog post

Installation

➡️ See the Installation for more details.

Try it out

➡️ Develop your First Data Job if you are impatient to start quickly.

Examples

➡️ Get your hands on interesting examples.

Interfaces

➡️ See Versatile Data Kit interfaces for a list of interfaces.

User Guide

➡️ User Guide is the most detailed documentation for data engineers and scientists wanting to use VDK. If you have a question and the answer is unclear, tell us (open an issue), and we will update it.

Operators Guide

➡️ Check details on how to deploy the VDK backend (Control Service) in this example here

Dictionary

➡️ Terms Dictionary contains definitions of all terms used in the documentation.