Skip to content

README.md

Latest commit

 

History

History
103 lines (73 loc) · 3.43 KB

README.md

File metadata and controls

103 lines (73 loc) · 3.43 KB

🚀 Lab 09: Job Registry with PostgreSQL

📌 Lab Summary

This lab focused on job registry with postgresql using a cloud-based Ubuntu 24.04 environment and a realistic hands-on workflow. I built the required project files, executed the setup and validation steps, captured outputs, and documented the final operational results.

🎯 Objectives

  • Design a PostgreSQL schema for job metadata persistence
  • Implement job status tracking with timestamps
  • Store and retrieve job execution results
  • Query job history and analyze execution patterns

🧰 Prerequisites

  • Basic Linux command-line skills
  • Understanding of SQL fundamentals (SELECT, INSERT, UPDATE)
  • Familiarity with Python programming
  • Basic knowledge of database concepts
  • Understanding of job scheduling concepts

💻 Lab Environment

  • Host: ip-172-31-10-184
  • Platform: Cloud-based Ubuntu 24.04 LTS lab VM
  • Shell: Bash
  • Database: PostgreSQL 16
  • Runtime: Python 3.12 virtual environment
  • Local User: toor (sudo access)

🛠️ Task Overview

  • Design and Implement Job Registry Schema
  • Implement Job Registry Manager

📁 Repository Structure

lab09-job-registry-with-postgresql/
├── commands.sh
├── interview_qna.md
├── job_manager.py
├── output.txt
├── schema.sql
├── test_job_registry.py
└── troubleshooting.md

✅ Verification and Validation

  • PostgreSQL schema created successfully
  • Lifecycle operations for pending, running, completed, and failed states verified
  • JSONB result storage and error persistence confirmed
  • Duration calculations and status-based queries worked as expected

📚 What I Learned

  • Design a PostgreSQL schema for job metadata persistence
  • Implement job status tracking with timestamps
  • Store and retrieve job execution results
  • Query job history and analyze execution patterns
  • How to document and validate the workflow in a portfolio-friendly structure

🌍 Why This Matters

A durable job registry gives automation workflows a reliable source of truth for status, timing, results, and failure reasons. That is a foundational pattern for schedulers, CI/CD jobs, ETL pipelines, and audit-friendly automation.

🧪 Real-World Applications

  • Batch processing and ETL orchestration
  • CI/CD execution tracking
  • Operational reporting and audit trails
  • Failure analysis and retry workflows

🏁 Result

  • PostgreSQL schema created successfully
  • Lifecycle operations for pending, running, completed, and failed states verified
  • JSONB result storage and error persistence confirmed
  • Duration calculations and status-based queries worked as expected

📘 Conclusion

You have successfully:

  • Designed a PostgreSQL schema for job metadata persistence
  • Implemented job lifecycle management (create, start, complete, fail)
  • Stored job execution timestamps and calculated durations
  • Persisted job results as JSONB data
  • Queried job history by status and analyzed execution patterns

This job registry pattern is essential for DevOps workflows, enabling:

  • Audit trails for automated processes
  • Performance monitoring and optimization
  • Failure analysis and debugging
  • Compliance and reporting requirements
  • Distributed system coordination

The skills learned here apply directly to building production job schedulers, CI/CD pipelines, and workflow orchestration systems. That structure and those goals are exactly what your uploaded Lab 9 asks you to implement and verify.