output.txt

Lab 12 - Worker Service for Job Execution
Outputs captured during setup, execution, testing, verification, and troubleshooting.

================================================================================
Update package manager
================================================================================
Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu noble InRelease
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
Get:3 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB]
Get:4 http://us-east-1.ec2.archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]
Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [812 kB]
Get:6 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [621 kB]
Get:7 http://us-east-1.ec2.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [516 kB]
Fetched 2,327 kB in 3s (776 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
18 packages can be upgraded. Run 'apt list --upgradable' to see them.

================================================================================
Install Python and pip
================================================================================
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3 is already the newest version (3.12.3-0ubuntu2).
python3-pip is already the newest version (24.0+dfsg-1ubuntu1).
python3-venv is already the newest version (3.12.3-0ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 18 not upgraded.

================================================================================
Install SQLite
================================================================================
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libsqlite3-0 libsqlite3-dev
Suggested packages:
  sqlite3-doc
The following NEW packages will be installed:
  sqlite3
The following packages will be upgraded:
  libsqlite3-0 libsqlite3-dev
2 upgraded, 1 newly installed, 0 to remove and 16 not upgraded.
Need to get 2,224 kB of archives.
After this operation, 547 kB of additional disk space will be used.
Fetched 2,224 kB in 1s (2,374 kB/s)
Setting up libsqlite3-0:amd64 (3.45.1-1ubuntu2.1) ...
Setting up libsqlite3-dev:amd64 (3.45.1-1ubuntu2.1) ...
Setting up sqlite3 (3.45.1-1ubuntu2.1) ...
Processing triggers for man-db (2.12.0-4build2) ...

================================================================================
Create project directory
================================================================================
/home/toor/worker-service-lab

================================================================================
Create Python virtual environment
================================================================================
(venv) toor@ip-172-31-10-226:~/worker-service-lab$

================================================================================
Install required Python packages
================================================================================
Collecting flask
  Downloading flask-3.1.0-py3-none-any.whl.metadata (2.7 kB)
Collecting requests
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting blinker>=1.9.0
Collecting click>=8.1.3
Collecting itsdangerous>=2.2.0
Collecting jinja2>=3.1.2
Collecting markupsafe>=2.1.1
Collecting werkzeug>=3.1
Collecting charset-normalizer<4,>=2
Collecting idna<4,>=2.5
Collecting urllib3<3,>=1.21.1
Collecting certifi>=2017.4.17
Downloading flask-3.1.0-py3-none-any.whl (102 kB)
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
Installing collected packages: urllib3, markupsafe, itsdangerous, idna, click, charset-normalizer, certifi, blinker, werkzeug, requests, jinja2, flask
Successfully installed blinker-1.9.0 certifi-2025.1.31 charset-normalizer-3.4.1 click-8.1.8 flask-3.1.0 idna-3.10 itsdangerous-2.2.0 jinja2-3.1.5 markupsafe-3.0.2 requests-2.32.3 urllib3-2.3.0 werkzeug-3.1.3

================================================================================
Verify Installation
================================================================================
Python 3.12.3
3.45.1 2024-01-30 16:01:20 e876e51a0ed5c5b3126f52e532044363a014bc594cfefa87ffb5b82257cc467a (64-bit)

================================================================================
Step 1: Initialize the Database
================================================================================
1|backup|pending|2026-04-12 18:47:18
2|cleanup|pending|2026-04-12 18:47:18
3|report|pending|2026-04-12 18:47:18

================================================================================
Step 4: Start the API Server
================================================================================
[1] 7214

================================================================================
Step 4: Start the API Server
================================================================================
 * Serving Flask app 'api_server'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://172.31.10.226:5000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 721-448-312

================================================================================
Step 4: Start the API Server
================================================================================
[
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 1,
    "job_type": "backup",
    "payload": "{\"path\": \"/var/log\", \"destination\": \"/tmp/backup\"}",
    "result": null,
    "status": "pending",
    "updated_at": "2026-04-12 18:47:18"
  },
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 2,
    "job_type": "cleanup",
    "payload": "{\"directory\": \"/tmp\", \"days\": 7}",
    "result": null,
    "status": "pending",
    "updated_at": "2026-04-12 18:47:18"
  },
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 3,
    "job_type": "report",
    "payload": "{\"type\": \"system\", \"format\": \"json\"}",
    "result": null,
    "status": "pending",
    "updated_at": "2026-04-12 18:47:18"
  }
]

================================================================================
Step 3: Test the Worker Service
================================================================================
Worker service started. Polling every 5 seconds...
Found 3 pending job(s)
[2026-04-12 18:47:32.104216] Processing job 1 (backup)
[2026-04-12 18:47:32.112991] Updated job 1 status to 'processing'
[2026-04-12 18:47:32.121084] Updated job 1 status to 'completed'
[2026-04-12 18:47:32.121137] Job 1 completed successfully
[2026-04-12 18:47:32.121144] Processing job 2 (cleanup)
[2026-04-12 18:47:32.127835] Updated job 2 status to 'processing'
[2026-04-12 18:47:32.134312] Updated job 2 status to 'completed'
[2026-04-12 18:47:32.134367] Job 2 completed successfully
[2026-04-12 18:47:32.134373] Processing job 3 (report)
[2026-04-12 18:47:32.141016] Updated job 3 status to 'processing'
[2026-04-12 18:47:32.156202] Updated job 3 status to 'completed'
[2026-04-12 18:47:32.156249] Job 3 completed successfully
[2026-04-12 18:47:37.164304] No pending jobs
[2026-04-12 18:47:42.171953] No pending jobs

================================================================================
Create a new backup job
================================================================================
{
  "created_at": "2026-04-12 18:47:45",
  "id": 4,
  "job_type": "backup",
  "payload": "{\"path\": \"/etc/hostname\", \"destination\": \"/tmp/backup\"}",
  "result": null,
  "status": "pending",
  "updated_at": "2026-04-12 18:47:45"
}

================================================================================
Create a cleanup job
================================================================================
{
  "created_at": "2026-04-12 18:47:46",
  "id": 5,
  "job_type": "cleanup",
  "payload": "{\"directory\": \"/tmp\", \"days\": 30}",
  "result": null,
  "status": "pending",
  "updated_at": "2026-04-12 18:47:46"
}

================================================================================
Create a report job
================================================================================
{
  "created_at": "2026-04-12 18:47:46",
  "id": 6,
  "job_type": "report",
  "payload": "{\"type\": \"system\", \"format\": \"json\"}",
  "result": null,
  "status": "pending",
  "updated_at": "2026-04-12 18:47:46"
}

================================================================================
Create a report job
================================================================================
Found 3 pending job(s)
[2026-04-12 18:47:47.179505] Processing job 4 (backup)
[2026-04-12 18:47:47.187263] Updated job 4 status to 'processing'
[2026-04-12 18:47:47.196840] Updated job 4 status to 'completed'
[2026-04-12 18:47:47.196893] Job 4 completed successfully
[2026-04-12 18:47:47.196900] Processing job 5 (cleanup)
[2026-04-12 18:47:47.203491] Updated job 5 status to 'processing'
[2026-04-12 18:47:47.210017] Updated job 5 status to 'completed'
[2026-04-12 18:47:47.210072] Job 5 completed successfully
[2026-04-12 18:47:47.210078] Processing job 6 (report)
[2026-04-12 18:47:47.216684] Updated job 6 status to 'processing'
[2026-04-12 18:47:47.232513] Updated job 6 status to 'completed'
[2026-04-12 18:47:47.232566] Job 6 completed successfully
[2026-04-12 18:47:52.240141] No pending jobs

================================================================================
Create a report job
================================================================================
[
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 1,
    "job_type": "backup",
    "payload": "{\"path\": \"/var/log\", \"destination\": \"/tmp/backup\"}",
    "result": "{\"success\": true, \"operation\": \"directory_backup\", \"source\": \"/var/log\", \"backup_file\": \"/tmp/backup/backup_20260412_184732.tar.gz\", \"message\": \"Backup archive created successfully for directory /var/log\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:32"
  },
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 2,
    "job_type": "cleanup",
    "payload": "{\"directory\": \"/tmp\", \"days\": 7}",
    "result": "{\"success\": true, \"directory\": \"/tmp\", \"days_threshold\": 7, \"files_found\": 0, \"files_removed\": 0, \"removed_files\": [], \"message\": \"Cleanup completed in /tmp\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:32"
  },
  {
    "created_at": "2026-04-12 18:47:18",
    "id": 3,
    "job_type": "report",
    "payload": "{\"type\": \"system\", \"format\": \"json\"}",
    "result": "{\"success\": true, \"report_type\": \"system\", \"format\": \"json\", \"generated_at\": \"2026-04-12T18:47:32.149601\", \"disk_usage\": \"Filesystem      Size  Used Avail Use% Mounted on\\n/dev/root        30G  9.9G   21G  33% /\\ntmpfs           1.9G     0  1.9G   0% /dev/shm\\ntmpfs           768M  1.4M  767M   1% /run\\n\", \"memory_usage\": \"               total        used        free      shared  buff/cache   available\\nMem:            3797         612        2348          12         989        2921\\nSwap:              0           0           0\\n\", \"uptime\": \"18:47:32 up 1 day,  3:16,  1 user,  load average: 0.03, 0.04, 0.01\", \"message\": \"System report generated successfully\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:32"
  },
  {
    "created_at": "2026-04-12 18:47:45",
    "id": 4,
    "job_type": "backup",
    "payload": "{\"path\": \"/etc/hostname\", \"destination\": \"/tmp/backup\"}",
    "result": "{\"success\": true, \"operation\": \"file_backup\", \"source\": \"/etc/hostname\", \"backup_file\": \"/tmp/backup/hostname_20260412_184747\", \"message\": \"File copied successfully from /etc/hostname to /tmp/backup/hostname_20260412_184747\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:47"
  },
  {
    "created_at": "2026-04-12 18:47:46",
    "id": 5,
    "job_type": "cleanup",
    "payload": "{\"directory\": \"/tmp\", \"days\": 30}",
    "result": "{\"success\": true, \"directory\": \"/tmp\", \"days_threshold\": 30, \"files_found\": 0, \"files_removed\": 0, \"removed_files\": [], \"message\": \"Cleanup completed in /tmp\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:47"
  },
  {
    "created_at": "2026-04-12 18:47:46",
    "id": 6,
    "job_type": "report",
    "payload": "{\"type\": \"system\", \"format\": \"json\"}",
    "result": "{\"success\": true, \"report_type\": \"system\", \"format\": \"json\", \"generated_at\": \"2026-04-12T18:47:47.226104\", \"disk_usage\": \"Filesystem      Size  Used Avail Use% Mounted on\\n/dev/root        30G  9.9G   21G  33% /\\ntmpfs           1.9G     0  1.9G   0% /dev/shm\\ntmpfs           768M  1.4M  767M   1% /run\\n\", \"memory_usage\": \"               total        used        free      shared  buff/cache   available\\nMem:            3797         614        2346          12         989        2919\\nSwap:              0           0           0\\n\", \"uptime\": \"18:47:47 up 1 day,  3:16,  1 user,  load average: 0.02, 0.04, 0.01\", \"message\": \"System report generated successfully\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:47"
  }
]

================================================================================
Verify API Functionality
================================================================================
[
    {
        "created_at": "2026-04-12 18:47:18",
        "id": 1,
        "job_type": "backup",
        "payload": "{\"path\": \"/var/log\", \"destination\": \"/tmp/backup\"}",
        "result": "{\"success\": true, \"operation\": \"directory_backup\", \"source\": \"/var/log\", \"backup_file\": \"/tmp/backup/backup_20260412_184732.tar.gz\", \"message\": \"Backup archive created successfully for directory /var/log\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:32"
    },
    {
        "created_at": "2026-04-12 18:47:18",
        "id": 2,
        "job_type": "cleanup",
        "payload": "{\"directory\": \"/tmp\", \"days\": 7}",
        "result": "{\"success\": true, \"directory\": \"/tmp\", \"days_threshold\": 7, \"files_found\": 0, \"files_removed\": 0, \"removed_files\": [], \"message\": \"Cleanup completed in /tmp\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:32"
    },
    {
        "created_at": "2026-04-12 18:47:18",
        "id": 3,
        "job_type": "report",
        "payload": "{\"type\": \"system\", \"format\": \"json\"}",
        "result": "{\"success\": true, \"report_type\": \"system\", \"format\": \"json\", \"generated_at\": \"2026-04-12T18:47:32.149601\", \"disk_usage\": \"Filesystem      Size  Used Avail Use% Mounted on\\n/dev/root        30G  9.9G   21G  33% /\\ntmpfs           1.9G     0  1.9G   0% /dev/shm\\ntmpfs           768M  1.4M  767M   1% /run\\n\", \"memory_usage\": \"               total        used        free      shared  buff/cache   available\\nMem:            3797         612        2348          12         989        2921\\nSwap:              0           0           0\\n\", \"uptime\": \"18:47:32 up 1 day,  3:16,  1 user,  load average: 0.03, 0.04, 0.01\", \"message\": \"System report generated successfully\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:32"
    },
    {
        "created_at": "2026-04-12 18:47:45",
        "id": 4,
        "job_type": "backup",
        "payload": "{\"path\": \"/etc/hostname\", \"destination\": \"/tmp/backup\"}",
        "result": "{\"success\": true, \"operation\": \"file_backup\", \"source\": \"/etc/hostname\", \"backup_file\": \"/tmp/backup/hostname_20260412_184747\", \"message\": \"File copied successfully from /etc/hostname to /tmp/backup/hostname_20260412_184747\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:47"
    },
    {
        "created_at": "2026-04-12 18:47:46",
        "id": 5,
        "job_type": "cleanup",
        "payload": "{\"directory\": \"/tmp\", \"days\": 30}",
        "result": "{\"success\": true, \"directory\": \"/tmp\", \"days_threshold\": 30, \"files_found\": 0, \"files_removed\": 0, \"removed_files\": [], \"message\": \"Cleanup completed in /tmp\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:47"
    },
    {
        "created_at": "2026-04-12 18:47:46",
        "id": 6,
        "job_type": "report",
        "payload": "{\"type\": \"system\", \"format\": \"json\"}",
        "result": "{\"success\": true, \"report_type\": \"system\", \"format\": \"json\", \"generated_at\": \"2026-04-12T18:47:47.226104\", \"disk_usage\": \"Filesystem      Size  Used Avail Use% Mounted on\\n/dev/root        30G  9.9G   21G  33% /\\ntmpfs           1.9G     0  1.9G   0% /dev/shm\\ntmpfs           768M  1.4M  767M   1% /run\\n\", \"memory_usage\": \"               total        used        free      shared  buff/cache   available\\nMem:            3797         614        2346          12         989        2919\\nSwap:              0           0           0\\n\", \"uptime\": \"18:47:47 up 1 day,  3:16,  1 user,  load average: 0.02, 0.04, 0.01\", \"message\": \"System report generated successfully\"}",
        "status": "completed",
        "updated_at": "2026-04-12 18:47:47"
    }
]

================================================================================
Verify API Functionality
================================================================================
{
    "created_at": "2026-04-12 18:47:18",
    "id": 1,
    "job_type": "backup",
    "payload": "{\"path\": \"/var/log\", \"destination\": \"/tmp/backup\"}",
    "result": "{\"success\": true, \"operation\": \"directory_backup\", \"source\": \"/var/log\", \"backup_file\": \"/tmp/backup/backup_20260412_184732.tar.gz\", \"message\": \"Backup archive created successfully for directory /var/log\"}",
    "status": "completed",
    "updated_at": "2026-04-12 18:47:32"
}

================================================================================
Verify Worker Processing
================================================================================
1|backup|completed|{"success": true, "operation": "directory_backup", "source": "/var/log", "backup_file": "/tmp/backup/backup_20260412_184732.tar.gz", "message": "Backup archive created successfully for directory /var/log"}
2|cleanup|completed|{"success": true, "directory": "/tmp", "days_threshold": 7, "files_found": 0, "files_removed": 0, "removed_files": [], "message": "Cleanup completed in /tmp"}
3|report|completed|{"success": true, "report_type": "system", "format": "json", "generated_at": "2026-04-12T18:47:32.149601", "disk_usage": "Filesystem      Size  Used Avail Use% Mounted on\n/dev/root        30G  9.9G   21G  33% /\ntmpfs           1.9G     0  1.9G   0% /dev/shm\ntmpfs           768M  1.4M  767M   1% /run\n", "memory_usage": "               total        used        free      shared  buff/cache   available\nMem:            3797         612        2348          12         989        2921\nSwap:              0           0           0\n", "uptime": "18:47:32 up 1 day,  3:16,  1 user,  load average: 0.03, 0.04, 0.01", "message": "System report generated successfully"}
4|backup|completed|{"success": true, "operation": "file_backup", "source": "/etc/hostname", "backup_file": "/tmp/backup/hostname_20260412_184747", "message": "File copied successfully from /etc/hostname to /tmp/backup/hostname_20260412_184747"}
5|cleanup|completed|{"success": true, "directory": "/tmp", "days_threshold": 30, "files_found": 0, "files_removed": 0, "removed_files": [], "message": "Cleanup completed in /tmp"}
6|report|completed|{"success": true, "report_type": "system", "format": "json", "generated_at": "2026-04-12T18:47:47.226104", "disk_usage": "Filesystem      Size  Used Avail Use% Mounted on\n/dev/root        30G  9.9G   21G  33% /\ntmpfs           1.9G     0  1.9G   0% /dev/shm\ntmpfs           768M  1.4M  767M   1% /run\n", "memory_usage": "               total        used        free      shared  buff/cache   available\nMem:            3797         614        2346          12         989        2919\nSwap:              0           0           0\n", "uptime": "18:47:47 up 1 day,  3:16,  1 user,  load average: 0.02, 0.04, 0.01", "message": "System report generated successfully"}

================================================================================
Verify Worker Processing
================================================================================
0

================================================================================
Verify Job Results
================================================================================
total 188
drwxrwxr-x  2 toor toor   4096 Apr 12 18:47 .
drwxrwxrwt 17 root root 176128 Apr 12 18:47 ..
-rw-rw-r--  1 toor toor  12984 Apr 12 18:47 backup_20260412_184732.tar.gz
-rw-r--r--  1 toor toor     20 Apr 12 18:47 hostname_20260412_184747

================================================================================
Verify Job Results
================================================================================
1|backup|completed|2026-04-12 18:47:18|2026-04-12 18:47:32|{"success": true, "operation": "directory_backup", "source": "/var/log", "backup_file": "/tmp/backup/backup_20260412_184732.tar.gz", "message": "Backup archive created successfully for directory /var/log"}
2|cleanup|completed|2026-04-12 18:47:18|2026-04-12 18:47:32|{"success": true, "directory": "/tmp", "days_threshold": 7, "files_found": 0, "files_removed": 0, "removed_files": [], "message": "Cleanup completed in /tmp"}
3|report|completed|2026-04-12 18:47:18|2026-04-12 18:47:32|{"success": true, "report_type": "system", "format": "json", "generated_at": "2026-04-12T18:47:32.149601", "disk_usage": "Filesystem      Size  Used Avail Use% Mounted on\n/dev/root        30G  9.9G   21G  33% /\ntmpfs           1.9G     0  1.9G   0% /dev/shm\ntmpfs           768M  1.4M  767M   1% /run\n", "memory_usage": "               total        used        free      shared  buff/cache   available\nMem:            3797         612        2348          12         989        2921\nSwap:              0           0           0\n", "uptime": "18:47:32 up 1 day,  3:16,  1 user,  load average: 0.03, 0.04, 0.01", "message": "System report generated successfully"}
4|backup|completed|2026-04-12 18:47:45|2026-04-12 18:47:47|{"success": true, "operation": "file_backup", "source": "/etc/hostname", "backup_file": "/tmp/backup/hostname_20260412_184747", "message": "File copied successfully from /etc/hostname to /tmp/backup/hostname_20260412_184747"}
5|cleanup|completed|2026-04-12 18:47:46|2026-04-12 18:47:47|{"success": true, "directory": "/tmp", "days_threshold": 30, "files_found": 0, "files_removed": 0, "removed_files": [], "message": "Cleanup completed in /tmp"}
6|report|completed|2026-04-12 18:47:46|2026-04-12 18:47:47|{"success": true, "report_type": "system", "format": "json", "generated_at": "2026-04-12T18:47:47.226104", "disk_usage": "Filesystem      Size  Used Avail Use% Mounted on\n/dev/root        30G  9.9G   21G  33% /\ntmpfs           1.9G     0  1.9G   0% /dev/shm\ntmpfs           768M  1.4M  767M   1% /run\n", "memory_usage": "               total        used        free      shared  buff/cache   available\nMem:            3797         614        2346          12         989        2919\nSwap:              0           0           0\n", "uptime": "18:47:47 up 1 day,  3:16,  1 user,  load average: 0.02, 0.04, 0.01", "message": "System report generated successfully"}

================================================================================
API server not responding
================================================================================
toor        7214  1.4  1.2 116532 48612 pts/0    S    18:47   0:00 python3 api_server.py
toor        7217  1.7  1.3 190264 51280 pts/0    Sl   18:47   0:00 /home/toor/worker-service-lab/venv/bin/python3 api_server.py
toor        7306  0.0  0.0   6828  2304 pts/2    S+   18:48   0:00 grep --color=auto api_server

================================================================================
API server not responding
================================================================================
LISTEN 0      128          0.0.0.0:5000       0.0.0.0:*    users:(("python3",pid=7217,fd=6),("python3",pid=7214,fd=6))

================================================================================
Worker not fetching jobs
================================================================================
[]

================================================================================
Clean up resources
================================================================================
[1]+  Terminated              python3 api_server.py

================================================================================
Clean up resources
================================================================================
^C
Shutting down worker service...