feature: major jobs refactor 2

[Anmol1696]

This pull request makes significant improvements to the job runtime and utilities by standardizing environment variable handling, consolidating configuration logic, and introducing structured logging. It removes the envalid dependency in favor of runtime helpers from @launchql/job-utils, updates Docker and development documentation for local builds, and improves the logging infrastructure for better observability.

Environment and Configuration Standardization:

• Replaced all usage of the envalid library in job-pg, job-scheduler, and job-utils with plain environment variable parsing or new runtime helpers from @launchql/job-utils, centralizing configuration logic and improving consistency across services. [1] [2] [3] [4] [5] [6] F3869fdaL1)
• Added and exported new runtime configuration helpers (e.g., getJobPgConfig, getJobSchema, getJobSupported, etc.) in job-utils, and updated all consumers to use these helpers instead of direct environment access. [1] [2] F3869fdaL1)

Logging Improvements:

• Introduced structured logging using @pgpmjs/logger in job-scheduler and job-utils, replacing all raw console.log and console.error statements with appropriate log.info, log.error, and log.warn calls for better traceability and log management. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Docker and Development Workflow Updates:

• Updated docker-compose.jobs.yml and Dockerfile to build and use local images (constructive-launchql:dev) instead of pulling from remote registries, and set CI=true for reproducible builds. Adjusted environment variables for local development, including new mailgun settings. [1] [2] [3] [4] [5] [6]
• Corrected documentation in DEVELOPMENT_JOBS.md to reference the correct directory (constructive-db/) for database commands.

API and Type Changes:

• Updated job utility methods to return null instead of undefined when no job is found, and improved error handling for scheduled jobs.

Dependency Management:

• Removed envalid from all packages and added new dependencies (@pgpmjs/logger, @pgpmjs/env, pg-env, pg-cache) where necessary. [1] [2] [3]

These changes modernize the configuration and logging approach, streamline local development, and improve maintainability across the job-related packages.

[Copilot]

Pull request overview

This PR modernizes the job system infrastructure by removing the envalid dependency and centralizing configuration through new runtime helpers in @launchql/job-utils. It also introduces structured logging via @pgpmjs/logger and updates Docker configuration for local development builds.

Key Changes:

• Replaced envalid library with custom environment variable parsing and centralized runtime helpers
• Introduced structured logging using @pgpmjs/logger throughout job-related packages
• Updated job utility return types from undefined to null for consistency
• Enhanced Docker setup to build local images instead of pulling from remote registries
• Removed OpenFaaS-specific terminology in favor of generic "job HTTP gateway" references

Reviewed changes

Copilot reviewed 34 out of 36 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
types/rimraf.d.ts	Added minimal TypeScript type stubs for rimraf
packages/types/src/pgpm.ts	Removed deprecated OpenFaaS configuration
packages/types/src/jobs.ts	Renamed OpenFaaS-specific types to generic job HTTP gateway types
jobs/knative-job-worker/src/run.ts	Migrated from envalid to runtime helpers for worker configuration
jobs/knative-job-worker/src/req.ts	Updated to use getCallbackBaseUrl() helper
jobs/knative-job-worker/src/index.ts	Replaced console logging with structured logger, updated to use runtime helpers
jobs/knative-job-worker/src/env.ts	Simplified env module, removed envalid dependency
jobs/knative-job-worker/package.json	Replaced envalid with @pgpmjs/logger dependency
jobs/knative-job-service/src/run.ts	Migrated to runtime helpers for service configuration
jobs/knative-job-service/src/env.ts	Replaced envalid with plain environment parsing
jobs/knative-job-service/package.json	Removed envalid, added job-utils dependency
jobs/knative-job-server/src/run.ts	Updated to use getJobsCallbackPort() helper
jobs/knative-job-server/src/index.ts	Enhanced error handling to support both error and message fields
jobs/knative-job-server/src/env.ts	Simplified env module, removed envalid
jobs/knative-job-server/package.json	Removed envalid dependency
jobs/knative-job-fn/src/index.ts	Added automatic job callback mechanism with HTTP client implementation
jobs/knative-job-example/src/env.ts	Simplified env module, removed envalid
jobs/knative-job-example/package.json	Removed envalid dependency
jobs/knative-job-example/Dockerfile	Updated base image from pyramation/openfaas-node to node:22-alpine
jobs/job-worker/src/env.ts	Replaced envalid with plain environment parsing
jobs/job-worker/package.json	Removed envalid dependency
jobs/job-utils/src/runtime.ts	Added new runtime configuration helpers for PG config, schema, hostname, gateway, and callback settings
jobs/job-utils/src/index.ts	Integrated runtime helpers, added structured logging, changed return types to null
jobs/job-utils/src/env.ts	Simplified to legacy compatibility stub
jobs/job-utils/package.json	Replaced envalid with @pgpmjs/env, @pgpmjs/logger, pg-env, and pg-cache
jobs/job-scheduler/src/run.ts	Migrated to runtime helpers for scheduler configuration
jobs/job-scheduler/src/index.ts	Replaced console logging with structured logger, updated to use runtime helpers
jobs/job-scheduler/src/env.ts	Replaced envalid with plain environment parsing
jobs/job-scheduler/package.json	Replaced envalid with @pgpmjs/logger dependency
jobs/job-pg/src/index.ts	Updated to use getJobPgConfig() runtime helper
jobs/job-pg/src/env.ts	Simplified to legacy compatibility stub
jobs/job-pg/package.json	Replaced envalid with job-utils dependency
docker-compose.jobs.yml	Updated to build local images, added mailgun configuration variables
Dockerfile	Added CI=true flag for reproducible builds
DEVELOPMENT_JOBS.md	Corrected directory reference from constructive/ to constructive-db/

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 35 out of 37 changed files in this pull request and generated 26 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

@Anmol1696 I've opened a new pull request, #405, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

@Anmol1696 I've opened a new pull request, #406, to work on those changes. Once the pull request is ready, I'll request review from you.