Go Reference API

⚠️ This is a reference implementation for learning and testing. Not intended for production use.

This Go application demonstrates production-grade best practices for integrating with the DevStack Core infrastructure using the Gin web framework. It showcases secure credential management, concurrent patterns, observability, and idiomatic Go code.

Features Overview
API Endpoints
Architecture
Go-Specific Features
Middleware
Monitoring & Observability
Quick Start
Development
Testing
Environment Variables

Features Overview

Core Capabilities

Vault Integration: Secure credential fetching with HashiCorp Vault
Database Connections: PostgreSQL, MySQL, MongoDB with proper context handling
Caching: Redis cluster integration with TTL support
Messaging: RabbitMQ pub/sub patterns with queue management
Health Monitoring: Comprehensive health checks for all services
Redis Cluster: Full cluster management and node monitoring

Go-Specific Features

Concurrency: Goroutines for async operations and concurrent health checks
Context Propagation: Proper context.Context usage throughout
Graceful Shutdown: Signal handling for clean termination
Structured Logging: Logrus with request ID correlation
Type Safety: Strong typing with proper error handling
Prometheus Metrics: Native Go Prometheus client integration

API Endpoints

Root Endpoint

Method	Endpoint	Description
GET	`/`	API information and available endpoints

Response:

{
  "name": "DevStack Core Reference API",
  "version": "1.0.0",
  "language": "Go",
  "framework": "Gin",
  "description": "Reference implementation for infrastructure integration",
  "endpoints": {
    "health": "/health",
    "vault_examples": "/examples/vault",
    "database_examples": "/examples/database",
    "cache_examples": "/examples/cache",
    "redis_management": "/redis",
    "messaging_examples": "/examples/messaging",
    "metrics": "/metrics"
  }
}

Health Checks (`/health`)

Comprehensive health monitoring for all infrastructure services using concurrent checks.

Method	Endpoint	Description
GET	`/health/`	Simple health check (no dependencies)
GET	`/health/all`	Aggregate health of all services (concurrent)
GET	`/health/vault`	Vault connectivity and status
GET	`/health/postgres`	PostgreSQL connection test
GET	`/health/mysql`	MySQL connection test
GET	`/health/mongodb`	MongoDB connection test
GET	`/health/redis`	Redis cluster health
GET	`/health/rabbitmq`	RabbitMQ connectivity

Health Check Response Format:

{
  "status": "healthy",
  "services": {
    "vault": {
      "status": "healthy",
      "initialized": true,
      "sealed": false,
      "version": "1.15.4"
    },
    "postgres": {
      "status": "healthy",
      "version": "PostgreSQL 16.6"
    },
    "redis": {
      "status": "healthy",
      "cluster_enabled": true,
      "cluster_state": "ok",
      "nodes": 3
    }
  }
}

Implementation Note: The /health/all endpoint uses goroutines to check all services concurrently, providing fast aggregate health status.

Vault Integration (`/examples/vault`)

Secure credential management using HashiCorp Vault KV v2 secrets engine.

Method	Endpoint	Description
GET	`/examples/vault/secret/:service_name`	Retrieve all secrets for a service
GET	`/examples/vault/secret/:service_name/:key`	Retrieve specific secret key

Examples:

# Get all PostgreSQL credentials
curl http://localhost:8002/examples/vault/secret/reference-api/postgres

# Get specific credential
curl http://localhost:8002/examples/vault/secret/reference-api/postgres/username

Response:

{
  "service": "reference-api/postgres",
  "secrets": {
    "username": "postgres",
    "password": "dynamic_password_from_vault",
    "database": "reference_db"
  }
}

Database Operations (`/examples/database`)

Demonstrates database connectivity with Vault-managed credentials.

Method	Endpoint	Description
GET	`/examples/database/postgres`	PostgreSQL query example
GET	`/examples/database/mysql`	MySQL query example
GET	`/examples/database/mongodb`	MongoDB query example

Implementation Pattern:

Fetch credentials from Vault using context
Create database connection
Execute query with timeout
Clean up resources with defer

Example Response:

{
  "database": "postgres",
  "status": "connected",
  "query_result": "PostgreSQL 16.6 on x86_64-pc-linux-musl",
  "timestamp": "2025-10-27T12:00:00Z"
}

Caching (`/examples/cache`)

Redis cache operations with TTL support.

Method	Endpoint	Description
GET	`/examples/cache/:key`	Get value from cache
POST	`/examples/cache/:key?value=X&ttl=3600`	Set cache value with TTL
DELETE	`/examples/cache/:key`	Delete cache key

Examples:

# Set cache value with 1 hour TTL
curl -X POST "http://localhost:8002/examples/cache/mykey?value=myvalue&ttl=3600"

# Get cache value
curl http://localhost:8002/examples/cache/mykey

# Delete cache key
curl -X DELETE http://localhost:8002/examples/cache/mykey

Response Format:

{
  "key": "mykey",
  "value": "myvalue",
  "operation": "set",
  "ttl": 3600,
  "success": true
}

Messaging (`/examples/messaging`)

RabbitMQ messaging operations with queue management.

Method	Endpoint	Description
POST	`/examples/messaging/publish/:queue`	Publish message to queue
GET	`/examples/messaging/queue/:queue_name/info`	Get queue information

Examples:

# Publish message to queue
curl -X POST http://localhost:8002/examples/messaging/publish/test-queue \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello from Go!", "timestamp": "2025-10-27T12:00:00Z"}'

# Get queue info
curl http://localhost:8002/examples/messaging/queue/test-queue/info

Publish Response:

{
  "status": "published",
  "queue": "test-queue",
  "message_id": "abc123",
  "timestamp": "2025-10-27T12:00:00Z"
}

Redis Cluster Management (`/redis`)

Advanced Redis cluster operations and node monitoring.

Method	Endpoint	Description
GET	`/redis/cluster/nodes`	List all cluster nodes
GET	`/redis/cluster/slots`	Show slot distribution
GET	`/redis/cluster/info`	Cluster information
GET	`/redis/nodes/:node_name/info`	Detailed node information

Examples:

# Get all cluster nodes
curl http://localhost:8002/redis/cluster/nodes

# Get slot distribution
curl http://localhost:8002/redis/cluster/slots

# Get specific node info
curl http://localhost:8002/redis/nodes/redis-node-1/info

Cluster Nodes Response:

{
  "cluster_enabled": true,
  "cluster_state": "ok",
  "cluster_size": 3,
  "nodes": [
    {
      "id": "abc123",
      "address": "172.20.0.21:6379",
      "role": "master",
      "slots": "0-5460",
      "flags": ["master"],
      "link_state": "connected"
    }
  ]
}

Metrics (`/metrics`)

Prometheus metrics endpoint exposing application and runtime metrics.

Method	Endpoint	Description
GET	`/metrics`	Prometheus metrics

Available Metrics:

http_requests_total - Total HTTP requests by method, endpoint, status
http_request_duration_seconds - Request latency histogram
go_goroutines - Number of goroutines
go_memstats_* - Go runtime memory statistics

Architecture

Project Structure

reference-apps/golang/
├── cmd/
│   └── api/
│       └── main.go              # Application entry point
├── internal/
│   ├── config/
│   │   └── config.go            # Configuration management
│   ├── handlers/
│   │   ├── health.go            # Health check handlers
│   │   ├── vault.go             # Vault integration handlers
│   │   ├── database.go          # Database handlers
│   │   ├── cache.go             # Redis cache handlers
│   │   ├── redis_cluster.go     # Redis cluster management
│   │   └── messaging.go         # RabbitMQ handlers
│   ├── middleware/
│   │   └── logging.go           # Logging and CORS middleware
│   └── services/
│       └── vault.go             # Vault client wrapper
├── go.mod                        # Go module definition
├── go.sum                        # Dependency checksums
├── Dockerfile                    # Multi-stage Docker build
├── init.sh                       # Initialization script
├── start.sh                      # Application startup script
└── README.md                     # This file

Design Principles

Clean Architecture: Separation of concerns with clear package boundaries
Dependency Injection: Handlers receive dependencies via constructors
Context Propagation: context.Context used throughout for cancellation and timeouts
Error Handling: Explicit error checks with proper HTTP status codes
Resource Cleanup: defer statements for closing connections
Concurrent Operations: Goroutines for parallelizable operations

Go-Specific Features

Concurrency Patterns

Concurrent Health Checks:

// All services checked in parallel using goroutines
var wg sync.WaitGroup
results := make(map[string]interface{})
mu := &sync.Mutex{}

for _, service := range services {
    wg.Add(1)
    go func(svc string) {
        defer wg.Done()
        result := checkService(ctx, svc)
        mu.Lock()
        results[svc] = result
        mu.Unlock()
    }(service)
}
wg.Wait()

Context Handling

All operations use context.Context for:

Request cancellation propagation
Timeouts (5s for database operations, 30s for health checks)
Distributed tracing correlation

Example:

ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
defer cancel()

conn, err := pgx.Connect(ctx, connStr)

Graceful Shutdown

srv := &http.Server{Addr: fmt.Sprintf(":%s", cfg.HTTPPort), Handler: router}

// Start server in goroutine
go func() {
    if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
        logger.Fatalf("Server error: %s", err)
    }
}()

// Wait for interrupt signal
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit

// Graceful shutdown with timeout
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
srv.Shutdown(ctx)

Middleware

Logging Middleware

Generates unique request ID (UUID)
Adds X-Request-ID header to responses
Logs all requests with structured fields:
- request_id, method, path, status, duration
Integration with logrus for JSON logging

CORS Middleware

Configurable allowed origins
Supports credentials
Standard headers: Content-Type, Authorization, X-Request-ID
Methods: GET, POST, PUT, DELETE, OPTIONS

Monitoring & Observability

Structured Logging

Uses logrus for structured logging:

logger.WithFields(logrus.Fields{
    "request_id": requestID,
    "method":     c.Request.Method,
    "path":       c.Request.URL.Path,
    "status":     c.Writer.Status(),
    "duration":   duration.Milliseconds(),
}).Info("HTTP request processed")

Prometheus Metrics

HTTP Metrics: Request counts and latency by endpoint
Go Runtime Metrics: Goroutines, memory, GC stats
Custom Metrics: Application-specific counters and gauges

Quick Start

Using Docker Compose

# Start the Go API service
docker-compose up -d golang-api

# View logs
docker-compose logs -f golang-api

# Test the API
curl http://localhost:8002/
curl http://localhost:8002/health/all

Local Development

# Install dependencies
cd reference-apps/golang
go mod download

# Run locally (requires services)
export VAULT_ADDR=http://localhost:8200
export VAULT_TOKEN=your-token
go run ./cmd/api

# Build binary
go build -o api ./cmd/api
./api

Development

Building

# Development build
go build -o api ./cmd/api

# Production build (optimized)
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o api ./cmd/api

Docker Build

# Build Docker image
docker build -t golang-api:latest .

# Run container
docker run -p 8002:8002 \
  -e VAULT_ADDR=http://vault:8200 \
  -e VAULT_TOKEN=your-token \
  golang-api:latest

Code Quality

# Format code
go fmt ./...

# Lint code
golangci-lint run

# Vet code
go vet ./...

# Run tests
go test ./...

Testing

Manual Testing

# Health checks
curl http://localhost:8002/health/
curl http://localhost:8002/health/all
curl http://localhost:8002/health/vault

# Vault integration
curl http://localhost:8002/examples/vault/secret/reference-api/postgres

# Database operations
curl http://localhost:8002/examples/database/postgres
curl http://localhost:8002/examples/database/mysql
curl http://localhost:8002/examples/database/mongodb

# Cache operations
curl -X POST "http://localhost:8002/examples/cache/test?value=hello&ttl=60"
curl http://localhost:8002/examples/cache/test
curl -X DELETE http://localhost:8002/examples/cache/test

# Redis cluster
curl http://localhost:8002/redis/cluster/nodes
curl http://localhost:8002/redis/cluster/info

# Messaging
curl -X POST http://localhost:8002/examples/messaging/publish/test-queue \
  -H "Content-Type: application/json" \
  -d '{"message": "test"}'

# Metrics
curl http://localhost:8002/metrics

Integration Testing

Automated tests are planned using:

Go's testing package
httptest for HTTP handler testing
Mock implementations for external services

Environment Variables

Required Variables

Variable	Description	Default
`VAULT_ADDR`	Vault server address	`http://vault:8200`
`VAULT_TOKEN`	Vault authentication token	Required

Service Configuration

Variable	Description	Default
`HTTP_PORT`	HTTP server port	`8002`
`HTTPS_PORT`	HTTPS server port	`8445`
`GOLANG_API_ENABLE_TLS`	Enable TLS	`false`

Database Configuration

Variable	Description	Default
`POSTGRES_HOST`	PostgreSQL host	`postgres`
`POSTGRES_PORT`	PostgreSQL port	`5432`
`MYSQL_HOST`	MySQL host	`mysql`
`MYSQL_PORT`	MySQL port	`3306`
`MONGODB_HOST`	MongoDB host	`mongodb`
`MONGODB_PORT`	MongoDB port	`27017`

Cache & Messaging

Variable	Description	Default
`REDIS_CLUSTER_NODES`	Redis cluster nodes	`redis-node-1:6379,...`
`RABBITMQ_HOST`	RabbitMQ host	`rabbitmq`
`RABBITMQ_PORT`	RabbitMQ port	`5672`

Application Settings

Variable	Description	Default
`ENVIRONMENT`	Environment name	`development`
`DEBUG`	Debug mode	`false`
`LOG_LEVEL`	Logging level	`info`

Comparison with Python Implementation

Feature	Go Implementation	Python Implementation
Web Framework	Gin	FastAPI
Concurrency	Goroutines	asyncio
Type System	Static typing	Type hints (runtime optional)
Performance	Compiled, very fast	Interpreted, fast with async
Resource Usage	Lower memory, faster startup	Higher memory, slower startup
Dependencies	Compiled into binary	Requires runtime + packages
Deployment	Single binary	Python + dependencies
Error Handling	Explicit error returns	Exceptions
Context	context.Context	async context

Dependencies

Main Go dependencies:

github.com/gin-gonic/gin - Web framework
github.com/hashicorp/vault/api - Vault client
github.com/jackc/pgx/v5 - PostgreSQL driver
github.com/go-sql-driver/mysql - MySQL driver
go.mongodb.org/mongo-driver - MongoDB driver
github.com/redis/go-redis/v9 - Redis client
github.com/rabbitmq/amqp091-go - RabbitMQ client
github.com/prometheus/client_golang - Prometheus metrics
github.com/sirupsen/logrus - Structured logging

Additional Resources

License

This reference implementation is part of the DevStack Core project. See the main project README for license information.

Uh oh!

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Go Reference API

Table of Contents

Features Overview

Core Capabilities

Go-Specific Features

API Endpoints

Root Endpoint

Health Checks (/health)

Vault Integration (/examples/vault)

Database Operations (/examples/database)

Caching (/examples/cache)

Messaging (/examples/messaging)

Redis Cluster Management (/redis)

Metrics (/metrics)

Architecture

Project Structure

Design Principles

Go-Specific Features

Concurrency Patterns

Context Handling

Graceful Shutdown

Middleware

Logging Middleware

CORS Middleware

Monitoring & Observability

Structured Logging

Prometheus Metrics

Quick Start

Using Docker Compose

Local Development

Development

Building

Docker Build

Code Quality

Testing

Manual Testing

Integration Testing

Environment Variables

Required Variables

Service Configuration

Database Configuration

Cache & Messaging

Application Settings

Comparison with Python Implementation

Dependencies

Additional Resources

License

Health Checks (`/health`)

Vault Integration (`/examples/vault`)

Database Operations (`/examples/database`)

Caching (`/examples/cache`)

Messaging (`/examples/messaging`)

Redis Cluster Management (`/redis`)

Metrics (`/metrics`)