Get the SynthoraAI Realtime Stream Processor running in 5 minutes!
- Docker and Docker Compose installed
- Git installed
- (Optional) Go 1.21+ for local development
# Clone the repository
git clone https://github.com/SynthoraAI-AI-News-Content-Curator/Realtime-Stream-Processor.git
cd Realtime-Stream-Processor
# Copy environment file
cp .env.example .envEdit .env and add your API keys:
# Required
GOOGLE_AI_API_KEY=your_google_ai_api_key_here
# Optional (for NewsAPI integration)
NEWS_API_KEY=your_news_api_key_here# Start all services with Docker Compose
docker-compose up -d
# Check if services are running
docker-compose ps# Check health
curl http://localhost:8080/health
# Expected output:
# {
# "status": "healthy",
# "timestamp": "2025-11-16T20:00:00Z",
# ...
# }// In browser console or Node.js
const ws = new WebSocket('ws://localhost:8080/ws');
ws.onmessage = (event) => {
const article = JSON.parse(event.data);
console.log('New article:', article);
};
ws.send(JSON.stringify({
action: 'subscribe',
topics: ['government', 'politics']
}));# List articles
curl http://localhost:8080/api/v1/articles?limit=5
# Stream a new article
curl -X POST http://localhost:8080/api/v1/stream \
-H "Content-Type: application/json" \
-d '{
"title": "Breaking: Important Government Announcement",
"content": "Full article content...",
"url": "https://example.com/article",
"source": "test-source",
"priority": "high",
"published_at": "2025-11-16T20:00:00Z"
}'Open http://localhost:8080/playground in your browser and try:
query {
recentArticles(limit: 5) {
id
title
summary
source
topics
publishedAt
}
}- Prometheus Metrics: http://localhost:9090/metrics
- Grafana Dashboard: http://localhost:3001 (admin/admin)
- Redis Commander: http://localhost:8081
- Jaeger Tracing: http://localhost:16686
# All services
docker-compose logs -f
# Stream processor only
docker-compose logs -f stream-processor
# Last 100 lines
docker-compose logs --tail=100 stream-processor# Stop all services
docker-compose down
# Stop and remove volumes (clean slate)
docker-compose down -v- Production Deployment: See README.md for Kubernetes deployment
- Configuration: Customize
config.yamlfor your needs - Integration: Connect to your existing SynthoraAI backend
- Monitoring: Set up alerts in Prometheus
- Development: See CONTRIBUTING.md to contribute
# Check service status
docker-compose ps
# View specific service logs
docker-compose logs <service-name>
# Restart a service
docker-compose restart <service-name># Check MongoDB logs
docker-compose logs mongodb
# Verify MongoDB is healthy
docker-compose exec mongodb mongosh --eval "db.adminCommand('ping')"# Check Redis logs
docker-compose logs redis
# Test Redis connection
docker-compose exec redis redis-cli ping- Verify
GOOGLE_AI_API_KEYis set correctly in.env - Check API quota and limits
- Review logs:
docker-compose logs stream-processor | grep "AI"
- Documentation: README.md
- Architecture: ARCHITECTURE.md
- Issues: https://github.com/SynthoraAI-AI-News-Content-Curator/Realtime-Stream-Processor/issues
- Email: hoangson091104@gmail.com
Happy streaming! 🚀