Skip to content

CLOUD_QUICK_START.md

Latest commit

 

History

History
258 lines (193 loc) · 5.72 KB

CLOUD_QUICK_START.md

File metadata and controls

258 lines (193 loc) · 5.72 KB

Cloud Migration - Quick Start

🚀 5-Minute Cloud Setup

Step 1: Create Elastic Cloud Account (2 minutes)

  1. Go to https://cloud.elastic.co

  2. Sign up (14-day free trial, no credit card required)

  3. Create a new deployment:

    • Name: phishnchips
    • Cloud Provider: AWS
    • Region: us-east-1 (or closest to you)
    • Version: 8.11.x
    • Size: 4GB RAM, 128GB storage (default)

  4. Save these credentials:

    Cloud ID: phishnchips:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyQ...
Username: elastic
Password: <generated-password>
Elasticsearch endpoint: https://xxxxx.es.us-east-1.aws.found.io:9243
Kibana endpoint: https://xxxxx.kb.us-east-1.aws.found.io:9243

Step 2: Configure Your Application (1 minute)

Create a .env file in the project root:

# Copy the example
cp env.example .env

# Edit .env with your cloud credentials
cat > .env << 'EOF'
DEPLOYMENT_TYPE=cloud
ELASTIC_CLOUD_ID=your-cloud-id-from-step-1
ELASTIC_USER=elastic
ELASTIC_PASSWORD=your-password-from-step-1
API_PORT=8000
EOF

Step 3: Test Connection (30 seconds)

# Install dependencies (if not already done)
pip3 install -r backend/requirements.txt python-dotenv

# Test connection
python3 scripts/test_cloud_connection.py

Expected output:

✓ Ping successful
✓ Elasticsearch version: 8.11.x
✓ Cluster status: green
✓ Number of nodes: 2
✅ All tests passed!

Step 4: Initialize Indices (30 seconds)

# Use cloud-ready setup script
python3 scripts/setup_elasticsearch.py

Step 5: Load Data (5-10 minutes for 1M records)

# Load 1M records to cloud
python3 scripts/bulk_load_1m.py \
  --count 1000000 \
  --batch-size 2000 \
  --verify

The script automatically uses credentials from .env file.

Step 6: Start API (locally connecting to cloud)

# Start API that connects to cloud Elasticsearch
cd backend
uvicorn main_cloud_ready:app --reload

Or use Docker:

# docker-compose-cloud.yml (API only)
docker-compose -f docker-compose-cloud.yml up

📝 What Changed?

Files Modified:

  1. .env - New file with cloud credentials
  2. backend/main.py → Use backend/main_cloud_ready.py instead
  3. All scripts - Automatically read from .env

Files NOT Changed:

  • ✓ Browser extension (just update API_ENDPOINT if deploying API to cloud)
  • ✓ Data generation logic
  • ✓ Index templates
  • ✓ Kibana dashboards

🔄 Switching Between Local and Cloud

To Use Local:

# .env file
DEPLOYMENT_TYPE=local
ELASTICSEARCH_HOSTS=http://localhost:9200

# Start local cluster
docker-compose up -d

# Use local API
cd backend && uvicorn main:app --reload

To Use Cloud:

# .env file
DEPLOYMENT_TYPE=cloud
ELASTIC_CLOUD_ID=your-cloud-id
ELASTIC_USER=elastic
ELASTIC_PASSWORD=your-password

# Use cloud-ready API
cd backend && uvicorn main_cloud_ready:app --reload

💰 Cost Comparison

Aspect Local (Docker) Elastic Cloud
Setup Time 5 minutes 5 minutes
Monthly Cost $0 ~$100 (can pause)
Maintenance You manage Fully managed
Scalability Limited Unlimited
High Availability No Yes (99.9% SLA)
Backups Manual Automatic
Best For Dev, testing, demos Production, demos

💡 Tip: Use cloud for impressive demos (shows production-ready), use local for development.

📊 Performance Differences

Metric Local Cloud
Network Latency < 1ms 20-50ms
Indexing Throughput ~3000 docs/sec ~2000 docs/sec
Query Latency ~10ms ~30-50ms
Data Safety At risk Replicated, backed up

🎯 When to Use Each?

Use Local When:

  • ✓ Developing and testing
  • ✓ Running demos without internet
  • ✓ Learning distributed concepts
  • ✓ Cost is a concern
  • ✓ Need fast iteration

Use Cloud When:

  • ✓ Demonstrating production-ready system
  • ✓ Need high availability
  • ✓ Want to impress with scale
  • ✓ Testing with remote clients
  • ✓ Need automatic backups

🐛 Troubleshooting

"Connection refused"

# Check cloud deployment status at cloud.elastic.co
# Verify credentials in .env
python3 scripts/test_cloud_connection.py

"Authentication failed"

# Regenerate password in Elastic Cloud console
# Update .env file with new password

"Slow performance"

# Check your deployment size (may need to upgrade)
# Optimize batch size: --batch-size 1000 (smaller for cloud)

"Timeout errors"

# Increase timeout in config_cloud.py
ES_REQUEST_TIMEOUT=60  # in .env

✅ Quick Checklist

To migrate to cloud:

  • Create Elastic Cloud account
  • Create deployment (save credentials)
  • Create .env file with cloud credentials
  • Test connection (test_cloud_connection.py)
  • Initialize indices (setup_elasticsearch.py)
  • Load data (bulk_load_1m.py)
  • Start API with main_cloud_ready.py
  • Test end-to-end
  • Update browser extension API endpoint (if needed)

Total time: ~10-15 minutes (excluding data load time)

📚 Related Documentation

💡 Pro Tips

  1. Keep Local for Dev: Use local for development, cloud for demos
  2. Use API Keys: More secure than password (see CLOUD_DEPLOYMENT.md)
  3. Monitor Costs: Pause deployment when not in use
  4. Optimize Batch Size: Use smaller batches (1000-2000) for cloud
  5. Test First: Always run test_cloud_connection.py before loading data