[GTI-129] Add unit tests and black formatter

Author: ajGingrich

.github/workflows/pr-checks.yml

@@ -0,0 +1,49 @@
+name: PR Checks
+
+on:
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  format-check:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.11'
+
+    - name: Install Black
+      run: |
+        python -m pip install --upgrade pip
+        pip install black
+
+    - name: Check code formatting with Black
+      run: |
+        black --check *.py
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.9, "3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+
+    - name: Run tests
+      run: |
+        pytest test_msstats.py -v

.gitignore

@@ -135,4 +135,7 @@ config.ini
 # Output files
 *.xlsx
 *.csv
-*.json
+*.json
+
+# Claude
+.claude

CLAUDE.md

@@ -0,0 +1,96 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+MSStats is a Python tool for extracting Google Cloud MemoryStore (Redis) database metrics using the Google Cloud Monitoring API. It processes Redis databases (single instance and replicated) across multiple GCP service accounts and generates Excel reports with usage statistics.
+
+## Architecture
+
+- **Single Python script**: `msstats.py` - Main application that processes Redis metrics
+- **Batch processing**: Shell scripts for bulk operations across multiple GCP projects
+- **Service account management**: Scripts to grant/revoke monitoring permissions
+- **Output format**: Excel files with detailed Redis command statistics and throughput data
+
+## Key Components
+
+- **Metric Processing**: Categorizes Redis commands by type (Get, Set, Hash, List, etc.) from monitoring data
+- **Multi-node handling**: Aggregates metrics across Redis cluster nodes taking maximum values
+- **Time series data**: Configurable duration (default 7 days) and step intervals (default 60s)
+- **GCP Integration**: Uses Google Cloud Monitoring API with service account authentication
+
+## Development Commands
+
+### Setup Environment
+```bash
+# Create and activate virtual environment
+python -m venv .env && source .env/bin/activate
+
+# Install dependencies
+pip install -r requirements.txt
+
+# Place service account JSON files in root directory
+cp path/to/service_account.json .
+```
+
+### Running the Tool
+```bash
+# Basic usage (7 days, 60s steps)
+python msstats.py
+
+# Custom duration and step size
+python msstats.py --duration 1800 --step 300
+
+# Specific project
+python msstats.py -p project-id
+```
+
+### Batch Operations
+```bash
+# Grant monitoring permissions to service account
+./grant_sa_monitoring_viewer.sh service-account@project.iam.gserviceaccount.com
+
+# Run across all accessible projects
+./batch_run_msstats.sh
+
+# Remove monitoring permissions
+./remove_sa_monitoring_viewer.sh service-account@project.iam.gserviceaccount.com
+
+# Clean up environment
+deactivate
+```
+
+## Command Line Options
+
+- `--duration SECONDS`: Time period to analyze (default: 604800 = 7 days)
+- `--step SECONDS`: Metric sampling interval (default: 60 seconds)
+- `-p PROJECT_ID`: Target specific GCP project
+
+### Testing and Code Quality
+```bash
+# Run all tests (unit and integration)
+pytest test_msstats.py
+
+# Format code with Black
+black *.py
+
+# Check formatting without making changes
+black --check *.py
+```
+
+## Dependencies
+
+- `openpyxl>=3.0.4`: Excel file generation
+- `google-cloud-monitoring==2.18.0`: GCP Monitoring API client
+- `black>=25.1.0`: Code formatter
+- `pytest>=8.4.0`: Testing framework
+- `pytest-mock>=3.14.0`: Mock utilities for testing
+- Python 3.9+ required
+
+## Security Notes
+
+- Service account JSON files must be placed in repository root
+- Tool uses read-only monitoring API access
+- Never connects directly to Redis instances
+- No impact on database performance or data

CODEOWNERS

@@ -1,7 +1,7 @@
 # These owners will be the default owners for everything in
 # the repo. Unless a later match takes precedence will be requested
 # for review when someone opens a pull request.
-*       @thomasefthymiou78 @erniavr @mar1boroman @ewpreston @gmflau
+*       @ajGingrich @kurtfm @sagile @thomasefthymiou78
 
 # Order is important; the last matching pattern takes the most
 # precedence. When someone opens a pull request that only

README.md

@@ -1,6 +1,6 @@
 # MSSTATS
 
-MSStats is a tool for extracting MemoryStore database metrics. The script is able to process all the Redis databases, both single instance and replicated (Basic or Standard) ones that belong to a specific service account. Multiple service accounts can be used at once. 
+MSStats is a tool for extracting MemoryStore database metrics. The script is able to process all the Redis databases, both single instance and replicated (Basic or Standard) ones that belong to a specific service account. Multiple service accounts can be used at once.
 
 The script will purely use google cloud monitoring api for getting the metrics. It will never connect to the Redis databases and it will NOT send any commands to the databases.
 
@@ -42,24 +42,24 @@ Copy your service account .json files in the root directory of the project:
 cp path/to/service_account.json .
 ```
 
-Execute 
+Execute
 
 ```
 python msstats.py
 ```
 
 This generates a file named <your project>.xlsx. You need to get that file and send it to Redis.
 By default, it uses steps of 60 seconds and a period of 7 days (604800 seconds).
-You can set different values as follows : 
+You can set different values as follows:
 
 ````
 python msstats.py --duration 1800 --step 300
 ````
 
-This can help solving issue like : 
+This can help solving issue like:
 
 ```
-google.api_core.exceptions.ResourceExhausted: 429 Maximum response size of 200000000 bytes reached. 
+google.api_core.exceptions.ResourceExhausted: 429 Maximum response size of 200000000 bytes reached.
 Consider querying less data by increasing the step or interval, using more filters and aggregations, or limiting the time duration.
 ```
 
@@ -87,8 +87,8 @@ Execute
 ```
 ./batch_run_msstats.sh
 ```
-    
-Remove monitoring.viewer role from the service account in all associated Google Cloud projects 
+
+Remove monitoring.viewer role from the service account in all associated Google Cloud projects
 
 ```
 ./remove_sa_monitoring_viewer.sh <service_account>
@@ -98,6 +98,18 @@ For example,
 ./remove_sa_monitoring_viewer.sh gmflau-sa@gcp-dev-day-nyc.iam.gserviceaccount.com
 ```
 
+## Testing and Code Quality
+
+Run all tests (unit and integration):
+```
+pytest test_msstats.py
+```
+
+Format code:
+```
+black *.py
+```
+
 When finished do not forget to deactivate the virtual environment
 
 ```