AI_stats_measurement

Pilot project for measuring official statistics in AI,

🔗 Live demo: https://ai-stats-measurement.lab.sspcloud.fr/

Overview

This project explores the reliability of Large Language Models (LLMs) when answering questions based on official statistical data. It compares model responses against trusted sources such as National Statistical Institutes (NSIs).

Goal

The goal of this project is to assess whether publicly available data from NSIs is machine-readable and findable. The aim is to identify whether NSIs need to take action to improve the machine-readability and discoverability of their data to better support AI systems.

Features

• Analytics dashboard Evaluate model performance using metrics such as ARR (Accuracy Rate Ratio) per NSI and per model.
• Response inspection Load and review all responses for a given prompt.

Access & Limitations

Submitting new prompts is currently limited to CBS researchers only. Public users can explore results and existing evaluations.

Quality badges

UML diagrams

component diagram

service diagram

ERD diagram

Installation on SSP Cloud

This guide explains how to deploy the AI Stats Measurement application on SSP Cloud using Kubernetes and Helm.

Step 1: Create an SSP Cloud account

Go to:

https://datalab.sspcloud.fr/

Create an account with your NSI account.

Step 2: Create secrets

Go to your SSP Cloud secrets page

Name the secret : ai-stats-measurement-secrets

Add the required secrets for the application.

Variable name	Description
CONNECTIONSTRINGS__DEFAULT	PostgreSQL connection string
IDENTITY__ADMINEMAIL	Admin email address for the application
IDENTITY__ADMINPASSWORD	Admin password for the application
JWT__AUDIENCE	JWT audience value
JWT__ISSUER	JWT issuer value
JWT__KEY	Secret key used to sign JWT tokens
LLMKEYS__GEMINI	Google Gemini API key
LLMKEYS__GROK	xAI Grok API key
LLMKEYS__OPENAI	OpenAI API key
POSTGRES_PASSWORD	Password for the PostgreSQL database

Example values:

CONNECTIONSTRINGS__DEFAULT=Host=ai-stats-postgres;Port=5432;Database=AIStatsMeasurementDB;Username=appuser;Password=<your-postgres-password>
IDENTITY__ADMINEMAIL=your-admin-email@example.com
IDENTITY__ADMINPASSWORD=<your-admin-password>
JWT__AUDIENCE=AIStatsUsers
JWT__ISSUER=AIStatsMeasurement
JWT__KEY=<your-jwt-key>
LLMKEYS__GEMINI=<your-gemini-api-key>
LLMKEYS__GROK=<your-grok-api-key>
LLMKEYS__OPENAI=<your-openai-api-key>
POSTGRES_PASSWORD=<your-postgres-password>

LLM API keys can be generated here:

• OpenAI: https://platform.openai.com/api-keys
• Google Gemini: https://aistudio.google.com/api-keys
• xAI: https://console.x.ai/team/d3e919c1-6450-4258-a690-15f7085a1b35/api-keys

Generate the JWT key in the terminal:

openssl rand -base64 32

Add this generated value as your JWT secret.

its should look like this

Step 3: Create a VS Code Python service

Create a new service in SSP Cloud.

Choose the service: vscode-python

This service has a terminal with the required permissions to deploy the application.

Choose the role: admin

This gives the service the correct rights in your Kubernetes namespace.

use name of the secret you created earlier: ai-stats-measurement-secrets

Step 4: Open the terminal

Open the terminal inside the VS Code client.

Step 5: Clone the repository

Download the repository with this command:

git clone https://github.com/SNStatComp/AI_stats_measurement.git

Navigate to the root of the project:

cd AI_stats_measurement

Step 6: Deploy the application with Helm

Deploy the Helm chart with this command:

helm upgrade --install ai-stats ./ai-stats-measurement-chart -n user-yourusername

Step 7: Check the pod status

Check if the pods are created:

kubectl get pods -n user-yourusername

If all pods are running, the application has been deployed successfully.

Optional: Configure CI/CD with GitHub Actions

This step is optional. Use this if you want GitHub Actions to deploy the application automatically.

Step 1: Create GitHub secrets

In your GitHub repository, go to: Settings > Secrets and variables > Actions

Create the required secrets for the CI/CD pipeline.

Add the following secrets:

Secret name	Description	Example value
DOCKERHUB_USERNAME	Your Docker Hub username	your-dockerhub-username
DOCKERHUB_TOKEN	Docker Hub access token used to push images	<your-dockerhub-token>
KUBE_NAMESPACE	Your SSP Cloud Kubernetes namespace	user-yourusername
KUBE_SERVER	Kubernetes API server URL from SSP Cloud	https://apiserver.kub.sspcloud.fr
KUBE_TOKEN	Kubernetes service account token used by GitHub Actions	<your-kubernetes-token>
SONAR_TOKEN	SonarCloud token used for code analysis	<your-sonar-token>

Step 2: Create a Kubernetes service account

To allow GitHub Actions to deploy the application, create a service account in Kubernetes.

kubectl apply -f sa-token.yaml

Step 3: Retrieve the service account token

After applying sa-token.yaml, retrieve the token and add it to GitHub Secrets.

Check the created secret:

kubectl describe secret github-actions-token -n user-yourusername

Copy the token and add it as a GitHub secret.

Usefull Kubernetes / SSP Cloud commands

Check pods

kubectl get pods -n user-yourusername
kubectl get pods -n user-yourusername -w

Check services

kubectl get svc -n user-yourusername

Check ingress

kubectl get ingress -n user-yourusername

Check logs

kubectl logs -f deployment/ai-stats-backend -n user-yourusername
kubectl logs --tail=200 deployment/ai-stats-backend -n user-yourusername

Restart deployment

kubectl rollout restart deployment ai-stats-backend -n user-yourusername

Deploy or update with Helm

helm upgrade --install ai-stats ./ai-stats-measurement-chart -n user-yourusername

Uninstall Helm release

helm uninstall ai-stats -n user-yourusername

Delete pod

kubectl delete pod ai-stats-postgres-0 -n user-yourusername

Delete database PVC

kubectl delete pvc postgres-data-ai-stats-postgres-0 -n user-yourusername

Fully reset app and database

helm uninstall ai-stats -n user-diegoespinosa
kubectl delete pvc postgres-data-ai-stats-postgres-0 -n user-yourusername
helm upgrade --install ai-stats ./ai-stats-measurement-chart -n user-yourusername

Check secrets

kubectl get secrets -n user-yourusername
kubectl describe secret ai-stats-measurement-secrets -n user-yourusername

After pushing a new latest image

kubectl rollout restart deployment ai-stats-backend -n user-yourusername
kubectl rollout restart deployment ai-stats-frontend -n user-yourusername

Pause

kubectl scale deployment ai-stats-backend --replicas=0 -n user-yourusername
kubectl scale deployment ai-stats-frontend --replicas=0 -n user-yourusername
kubectl scale statefulset ai-stats-postgres --replicas=0 -n user-yourusername

Resume

kubectl scale statefulset ai-stats-postgres --replicas=1 -n user-yourusername
kubectl scale deployment ai-stats-backend --replicas=1 -n user-yourusername
kubectl scale deployment ai-stats-frontend --replicas=1 -n user-yourusername