Skip to content

Commit f1f90ae

Browse files
allan-dgallan-dg
committed
fix: interference api upload issues
1 parent ceafd7c commit f1f90ae

File tree

10 files changed

+1677
-366
lines changed

10 files changed

+1677
-366
lines changed

‎docker-compose-test.README.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Docker Compose Test Setup
2+
3+
This docker-compose file provides a local testing environment for the TCDI Admin application with all required dependencies.
4+
5+
## Services
6+
7+
| Service | Port | Description |
8+
|---------|------|-------------|
9+
| postgres | 5432 | PostgreSQL database for the admin application |
10+
| postgres-interference | 5433 | PostgreSQL database for the interference service |
11+
| eureka | 8761 | Netflix Eureka service registry |
12+
| mock-interference-service | 8090 | MockServer simulating the interference service API |
13+
14+
## Prerequisites
15+
16+
- Docker and Docker Compose installed
17+
- The admin application built (`mvn install -Dcheckstyle.skip=true -DskipTests`)
18+
19+
## Quick Start
20+
21+
1. **Start the infrastructure:**
22+
```bash
23+
docker-compose -f docker-compose-test.yml up -d
24+
```
25+
26+
2. **Wait for services to be healthy:**
27+
```bash
28+
docker-compose -f docker-compose-test.yml ps
29+
```
30+
31+
3. **Run the admin application:**
32+
```bash
33+
mvn spring-boot:run -pl forms -Dcheckstyle.skip=true \
34+
-Dspring-boot.run.profiles=dev \
35+
-Dspring-boot.run.arguments="--eureka.client.serviceUrl.defaultZone=http://localhost:8761/eureka/"
36+
```
37+
38+
4. **Access the application:**
39+
- Admin UI: http://localhost:8080
40+
- Eureka Dashboard: http://localhost:8761
41+
42+
## Testing CSV Dataset Upload
43+
44+
### Scenario: Test normal upload flow
45+
46+
1. Navigate to the CSV Datasets section in the admin UI
47+
2. Create a new dataset for the "INTERFERENCE" service
48+
3. Upload the test CSV file: `forms/src/test/resources/world_industry_interference_2025.csv`
49+
4. Click "Save and Publish"
50+
5. The mock service will return a successful response
51+
52+
### Scenario: Test stuck publishing detection
53+
54+
1. Create and publish a dataset
55+
2. The job checker runs every minute (`@Scheduled(cron = "0 * * * * *")`)
56+
3. If a job is in PUBLISHING state for longer than `dataset.publishing.timeout.minutes` (default: 30), it will be marked as ERROR_IN_PUBLISHING
57+
58+
### Scenario: Test Cancel Publishing button
59+
60+
1. Create a dataset and start publishing
61+
2. While in PUBLISHING state, a "Cancel Publishing" button should appear
62+
3. Click the button to cancel and mark as ERROR_IN_PUBLISHING
63+
4. This allows retrying the upload after fixing issues
64+
65+
## Configuration
66+
67+
### Publishing Timeout
68+
69+
Configure the timeout for stuck jobs in `application.properties` or via environment variable:
70+
71+
```properties
72+
dataset.publishing.timeout.minutes=30
73+
```
74+
75+
### Mock Service Responses
76+
77+
The mock interference service (MockServer) is configured via `mockserver-init.json`. It simulates:
78+
79+
- `GET /health` - Health check endpoint
80+
- `POST /datasets` - Dataset upload (returns PROCESSING status)
81+
- `GET /jobs/code/tcdi-*` - Job status check (returns COMPLETED)
82+
- `DELETE /datasets/tcdi-*` - Dataset deletion
83+
- `GET /template/download` - CSV template download
84+
- `GET /dimensions` - Get available dimensions
85+
- `GET /measures` - Get available measures
86+
87+
## Cleanup
88+
89+
Stop and remove all containers and volumes:
90+
91+
```bash
92+
docker-compose -f docker-compose-test.yml down -v
93+
```
94+
95+
## Troubleshooting
96+
97+
### Database connection issues
98+
99+
Check if PostgreSQL is healthy:
100+
```bash
101+
docker-compose -f docker-compose-test.yml logs postgres
102+
```
103+
104+
### Eureka connection issues
105+
106+
Ensure Eureka is running and accessible:
107+
```bash
108+
curl http://localhost:8761/actuator/health
109+
```
110+
111+
### Mock service not responding
112+
113+
Check MockServer logs:
114+
```bash
115+
docker-compose -f docker-compose-test.yml logs mock-interference-service
116+
```
117+
118+
## Running Unit Tests
119+
120+
The `DatasetClientServiceTest` class contains unit tests for the new functionality:
121+
122+
```bash
123+
mvn test -pl forms -Dtest=DatasetClientServiceTest -Dcheckstyle.skip=true
124+
```
125+
126+
Tests cover:
127+
- Cancel publishing functionality
128+
- Status transitions (PUBLISHING → PUBLISHED, UNPUBLISHING → DRAFT)
129+
- Error status transitions (PUBLISHING → ERROR_IN_PUBLISHING, UNPUBLISHING → ERROR_IN_UNPUBLISHING)
130+
- Timeout detection for stuck jobs
131+
- Dataset saving based on type (CSV vs Tetsim)

‎docker-compose-test.yml‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
services:
2+
# PostgreSQL database for the admin application
3+
postgres:
4+
image: postgis/postgis:15-3.3-alpine
5+
restart: unless-stopped
6+
ports:
7+
- "5432:5432"
8+
volumes:
9+
- postgres_data:/var/lib/postgresql/data
10+
environment:
11+
POSTGRES_PASSWORD: admin
12+
POSTGRES_USER: postgres
13+
POSTGRES_DB: tcdi_admin
14+
healthcheck:
15+
test: ["CMD-SHELL", "pg_isready -U postgres -d tcdi_admin"]
16+
interval: 10s
17+
timeout: 5s
18+
retries: 5
19+
networks:
20+
- tcdi-network
21+
22+
# PostgreSQL database for the interference service (remote service)
23+
postgres-interference:
24+
image: postgis/postgis:15-3.3-alpine
25+
restart: unless-stopped
26+
ports:
27+
- "5433:5432"
28+
volumes:
29+
- postgres_interference_data:/var/lib/postgresql/data
30+
environment:
31+
POSTGRES_PASSWORD: admin
32+
POSTGRES_USER: postgres
33+
POSTGRES_DB: interference
34+
healthcheck:
35+
test: ["CMD-SHELL", "pg_isready -U postgres -d interference"]
36+
interval: 10s
37+
timeout: 5s
38+
retries: 5
39+
networks:
40+
- tcdi-network
41+
42+
# Eureka service registry
43+
eureka:
44+
image: steeltoeoss/eureka-server:latest
45+
restart: unless-stopped
46+
ports:
47+
- "8761:8761"
48+
environment:
49+
EUREKA_SERVER_ENABLE_SELF_PRESERVATION: "false"
50+
healthcheck:
51+
test:
52+
[
53+
"CMD",
54+
"wget",
55+
"-q",
56+
"--spider",
57+
"http://localhost:8761/actuator/health",
58+
]
59+
interval: 10s
60+
timeout: 5s
61+
retries: 10
62+
networks:
63+
- tcdi-network
64+
65+
# Mock interference service for testing CSV uploads
66+
# This is a simple mock that simulates the remote interference service API
67+
mock-interference-service:
68+
image: mockserver/mockserver:5.15.0
69+
restart: unless-stopped
70+
ports:
71+
- "8090:1080"
72+
environment:
73+
MOCKSERVER_INITIALIZATION_JSON_PATH: /config/mockserver-init.json
74+
MOCKSERVER_LOG_LEVEL: INFO
75+
volumes:
76+
- ./mockserver-init.json:/config/mockserver-init.json:ro
77+
healthcheck:
78+
test:
79+
[
80+
"CMD",
81+
"wget",
82+
"-q",
83+
"--spider",
84+
"http://localhost:1080/mockserver/status",
85+
]
86+
interval: 10s
87+
timeout: 5s
88+
retries: 5
89+
networks:
90+
- tcdi-network
91+
92+
networks:
93+
tcdi-network:
94+
driver: bridge
95+
96+
volumes:
97+
postgres_data:
98+
postgres_interference_data:

0 commit comments

Comments
 (0)