Skip to content

Commit 96fbc9a

Browse files
elasticroentgenelasticroentgen
committed
switch proxmox controlelr to direct HTTP
1 parent cbe443c commit 96fbc9a

16 files changed

Lines changed: 1574 additions & 140 deletions

‎CloudControllers/ProxmoxCloudController.cs‎

Lines changed: 372 additions & 135 deletions
Large diffs are not rendered by default.

‎DATABASE_INTEGRATION_TESTS_SUMMARY.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# Database Integration Tests Implementation Summary
2+
3+
## ✅ Problem Resolved
4+
5+
### **Issue**: PostgreSQL Database Connection Required
6+
The integration tests were failing because the `GenerateName()` method in `BaseCloudController` uses Entity Framework to check for hostname collisions in the database:
7+
8+
```csharp
9+
// CloudControllers/BaseCloudController.cs:81
10+
while (await db.Runners.AnyAsync(x => x.Hostname == name));
11+
```
12+
13+
**Error**: `The ConnectionString property has not been initialized.`
14+
15+
### **Solution**: Complete Database Integration
16+
✅ **Added PostgreSQL Test Database Support**
17+
✅ **Docker Compose Configuration**
18+
✅ **Automatic Database Setup**
19+
✅ **Version Compatibility Fixes**
20+
21+
## 🔧 Implementation Details
22+
23+
### **1. Database Configuration**
24+
- **Connection String**: Uses port 5433 to avoid conflicts with local PostgreSQL
25+
- **Database**: `github_actions_orchestrator_test`
26+
- **User**: `test_user` / `test_password`
27+
- **Docker Container**: `github-actions-orchestrator-test-db`
28+
29+
### **2. Files Created/Modified**
30+
31+
**New Files:**
32+
- `docker-compose.test.yml` - PostgreSQL test database configuration
33+
- `setup-database.sh` - Database setup script (executable)
34+
35+
**Modified Files:**
36+
- `TestConfiguration/TestAutoScalerConfiguration.cs` - Added `DbConnectionString`
37+
- `Fixtures/ProxmoxTestFixture.cs` - Added database initialization
38+
- `run-tests.sh` - Automatic database startup and health checks
39+
- `README.md` - Added database setup documentation
40+
41+
### **3. Version Compatibility Fixes**
42+
Updated to Entity Framework Core 8.0.8 across all projects:
43+
- **Main Project**: `Microsoft.EntityFrameworkCore.Design` 8.0.7 → 8.0.8
44+
- **Main Project**: `Npgsql.EntityFrameworkCore.PostgreSQL` 8.0.4 → 8.0.8
45+
- **Test Project**: Added `Microsoft.EntityFrameworkCore` 8.0.8
46+
- **Test Project**: Added `Npgsql.EntityFrameworkCore.PostgreSQL` 8.0.8
47+
48+
### **4. Database Initialization**
49+
The `ProxmoxTestFixture` now:
50+
- Validates test environment (Proxmox + Database)
51+
- Sets up `Program.Config` for tests
52+
- Initializes PostgreSQL test database
53+
- Runs `EnsureCreatedAsync()` to create schema
54+
- Handles database connection failures gracefully
55+
56+
## 🚀 Usage Instructions
57+
58+
### **Quick Start (Recommended)**
59+
```bash
60+
cd GithubActionsOrchestrator.IntegrationTests
61+
62+
# Set Proxmox credentials
63+
export PVE_HOST="your-proxmox-host.com"
64+
export PVE_USERNAME="root@pam"
65+
export PVE_PASSWORD="your-password"
66+
67+
# Run tests (automatically starts database)
68+
./run-tests.sh
69+
```
70+
71+
### **Manual Database Setup**
72+
```bash
73+
# Start PostgreSQL manually
74+
./setup-database.sh
75+
76+
# Or with Docker Compose directly
77+
docker-compose -f docker-compose.test.yml up -d
78+
79+
# Run tests
80+
dotnet test GithubActionsOrchestrator.IntegrationTests.csproj
81+
```
82+
83+
### **Custom Database**
84+
```bash
85+
export TEST_DB_CONNECTION_STRING="Host=myhost;Port=5432;Database=test_db;Username=user;Password=pass"
86+
./run-tests.sh
87+
```
88+
89+
## 🛡️ Safety Features
90+
91+
### **Isolation**
92+
- **Database**: Separate test database (`github_actions_orchestrator_test`)
93+
- **Port**: Uses 5433 to avoid production PostgreSQL conflicts
94+
- **VM IDs**: Test range 20000+ vs production 5000+
95+
- **Runner Names**: Test prefix `ghr-test-*` vs production `ghr-*`
96+
97+
### **Validation**
98+
- Environment variable validation (PVE_HOST, PVE_USERNAME, PVE_PASSWORD)
99+
- Database connection validation with helpful error messages
100+
- Production settings detection and prevention
101+
- VM ID range enforcement
102+
103+
### **Cleanup**
104+
- Automatic test VM deletion after tests
105+
- Docker volumes for easy database reset
106+
- Comprehensive error handling and troubleshooting
107+
108+
## 📊 Test Results
109+
110+
With PostgreSQL database running, all integration tests should pass:
111+
- ✅ **9 integration tests** covering all ProxmoxCloudController operations
112+
- ✅ **VM creation** with hostname collision detection
113+
- ✅ **Database integration** for unique name generation
114+
- ✅ **Isolated test environment** (VM IDs 20000+, prefix `ghr-test`)
115+
- ✅ **Automatic cleanup** of test VMs and data
116+
117+
## 🔧 Troubleshooting
118+
119+
### **Database Issues**
120+
```bash
121+
# Database not starting
122+
docker-compose -f docker-compose.test.yml logs postgres-test
123+
124+
# Reset database
125+
docker-compose -f docker-compose.test.yml down -v
126+
./setup-database.sh restart
127+
128+
# Connect to database manually
129+
docker-compose -f docker-compose.test.yml exec postgres-test psql -U test_user -d github_actions_orchestrator_test
130+
```
131+
132+
### **Entity Framework Errors**
133+
The EF Core version conflicts have been resolved by aligning all packages to version 8.0.8.
134+
135+
### **Connection String Issues**
136+
Default connection string:
137+
```
138+
Host=localhost;Port=5433;Database=github_actions_orchestrator_test;Username=test_user;Password=test_password
139+
```
140+
141+
## 🎯 Next Steps
142+
143+
The integration tests are now fully functional with database support:
144+
145+
1. **Ready to Use**: Set Proxmox credentials and run `./run-tests.sh`
146+
2. **CI/CD Ready**: Docker-based database can be integrated into CI pipelines
147+
3. **Production Safe**: Comprehensive isolation and validation measures
148+
4. **Maintainable**: Clear documentation and troubleshooting guides
149+
150+
The database integration completes the integration test suite, making it a comprehensive testing solution for the ProxmoxCloudController with full production safety measures.

‎GithubActionsOrchestrator.IntegrationTests/Fixtures/ProxmoxTestFixture.cs‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
using GithubActionsOrchestrator.CloudControllers;
2+
using GithubActionsOrchestrator.Database;
3+
using GithubActionsOrchestrator.IntegrationTests.TestConfiguration;
4+
using GithubActionsOrchestrator.Models;
5+
using Microsoft.EntityFrameworkCore;
6+
using Microsoft.Extensions.Logging;
7+
8+
namespace GithubActionsOrchestrator.IntegrationTests.Fixtures;
9+
10+
public class ProxmoxTestFixture : IDisposable
11+
{
12+
private readonly List<long> _createdVmIds = new List<long>();
13+
private readonly ILogger<ProxmoxCloudController> _logger;
14+
15+
public ProxmoxCloudController Controller { get; }
16+
public AutoScalerConfiguration TestConfig { get; }
17+
18+
public ProxmoxTestFixture()
19+
{
20+
// Validate environment
21+
ValidateTestEnvironment();
22+
23+
// Setup logging
24+
using var loggerFactory = LoggerFactory.Create(builder =>
25+
builder.AddConsole().SetMinimumLevel(LogLevel.Information));
26+
_logger = loggerFactory.CreateLogger<ProxmoxCloudController>();
27+
28+
// Create test configuration
29+
TestConfig = TestAutoScalerConfiguration.CreateTestConfiguration();
30+
31+
// Set the Program.Config for the controller to use
32+
Program.Config = TestConfig;
33+
34+
// Initialize test database synchronously (xUnit doesn't support async constructors)
35+
InitializeDatabaseAsync().GetAwaiter().GetResult();
36+
37+
// Create controller with test configuration
38+
Controller = new ProxmoxCloudController(
39+
_logger,
40+
TestConfig.Sizes,
41+
TestConfig.ProvisionScriptBaseUrl,
42+
TestConfig.MetricUser,
43+
TestConfig.MetricPassword,
44+
TestConfig.PveHost,
45+
TestConfig.PveUsername,
46+
TestConfig.PvePassword,
47+
TestConfig.PveTemplate,
48+
TestConfig.MinVmId
49+
);
50+
}
51+
52+
public void TrackCreatedVm(long vmId)
53+
{
54+
_createdVmIds.Add(vmId);
55+
}
56+
57+
private async Task InitializeDatabaseAsync()
58+
{
59+
try
60+
{
61+
using var context = new ActionsRunnerContext();
62+
63+
// Ensure the database exists and is up to date
64+
await context.Database.EnsureCreatedAsync();
65+
66+
// Optionally run migrations if needed
67+
// await context.Database.MigrateAsync();
68+
69+
Console.WriteLine("Test database initialized successfully");
70+
}
71+
catch (Exception ex)
72+
{
73+
throw new InvalidOperationException($"Failed to initialize test database: {ex.Message}. Make sure PostgreSQL test database is running on port 5433.", ex);
74+
}
75+
}
76+
77+
private void ValidateTestEnvironment()
78+
{
79+
var requiredEnvVars = new[] { "PVE_HOST", "PVE_USERNAME", "PVE_PASSWORD" };
80+
var missingVars = requiredEnvVars.Where(var => string.IsNullOrEmpty(Environment.GetEnvironmentVariable(var))).ToList();
81+
82+
if (missingVars.Any())
83+
{
84+
throw new InvalidOperationException($"Missing required environment variables: {string.Join(", ", missingVars)}");
85+
}
86+
87+
// Validate test settings to ensure we're not running against production
88+
var runnerPrefix = Environment.GetEnvironmentVariable("TEST_RUNNER_PREFIX") ?? "ghr-test";
89+
var minVmId = int.Parse(Environment.GetEnvironmentVariable("TEST_MIN_VM_ID") ?? "20000");
90+
91+
if (runnerPrefix == "ghr")
92+
{
93+
throw new InvalidOperationException("Cannot run integration tests with production runner prefix 'ghr'. Use 'ghr-test' or another test prefix.");
94+
}
95+
96+
if (minVmId < 20000)
97+
{
98+
throw new InvalidOperationException($"Test MinVmId ({minVmId}) must be >= 20000 to avoid production VM ID range collision.");
99+
}
100+
}
101+
102+
public async Task CleanupCreatedVmsAsync()
103+
{
104+
foreach (var vmId in _createdVmIds)
105+
{
106+
try
107+
{
108+
await Controller.DeleteRunner(vmId);
109+
Console.WriteLine($"Cleaned up test VM: {vmId}");
110+
}
111+
catch (Exception ex)
112+
{
113+
Console.WriteLine($"Failed to cleanup VM {vmId}: {ex.Message}");
114+
}
115+
}
116+
_createdVmIds.Clear();
117+
}
118+
119+
public async Task<List<CspServer>> GetTestVmsAsync()
120+
{
121+
var allServers = await Controller.GetAllServersFromCsp();
122+
return allServers.Where(s => s.Name.StartsWith(TestConfig.RunnerPrefix)).ToList();
123+
}
124+
125+
public void Dispose()
126+
{
127+
try
128+
{
129+
CleanupCreatedVmsAsync().GetAwaiter().GetResult();
130+
}
131+
catch (Exception ex)
132+
{
133+
Console.WriteLine($"Error during fixture cleanup: {ex.Message}");
134+
}
135+
}
136+
}

‎GithubActionsOrchestrator.IntegrationTests/GithubActionsOrchestrator.IntegrationTests.csproj‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
<Project Sdk="Microsoft.NET.Sdk">
2+
3+
<PropertyGroup>
4+
<TargetFramework>net9.0</TargetFramework>
5+
<Nullable>disable</Nullable>
6+
<ImplicitUsings>enable</ImplicitUsings>
7+
<IsPackable>false</IsPackable>
8+
<IsTestProject>true</IsTestProject>
9+
</PropertyGroup>
10+
11+
<ItemGroup>
12+
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
13+
<PackageReference Include="xunit" Version="2.6.1" />
14+
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.3">
15+
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
16+
<PrivateAssets>all</PrivateAssets>
17+
</PackageReference>
18+
<PackageReference Include="xunit.runner.console" Version="2.6.1" />
19+
<PackageReference Include="coverlet.collector" Version="6.0.0">
20+
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
21+
<PrivateAssets>all</PrivateAssets>
22+
</PackageReference>
23+
<PackageReference Include="Microsoft.Extensions.Configuration" Version="8.0.0" />
24+
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="8.0.0" />
25+
<PackageReference Include="Microsoft.Extensions.Logging" Version="8.0.0" />
26+
<PackageReference Include="Microsoft.Extensions.Logging.Console" Version="8.0.0" />
27+
<PackageReference Include="Microsoft.EntityFrameworkCore" Version="8.0.8" />
28+
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="8.0.8" />
29+
<PackageReference Include="Moq" Version="4.20.70" />
30+
</ItemGroup>
31+
32+
<ItemGroup>
33+
<ProjectReference Include="..\GithubActionsOrchestrator.csproj" />
34+
</ItemGroup>
35+
36+
<ItemGroup>
37+
<None Update="appsettings.test.json">
38+
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
39+
</None>
40+
</ItemGroup>
41+
42+
</Project>

0 commit comments

Comments
 (0)