Merge pull request #29 from OpenElementsLabs/feat/103-tests-postgres-testcontainers

feat(spec-103): backend tests on PostgreSQL via Testcontainers (drop H2)

Author: hendrikebbers

.claude/conventions/backend.md

@@ -32,7 +32,7 @@ When building libraries that target backend applications, provide support for Sp
 - **IMPORTANT**: Use **JPA** (Jakarta Persistence API) for data access. Do not use implementation-specific APIs (e.g., Hibernate session or criteria API directly) — always program against the JPA interfaces.
 - Use **[Flyway](https://flywaydb.org/)** for database schema management and migrations in all projects with a database.
 - **PostgreSQL** is the preferred database for test environments and production.
-- **H2** (in-memory) is the preferred database for fast, automated unit/integration tests. In the future, we plan to replace H2 with [Testcontainers](https://www.testcontainers.org/)-based PostgreSQL to test against the same database in all environments.
+- Automated tests run against the same PostgreSQL version as production via [Testcontainers](https://testcontainers.com/). Docker must be available on the developer machine and on CI runners (Linux `ubuntu-latest` ships with it). H2 is no longer used.
 - **IMPORTANT**: Database connection URLs, credentials, and other settings must be configurable via environment variables (see [fullstack-architecture.md](fullstack-architecture.md#configuration)).
 
 ## Data Privacy and GDPR
@@ -60,24 +60,24 @@ Every backend layer must have its own tests. Tests at a higher layer do not repl
 
 **Repository tests (`@DataJpaTest`)**
 - Every repository interface with custom query methods must have a test class.
-- Use `@DataJpaTest` — it auto-configures an embedded H2 database and rolls back after each test. No `@ActiveProfiles` annotation needed.
+- Use `@DataJpaTest` and the project's `AbstractDbTest` so the test runs against a real PostgreSQL container with Flyway-managed schema.
 - Use `TestEntityManager` (`persistAndFlush` + `clear()`) to force real database roundtrips instead of hitting the first-level cache.
 - Test custom query methods, pagination, and database constraints (NOT NULL, unique).
 
 **Service tests (`@SpringBootTest`)**
 - Every service class must have a test class that covers all public methods.
-- Use `@SpringBootTest` with `@ActiveProfiles("test")` and a real H2 database — do not mock repositories.
-- Clean up test data in `@BeforeEach` by deleting via repositories in foreign-key order. Do not use `@Transactional` on service tests — this hides transaction boundary bugs in service methods that are themselves `@Transactional`.
+- Extend `AbstractDbTest` — it provides `@SpringBootTest`, `@AutoConfigureMockMvc`, `@ActiveProfiles("test")`, a shared Postgres container, and per-test `TRUNCATE` cleanup. Do not redeclare those annotations.
+- Do not use `@Transactional` on service tests — this hides transaction boundary bugs in service methods that are themselves `@Transactional`.
 - Test happy paths, validation errors, cross-entity business logic, and edge cases.
 
 **Controller tests (`@SpringBootTest` + `MockMvc`)**
 - Every controller must have a test class that verifies HTTP status codes, request/response serialization, and validation.
-- Use `@SpringBootTest` with `@AutoConfigureMockMvc` and `@ActiveProfiles("test")`.
+- Extend `AbstractDbTest` (which already configures MockMvc).
 
 ### General test rules
 
 - Do not mock repositories or other internal dependencies when the real implementation is fast and available. Mocks add complexity without proportional value in small codebases and miss integration bugs.
-- Use H2 for all automated tests. In the future, Testcontainers with PostgreSQL will replace H2 (see Data Access section).
+- All Spring-based tests run against a real PostgreSQL container via `AbstractDbTest` (see Data Access section). Pure unit tests (no Spring context) do not need to extend it.
 - Test classes live in the same package structure as the code they test.
 
 ## Observability

README.md

@@ -146,6 +146,19 @@ docker compose down -v    # removes database data
 cd backend && ./mvnw clean verify
 ```
 
+> Backend tests run against a real PostgreSQL 17 container started by
+> [Testcontainers](https://testcontainers.com/). **Docker (Docker Desktop or
+> any Docker-compatible daemon) must be running** before `./mvnw test` /
+> `./mvnw verify` is invoked. For faster local iteration, enable container
+> reuse by adding the following line to `~/.testcontainers.properties`:
+>
+> ```properties
+> testcontainers.reuse.enable=true
+> ```
+>
+> With reuse on, the same Postgres container survives between `mvn test`
+> runs and the per-suite startup cost drops from ~3–5 s to ~200 ms.
+
 **Frontend:**
 
 ```bash

backend/pom.xml

@@ -131,8 +131,20 @@
         </dependency>
 
         <dependency>
-            <groupId>com.h2database</groupId>
-            <artifactId>h2</artifactId>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-testcontainers</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.testcontainers</groupId>
+            <artifactId>junit-jupiter</artifactId>
+            <scope>test</scope>
+        </dependency>
+
+        <dependency>
+            <groupId>org.testcontainers</groupId>
+            <artifactId>postgresql</artifactId>
             <scope>test</scope>
         </dependency>
     </dependencies>

backend/src/main/java/com/openelements/crm/auditlog/AuditLogController.java

@@ -68,6 +68,9 @@ public Page<AuditLogDto> listAuditLogs(
             + "Used to populate the entity-type filter dropdown. Requires the IT-ADMIN role."
     )
     public List<String> listEntityTypes() {
-        return auditLogService.findAllEntityTypes();
+        // Sort explicitly: the upstream findAllEntityTypes() returns rows in
+        // the database's natural order (insertion order on H2, undefined on
+        // Postgres) — the endpoint's contract is "sorted alphabetically".
+        return auditLogService.findAllEntityTypes().stream().sorted().toList();
     }
 }

backend/src/test/java/com/openelements/crm/AbstractDbTest.java

@@ -0,0 +1,98 @@
+package com.openelements.crm;
+
+import com.openelements.spring.base.services.user.SystemUser;
+import com.openelements.spring.base.services.user.UserRepository;
+import org.junit.jupiter.api.AfterEach;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
+import org.springframework.boot.test.context.SpringBootTest;
+import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
+import org.springframework.jdbc.core.JdbcTemplate;
+import org.springframework.test.context.ActiveProfiles;
+import org.testcontainers.containers.PostgreSQLContainer;
+
+/**
+ * Shared test base for Spring Boot tests that touch the database. Starts a
+ * single {@code postgres:17-alpine} container once per JVM, registers it as
+ * the application {@link javax.sql.DataSource} via {@link ServiceConnection},
+ * and truncates all application tables between tests for per-method isolation.
+ *
+ * <p>The container is reused across runs when {@code testcontainers.reuse.enable=true}
+ * is set in {@code ~/.testcontainers.properties} (local dev). CI runs ignore
+ * the reuse hint, so each build gets a fresh container.
+ *
+ * <p>Adding a new table in a Flyway migration requires extending {@link #TABLES_TO_TRUNCATE}.
+ * This is intentional — the explicit list keeps the truncate fast and predictable.
+ *
+ * <p>The {@code @Testcontainers} JUnit extension is intentionally not used —
+ * it discovers {@code @Container} fields and does nothing for a manually
+ * started container. The {@code static { POSTGRES.start(); }} block is what
+ * actually brings the container up before any subclass context loads.
+ */
+@SpringBootTest
+@AutoConfigureMockMvc
+@ActiveProfiles("test")
+public abstract class AbstractDbTest {
+
+    /**
+     * Tables truncated between tests. Order is irrelevant because
+     * {@code CASCADE} drops dependent rows.
+     */
+    private static final String TABLES_TO_TRUNCATE = String.join(", ",
+        "api_keys",
+        "audit_log",
+        "comments",
+        "company_comments",
+        "company_tags",
+        "contact_comments",
+        "contact_social_links",
+        "contact_tags",
+        "contacts",
+        "companies",
+        "settings",
+        "task_comments",
+        "task_tags",
+        "tasks",
+        "tags",
+        "users",
+        "webhooks"
+    );
+
+    @ServiceConnection
+    static final PostgreSQLContainer<?> POSTGRES =
+        new PostgreSQLContainer<>("postgres:17-alpine")
+            .withReuse(true);
+
+    static {
+        POSTGRES.start();
+    }
+
+    @Autowired
+    private JdbcTemplate jdbcTemplate;
+
+    @Autowired
+    private UserRepository userRepository;
+
+    @AfterEach
+    void truncateAllTables() {
+        // RESTART IDENTITY keeps sequence-backed columns deterministic; CASCADE
+        // drops dependent rows so order of tables is irrelevant.
+        jdbcTemplate.execute("TRUNCATE TABLE " + TABLES_TO_TRUNCATE + " RESTART IDENTITY CASCADE");
+    }
+
+    /**
+     * (Re-)inserts the {@link SystemUser} row needed by audit-log / comment
+     * tests that reference {@code SystemUser.ID} as a foreign key. Idempotent:
+     * subclasses call this from their own {@code @BeforeEach} when needed.
+     * The {@code @AfterEach} truncate wipes {@code users}, so the seed must
+     * be re-applied per test method.
+     */
+    protected void seedSystemUser() {
+        if (userRepository.findBySub(SystemUser.SUB).isEmpty()) {
+            jdbcTemplate.update(
+                "INSERT INTO users (id, sub, name, created_at, updated_at) "
+                    + "VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
+                SystemUser.ID, SystemUser.SUB, SystemUser.NAME);
+        }
+    }
+}

backend/src/test/java/com/openelements/crm/auditlog/AuditLogControllerTest.java

@@ -1,19 +1,17 @@
 package com.openelements.crm.auditlog;
 
+import com.openelements.crm.AbstractDbTest;
 import com.openelements.spring.base.services.audit.AuditAction;
 import com.openelements.spring.base.services.audit.AuditLogDataService;
 import com.openelements.spring.base.services.user.UserEntity;
 import com.openelements.spring.base.services.user.UserRepository;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
-import org.springframework.boot.test.context.SpringBootTest;
 import org.springframework.jdbc.core.JdbcTemplate;
 import org.springframework.security.core.GrantedAuthority;
 import org.springframework.security.core.authority.SimpleGrantedAuthority;
 import org.springframework.security.oauth2.jwt.Jwt;
-import org.springframework.test.context.ActiveProfiles;
 import org.springframework.test.web.servlet.MockMvc;
 import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
 
@@ -36,10 +34,7 @@
  * focuses on the API contract — pagination, filter combinations, and the
  * entity-types listing.
  */
-@SpringBootTest
-@AutoConfigureMockMvc
-@ActiveProfiles("test")
-class AuditLogControllerTest {
+class AuditLogControllerTest extends AbstractDbTest {
 
     @Autowired
     private MockMvc mockMvc;

backend/src/test/java/com/openelements/crm/comment/CommentEndpointsIntegrationTest.java

@@ -2,6 +2,7 @@
 
 import com.fasterxml.jackson.databind.JsonNode;
 import com.fasterxml.jackson.databind.ObjectMapper;
+import com.openelements.crm.AbstractDbTest;
 import com.openelements.crm.company.CompanyEntity;
 import com.openelements.crm.company.CompanyRepository;
 import com.openelements.crm.contact.ContactEntity;
@@ -11,17 +12,13 @@
 import com.openelements.spring.base.services.comment.CommentService;
 import com.openelements.spring.base.services.user.SystemUser;
 import com.openelements.spring.base.services.user.UserRepository;
-import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
-import org.springframework.boot.test.context.SpringBootTest;
 import org.springframework.http.MediaType;
 import org.springframework.security.core.GrantedAuthority;
 import org.springframework.security.core.authority.SimpleGrantedAuthority;
 import org.springframework.security.oauth2.jwt.Jwt;
-import org.springframework.test.context.ActiveProfiles;
 import org.springframework.test.web.servlet.MockMvc;
 import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
 
@@ -43,10 +40,7 @@
  * spec 094. Covers happy paths, validation, mismatched ownership, listing, and
  * the cascade behaviour on owner deletion.
  */
-@SpringBootTest
-@AutoConfigureMockMvc
-@ActiveProfiles("test")
-class CommentEndpointsIntegrationTest {
+class CommentEndpointsIntegrationTest extends AbstractDbTest {
 
     @Autowired
     private MockMvc mockMvc;
@@ -76,22 +70,8 @@ class CommentEndpointsIntegrationTest {
     private ObjectMapper objectMapper;
 
     @BeforeEach
-    void seedSystemUser() {
-        if (userRepository.findBySub(SystemUser.SUB).isEmpty()) {
-            jdbcTemplate.update(
-                "INSERT INTO users (id, sub, name, created_at, updated_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
-                SystemUser.ID, SystemUser.SUB, SystemUser.NAME);
-        }
-    }
-
-    @AfterEach
-    void clean() {
-        // Delete owners first to clear join-table rows, then the orphaned comments.
-        jdbcTemplate.update("DELETE FROM company_comments");
-        jdbcTemplate.update("DELETE FROM contact_comments");
-        contactRepository.deleteAll();
-        companyRepository.deleteAll();
-        commentRepository.deleteAll();
+    void seed() {
+        seedSystemUser();
     }
 
     private static MockHttpServletRequestBuilder asUser(MockHttpServletRequestBuilder builder, List<String> roles) {

backend/src/test/java/com/openelements/crm/contact/ContactPhotoHeicWebpIntegrationTest.java

@@ -8,23 +8,20 @@
 import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.multipart;
 import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
 
+import com.openelements.crm.AbstractDbTest;
 import com.openelements.spring.base.services.user.SystemUser;
 import com.openelements.spring.base.services.user.UserRepository;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.List;
-import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Disabled;
 import org.junit.jupiter.api.Test;
 import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
-import org.springframework.boot.test.context.SpringBootTest;
 import org.springframework.mock.web.MockMultipartFile;
 import org.springframework.security.core.GrantedAuthority;
 import org.springframework.security.core.authority.SimpleGrantedAuthority;
 import org.springframework.security.oauth2.jwt.Jwt;
-import org.springframework.test.context.ActiveProfiles;
 import org.springframework.test.web.servlet.MockMvc;
 import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
 import org.springframework.test.web.servlet.request.MockMultipartHttpServletRequestBuilder;
@@ -41,10 +38,7 @@
  * has no probe sample on classpath, so {@code HeicSupportCheck.isHeicAvailable()}
  * naturally returns false) and the malformed-WebP rejection path.
  */
-@SpringBootTest
-@AutoConfigureMockMvc
-@ActiveProfiles("test")
-class ContactPhotoHeicWebpIntegrationTest {
+class ContactPhotoHeicWebpIntegrationTest extends AbstractDbTest {
 
     private static final String FIXTURE_TODO =
         "Awaiting test fixtures — see TODO.md: HEIC- und WebP-Testfixtures bereitstellen";
@@ -65,17 +59,8 @@ class ContactPhotoHeicWebpIntegrationTest {
     private org.springframework.jdbc.core.JdbcTemplate jdbcTemplate;
 
     @BeforeEach
-    void seedSystemUser() {
-        if (userRepository.findBySub(SystemUser.SUB).isEmpty()) {
-            jdbcTemplate.update(
-                "INSERT INTO users (id, sub, name, created_at, updated_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
-                SystemUser.ID, SystemUser.SUB, SystemUser.NAME);
-        }
-    }
-
-    @AfterEach
-    void clean() {
-        contactRepository.deleteAll();
+    void seed() {
+        seedSystemUser();
     }
 
     private static <T extends MockHttpServletRequestBuilder> T asUser(final T builder) {