Skip to content

Commit eb2803e

Browse files
jarlahclaude
jarlah
and
claude
authored
Improve Docker-in-Docker (DinD) support (#248)
* Improve Docker-in-Docker (DinD) support - Add TESTCONTAINERS_HOST_OVERRIDE env var and tc.host.override property to allow explicit host override, bypassing auto-detection - Add TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE env var to override the Docker socket path mounted into Ryuk - Improve gateway fallback by parsing /proc/net/route when bridge gateway inspection fails, instead of falling back to localhost - Improve container detection by checking /proc/1/cgroup as fallback when /.dockerenv doesn't exist (handles more container runtimes) - Add unit tests for route parsing and container detection logic Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add connect timeout to Ryuk TCP socket gen_tcp.connect had no timeout (default :infinity), causing an indefinite hang when bridge gateway IP is unreachable due to hairpin NAT issues in DooD environments. Add 5-second connect timeout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fall back to container internal IP for Ryuk connection When connecting to Ryuk via docker_hostname:mapped_port fails (common in DooD environments due to hairpin NAT), fall back to connecting via the container's internal IP on its internal port (8080). Both the test runner container and Ryuk are on the same bridge network by default, so direct IP access works reliably. Also extracts try_tcp_connect/2 helper to reduce duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Use container internal IPs in DooD environments When running inside a container with a shared Docker socket (DooD), the bridge gateway's mapped ports may be unreachable due to hairpin NAT. This change detects that scenario at startup by probing the gateway, and switches to "container networking mode" where: - get_host(container) returns container.ip_address instead of gateway - get_port(container, port) returns the internal port directly All built-in container modules (postgres, mysql, redis, kafka, etc.) now use these DooD-aware APIs, so tests work automatically in both standard and DooD environments without any configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix hardcoded hosts in tests and PortWaitStrategy - PortWaitStrategy now uses get_host(container) and get_port(container) at wait time, overriding the IP set at construction. This fixes Selenium and EMQX containers in DooD. - Update tests that hardcoded 127.0.0.1 or localhost to use the DooD-aware APIs instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix remaining test failures in DooD - Remove unused Container alias from kafka_container_test - Skip host network test in DooD (inherently incompatible since host networking binds to Docker host, not the test container) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fall back to mapped ports for containers on custom networks In container_ip mode, containers on custom Docker networks are not reachable from the test container via internal IP (different network). Detect this via the container.network field and fall back to the standard docker_hostname:mapped_port approach for those containers. Also fix unused alias warnings in port_wait_strategy and kafka_container_test. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix Kafka and Toxiproxy in DooD environments Kafka: In DooD, clients connect to the container's internal IP so Kafka must advertise on the internal port. Use a BROKER listener name in container mode to avoid advertising the unreachable bridge gateway address. Toxiproxy test: Use get_host(container) instead of get_host() so the API URL uses the correct host in DooD. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Tag DooD-incompatible tests with :dood_limitation Tests that require custom Docker networks or have slow container startup in nested Docker are tagged with @tag :dood_limitation and automatically excluded when running inside a container. Affected tests: - Toxiproxy integration (custom network) - Network hostname communication (custom network) - EMQX custom config (slow startup timeout) - Selenium (slow startup timeout) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix Kafka advertised listeners in DooD with after_start Kafka needs to advertise an address reachable by clients. In DooD, the container IP is only known after startup. Use after_start to run kafka-configs.sh and update the advertised listener to BROKER://container_ip:internal_port, so KafkaEx clients can resolve the broker address correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Tag Kafka integration tests as dood_limitation Kafka's advertised.listeners cannot be dynamically updated to use the container's internal IP after startup. This is a known limitation that testcontainers-java solves with custom startup script injection. Tag these tests for exclusion in DooD environments. Also reverts the kafka-configs.sh after_start approach which doesn't work reliably (not a dynamic config in KRaft mode). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert Kafka DooD listener changes Since Kafka tests are tagged dood_limitation, the DooD-specific listener config is dead code. Revert to original with_kraft_config. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Use @tag :dood_limitation for host network test Replace runtime container check with tag-based exclusion, consistent with all other DooD-incompatible tests. Restore original 127.0.0.1 since this test only runs outside containers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c61dcbf commit eb2803e

24 files changed

+433
-72
lines changed

‎lib/container/cassandra_container.ex‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,13 +62,13 @@ defmodule Testcontainers.CassandraContainer do
6262
@doc """
6363
Retrieves the port mapped by the Docker host for the Cassandra container.
6464
"""
65-
def port(%Container{} = container), do: Container.mapped_port(container, @default_port)
65+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_port)
6666

6767
@doc """
6868
Generates the connection URL for accessing the Cassandra service running within the container.
6969
"""
7070
def connection_uri(%Container{} = container) do
71-
"#{Testcontainers.get_host()}:#{port(container)}"
71+
"#{Testcontainers.get_host(container)}:#{port(container)}"
7272
end
7373

7474
defimpl ContainerBuilder do

‎lib/container/ceph_container.ex‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ defmodule Testcontainers.CephContainer do
175175
iex> CephContainer.port(container)
176176
32768 # This value will be different depending on the mapped port.
177177
"""
178-
def port(%Container{} = container), do: Container.mapped_port(container, @default_port)
178+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_port)
179179

180180
@doc """
181181
Generates the connection URL for accessing the Ceph service running within the container.
@@ -192,7 +192,7 @@ defmodule Testcontainers.CephContainer do
192192
"http://localhost:32768" # This value will be different depending on the mapped port.
193193
"""
194194
def connection_url(%Container{} = container) do
195-
"http://#{Testcontainers.get_host()}:#{port(container)}"
195+
"http://#{Testcontainers.get_host(container)}:#{port(container)}"
196196
end
197197

198198
@doc """
@@ -203,7 +203,7 @@ defmodule Testcontainers.CephContainer do
203203
[
204204
port: CephContainer.port(container),
205205
scheme: "http://",
206-
host: Testcontainers.get_host(),
206+
host: Testcontainers.get_host(container),
207207
access_key_id: container.environment[:CEPH_DEMO_ACCESS_KEY],
208208
secret_access_key: container.environment[:CEPH_DEMO_SECRET_KEY]
209209
]

‎lib/container/emqx_container.ex‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ defmodule Testcontainers.EmqxContainer do
108108
Returns the port on the _host machine_ where the Emqx container is listening.
109109
"""
110110
def mqtt_port(%Container{} = container),
111-
do: Container.mapped_port(container, @default_mqtt_port)
111+
do: Testcontainers.get_port(container, @default_mqtt_port)
112112

113113
defimpl ContainerBuilder do
114114
import Container

‎lib/container/kafka_container.ex‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -156,15 +156,15 @@ defmodule Testcontainers.KafkaContainer do
156156
Returns the bootstrap servers string for connecting to the Kafka container.
157157
"""
158158
def bootstrap_servers(%Container{} = container) do
159-
port = Container.mapped_port(container, @default_internal_kafka_port)
160-
"#{Testcontainers.get_host()}:#{port}"
159+
port = Testcontainers.get_port(container, @default_internal_kafka_port)
160+
"#{Testcontainers.get_host(container)}:#{port}"
161161
end
162162

163163
@doc """
164164
Returns the port on the host machine where the Kafka container is listening.
165165
"""
166166
def port(%Container{} = container),
167-
do: Container.mapped_port(container, @default_internal_kafka_port)
167+
do: Testcontainers.get_port(container, @default_internal_kafka_port)
168168

169169
defimpl Testcontainers.ContainerBuilder do
170170
import Container

‎lib/container/minio_container.ex‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,13 @@ defmodule Testcontainers.MinioContainer do
4949
@doc """
5050
Retrieves the port mapped by the Docker host for the Minio container.
5151
"""
52-
def port(%Container{} = container), do: Container.mapped_port(container, @default_s3_port)
52+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_s3_port)
5353

5454
@doc """
5555
Generates the connection URL for accessing the Minio service running within the container.
5656
"""
5757
def connection_url(%Container{} = container) do
58-
"http://#{Testcontainers.get_host()}:#{port(container)}"
58+
"http://#{Testcontainers.get_host(container)}:#{port(container)}"
5959
end
6060

6161
@doc """
@@ -66,7 +66,7 @@ defmodule Testcontainers.MinioContainer do
6666
[
6767
port: MinioContainer.port(container),
6868
scheme: "http://",
69-
host: Testcontainers.get_host(),
69+
host: Testcontainers.get_host(container),
7070
access_key_id: container.environment[:MINIO_ROOT_USER],
7171
secret_access_key: container.environment[:MINIO_ROOT_PASSWORD]
7272
]

‎lib/container/mysql_container.ex‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -175,14 +175,14 @@ defmodule Testcontainers.MySqlContainer do
175175
@doc """
176176
Returns the port on the _host machine_ where the MySql container is listening.
177177
"""
178-
def port(%Container{} = container), do: Container.mapped_port(container, @default_port)
178+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_port)
179179

180180
@doc """
181181
Returns the connection parameters to connect to the database from the _host machine_.
182182
"""
183183
def connection_parameters(%Container{} = container) do
184184
[
185-
hostname: Testcontainers.get_host(),
185+
hostname: Testcontainers.get_host(container),
186186
port: port(container),
187187
username: container.environment[:MYSQL_USER],
188188
password: container.environment[:MYSQL_PASSWORD],

‎lib/container/postgres_container.ex‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -175,14 +175,14 @@ defmodule Testcontainers.PostgresContainer do
175175
@doc """
176176
Returns the port on the _host machine_ where the Postgres container is listening.
177177
"""
178-
def port(%Container{} = container), do: Container.mapped_port(container, @default_port)
178+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_port)
179179

180180
@doc """
181181
Returns the connection parameters to connect to the database from the _host machine_.
182182
"""
183183
def connection_parameters(%Container{} = container) do
184184
[
185-
hostname: Testcontainers.get_host(),
185+
hostname: Testcontainers.get_host(container),
186186
port: port(container),
187187
username: container.environment[:POSTGRES_USER],
188188
password: container.environment[:POSTGRES_PASSWORD],

‎lib/container/rabbitmq_container.ex‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@ defmodule Testcontainers.RabbitMQContainer do
187187
"""
188188
def port(%Container{} = container),
189189
do:
190-
Container.mapped_port(
190+
Testcontainers.get_port(
191191
container,
192192
String.to_integer(container.environment[:RABBITMQ_NODE_PORT])
193193
)
@@ -210,7 +210,7 @@ defmodule Testcontainers.RabbitMQContainer do
210210
"amqp://guest:guest@localhost:32768/vhost"
211211
"""
212212
def connection_url(%Container{} = container) do
213-
"amqp://#{container.environment[:RABBITMQ_DEFAULT_USER]}:#{container.environment[:RABBITMQ_DEFAULT_PASS]}@#{Testcontainers.get_host()}:#{port(container)}#{virtual_host_segment(container)}"
213+
"amqp://#{container.environment[:RABBITMQ_DEFAULT_USER]}:#{container.environment[:RABBITMQ_DEFAULT_PASS]}@#{Testcontainers.get_host(container)}:#{port(container)}#{virtual_host_segment(container)}"
214214
end
215215

216216
@doc """
@@ -233,7 +233,7 @@ defmodule Testcontainers.RabbitMQContainer do
233233
"""
234234
def connection_parameters(%Container{} = container) do
235235
[
236-
host: Testcontainers.get_host(),
236+
host: Testcontainers.get_host(container),
237237
port: port(container),
238238
username: container.environment[:RABBITMQ_DEFAULT_USER],
239239
password: container.environment[:RABBITMQ_DEFAULT_PASS],

‎lib/container/redis_container.ex‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ defmodule Testcontainers.RedisContainer do
122122
@doc """
123123
Returns the port on the _host machine_ where the Redis container is listening.
124124
"""
125-
def port(%Container{} = container), do: Container.mapped_port(container, @default_port)
125+
def port(%Container{} = container), do: Testcontainers.get_port(container, @default_port)
126126

127127
@doc """
128128
Generates the connection URL for accessing the Redis service running within the container.
@@ -141,7 +141,7 @@ defmodule Testcontainers.RedisContainer do
141141
def connection_url(%Container{} = container) do
142142
password = container.environment[:REDIS_PASSWORD]
143143
auth_part = if password, do: ":#{password}@", else: ""
144-
"redis://#{auth_part}#{Testcontainers.get_host()}:#{port(container)}/"
144+
"redis://#{auth_part}#{Testcontainers.get_host(container)}:#{port(container)}/"
145145
end
146146

147147
defimpl ContainerBuilder do

‎lib/container/toxiproxy_container.ex‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ defmodule Testcontainers.ToxiproxyContainer do
8080
Returns the mapped control port on the host for the running container.
8181
"""
8282
def mapped_control_port(%Container{} = container) do
83-
Container.mapped_port(container, @control_port)
83+
Testcontainers.get_port(container, @control_port)
8484
end
8585

8686
@doc """
@@ -92,7 +92,7 @@ defmodule Testcontainers.ToxiproxyContainer do
9292
|> then(&Application.put_env(:toxiproxy_ex, :host, &1))
9393
"""
9494
def api_url(%Container{} = container) do
95-
host = Testcontainers.get_host()
95+
host = Testcontainers.get_host(container)
9696
port = mapped_control_port(container)
9797
"http://#{host}:#{port}"
9898
end
@@ -130,7 +130,7 @@ defmodule Testcontainers.ToxiproxyContainer do
130130
def create_proxy(%Container{} = container, name, upstream, opts \\ []) do
131131
listen_port = Keyword.get(opts, :listen_port, @first_proxy_port)
132132

133-
host = Testcontainers.get_host()
133+
host = Testcontainers.get_host(container)
134134
api_port = mapped_control_port(container)
135135

136136
:inets.start()
@@ -149,11 +149,11 @@ defmodule Testcontainers.ToxiproxyContainer do
149149
case httpc_request_with_retry(:post, {url, headers, ~c"application/json", body}) do
150150
{:ok, {{_, code, _}, _, _}} when code in [200, 201] ->
151151
# Return the mapped port on the host
152-
{:ok, Container.mapped_port(container, listen_port)}
152+
{:ok, Testcontainers.get_port(container, listen_port)}
153153

154154
{:ok, {{_, 409, _}, _, _}} ->
155155
# Proxy already exists, return the port
156-
{:ok, Container.mapped_port(container, listen_port)}
156+
{:ok, Testcontainers.get_port(container, listen_port)}
157157

158158
{:ok, {{_, code, _}, _, response_body}} ->
159159
{:error, {:http_error, code, response_body}}
@@ -193,7 +193,7 @@ defmodule Testcontainers.ToxiproxyContainer do
193193
Deletes a proxy from Toxiproxy.
194194
"""
195195
def delete_proxy(%Container{} = container, name) do
196-
host = Testcontainers.get_host()
196+
host = Testcontainers.get_host(container)
197197
api_port = mapped_control_port(container)
198198

199199
:inets.start()
@@ -212,7 +212,7 @@ defmodule Testcontainers.ToxiproxyContainer do
212212
Resets Toxiproxy, removing all toxics and re-enabling all proxies.
213213
"""
214214
def reset(%Container{} = container) do
215-
host = Testcontainers.get_host()
215+
host = Testcontainers.get_host(container)
216216
api_port = mapped_control_port(container)
217217

218218
:inets.start()
@@ -232,7 +232,7 @@ defmodule Testcontainers.ToxiproxyContainer do
232232
Returns a map of proxy names to their configurations.
233233
"""
234234
def list_proxies(%Container{} = container) do
235-
host = Testcontainers.get_host()
235+
host = Testcontainers.get_host(container)
236236
api_port = mapped_control_port(container)
237237

238238
:inets.start()

0 commit comments

Comments
 (0)