feat: add python remote debug support by morgan-dgk · Pull Request #138 · GoogleCloudPlatform/composer-local-dev

morgan-dgk · 2026-05-28T03:18:26Z

This pull request is based on #53. The majority of the changes introduced have been cherry-picked unchanged from that branch.

Some changes to the original commits included in #53 were required to resolve conflicts and bring the original feature branch up-to-date with the current state of the development branch.

Resolves #51.

All tests passed successfully.

Change-Id: Id1ab768dfaabb7868713ac238c8840406e2c98c0 GitOrigin-RevId: daf3412c40c0d2509e543470f02608b6c4761fe3

morgan-dgk · 2026-05-28T06:45:29Z

~~Converting back to a draft, there are some permission issues related to postgres data folder for some reason when attempting to install openssh-server.~~

Was a local build caching issue 🥲, works as expected.

viktorl-dev · 2026-06-03T12:37:38Z

Hi, Morgan. Thank you for the PR. I'm going to review it and provide some feedback.

Viktor

viktorl-dev

The PR looks good overall. Please address the comments above, and I think we'll be good to go.

viktorl-dev · 2026-06-03T12:45:00Z

+@option_enable_ssh
+@option_ssh_port


I think it also makes sense to enable these 2 options for start and restart.

I've pushed up some changes that aim to address this.

I assume this requires modifying load_from_config and EnvironmentConfig to respect the value of these variables.

Sure, the changes look good to me 👍

morgan-dgk · 2026-06-04T00:44:11Z

Thanks your feedback @viktorl-dev. I've addressed all of the above points with the exception of adding --enable-ssh and --ssh-port to the start and restart commands.

I've added my questions on that point above 🙂.

`install_and_run_sshd` previously failed as the base Airflow images do not include the default ubuntu package repositories. As such, the openssh-server package could not be found. Additionally, the ssh_host_keys generated by sshd are owned by root. This causes permission related failures since the container runs as user airflow.

Also add these arguments to `start` and `restart` commands

viktorl-dev

I've added a few more comments. A quick summary of next steps:

Fixing these issues may cause some existing tests to break, so please verify them.
Do you think we should add additional test cases for the new features?

viktorl-dev · 2026-06-10T11:10:09Z

            # By default, the container runs as the user `airflow` with UID 999. Set
            # this env variable to "True" to make it run as the current host user.
            "COMPOSER_CONTAINER_RUN_AS_HOST_USER": "False",
+            "COMPOSER_CONTAINER_ENABLE_SSHD": str(enable_ssh),


I'd suggest making the naming consistent: if we add an option named --enable-ssh, the corresponding environment variable should also be named COMPOSER_CONTAINER_ENABLE_SSH

viktorl-dev · 2026-06-10T11:36:52Z

+        if "enable_ssh" in self.config:
+            self.enable_ssh = (
+                enable_ssh
+                if enable_ssh is not None
+                else self.get_str_param("enable_ssh")
+            )
+            self.ssh_port = (
+                ssh_port
+                if ssh_port is not None
+                else self.parse_int_param("ssh_port", allowed_range=(0, 65536))
+            )
+        else:
+            self.enable_ssh = False
+            self.ssh_port = None


This looks a bit convoluted. I suggest refactoring it as follows:

self.enable_ssh = self.config.get("enable_ssh", constants.DEFAULT_ENABLE_SSH) self.ssh_port = self.config.get("ssh_port", constants.DEFAULT_SSH_PORT) if enable_ssh is not None: self.enable_ssh = enable_ssh if ssh_port is not None: self.ssh_port = ssh_port

So you'd also need to add DEFAULT_ENABLE_SSH = False to constants.py

I just realized I missed this part in my suggestion:

self.parse_int_param("ssh_port", allowed_range=(0, 65536))

Could you please incorporate this.

P.S.: though it should be 65535 and not 65536. Yeah, I can see there are many places with the same wrong number. We'll handle this later on.

viktorl-dev · 2026-06-10T11:50:56Z

            self.database_engine == constants.DatabaseEngine.sqlite3
        )
        self.port: int = port if port is not None else 8080
+        self.enable_ssh: bool = enable_ssh


I'd suggest setting it to the default value as well:

Suggested change

self.enable_ssh: bool = enable_ssh

self.enable_ssh: bool = enable_ssh if enable_ssh is not None else constants.DEFAULT_ENABLE_SSH

morgan-dgk · 2026-06-16T05:22:09Z

@viktorl-dev I've made the required changes and confirmed that the existing tests continue to pass.

morgan-dgk · 2026-06-16T23:54:57Z

I see periodic failures starting the database container:

Failed to log action (psycopg2.OperationalError) could not translate 
host name "composer-local-dev-db-testenv" to address: Name or service not known

Unsure why this is occuring.

viktorl-dev · 2026-06-17T10:39:25Z

    sudo chown -R airflow:airflow "$FAST_API_DIR"
  fi

+  if [ "${COMPOSER_CONTAINER_ENABLE_SSHD}" = "True" ]; then


Here, a variable hasn't been renamed. What is more interesting is that all tests are green even with this "not yet renamed" variable. Could you please add a test case that specifically catches this issue?

Bump requirements in e2e test

c29a1ac

Change-Id: Id1ab768dfaabb7868713ac238c8840406e2c98c0 GitOrigin-RevId: daf3412c40c0d2509e543470f02608b6c4761fe3

morgan-dgk changed the title ~~Feat/add python remote debug support~~ feat: add python remote debug support May 28, 2026

morgan-dgk force-pushed the feat/add-python-remote-debug-support branch 2 times, most recently from aa5a330 to d73ddc9 Compare May 28, 2026 03:31

morgan-dgk marked this pull request as draft May 28, 2026 06:45

morgan-dgk marked this pull request as ready for review May 28, 2026 07:16

morgan-dgk force-pushed the feat/add-python-remote-debug-support branch from f6444d5 to b948b44 Compare June 2, 2026 06:14

viktorl-dev suggested changes Jun 3, 2026

View reviewed changes

morgan-dgk force-pushed the feat/add-python-remote-debug-support branch from b907555 to 408d395 Compare June 4, 2026 01:04

strobil and others added 10 commits June 4, 2026 11:05

WIP

cdfb2ba

added Remote debugging section to README

f684578

fix: check if sshd is already installed in container

803c5b8

Remove redundant f-string

4a132ee

Improve logging in docker entrypoint

9f2c450

Add constant for default ssh port

5cbda16

docs: add description for --ssh-port option

320d77b

fix: prevent failures when ssh opts not in config.json

6d33fc5

misc: pass enable_ssh to EnvironmentConfig

a2ea548

Also add these arguments to `start` and `restart` commands

morgan-dgk force-pushed the feat/add-python-remote-debug-support branch from 408d395 to a2ea548 Compare June 4, 2026 01:05

morgan-dgk requested a review from viktorl-dev June 4, 2026 23:13

ahidalgob requested review from ahidalgob and removed request for viktorl-dev June 9, 2026 09:49

viktorl-dev suggested changes Jun 10, 2026

View reviewed changes

fixup! Add constant for default ssh port

2cdd76d

Add e2e test

0fa9fc4

viktorl-dev suggested changes Jun 18, 2026

View reviewed changes

	self.enable_ssh: bool = enable_ssh
	self.enable_ssh: bool = enable_ssh if enable_ssh is not None else constants.DEFAULT_ENABLE_SSH

		@option_enable_ssh
		@option_ssh_port

Uh oh!

Conversation

morgan-dgk commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

morgan-dgk commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

viktorl-dev commented Jun 3, 2026

Uh oh!

viktorl-dev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

morgan-dgk commented Jun 4, 2026

Uh oh!

viktorl-dev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

morgan-dgk commented Jun 16, 2026

Uh oh!

morgan-dgk commented Jun 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

morgan-dgk commented May 28, 2026 •

edited

Loading

morgan-dgk commented May 28, 2026 •

edited

Loading