vdi module - idempotence update

Author: scott-nag

community/modules/scripts/vdi-setup/README.md

@@ -1,15 +1,16 @@
 ## Description
 
-Creates a containerised Guacamole instance. Works with the Rocky, Debian and Ubuntu images shown in blueprint example below.
+Creates a containerised Guacamole instance. Currently designed to work mainly with the Rocky image in the blueprint example below. Ubuntu/Debian support previously worked too but fixes may be required due to updates.
 
 ## Features
 
 - **VDI Tool Support**: Currently supports Guacamole for web-based VDI access
-- **User Provisioning**: Supports local user creation with secure password management
-- **VNC Integration**: Configures VNC servers for desktop access
+- **User Provisioning**: Supports local user creation with secure password management and group management
+- **VNC Integration**: Configures VNC servers for desktop access with port change support
 - **Secret Manager Integration**: Secure password storage and retrieval for VDI users and webapp admin
-- **VDI Monitoring System**: Automatic monitoring and reconfiguration capabilities with password reset flags and deployment status updates
+- **VDI Monitoring System**: Automatic monitoring and reconfiguration capabilities with password reset flags and deployment status updates. [More info](roles/vdi_monitor/README.md).
 - **Debug Mode**: Comprehensive logging when enabled via the `debug` variable
+- **Port Change Support**: Handles both webapp port changes (container recreation) and user VNC port changes (SQL updates)
 
 ## Secret Manager Integration
 
@@ -21,11 +22,31 @@ The module integrates with Google Cloud Secret Manager for secure password handl
 - **Reset Functionality**: Both individual user passwords (`reset_password`) and webapp admin password (`reset_webapp_admin_password`) can be forced to regenerate
 - **Database Updates**: Password resets update existing database records without re-initializing the entire database
 
+## Port Change and User Group Management
+
+The module supports dynamic configuration changes without full redeployment:
+
+### **Webapp Port Changes**
+- **Container Recreation**: When `vdi_webapp_port` changes, the Guacamole webapp container is recreated with the new port
+- **Database Synchronization**: Database passwords are automatically synchronized during port changes
+- **Service Continuity**: VNC services remain unaffected during webapp port changes
+
+### **User VNC Port Changes**
+- **SQL Updates**: User VNC port changes are handled via database updates without container recreation
+- **VNC Service Management**: VNC services are automatically stopped and restarted with new ports
+- **X Server Cleanup**: Old X server processes are properly terminated before starting new ones
+
+### **User Group Management**
+- **Dynamic Group Changes**: When `vdi_user_group` changes, users are automatically migrated to the new group
+- **Group Removal**: Users are removed from the old group and added to the new group
+- **Idempotent Operations**: Group changes are safe to apply multiple times
+
 ## VDI Monitoring System
 
 The module includes a monitoring system that:
 
-- **Deployment Status**: Updates instance metadata to reflect deployment state (`available`, `reconfiguring`)
+- **Deployment Status**: Updates instance metadata to reflect deployment state (`available`, `reconfiguring`, `failed`)
+- **Targeted Updates**: Performs database-only updates for password resets and user changes without container recreation
 - **Password Reset Flags**: Supports `reset_password` for individual users and `reset_webapp_admin_password` for the webapp admin account
 - **Automatic Reconfiguration**: Detects changes and triggers reconfiguration when needed
 - **Status Tracking**: Maintains deployment status through instance metadata
@@ -101,14 +122,13 @@ deployment_groups:
     source: modules/compute/vm-instance
     settings:
       instance_image:
-        # Several supported image families:
         family: hpc-rocky-linux-8
         project: cloud-hpc-image-public
         #family: debian-11
         #project: debian-cloud
         #family: ubuntu-2204-lts
         #project: ubuntu-os-cloud
-        name_prefix: debian
+        name_prefix: guacamole
         add_deployment_name_before_prefix: true
       machine_type: e2-highcpu-8
       tags: ["guacamole"]
@@ -156,3 +176,70 @@ After deployment, you can access the VDI in several ways:
    ```
 
    - The web interface will then be accessible from http://localhost:8080/guacamole/ (for Guacamole)
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
+| <a name="requirement_archive"></a> [archive](#requirement\_archive) | ~> 2.0 |
+| <a name="requirement_google"></a> [google](#requirement\_google) | >= 3.83 |
+| <a name="requirement_random"></a> [random](#requirement\_random) | ~> 3.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_archive"></a> [archive](#provider\_archive) | ~> 2.0 |
+| <a name="provider_google"></a> [google](#provider\_google) | >= 3.83 |
+| <a name="provider_random"></a> [random](#provider\_random) | ~> 3.0 |
+| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_startup_script"></a> [startup\_script](#module\_startup\_script) | ../../../../modules/scripts/startup-script | n/a |
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [google_storage_bucket.bucket](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket) | resource |
+| [random_id.resource_name_suffix](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource |
+| [terraform_data.input_validation](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
+| [archive_file.roles_tar](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_debug"></a> [debug](#input\_debug) | Enable debug mode for verbose logging during VDI setup. | `bool` | `false` | no |
+| <a name="input_deployment_name"></a> [deployment\_name](#input\_deployment\_name) | The name of the deployment. | `string` | n/a | yes |
+| <a name="input_force_rerun"></a> [force\_rerun](#input\_force\_rerun) | Force complete container recreation and database re-initialization, bypassing all idempotency checks. Use only when troubleshooting or when the system is in a broken state. | `bool` | `false` | no |
+| <a name="input_labels"></a> [labels](#input\_labels) | Key-value pairs of labels to be added to created resources. | `map(string)` | n/a | yes |
+| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | Project in which the HPC deployment will be created. | `string` | n/a | yes |
+| <a name="input_region"></a> [region](#input\_region) | Region to place bucket containing startup script. | `string` | n/a | yes |
+| <a name="input_reset_webapp_admin_password"></a> [reset\_webapp\_admin\_password](#input\_reset\_webapp\_admin\_password) | Force reset of the webapp admin password during reconfiguration. If true, a new password will be generated and stored in Secret Manager, even if an existing password exists. | `bool` | `false` | no |
+| <a name="input_user_provision"></a> [user\_provision](#input\_user\_provision) | User type to create (local\_users supported. os-login to do. | `string` | `"local_users"` | no |
+| <a name="input_vdi_resolution"></a> [vdi\_resolution](#input\_vdi\_resolution) | Desktop resolution for VNC sessions (e.g. 1920x1080). | `string` | `"1920x1080"` | no |
+| <a name="input_vdi_tool"></a> [vdi\_tool](#input\_vdi\_tool) | VDI tool to deploy (guacamole currently supported). | `string` | `"guacamole"` | no |
+| <a name="input_vdi_user_group"></a> [vdi\_user\_group](#input\_vdi\_user\_group) | Unix group to create/use for VDI users. | `string` | `"vdiusers"` | no |
+| <a name="input_vdi_users"></a> [vdi\_users](#input\_vdi\_users) | List of VDI users to configure. Passwords are handled securely by the Ansible roles: if secret\_name is provided, the password is fetched from Secret Manager; if neither password nor secret\_name is provided, a random password is generated and stored in Secret Manager. If secret\_project is provided, it specifies the GCP project where the secret is stored (defaults to the deployment project). Set reset\_password to true to trigger password regeneration for auto-generated passwords. | <pre>list(object({<br/>    username       = string<br/>    port           = number<br/>    secret_name    = optional(string)<br/>    secret_project = optional(string)<br/>    reset_password = optional(bool)<br/>  }))</pre> | `[]` | no |
+| <a name="input_vdi_webapp_port"></a> [vdi\_webapp\_port](#input\_vdi\_webapp\_port) | Port to serve the Webapp interface from if applicable (note: containers will be recreated if changed) | `string` | `"8080"` | no |
+| <a name="input_vnc_flavor"></a> [vnc\_flavor](#input\_vnc\_flavor) | The VNC server flavor to use (tigervnc currently supported) | `string` | `"tigervnc"` | no |
+| <a name="input_vnc_port_max"></a> [vnc\_port\_max](#input\_vnc\_port\_max) | Maximum valid VNC port. | `number` | `5999` | no |
+| <a name="input_vnc_port_min"></a> [vnc\_port\_min](#input\_vnc\_port\_min) | Minimum valid VNC port. | `number` | `5901` | no |
+| <a name="input_zone"></a> [zone](#input\_zone) | Zone in which the VDI instances are created. | `string` | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_guacamole_admin_password_secret"></a> [guacamole\_admin\_password\_secret](#output\_guacamole\_admin\_password\_secret) | The name of the Secret Manager secret containing the Guacamole admin password |
+| <a name="output_guacamole_admin_username"></a> [guacamole\_admin\_username](#output\_guacamole\_admin\_username) | The admin username for Guacamole |
+| <a name="output_startup_script"></a> [startup\_script](#output\_startup\_script) | Combined startup script that installs VDI (VNC, Guacamole, users). |
+| <a name="output_vdi_runner"></a> [vdi\_runner](#output\_vdi\_runner) | Shell runner wrapping Ansible playbook + roles (for custom-image or direct use). |
+| <a name="output_vdi_user_credentials"></a> [vdi\_user\_credentials](#output\_vdi\_user\_credentials) | Map of VDI user credentials stored in Secret Manager |
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

community/modules/scripts/vdi-setup/main.tf

@@ -99,7 +99,7 @@ locals {
         }
       )
       destination = "/opt/vdi-setup/install.yaml"
-      args = var.debug ? "--extra-vars @/opt/vdi-setup/vars.yaml -v --extra-vars debug=true" : "--extra-vars @/opt/vdi-setup/vars.yaml"
+      args        = var.debug ? "--extra-vars @/opt/vdi-setup/vars.yaml -v --extra-vars debug=true" : "--extra-vars @/opt/vdi-setup/vars.yaml"
     },
     # Clean up temporary directory
     {

community/modules/scripts/vdi-setup/roles/base_os/tasks/main.yaml

@@ -26,15 +26,30 @@
   ansible.builtin.import_role:
     name: lock_manager
     tasks_from: check_lock
-  register: lock_check_result
 
 - name: DEBUG Finished import_role for base_os
   debug:
     msg: "Finished import_role for base_os"
 
-- name: Set role_should_run from lock check result
+- name: Set role variables from lock manager facts
   ansible.builtin.set_fact:
-    role_should_run: "{{ lock_check_result.ansible_facts.role_should_run | default(true) }}"
+    role_should_run: "{{ role_should_run | default(false) }}"
+    is_fresh_deployment: "{{ is_fresh_deployment | default(false) }}"
+    deployment_hash: "{{ current_deployment_hash | default('none') }}"
+    user_hash: "{{ current_user_secrets_hash | default('none') }}"
+    lock_file_stat: "{{ lock_file_stat | default({}) }}"
+    vdi_setup_status: "{{ vdi_setup_status | default({}) }}"
+
+- name: Debug role variables
+  ansible.builtin.debug:
+    msg: |
+      Base OS role variables:
+        - role_should_run: {{ role_should_run }}
+        - is_fresh_deployment: {{ is_fresh_deployment }}
+        - deployment_hash: {{ deployment_hash }}
+        - user_hash: {{ user_hash }}
+        - execution_reason: {{ 'Fresh deployment' if is_fresh_deployment else ('Hash mismatch' if not role_should_run else 'Normal role execution') }}
+  when: debug
 
 # Skip all tasks if role should not run
 - name: Skip base_os tasks if role should not run
@@ -57,21 +72,21 @@
 # Include distribution-specific tasks
 - name: Include Ubuntu-specific tasks
   ansible.builtin.include_tasks: ubuntu.yaml
-  when: 
-    - ansible_facts.distribution == 'Ubuntu'
-    - role_should_run
+  when:
+  - ansible_facts.distribution == 'Ubuntu'
+  - role_should_run
 
 - name: Include Rocky Linux specific tasks
   ansible.builtin.include_tasks: rocky.yaml
-  when: 
-    - ansible_facts.distribution == 'Rocky'
-    - role_should_run
+  when:
+  - ansible_facts.distribution == 'Rocky'
+  - role_should_run
 
 - name: Include Debian-specific tasks
   ansible.builtin.include_tasks: debian.yaml
-  when: 
-    - ansible_facts.distribution == 'Debian'
-    - role_should_run
+  when:
+  - ansible_facts.distribution == 'Debian'
+  - role_should_run
 
 # Common tasks for all distributions
 - name: Ensure requests package is installed
@@ -92,9 +107,9 @@
 # GPU / vWS driver setup
 - name: Provision GPU setup
   ansible.builtin.include_tasks: gpu_driver_setup.yaml
-  when: 
-    - nvidia_gpu_present.rc == 0
-    - role_should_run
+  when:
+  - nvidia_gpu_present.rc == 0
+  - role_should_run
 
 # Mark role as completed
 - name: Set current role for lock manager completion

community/modules/scripts/vdi-setup/roles/lock_manager/README.md

@@ -0,0 +1,150 @@
+# Lock Manager Role
+
+The `lock_manager` role is the central coordination mechanism for the VDI setup module. It manages deployment state, change detection, and ensures idempotent role execution.
+
+## Overview
+
+The lock manager uses a hash-based change detection system to determine when roles should run, preventing unnecessary re-execution while ensuring changes are properly applied.
+
+## Key Features
+
+### 1. **Hash-Based Change Detection**
+
+- **Deployment Hash**: Tracks changes to deployment configuration (ports, tools, etc.)
+- **User Hash**: Tracks changes to user configurations (additions, removals, password resets)
+- **Smart Detection**: Only re-runs roles when relevant changes are detected
+
+### 2. **Centralized Variable Management**
+
+- **User Management Variables**: Calculates and propagates user change information to all roles
+- **Port Change Detection**: Detects and tracks VNC port changes for existing users
+- **Password Reset Tracking**: Identifies users requiring password resets
+
+### 3. **Targeted Updates**
+
+- **Database-Only Updates**: Performs user additions/deletions/password resets without container recreation
+- **Port Change Handling**: Updates VNC ports and Guacamole connections without full re-initialization
+- **Webapp Admin Password Reset**: Updates admin password without container recreation
+
+### 4. **Deployment Status Management**
+
+- **Lock File**: `.vdi-lock.yaml` tracks deployment state and completed roles
+- **Status Tracking**: `available`, `configuring`, `reconfiguring`, `failed` states
+- **Role Completion**: Tracks which roles have completed successfully
+
+## Variable Propagation
+
+### **Variables Calculated by Lock Manager:**
+
+| Variable | Description | Usage |
+|----------|-------------|-------|
+| `new_users` | Users added since last deployment | `vdi_tool`, `user_provision`, `vnc` |
+| `removed_users` | Users removed since last deployment | `vdi_tool`, `user_provision`, `vnc` |
+| `users_needing_reset` | Users with `reset_password: true` | `vdi_tool`, `secret_manager` |
+| `users_with_port_changes` | Users whose VNC ports changed | `vdi_tool`, `vnc` |
+| `webapp_port_changed` | Whether webapp port changed | `vdi_tool` |
+| `users_changed` | Any user-related changes detected | `vdi_tool` |
+| `db_needs_init` | Whether database needs re-initialization | `vdi_tool` |
+| `vdi_user_group` | Current VDI user group for group management | `user_provision` |
+
+### **Variable Propagation Flow:**
+
+1. **Lock Manager Calculation**: Variables calculated once during initial `lock_manager` run
+2. **Role Import**: Other roles import `lock_manager` and receive these variables
+3. **Defensive Programming**: Variables preserved with `default()` filters, not overwritten
+4. **Role Usage**: Each role uses only the variables it needs
+
+## Data Flows
+
+### **Initial Deployment:**
+
+```mermaid
+1. lock_manager → Calculates all variables (empty for fresh deployment)
+2. base_os → OS setup
+3. secret_manager → Password generation and Secret Manager setup
+4. user_provision → Local user creation
+5. vnc → VNC service setup
+6. vdi_tool → Guacamole deployment
+7. vdi_monitor → Monitoring service setup
+```
+
+### **Reconfiguration (User Changes):**
+
+```mermaid
+1. lock_manager → Detects changes, calculates user management variables
+2. user_provision → Handles user additions/removals
+3. vnc → Handles VNC port changes and user removal
+4. vdi_tool → Handles database updates (targeted, no container recreation)
+5. vdi_monitor → No changes needed
+```
+
+### **Reconfiguration (Port Changes):**
+
+```mermaid
+1. lock_manager → Detects port changes
+2. vnc → Stops VNC services for users with port changes
+3. vdi_tool → Updates Guacamole connection ports in database
+4. user_provision → Updates local user configurations
+```
+
+## Lock File Structure
+
+```yaml
+vdi_setup_status:
+  completed_roles:
+    base_os: true
+    secret_manager: true
+    user_provision: true
+    vdi_monitor: true
+    vdi_tool: true
+    vnc: true
+  created_at: '2025-08-07T16:20:26Z'
+  current_users: ['alice', 'bob', 'charlie', 'david']
+  deployment_hash: 'abc123...'
+  deployment_name: 'vdi-test-scott'
+  force_rerun: false
+  last_updated: '2025-08-07T16:28:42Z'
+  lock_version: '1.0'
+  setup_status: 'available'
+  user_hash: 'def456...'
+  user_ports:
+    alice: 5901
+    bob: 5902
+    charlie: 5903
+    david: 5904
+  webapp_port: 8081
+  vdi_user_group: vdiusers
+```
+
+## Role Execution Logic
+
+### **When Roles Run:**
+- **Fresh Deployment**: All roles run regardless of hash
+- **Hash Mismatch**: Only roles with changed hashes run
+- **Force Rerun**: All roles run when `force_rerun: true`
+
+### **Role Completion:**
+- Roles mark themselves as completed in lock file
+- Lock file tracks completion status per role
+- Deployment status updated based on all role completions
+
+## Error Handling
+
+### **Failed Deployments:**
+- VDI monitor sets status to `"failed"` when Ansible exits with non-zero code
+- Lock file preserves failed state for debugging
+- Valid blueprint can be re-applied to fix `failed` deployments
+
+### **Variable Safety:**
+- Defensive programming ensures all variables have safe defaults
+- `default()` filters prevent undefined variable errors
+- Variables preserved across role imports
+
+## Debug Output
+
+When `debug: true` is set, the lock manager provides detailed information about:
+- Variable calculations and propagation
+- Change detection results
+- Hash comparisons
+- Role execution decisions
+- Port change detection details