[GCP Compute Engine] Update image build

Author: JosepSampe

config/README.md

@@ -69,6 +69,7 @@ Storage Backends
 - [IBM Virtual Private Cloud](../docs/source/compute_config/ibm_vpc.md)
 - [AWS Elastic Compute Cloud (EC2)](../docs/source/compute_config/aws_ec2.md)
 - [Azure Virtual Machines](../docs/source/compute_config/azure_vms.md)
+- [Google Compute Engine](../docs/source/compute_config/gcp_compute_engine.md)
 
 </td>
 <td valign="top">

config/config_template.yaml

@@ -137,6 +137,7 @@
 # Google Cloud Platform – credentials shared by all gcp_* sections
 # =============================================================================
 #gcp:
+    #project_name: <GCP_PROJECT_ID>
     #region: <REGION_NAME>                # e.g. us-east1
     #credentials_path: <ABSOLUTE_PATH_TO_SERVICE_ACCOUNT_JSON>
 
@@ -172,6 +173,26 @@
     #region: <REGION_NAME>                # Falls back to gcp.region
     #storage_bucket: <BUCKET_NAME>        # Auto-created if not provided
 
+# Google Compute Engine – standalone compute
+#gcp_compute_engine:
+    #project_name: <GCP_PROJECT_ID>       # Falls back to gcp.project_name
+    #zone: <ZONE_NAME>                    # e.g. us-east1-b. Mandatory
+    #region: <REGION_NAME>                # Falls back to gcp.region; derived from zone if omitted
+    #service_account: <SERVICE_ACCOUNT_EMAIL>
+    #network_name: <VPC_NETWORK_NAME>     # Lithops creates a VPC if not provided
+    #subnet_name: <SUBNET_NAME>
+    #source_image: projects/ubuntu-os-cloud/global/images/family/ubuntu-2404-lts-amd64
+    #instance_name: <INSTANCE_NAME>       # Mandatory in consume mode
+    #master_instance_type: e2-small
+    #worker_instance_type: e2-standard-2
+    #ssh_username: ubuntu
+    #ssh_password: <RANDOM_IF_NOT_PROVIDED>
+    #ssh_key_filename: ~/.ssh/id_rsa
+    #request_spot_instances: False
+    #delete_on_dismantle: True
+    #max_workers: 100
+    #worker_processes: AUTO               # Default: number of CPUs of the worker VM
+
 
 # =============================================================================
 # Microsoft Azure – credentials shared by all azure_* sections
@@ -460,7 +481,7 @@
 
 
 # =============================================================================
-# Standalone shared settings (applies to ibm_vpc, aws_ec2, azure_vms, vm)
+# Standalone shared settings (applies to ibm_vpc, aws_ec2, azure_vms, gcp_compute_engine, vm)
 # =============================================================================
 #standalone:
     #runtime: python3

docs/source/compute_backends.rst

@@ -44,4 +44,4 @@ Compute Backends
    compute_config/ibm_vpc.md
    compute_config/aws_ec2.md
    compute_config/azure_vms.md
-   compute_config/gcp_compute_engie.md
+   compute_config/gcp_compute_engine.md

…urce/compute_config/gcp_compute_engie.md …rce/compute_config/gcp_compute_engine.mddocs/source/compute_config/gcp_compute_engie.md renamed to docs/source/compute_config/gcp_compute_engine.md docs/source/compute_config/gcp_compute_engie.md renamed to docs/source/compute_config/gcp_compute_engine.md

@@ -2,19 +2,27 @@
 
 The GCP Compute Engine backend of Lithops can provide a serverless user experience on top of GCE where Lithops creates new Virtual Machines (VMs) dynamically at runtime and scales Lithops jobs against them (create and reuse modes). Alternatively Lithops can start and stop an existing VM instance (consume mode).
 
-The backend key is `gcp_compute_engie` (matches the Lithops module name).
+The backend key is `gcp_compute_engine` (matches the Lithops module name).
 
 ## Choose an operating system image for the VM
 
-Any VM needs an operating system image. By default Lithops uses Ubuntu 24.04 (`ubuntu-2404-lts-amd64`). Lithops installs required dependencies on the VM on first use (this can take a few minutes).
+Any VM needs an operating system image. By default Lithops uses Ubuntu 24.04 (`projects/ubuntu-os-cloud/global/images/family/ubuntu-2404-lts-amd64`). Lithops installs required dependencies on the VM on first use (this can take a few minutes).
+
+For faster startups, build a pre-configured custom image (see [runtime/gcp_compute_engine](https://github.com/lithops-cloud/lithops/tree/master/runtime/gcp_compute_engine)):
+
+```bash
+lithops image build -b gcp_compute_engine
+```
+
+This creates `lithops-ubuntu-2404-lts-amd64-server` in your project; Lithops uses it automatically when present.
 
 To list available images:
 
 ```bash
-lithops image list -b gcp_compute_engie
+lithops image list -b gcp_compute_engine
 ```
 
-Use the **Image ID** column as `source_image` in your config.
+Use the **Image ID** column as `source_image` in your config when using a custom image name.
 
 ## Installation
 
@@ -54,12 +62,12 @@ gcloud projects add-iam-policy-binding <PROJECT_ID> \
 
 ```yaml
 lithops:
-  backend: gcp_compute_engie
+  backend: gcp_compute_engine
 
 gcp:
   credentials_path: <FULL_PATH_TO_SERVICE_ACCOUNT_JSON>
 
-gcp_compute_engie:
+gcp_compute_engine:
   project_name: <GCP_PROJECT_ID>
   zone: <ZONE>
   exec_mode: reuse
@@ -78,29 +86,29 @@ Lithops attaches the service account from `credentials_path` to master and worke
 
 |Group|Key|Default|Mandatory|Additional info|
 |---|---|---|---|---|
-|gcp_compute_engie | project_name | |yes | GCP project ID |
-|gcp_compute_engie | zone | |yes | Compute Engine zone, for example `us-east1-b` |
-|gcp_compute_engie | region | derived from zone |no | Region used for subnet and NAT |
-|gcp_compute_engie | service_account | |no | Service account email attached to VMs. Default: `client_email` from `credentials_path` |
-|gcp_compute_engie | network_name | |no | Existing VPC name. If not provided, Lithops creates a new network |
-|gcp_compute_engie | subnet_name | |no | Existing subnet name when using a custom VPC |
-|gcp_compute_engie | source_image | ubuntu-2404-lts-amd64 |no | Boot image reference |
-|gcp_compute_engie | master_instance_type | e2-small |no | Master VM machine type |
-|gcp_compute_engie | worker_instance_type | e2-standard-2 |no | Worker VM machine type |
-|gcp_compute_engie | ssh_username | ubuntu |no | Username to access the VM |
-|gcp_compute_engie | ssh_password | |no | Password for worker VMs. If not provided, it is created randomly |
-|gcp_compute_engie | ssh_key_filename | ~/.ssh/id_rsa |no | SSH private key for the master VM. If not provided, Lithops creates one |
-|gcp_compute_engie | request_spot_instances | False |no | Use Spot VMs for workers |
-|gcp_compute_engie | delete_on_dismantle | True |no | Delete worker VMs when stopped. Master VM is never deleted when stopped |
-|gcp_compute_engie | max_workers | 100 |no | Max number of workers per `FunctionExecutor()` |
-|gcp_compute_engie | worker_processes | AUTO |no | Parallel Lithops processes per worker. Default: CPUs of `worker_instance_type` |
-|gcp_compute_engie | runtime | python3 |no | Runtime name. Default: python3 on the VM |
-|gcp_compute_engie | auto_dismantle | True |no | If False, VMs are not stopped automatically |
-|gcp_compute_engie | soft_dismantle_timeout | 300 |no | Seconds to stop the VM after a job **completed** |
-|gcp_compute_engie | hard_dismantle_timeout | 3600 |no | Seconds to stop the VM after a job **started** |
-|gcp_compute_engie | exec_mode | reuse |no | One of: **consume**, **create** or **reuse** |
-|gcp_compute_engie | extra_apt_packages | [] |no | Extra apt packages on master/worker VMs during setup |
-|gcp_compute_engie | extra_python_packages | [] |no | Extra pip packages on master/worker VMs after Lithops |
+|gcp_compute_engine | project_name | |yes | GCP project ID |
+|gcp_compute_engine | zone | |yes | Compute Engine zone, for example `us-east1-b` |
+|gcp_compute_engine | region | derived from zone |no | Region used for subnet and NAT |
+|gcp_compute_engine | service_account | |no | Service account email attached to VMs. Default: `client_email` from `credentials_path` |
+|gcp_compute_engine | network_name | |no | Existing VPC name. If not provided, Lithops creates a new network |
+|gcp_compute_engine | subnet_name | |no | Existing subnet name when using a custom VPC |
+|gcp_compute_engine | source_image | ubuntu-2404-lts-amd64 |no | Boot image reference |
+|gcp_compute_engine | master_instance_type | e2-small |no | Master VM machine type |
+|gcp_compute_engine | worker_instance_type | e2-standard-2 |no | Worker VM machine type |
+|gcp_compute_engine | ssh_username | ubuntu |no | Username to access the VM |
+|gcp_compute_engine | ssh_password | |no | Password for worker VMs. If not provided, it is created randomly |
+|gcp_compute_engine | ssh_key_filename | ~/.ssh/id_rsa |no | SSH private key for the master VM. If not provided, Lithops creates one |
+|gcp_compute_engine | request_spot_instances | False |no | Use Spot VMs for workers |
+|gcp_compute_engine | delete_on_dismantle | True |no | Delete worker VMs when stopped. Master VM is never deleted when stopped |
+|gcp_compute_engine | max_workers | 100 |no | Max number of workers per `FunctionExecutor()` |
+|gcp_compute_engine | worker_processes | AUTO |no | Parallel Lithops processes per worker. Default: CPUs of `worker_instance_type` |
+|gcp_compute_engine | runtime | python3 |no | Runtime name. Default: python3 on the VM |
+|gcp_compute_engine | auto_dismantle | True |no | If False, VMs are not stopped automatically |
+|gcp_compute_engine | soft_dismantle_timeout | 300 |no | Seconds to stop the VM after a job **completed** |
+|gcp_compute_engine | hard_dismantle_timeout | 3600 |no | Seconds to stop the VM after a job **started** |
+|gcp_compute_engine | exec_mode | reuse |no | One of: **consume**, **create** or **reuse** |
+|gcp_compute_engine | extra_apt_packages | [] |no | Extra apt packages on master/worker VMs during setup |
+|gcp_compute_engine | extra_python_packages | [] |no | Extra pip packages on master/worker VMs after Lithops |
 
 ## Consume mode
 
@@ -110,12 +118,12 @@ In this mode, Lithops uses an existing VM. The VM must be reachable by SSH and h
 
 ```yaml
 lithops:
-  backend: gcp_compute_engie
+  backend: gcp_compute_engine
 
 gcp:
   credentials_path: <FULL_PATH_TO_SERVICE_ACCOUNT_JSON>
 
-gcp_compute_engie:
+gcp_compute_engine:
   exec_mode: consume
   project_name: <GCP_PROJECT_ID>
   zone: <ZONE>
@@ -126,19 +134,19 @@ gcp_compute_engie:
 
 |Group|Key|Default|Mandatory|Additional info|
 |---|---|---|---|---|
-|gcp_compute_engie | instance_name | |yes | Existing VM instance name |
-|gcp_compute_engie | project_name | |yes | GCP project ID |
-|gcp_compute_engie | zone | |yes | Compute Engine zone |
-|gcp_compute_engie | ssh_username | ubuntu |no | Username to access the VM |
-|gcp_compute_engie | ssh_key_filename | ~/.ssh/id_rsa |no | Path to the SSH private key |
-|gcp_compute_engie | worker_processes | AUTO |no | Parallel Lithops processes per worker |
+|gcp_compute_engine | instance_name | |yes | Existing VM instance name |
+|gcp_compute_engine | project_name | |yes | GCP project ID |
+|gcp_compute_engine | zone | |yes | Compute Engine zone |
+|gcp_compute_engine | ssh_username | ubuntu |no | Username to access the VM |
+|gcp_compute_engine | ssh_key_filename | ~/.ssh/id_rsa |no | Path to the SSH private key |
+|gcp_compute_engine | worker_processes | AUTO |no | Parallel Lithops processes per worker |
 
 ## Test Lithops
 
 Once you have your compute and storage backends configured, you can run a hello world function with:
 
 ```bash
-lithops hello -b gcp_compute_engie -s gcp_storage
+lithops hello -b gcp_compute_engine -s gcp_storage
 ```
 
 ## Viewing the execution logs
@@ -158,33 +166,33 @@ All VMs, including the master, are automatically stopped after a configurable ti
 You can open an SSH session to the master VM with:
 
 ```bash
-lithops attach -b gcp_compute_engie
+lithops attach -b gcp_compute_engine
 ```
 
 The master and worker VMs store Lithops service logs in `/tmp/lithops-root/*-service.log`.
 
 To list available workers:
 
 ```bash
-lithops worker list -b gcp_compute_engie
+lithops worker list -b gcp_compute_engine
 ```
 
 To list submitted jobs:
 
 ```bash
-lithops job list -b gcp_compute_engie
+lithops job list -b gcp_compute_engine
 ```
 
 To delete workers only:
 
 ```bash
-lithops clean -b gcp_compute_engie -s gcp_storage
+lithops clean -b gcp_compute_engine -s gcp_storage
 ```
 
 To delete workers, the master VM, and Lithops-created network resources:
 
 ```bash
-lithops clean -b gcp_compute_engie -s gcp_storage --all
+lithops clean -b gcp_compute_engine -s gcp_storage --all
 ```
 
 ## Architecture diagram

docs/source/configuration.rst

@@ -37,7 +37,7 @@ Choose your compute and storage engines from the table below:
 || `IBM Virtual Private Cloud <compute_config/ibm_vpc.html>`_        ||                                                                   |
 || `AWS Elastic Compute Cloud (EC2) <compute_config/aws_ec2.html>`_  ||                                                                   |
 || `Azure Virtual Machines <compute_config/azure_vms.html>`_         ||                                                                   |
-|| `Google Compute Engine <compute_config/gcp_compute_engie.html>`_  ||                                                                   |
+|| `Google Compute Engine <compute_config/gcp_compute_engine.html>`_  ||                                                                   |
 +--------------------------------------------------------------------+--------------------------------------------------------------------+
 
 Configuration File

docs/source/execution_modes.rst

@@ -83,4 +83,4 @@ underlying infrastructure.
 
     fexec = lithops.StandaloneExecutor()
 
-- Available backends: `IBM Virtual Private Cloud <compute_config/ibm_vpc.html>`_, `AWS Elastic Compute Cloud (EC2) <compute_config/aws_ec2.html>`_, `Azure Virtual Machines <compute_config/azure_vms.html>`_, `Google Compute Engine <compute_config/gcp_compute_engie.html>`_, `Virtual Machine <compute_config/vm.html>`_
+- Available backends: `IBM Virtual Private Cloud <compute_config/ibm_vpc.html>`_, `AWS Elastic Compute Cloud (EC2) <compute_config/aws_ec2.html>`_, `Azure Virtual Machines <compute_config/azure_vms.html>`_, `Google Compute Engine <compute_config/gcp_compute_engine.html>`_, `Virtual Machine <compute_config/vm.html>`_

lithops/constants.py

@@ -120,6 +120,6 @@
     'ibm_vpc',
     'aws_ec2',
     'azure_vms',
-    'gcp_compute_engie',
+    'gcp_compute_engine',
     'vm'
 ]

lithops/standalone/backends/gcp_compute_engie/__init__.py

@@ -0,0 +1,3 @@
+from .gcp_compute_engine import GCPComputeEngineBackend as StandaloneBackend
+
+__all__ = ['StandaloneBackend']

lithops/standalone/backends/gcp_compute_engine/__init__.py

@@ -43,48 +43,48 @@
 
 
 def load_config(config_data):
-    if 'gcp_compute_engie' not in config_data or not config_data['gcp_compute_engie']:
-        raise Exception("'gcp_compute_engie' section is mandatory in the configuration")
+    if not config_data['gcp_compute_engine']:
+        raise Exception("'gcp_compute_engine' section is mandatory in the configuration")
 
     if 'gcp' not in config_data:
         config_data['gcp'] = {}
 
-    temp = copy.deepcopy(config_data['gcp_compute_engie'])
-    config_data['gcp_compute_engie'].update(config_data['gcp'])
-    config_data['gcp_compute_engie'].update(temp)
+    temp = copy.deepcopy(config_data['gcp_compute_engine'])
+    config_data['gcp_compute_engine'].update(config_data['gcp'])
+    config_data['gcp_compute_engine'].update(temp)
 
-    if 'credentials_path' in config_data['gcp_compute_engie']:
-        config_data['gcp_compute_engie']['credentials_path'] = os.path.expanduser(
-            config_data['gcp_compute_engie']['credentials_path']
+    if 'credentials_path' in config_data['gcp_compute_engine']:
+        config_data['gcp_compute_engine']['credentials_path'] = os.path.expanduser(
+            config_data['gcp_compute_engine']['credentials_path']
         )
 
     for key in DEFAULT_CONFIG_KEYS:
-        if key not in config_data['gcp_compute_engie']:
-            config_data['gcp_compute_engie'][key] = DEFAULT_CONFIG_KEYS[key]
+        if key not in config_data['gcp_compute_engine']:
+            config_data['gcp_compute_engine'][key] = DEFAULT_CONFIG_KEYS[key]
 
     if 'standalone' not in config_data or config_data['standalone'] is None:
         config_data['standalone'] = {}
 
     for key in SA_DEFAULT_CONFIG_KEYS:
-        if key in config_data['gcp_compute_engie']:
-            config_data['standalone'][key] = config_data['gcp_compute_engie'].pop(key)
+        if key in config_data['gcp_compute_engine']:
+            config_data['standalone'][key] = config_data['gcp_compute_engine'].pop(key)
         elif key not in config_data['standalone']:
             config_data['standalone'][key] = SA_DEFAULT_CONFIG_KEYS[key]
 
     if config_data['standalone']['exec_mode'] == 'consume':
         params_to_check = MANDATORY_PARAMETERS_1
-        config_data['gcp_compute_engie']['max_workers'] = 1
+        config_data['gcp_compute_engine']['max_workers'] = 1
     else:
         params_to_check = MANDATORY_PARAMETERS_2
 
     for param in params_to_check:
-        if param not in config_data['gcp_compute_engie']:
-            msg = f"'{param}' is mandatory in the 'gcp_compute_engie' section of the configuration"
+        if param not in config_data['gcp_compute_engine']:
+            msg = f"'{param}' is mandatory in the 'gcp_compute_engine' or 'gcp' section of the configuration"
             raise Exception(msg)
 
-    if 'region' not in config_data['gcp_compute_engie']:
-        zone = config_data['gcp_compute_engie']['zone']
-        config_data['gcp_compute_engie']['region'] = '-'.join(zone.split('-')[:-1])
+    if 'region' not in config_data['gcp_compute_engine']:
+        zone = config_data['gcp_compute_engine']['zone']
+        config_data['gcp_compute_engine']['region'] = '-'.join(zone.split('-')[:-1])
 
-    if 'region' not in config_data['gcp'] and 'region' in config_data['gcp_compute_engie']:
-        config_data['gcp']['region'] = config_data['gcp_compute_engie']['region']
+    if 'region' not in config_data['gcp'] and 'region' in config_data['gcp_compute_engine']:
+        config_data['gcp']['region'] = config_data['gcp_compute_engine']['region']