{confcom}: Simplify logic in map_image_from_tar(_compatibility)?, new integrity-vhd (Azure#9838)

* {confcom}: Simplify logic in map_image_from_tar(_compatibility)?

The current code extracts an item in the tar based on a path controlled by the
manifest within the tar, which is prone to path traversal on Linux and Windows,
and potentially other path confusion attacks on Windows if the Config field in
the manifest contains special names.

This extraction is unnecessary as we can read the content of the config directly
from the tar file using extractfile.  This commit simplifies it to do that.

Also, there are two versions of the map_image_from_tar function, with the second
one introduced in Azure#7414. Later code changes in Azure#8238 means that these functions
now do basically the same thing (with different clean up code). After this
simplification, these two functions are exactly the same, so let's also remove
the _compatibility one.

Before:

    > az confcom acipolicygen --image mcr.microsoft.com/aci/skr:2.14 --tar image-docker-malformed.tar --outraw-pretty-print
    ...
    Pulling and hashing images...:   0%|                                                                                                        | 0/2 [00:00<?, ?percent/s]
    ERROR: The command failed with an unexpected error. Here is the traceback:
    ERROR: [Errno 13] Permission denied: '../../../../../blobs'
    Traceback (most recent call last):
    File "/home/mao/.az-cli-newpy.venv/lib/python3.13/site-packages/knack/cli.py", line 233, in invoke
        cmd_result = self.invocation.execute(args)
    File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py", line 677, in execute
        raise ex
    File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py", line 820, in _run_jobs_serially
        results.append(self._run_job(expanded_arg, cmd_copy))
                    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py", line 789, in _run_job
        result = cmd_copy(params)
    File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/commands/__init__.py", line 335, in __call__
        return self.handler(*args, **kwargs)
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
    File "/home/mao/src/github.com/Microsoft/azure-cli/src/azure-cli-core/azure/cli/core/commands/command_operation.py", line 120, in handler
        return op(**command_args)
    File "/home/mao/.azure/cliextensions/confcom/azext_confcom/custom.py", line 197, in acipolicygen_confcom
        policy.populate_policy_content_for_all_images(
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
            individual_image=bool(image_name), tar_mapping=tar_mapping, faster_hashing=faster_hashing
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
    File "/home/mao/.azure/cliextensions/confcom/azext_confcom/security_policy.py", line 485, in populate_policy_content_for_all_images
        image_info, tar = get_image_info(progress, message_queue, tar_mapping, image)
                        ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/mao/.azure/cliextensions/confcom/azext_confcom/template_util.py", line 111, in get_image_info
        image_info = os_util.map_image_from_tar_backwards_compatibility(
            image_name, tar_file, tar_location
        )
    File "/home/mao/.azure/cliextensions/confcom/azext_confcom/os_util.py", line 190, in map_image_from_tar_backwards_compatibility
        tar.extract(info_file.name, path=tar_dir)
        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/mao/.local/share/uv/python/cpython-3.13.12-linux-x86_64-gnu/lib/python3.13/tarfile.py", line 2416, in extract
        self._extract_one(tarinfo, path, set_attrs, numeric_owner)
        ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/mao/.local/share/uv/python/cpython-3.13.12-linux-x86_64-gnu/lib/python3.13/tarfile.py", line 2464, in _extract_one
        self._handle_fatal_error(e)
        ~~~~~~~~~~~~~~~~~~~~~~~~^^^
    File "/home/mao/.local/share/uv/python/cpython-3.13.12-linux-x86_64-gnu/lib/python3.13/tarfile.py", line 2458, in _extract_one
        self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
        ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                            set_attrs=set_attrs,
                            ^^^^^^^^^^^^^^^^^^^^
                            numeric_owner=numeric_owner,
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                            filter_function=filter_function,
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                            extraction_root=path)
                            ^^^^^^^^^^^^^^^^^^^^^
    File "/home/mao/.local/share/uv/python/cpython-3.13.12-linux-x86_64-gnu/lib/python3.13/tarfile.py", line 2539, in _extract_member
        os.makedirs(upperdirs, exist_ok=True)
        ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<frozen os>", line 218, in makedirs
    File "<frozen os>", line 228, in makedirs
    PermissionError: [Errno 13] Permission denied: '../../../../../blobs'
    To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

After:

    > az confcom acipolicygen --image mcr.microsoft.com/aci/skr:2.14 --tar image-docker-malformed.tar --outraw-pretty-print | grep -C10 layers
    ...
    Pulling and hashing images...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.38percent/s]
    WARNING: mcr.microsoft.com/aci/skr:2.14 read from local tar file
            "strategy": "re2"
          },
          {
            "pattern": "azurecontainerinstance_restarted_by=.+",
            "required": false,
            "strategy": "re2"
          }
        ],
        "exec_processes": [],
        "id": "mcr.microsoft.com/aci/skr:2.14",
        "layers": [
          "a189b02d4858578459fda1dfbd7c6a4557c44208b9829e02b931771a6d611c39",
          "300f9661fb3d46c0f299ad6f552b7ad0c41ea5141755b0b3feaca3081a108f7a",
          "0afffca98bacf8e7b6e6f7982459a03219f60555523163c73c4b092e0a3deef2",
          "eefefd5009aed4ba4478876995d1a18aa3a670661fcc61d2e4cba6e2b79da0a1",
          "b868a7e1bebef40e5bf4d58fe271c0a10a351e68b12179ec019af9f6c75781ae",
          "8b4842f06982817534a75bcf71865213b09dfa8313229c384e5201dadbd75e25"
        ],
    ...

(Validated that this matches the result of using the image reference directly
without a tar, and also matches with the result of using the oci tar from `oras
backup`)

Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>

* {confcom} Bump version to 2.0.0b3

* {confcom}: Fix type on load_json_from_str and read_file_from_tar

* {confcom}: Release 2.0.0 (non-preview)

* {confcom}: Bump integrity-vhd to v2.1

Take Windows version check for C-WCOW policy generation

---------

Signed-off-by: Tingmao Wang <tingmaowang@microsoft.com>

Author: micromaomao

src/confcom/HISTORY.rst

@@ -2,6 +2,11 @@
 
 Release History
 ===============
+
+2.0.0
++++++
+* Fix path traversal when generating policies from untrusted image tar files
+
 2.0.0b2
 +++++++
 * Fix default working directory for Windows containers being set to C:\\ if the image doesn't specify one.

src/confcom/azext_confcom/azext_metadata.json

@@ -1,4 +1,3 @@
 {
-    "azext.minCliCoreVersion": "2.26.2",
-    "azext.isPreview": true
+    "azext.minCliCoreVersion": "2.26.2"
 }

src/confcom/azext_confcom/os_util.py

@@ -46,7 +46,7 @@ def clean_up_temp_folder(temp_file_path: str) -> None:
     shutil.rmtree(folder_name)
 
 
-def load_json_from_str(data: str) -> dict:
+def load_json_from_str(data: str | bytes | bytearray) -> dict:
     if data:
         try:
             return json.loads(data)
@@ -159,56 +159,13 @@ def load_tar_mapping_from_config_file(path: str) -> dict:
     return output_dict
 
 
-def map_image_from_tar_backwards_compatibility(image_name: str, tar: TarFile, tar_location: str):
-    tar_dir = os.path.dirname(tar_location)
-    # grab all files in the folder and only take the one that's named with hex values and a json extension
-    members = tar.getmembers()
-
-    info_file = None
-    # if there's more than one image in the tarball, we need to do some more logic
-    if len(members) > 0:
-        # extract just the manifest file and see if any of the RepoTags match the image_name we're searching for
-        # the manifest.json should have a list of all the image tags
-        # and what json files they map to to get env vars, startup cmd, etc.
-        tar.extract("manifest.json", path=tar_dir)
-        manifest_path = os.path.join(tar_dir, "manifest.json")
-        manifest = load_json_from_file(manifest_path)
-        # if we match a RepoTag to the image, stop searching
-        for image in manifest:
-            if image_name in image.get("RepoTags"):
-                info_file = [
-                    item for item in members if item.name == image.get("Config")
-                ][0]
-                break
-        # remove the extracted manifest file to clean up
-        force_delete_silently(manifest_path)
-    else:
-        eprint(f"Tarball at {tar_location} contains no images")
-
-    if not info_file:
-        return None
-    tar.extract(info_file.name, path=tar_dir)
-
-    # get the path of the json file and read it in
-    image_info_file_path = os.path.join(tar_dir, info_file.name)
-    image_info_raw = load_json_from_file(image_info_file_path)
-    # delete the extracted json file to clean up
-    force_delete_silently(image_info_file_path)
-    image_info = image_info_raw.get("config")
-    # importing the constant from config.py gives a circular dependency error
-    image_info["Architecture"] = image_info_raw.get("architecture")
-
-    shutil.rmtree("blobs", ignore_errors=True)
-    return image_info
-
-
 def get_oci_image_name(image_name: str) -> str:
     if "/" not in image_name:
         return f"docker.io/library/{image_name}"
     return image_name
 
 
-def read_file_from_tar(tar: TarFile, filename: str) -> str:
+def read_file_from_tar(tar: TarFile, filename: str) -> bytes:
     try:
         return tar.extractfile(filename).read()
     except KeyError:
@@ -251,36 +208,33 @@ def map_image_from_tar_oci_layout_v1(image_name: str, tar: TarFile, tar_location
     eprint(f"Image '{image_name}' is not found in '{tar_location}'")
 
 
-def map_image_from_tar(image_name: str, tar: TarFile, tar_location: str):
-    tar_dir = os.path.dirname(tar_location)
+def map_image_from_tar(image_name: str, tar: TarFile, _tar_location: str):
+    # Inspect the manifest file and see if any of the RepoTags match the
+    # image_name we're searching for.  For each manifest in the JSON, it should
+    # also have a Config field for what json files they map to to get env vars,
+    # startup cmd, etc.
+    #
+    # NOTE: read manifest.json directly (not via read_file_from_tar) so that a
+    # missing manifest.json raises KeyError. The caller relies on that to fall
+    # back to the OCI layout v1 reader.
+    manifest_bytes = tar.extractfile("manifest.json").read()
+    manifest = load_json_from_str(manifest_bytes)
+
     info_file = None
-    info_file_name = "manifest.json"
-
-    # extract just the manifest file and see if any of the RepoTags match the image_name we're searching for
-    # the manifest.json should have a list of all the image tags
-    # and what json files they map to to get env vars, startup cmd, etc.
-    tar.extract(info_file_name, path=tar_dir)
-    manifest_path = os.path.join(tar_dir, info_file_name)
-    manifest = load_json_from_file(manifest_path)
-    try:
-        # if we match a RepoTag to the image, stop searching
-        for image in manifest:
-            if image_name in image.get("RepoTags"):
-                info_file = image.get("Config")
-                break
-    finally:
-        # remove the extracted manifest file to clean up
-        force_delete_silently(manifest_path)
+    # if we match a RepoTag to the image, stop searching
+    for image in manifest:
+        if image_name in image.get("RepoTags"):
+            info_file = image.get("Config")
+            break
 
     if not info_file:
         return None
-    tar.extract(info_file, path=tar_dir)
 
-    # get the path of the json file and read it in
-    image_info_file_path = os.path.join(tar_dir, info_file)
-    image_info_raw = load_json_from_file(image_info_file_path)
-    # delete the extracted json file to clean up
-    force_delete_silently(image_info_file_path)
+    # Read config file directly from the tar stream (without extracting
+    # anything) so that malicious paths in the manifest cannot cause any actual
+    # writes.
+    image_info_raw_bytes = read_file_from_tar(tar, info_file)
+    image_info_raw = load_json_from_str(image_info_raw_bytes)
     image_info = image_info_raw.get("config")
     # importing the constant from config.py gives a circular dependency error
     image_info["Architecture"] = image_info_raw.get("architecture")

src/confcom/azext_confcom/rootfs_proxy.py

@@ -28,13 +28,13 @@
 _dmverity_vhd_binaries = {
     "Linux": {
         "path": _binaries_dir / "dmverity-vhd",
-        "url": "https://github.com/microsoft/integrity-vhd/releases/download/v2.0/dmverity-vhd",
-        "sha256": "e7ad858fef018acd7d8a4ccb74f1b7a9cc1b3d6db5a7f8da5a259f71b26c12ea",
+        "url": "https://github.com/microsoft/integrity-vhd/releases/download/v2.1/dmverity-vhd",
+        "sha256": "a75eb11f3ad3058bfdef0b5cf0bf64c1bef714a2afa054f3e242932d25d9e57d",
     },
     "Windows": {
         "path": _binaries_dir / "dmverity-vhd.exe",
-        "url": "https://github.com/microsoft/integrity-vhd/releases/download/v2.0/dmverity-vhd.exe",
-        "sha256": "6ef425c4bd07739d9cc90e57488985c1fca41f8d106fc816123b95b6305ee0af",
+        "url": "https://github.com/microsoft/integrity-vhd/releases/download/v2.1/dmverity-vhd.exe",
+        "sha256": "5e371f86a2b552e5e69759421a81a26a07450dc404c1c88a08a4f983322598a2",
     },
 }
 

src/confcom/azext_confcom/template_util.py

@@ -107,11 +107,6 @@ def get_image_info(progress, message_queue, tar_mapping, image):
             with tarfile.open(tar_location) as tar_file:
                 # get all the info out of the tarfile
                 try:
-                    logger.info("using backwards compatibility tar file")
-                    image_info = os_util.map_image_from_tar_backwards_compatibility(
-                        image_name, tar_file, tar_location
-                    )
-                except IndexError:
                     logger.info("using docker formatted tar file")
                     image_info = os_util.map_image_from_tar(
                         image_name, tar_file, tar_location

src/confcom/setup.py

@@ -19,7 +19,7 @@
 
     logger.warn("Wheel is not available, disabling bdist_wheel hook")
 
-VERSION = "2.0.0b2"
+VERSION = "2.0.0"
 
 # The full list of classifiers is available at
 # https://pypi.python.org/pypi?%3Aaction=list_classifiers