Skip to content

d2d pipeline bugs from ecosystem specific configurations #1715

@AyanSinhaMahapatra

Description

@AyanSinhaMahapatra

There are acouple issues:

  1. If the d2d pipelines are run without any optional steps selected, we fail with:
'NoneType' object is not iterable

Traceback:
  File "/opt/scancodeio/aboutcode/pipeline/__init__.py", line 199, in execute
    step(self)
    ~~~~^^^^^^
  File "/opt/scancodeio/scanpipe/pipelines/deploy_to_develop.py", line 138, in load_ecosystem_config
    d2d_config.load_ecosystem_config(pipeline=self, options=self.selected_groups)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/scanpipe/pipes/d2d_config.py", line 158, in load_ecosystem_config
    for selected_option in options:
                           ^^^^^^^
  1. If purldb is configured with the d2d pipeline, there is an exception with:
'DeployToDevelop' object has no attribute 'matchable_package_extensions'

Traceback:
  File "/opt/scancodeio/aboutcode/pipeline/__init__.py", line 199, in execute
    step(self)
    ~~~~^^^^^^
  File "/opt/scancodeio/scanpipe/pipelines/deploy_to_develop.py", line 161, in match_archives_to_purldb
    extensions=self.matchable_package_extensions,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  1. There is an issue with setting of matchable package archives:
'NoneType' object is not iterable

Traceback:
  File "/opt/scancodeio/aboutcode/pipeline/__init__.py", line 199, in execute
    step(self)
    ~~~~^^^^^^
  File "/opt/scancodeio/scanpipe/pipelines/deploy_to_develop.py", line 159, in match_archives_to_purldb
    d2d.match_purldb_resources(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        project=self.project,
        ^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        logger=self.log,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/scancodeio/scanpipe/pipes/d2d.py", line 635, in match_purldb_resources
    .filter(extension__in=extensions)
     ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^

These are introduced by #1550

We should also introduce tests to cover against there issues:

  1. Running the d2d pipeline without any selected options
  2. Running the d2d pipeline with the purldb calls mocked

Metadata

Metadata

Labels

Type

No type
No fields configured for issues without a type.

Projects

Status

Validated

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions