fix: Reporting of malformed licenses

[dominicmh]

Description
very_good packages check licenses fails to report pubs with unknown licenses (e.g. rxdart 0.27.7):

$ very_good packages check licenses
✓ Retrieved 0 licenses from 1 package. (0.7s)

Steps To Reproduce

1. Clone example repo (includes rxdart dependency in pubspec.lock)
2. Execute dart pub global activate very_good_cli; very_good packages check licenses

Expected Behavior
✓ Retrieved 1 license from 1 package of type: unknown (1).

Actual Behavior
✓ Retrieved 0 licenses from 1 package.

[alestiago]

Hi @dominicmh thanks for opening an issue! And sorry for the late reply, I haven't been able to catch-up recently.

I've been able to reproduce this issue by using the following pubspec.yaml.

puspec.yaml

name: example
description: An example project.
version: 1.0.0+1
publish_to: none

environment:
  sdk: "^3.3.0"

dependencies:
  rxdart: ^0.27.7

@dominicmh are you interested in contributing with a fix? If so, let me know and I can assign you to this item and assist you on landing it 🙌

[dominicmh]

Hi @alestiago – I can't really promise any delivery date, could be weeks or even months. But generally I'm open to fix this.

[tomsp-01]

Hi, following PR introduced this issue: #883

[tomsp-01]

So you can use version 0.17.1 until it is fixed. Maybe i can look into this issue :)

[dominicmh]

I analyzed the issue und added a failing test for the bug.

Analysis

• PANA's detectLicense is used to parse all detected license types from the LICENSE file (2ec4fc4)
• Text from the LICENSE file that doesn't match to a standardized license format is simply ignored in the returned result.
• The answer what to do with ignored text is not straight-foward. Options:
  1. If there is a LICENSE file, but the result doesn't include any matched license types (Result.matches.isEmpty), add license type SpdxLicense.$unknown.value for the given dependency
  2. If some proportion of the LICENSE file's text cannot be matched to a license type, add (an additional) license type SpdxLicense.$unknown.value for the given dependency. This proportion can be retrieved through Result.unclaimedTokenPercentage and a threshold would need to be defined. This option extends option 1 with the ability to detect unknown licenses if multiple licenses are listed (usually through transitive dependencies that require the consumer's license to include their license). Examples:
     • If no license type can be detected at all (like in rxdart 0.27.7), Result.unclaimedTokenPercentage is 1.
     • If the LICENSE file conforms to license format standards, Result.unclaimedTokenPercentage is somewhere between 0.01 and 0.05 (based on a test run with 40 dependencies).
     • If one standard license and one non-standard license is included, Result.unclaimedTokenPercentage will be around 0.5.
  3. We keep it as is. Non-standard licenses are ignored and not reported.
• A 100% confidence of reporting all licenses cannot be reached, independent of the chosen option.
• While libraries should ideally conform to recognized licensing standards, it is ultimately the consumer's responsibility to review and understand any non-standard licenses. If very_good_license ignores licenses, it may impose legal risks to the consumer.
• Proposal: Go with option 1. It handles most of the cases and the behaviour is comprehensible. What do you think, @alestiago? If you agree, I can implement it.

Testing

A note on testing: PANA's detectLicense is currently mocked in the tests. I would recommend using the real implementation. Motivation: We rely on PANA's internal license detection algorithm, which is not exposed as a public API. If something in their implementation changes, we would neither notice through changelogs, nor our tests – unless we use the real implementation in our tests.

[alestiago]

Hi @dominicmh thanks for the analysis!

I'll try having a read to the entire issue again next week and provide my two cents asap!