Skip to content

diff for debian 1.6.2-4#1582

Open
neolynx wants to merge 55 commits into
debian-masterfrom
debian/1.6.2-4
Open

diff for debian 1.6.2-4#1582
neolynx wants to merge 55 commits into
debian-masterfrom
debian/1.6.2-4

Conversation

@neolynx

@neolynx neolynx commented Jun 14, 2026

Copy link
Copy Markdown
Member

Description of the Change

  • backport fixes from master to debian 1.6.2-3

@neolynx neolynx force-pushed the debian/1.6.2-4 branch 3 times, most recently from c5e770f to 2d78c46 Compare June 14, 2026 22:50
@codecov

codecov Bot commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 32.56743% with 675 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (debian-master@d90825f). Learn more about missing BASE report.

Files with missing lines Patch % Lines
api/publish.go 13.52% 236 Missing and 7 partials ⚠️
api/snapshot.go 6.66% 125 Missing and 1 partial ⚠️
api/repos.go 18.24% 118 Missing and 3 partials ⚠️
api/mirror.go 2.38% 41 Missing ⚠️
deb/publish.go 27.65% 31 Missing and 3 partials ⚠️
task/list.go 55.10% 21 Missing and 1 partial ⚠️
cmd/publish.go 0.00% 20 Missing ⚠️
azure/public.go 71.42% 14 Missing and 2 partials ⚠️
api/files.go 64.51% 9 Missing and 2 partials ⚠️
s3/public.go 62.06% 9 Missing and 2 partials ⚠️
... and 11 more
Additional details and impacted files
@@               Coverage Diff                @@
##             debian-master    #1582   +/-   ##
================================================
  Coverage                 ?   38.17%           
================================================
  Files                    ?      159           
  Lines                    ?    18888           
  Branches                 ?        0           
================================================
  Hits                     ?     7211           
  Misses                   ?    10980           
  Partials                 ?      697           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@neolynx neolynx changed the base branch from release/1.6.2 to debian-master June 15, 2026 10:04
@neolynx neolynx force-pushed the debian/1.6.2-4 branch 6 times, most recently from 2bbfc8b to 3bbe8d6 Compare June 19, 2026 17:26
JupiterRider and others added 11 commits June 19, 2026 19:39
`orig` isn't necessarily a string, so the string concatenation here can
raise a TypeError.
When using rootless podman, the *current user* gets mapped to uid 0,
which results in the aptly user being unable to write to the build
directory. We can instead map the current user to the corresponding uid
in the container via `PODMAN_USERNS=keep-id`, which matches up with what
docker-wrapper wants...but then that will *enter the container as the
current uid*, which messes with the ability to set permissions on
`/var/lib/aptly`. That can be fixed by explicitly passing `--user 0:0`,
which should be a no-op on docker (since the container's default user is
already root).

Additionally, this adds `--security-opt label=disable` to avoid
permission errors when running on systems with SELinux enforcing.
Otherwise, every `make docker-...` invocation will need to rebuild
everything from scratch.
The code was only forwarding TEST, but CAPTURE is useful too.
Enabling coverage near-doubles the incremental build time and adds
overhead to individual tests on the order of **5-10x** or more. It's not
essential to have this for quick local system-test runs, so add an option
to disable it.
Initially found by automated repository health checks used by Termux
in termux/termux-packages#27472

The root problem was 4.3.5a comparing less than 4.3.5-rc1-1 by aptly
According to debian "4.3.5a" > "4.3.5-rc1-1"

This is because dpkg splits hyphen for revision at the first hyphen,
whereas aptly was splitting at the last hyphen which is different from
dpkg's behaviour.

dpkg behaviour: https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/lib/dpkg/parsehelp.c#n242

Perhaps this wasn't detected as there was broken tests in the repository
since the initial commit of aptly. This also fixes those tests
Ales Bregar and others added 23 commits June 19, 2026 20:20
Implement support for the SOURCE_DATE_EPOCH environment variable as
specified by reproducible-builds.org. When set, this variable overrides
the current timestamp in the Release file's Date and Valid-Until fields,
enabling reproducible filesystem publishes.

- Read SOURCE_DATE_EPOCH environment variable in Publish()
- Use the epoch timestamp for both Date and Valid-Until fields
- Gracefully fallback to current time if unset or invalid
- Add comprehensive tests for valid and invalid SOURCE_DATE_EPOCH values
* enable in dev and test env
* fix api/repos doc
* remove useless resource lock
  Resource locks need to be before the background task. creating same publish endpoint at the same time is unlikely...
* load data inside background tasks
  This fixes a flaw in async apis, which loaded the published repo from the DB and mutated it outside the task closure, before the task lock was acquired.
  Perform collection.LoadComplete inside maybeRunTaskInBackground and have tasks use a fresh copy of taskCollectionFactory, taskCollection
* lock source repos/snapshots for publish operations
  Concurrent tasks were not properly locking their resources, leading to inconsistent published indexes:
  SourceLocalRepo: iterate published.Sources (component -> source UUID), look up each local repo via localRepoCollection.ByUUID and append string(repo.Key()) to resources
  SourceSnapshot: iterate b.Snapshots,look up each snapshot via snapshotCollection.ByName and append string(snapshot.ResourceKey()) to resources.
* lock pool on non MultiDist publish
* revert mutex on LinkFromPool
* use uuids, since names can be renamed
* add test for MultiDist change
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* use uuids, since names can be renamed
 * perform collection.LoadComplete inside maybeRunTaskInBackground
 * have tasks use a fresh copy of taskCollectionFactory, taskCollection
 * fix locking for snapshots of snapshots by locking SourceSnapshots
 * use uuids, since names can be renamed
* load data inside background tasks
  Perform collection.LoadComplete inside maybeRunTaskInBackground
  Have tasks use a fresh copy of taskCollectionFactory, taskCollection
* show resources in task details
* fix task state locking
* return task object consistently

Race condition iexisted where task State, err, and processReturnValue fields
were written by consumer goroutine and read by concurrent accessors without
proper synchronization, causing torn reads and data races.
and path escape the new name param
Make sure pathCache is properly locked for concurrent access.

Add RWMutex to the PublishedStorage struct:
- Cache initialization
  Read-lock to test for nil, then write-lock with a second nil check before populating
- Cache reads
  RLock/RUnlock, allowing concurrent readers
- Cache writes / deletes
  Lock/Unlock
@neolynx neolynx changed the title diff diff for debian 1.6.2-4 Jun 19, 2026
@neolynx neolynx force-pushed the debian/1.6.2-4 branch 3 times, most recently from 9ee5a02 to aeeac54 Compare June 19, 2026 19:07
- run separate unit-test job
- build docker
- allow make docker-unit-tests in ci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.