Commit b243142
authored
feat(nextcloud): trigger endpoint for bulk migrations (#4728)
## Summary
Adds `POST /remote/nextcloud/migration` so the Settings UI can start a
Nextcloud-to-Cozy bulk migration end-to-end. The endpoint probes the
supplied credentials, upserts an `io.cozy.accounts` document with the
resolved WebDAV user ID, creates an `io.cozy.nextcloud.migrations`
tracking document in `pending` state, and publishes a
`nextcloud.migration.requested` command to the `migration` RabbitMQ
exchange. A separate service consumes the command and drives the
existing `/remote/nextcloud/:account/*` routes, writing progress back to
the tracking document so the UI can render a real-time progress bar.
Only one migration per instance can be in flight at a time; failed
migrations do not block retries.
## API
`POST /remote/nextcloud/migration` (requires `POST
io.cozy.nextcloud.migrations`)
```json
{
"nextcloud_url": "https://nextcloud.example.com",
"nextcloud_login": "alice",
"nextcloud_app_password": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx",
"source_path": "/",
"target_dir": "/Nextcloud"
}
```
`source_path` and `target_dir` are optional. `source_path` defaults to
`/`. `target_dir` defaults to `/Nextcloud` and must be a clean absolute
path (validated with `path.Clean`, so `..`, double slashes, and relative
segments are rejected). `nextcloud_app_password` should be a Nextcloud
app password, not the account password.
### Error responses
| Code | When |
| ---- | ---- |
| 400 Bad Request | Required fields missing or `target_dir` is not a
clean absolute path |
| 401 Unauthorized | The Nextcloud server rejected the supplied
credentials |
| 409 Conflict | A `pending` or `running` migration already exists for
this instance |
| 500 Internal Server Error | Account upsert or tracking document
creation failed |
| 502 Bad Gateway | The Nextcloud instance is unreachable |
| 503 Service Unavailable | RabbitMQ publish failed; the tracking
document is marked `failed` before returning |
## Companion route: recursive size
`GET /remote/nextcloud/:account/size/*path` returns the recursive byte
total of a Nextcloud folder via a single PROPFIND on `oc:size`. The
migration consumer uses it as a pre-flight quota check so it knows the
true source size before starting; the previous shallow-sum estimate
could be off by several orders of magnitude on deep trees.
## RabbitMQ contract
Exchange `migration`, routing key `nextcloud.migration.requested`. The
consumer is responsible for declaring its queue and binding.
```json
{
"migrationId": "d4e5f6a7b8c94d0ea1b2c3d4e5f6a7b8",
"workplaceFqdn": "alice.cozy.example.com",
"accountId": "a1b2c3d4e5f6",
"sourcePath": "/",
"timestamp": 1712563200
}
```
Credentials are never in the payload; they live in the
`io.cozy.accounts` document referenced by `accountId`. `target_dir` is
read by the consumer from the tracking document so legacy messages and
new ones both flow through the same path. `MessageID` is set to the
migration ID for cross-system tracing.
## Credentials probe
The OCS probe itself is not new. It already existed as a lazy fallback
behind `(nc *NextCloud).fetchUserID()`, called from `fillUserID` only
when an existing account document was missing its cached
`webdav_user_id`. In practice that path was almost never exercised, so a
latent bug sat there undetected: the probe targeted `apps/user_status`,
and any non-200 (including a 404 from a managed Nextcloud host that
strips the optional `user_status` app) was turned into
`webdav.ErrInvalidAuth`.
What this PR adds is the synchronous, user-facing path: the migration
trigger endpoint calls `FetchUserIDWithCredentials` before persisting
anything, so the probe is now on the critical path of every trigger.
That exposure turned the latent classification bug into a user-visible
401 for anyone pointing the feature at a managed Nextcloud like
`thegood.cloud`.
A follow-up commit on this branch fixes it by switching the probe to OCS
Core (`/ocs/v2.php/cloud/user`, which cannot be disabled by an admin)
and narrowing the auth-failure classification to 401/403 only, so other
non-2xx statuses bubble up with their real cause. Both code paths (the
old lazy fallback and the new synchronous call) benefit from the fix.
Validated end-to-end against a real Nextcloud instance: the probe, the
encrypted-at-rest account persistence, the tracking document, the
`/size/*path` pre-flight, the RabbitMQ publish with no credentials in
the payload, and the consumer-driven transfer all behave as documented.18 files changed
Lines changed: 1969 additions & 30 deletions
File tree
- docs
- model/nextcloud
- pkg
- consts
- couchdb
- logger
- rabbitmq
- webdav
- web
- remote
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
579 | 579 | | |
580 | 580 | | |
581 | 581 | | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
134 | 134 | | |
135 | 135 | | |
136 | 136 | | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
137 | 177 | | |
138 | 178 | | |
139 | 179 | | |
| |||
0 commit comments