Commit 2bd3dc0
authored
feat(sharepoint): pass through ACL permission metadata (#699)
## Summary
- Extract SharePoint permission data from the Graph API and normalize to
the standard `[{read: {users, groups}}, {update: ...}, {delete: ...}]`
schema used by Google Drive and Confluence connectors.
- Fetch permissions via Graph JSON batching (`/$batch`, up to 20 per
round-trip) with automatic per-item fallback on batch failure.
- Parse identities from raw `permission.properties` JSON to avoid
cross-field data bleed in the office365 typed accessors.
## Details
- Added `MICROSOFT_ROLE_MAPPING` covering Graph standard roles and
SharePoint-specific `sp.*` roles.
- Added `_extract_identity_ids_from_raw()` to pull Azure AD user/group
IDs from `grantedToV2` (preferred) with `grantedTo` fallback. SharePoint
`siteGroup` numeric IDs are excluded (not resolvable via Graph).
- Added `extract_permissions()` to normalize raw Graph permissions into
the canonical schema.
- Added `_fetch_permissions_batched()` with queue-drain guard on batch
failure and `ClientRequestException` handling per item in fallback.
- Refactored `run_async` to window-and-yield in chunks of 20 instead of
materializing the full file list.
- 32 unit tests covering identity extraction, role mapping, batching,
fallback, and edge cases.
## Limitations
- Requires SharePoint Online (M365) with Azure AD OAuth. SharePoint
Server on-prem is not supported (no Graph API).
- Graph API does not support `$expand=permissions` on DriveItem
collections, so permissions are fetched separately per batch.
- ACL metadata is a best-effort snapshot; it is not runtime
authorization.
Closes PLU-331
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Touches SharePoint indexing flow and adds external Graph
batching/fallback logic; failures could impact metadata completeness or
indexing performance, though behavior is largely additive and covered by
extensive tests.
>
> **Overview**
> Adds **SharePoint ACL pass-through** by fetching Graph `DriveItem`
permissions and normalizing them into the canonical
`[{read},{update},{delete}]` `permissions_data` schema.
>
> Implements Graph `/$batch` permission hydration (20 items per batch)
with a per-item fallback on batch failure, plus raw-JSON identity
extraction and a `MICROSOFT_ROLE_MAPPING` to translate
Microsoft/SharePoint roles into operations. Updates `run_async` to
process files in permission-hydrated chunks, and extends unit +
integration fixtures to validate deterministic, merged ACL output.
>
> Bumps package version to `1.5.0` and documents the enhancement in
`CHANGELOG.md`.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
7cb9a08. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->1 parent 6433ee5 commit 2bd3dc0
21 files changed
Lines changed: 1195 additions & 116 deletions
File tree
- test
- integration/connectors/expected_results
- unit/connectors
- unstructured_ingest
- processes/connectors
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
1 | 7 | | |
2 | 8 | | |
3 | 9 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
Lines changed: 38 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
20 | 51 | | |
21 | 52 | | |
22 | 53 | | |
23 | | - | |
| 54 | + | |
24 | 55 | | |
25 | 56 | | |
26 | 57 | | |
| |||
30 | 61 | | |
31 | 62 | | |
32 | 63 | | |
33 | | - | |
| 64 | + | |
34 | 65 | | |
35 | | - | |
| 66 | + | |
Lines changed: 38 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
20 | 51 | | |
21 | 52 | | |
22 | 53 | | |
23 | | - | |
| 54 | + | |
24 | 55 | | |
25 | 56 | | |
26 | 57 | | |
| |||
30 | 61 | | |
31 | 62 | | |
32 | 63 | | |
33 | | - | |
| 64 | + | |
34 | 65 | | |
35 | | - | |
| 66 | + | |
Lines changed: 38 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
20 | 51 | | |
21 | 52 | | |
22 | 53 | | |
23 | | - | |
| 54 | + | |
24 | 55 | | |
25 | 56 | | |
26 | 57 | | |
| |||
30 | 61 | | |
31 | 62 | | |
32 | 63 | | |
33 | | - | |
| 64 | + | |
34 | 65 | | |
35 | | - | |
| 66 | + | |
Lines changed: 38 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
20 | 51 | | |
21 | 52 | | |
22 | 53 | | |
23 | | - | |
| 54 | + | |
24 | 55 | | |
25 | 56 | | |
26 | 57 | | |
| |||
30 | 61 | | |
31 | 62 | | |
32 | 63 | | |
33 | | - | |
| 64 | + | |
34 | 65 | | |
35 | | - | |
| 66 | + | |
0 commit comments