Skip to content

Commit 74fb2d3

Browse files
benjdlambertbenjdlambert
authored
Merge pull request backstage#18513 from taras/patch-7
Correct example and api endpoint in Incremental Ingestion
2 parents 7e011f7 + 2acc8ee commit 74fb2d3

2 files changed

Lines changed: 21 additions & 21 deletions

File tree

‎.changeset/pretty-apes-know.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@backstage/plugin-catalog-backend-module-incremental-ingestion': patch
3+
---
4+
5+
Update readme and instructions

‎plugins/catalog-backend-module-incremental-ingestion/README.md‎

Lines changed: 16 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -99,18 +99,18 @@ export default async function createPlugin(
9999

100100
If you want to manage your incremental entity providers via REST endpoints, the following endpoints are available:
101101

102-
| Method | Path | Description |
103-
| ------ | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
104-
| GET | `/incremental/health` | Checks the health of all incremental providers. Returns array of any unhealthy ones. |
105-
| GET | `/incremental/providers` | Get a list of all known incremental entity providers |
106-
| GET | `/incremental/providers/:provider` | Checks the status of an incremental provider (resting, interstitial, etc). |
107-
| POST | `/incremental/providers/:provider/trigger` | Triggers a provider's next action immediately. E.g., if it's currently interstitial, it will trigger the next burst. |
108-
| POST | `/incremental/providers/:provider/start` | Stop the current ingestion cycle and start a new one immediately. |
109-
| POST | `/incremental/providers/:provider/cancel` | Stop the current ingestion cycle and start a new one in 24 hours. |
110-
| DELETE | `/incremental/providers/:provider` | Completely remove all records for the provider and schedule it to start again in 24 hours. |
111-
| GET | `/incremental/providers/:provider/marks` | Retrieve a list of all ingestion marks for the current ingestion cycle. |
112-
| DELETE | `/incremental/providers/:provider/marks` | Remove all ingestion marks for the current ingestion cycle. |
113-
| POST | `/incremental/cleanup` | Completely remove all records for ALL providers and schedule them to start again in 24 hours. (CAUTION! Can cause orphans!) |
102+
| Method | Path | Description |
103+
| ------ | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
104+
| GET | `/api/catalog/incremental/health` | Checks the health of all incremental providers. Returns array of any unhealthy ones. |
105+
| GET | `/api/catalog/incremental/providers` | Get a list of all known incremental entity providers |
106+
| GET | `/api/catalog/incremental/providers/:provider` | Checks the status of an incremental provider (resting, interstitial, etc). |
107+
| POST | `/api/catalog/incremental/providers/:provider/trigger` | Triggers a provider's next action immediately. E.g., if it's currently interstitial, it will trigger the next burst. |
108+
| POST | `/api/catalog/incremental/providers/:provider/start` | Stop the current ingestion cycle and start a new one immediately. |
109+
| POST | `/api/catalog/incremental/providers/:provider/cancel` | Stop the current ingestion cycle and start a new one in 24 hours. |
110+
| DELETE | `/api/catalog/incremental/providers/:provider` | Completely remove all records for the provider and schedule it to start again in 24 hours. |
111+
| GET | `/api/catalog/incremental/providers/:provider/marks` | Retrieve a list of all ingestion marks for the current ingestion cycle. |
112+
| DELETE | `/api/catalog/incremental/providers/:provider/marks` | Remove all ingestion marks for the current ingestion cycle. |
113+
| POST | `/api/catalog/incremental/cleanup` | Completely remove all records for ALL providers and schedule them to start again in 24 hours. (CAUTION! Can cause orphans!) |
114114

115115
In all cases, `:provider` is the name of the incremental entity provider.
116116

@@ -323,22 +323,17 @@ incrementalBuilder.addIncrementalEntityProvider(myEntityProvider, {
323323
// How long should it attempt to read pages from the API in a
324324
// single burst? Keep this short. The Incremental Entity Provider
325325
// will attempt to read as many pages as it can in this time
326-
burstLength: Duration.fromObject({ seconds: 3 }),
326+
burstLength: { seconds: 3 },
327327

328328
// How long should it wait between bursts?
329-
burstInterval: Duration.fromObject({ seconds: 3 }),
329+
burstInterval: { seconds: 3 },
330330

331331
// How long should it rest before re-ingesting again?
332-
restLength: Duration.fromObject({ day: 1 }),
332+
restLength: { day: 1 },
333333

334334
// Optional back-off configuration - how long should it wait to retry
335335
// in the event of an error?
336-
backoff: [
337-
Duration.fromObject({ seconds: 5 }),
338-
Duration.fromObject({ seconds: 30 }),
339-
Duration.fromObject({ minutes: 10 }),
340-
Duration.fromObject({ hours: 3 }),
341-
],
336+
backoff: [{ seconds: 5 }, { seconds: 30 }, { minutes: 10 }, { hours: 3 }],
342337

343338
// Optional. Use this to prevent removal of entities above a given
344339
// percentage. This can be helpful if a data source is flaky and

0 commit comments

Comments
 (0)