Skip to content

Add lookback time adjustment for Office 365 source plugin partition e…#6266

Merged
graytaylor0 merged 1 commit into
opensearch-project:mainfrom
alparish:Office365AdjustStartTime
Nov 13, 2025
Merged

Add lookback time adjustment for Office 365 source plugin partition e…#6266
graytaylor0 merged 1 commit into
opensearch-project:mainfrom
alparish:Office365AdjustStartTime

Conversation

@alparish

Copy link
Copy Markdown
Contributor

…xecution

What:

  • Added time window adjustment logic in Office365Service to correctly handle configured range parameter
  • Ensures exact time window compliance when fetching audit logs

Example time window adjustment:
If current time is 9:30 AM and range is 4 days:

  • Crawler creates partition with startTime = 9:00 AM (4 days ago)
  • Service adjusts to exact 9:30 AM (4 days ago)
    → Ensures we fetch exactly 4 days of data, not 4 days + 30 minutes

Why:

  • The dimensional time slice crawler creates hourly partitions starting from truncated hours
  • Without adjustment, we would fetch extra data beyond the configured range
  • For example, if current time is 9:30 AM:
    • Crawler starts from 9:00 AM (extra 30 minutes)

Issues Resolved

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Instant sevenDaysAgo = Instant.now().minus(SEVEN_DAYS);
if (startTime.isBefore(sevenDaysAgo)) {
adjustedStartTime = sevenDaysAgo;
Instant lookBackHoursAgo = Instant.now().minus(Duration.ofHours(office365SourceConfig.getLookBackHours()));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add corresponding unit tests coverage!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added unit test

@alparish alparish force-pushed the Office365AdjustStartTime branch 2 times, most recently from a5a66c4 to 72f6bec Compare November 13, 2025 21:34
…xecution

Signed-off-by: Alekhya Parisha <aparisha@amazon.com>
@alparish alparish force-pushed the Office365AdjustStartTime branch from 72f6bec to 9e3f781 Compare November 13, 2025 21:58
@graytaylor0 graytaylor0 merged commit 0570c8e into opensearch-project:main Nov 13, 2025
45 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants