fix(warehouses): Add write_dispositions to each resource

[martyngigg]

Summary

Fix problems arising from removal of cli parameter in #177. Set write_dispositions explicitly on each resource.

Summary by CodeRabbit

• Documentation

  • Removed parameter documentation from CLI docstring.

• Refactor

  • Migrated write disposition configuration from global defaults to resource-level settings across data source connectors, enabling granular control over how data is written to targets (replace vs. merge operations).

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough

Walkthrough

The pull request refactors write disposition handling across multiple extract-load modules, transitioning from global default parameters to targeted resource-level specifications via apply_hints and resource_defaults configurations. Documentation updates accompany these structural changes.

Changes

Cohort / File(s)	Summary
CLI Documentation elt-common/src/elt_common/cli.py	Removed docstring parameter documentation for default_write_disposition; function signature and behaviour unchanged.
Extract-Load Resource Configuration warehouses/accelerator/extract_load/accelerator_sharepoint/extract_and_load.py, warehouses/accelerator/extract_load/electricity_sharepoint/extract_and_load.py, warehouses/accelerator/extract_load/statusdisplay/extract_and_load.py	Removed global default_write_disposition parameters from CLI entry points and replaced with targeted write disposition specifications: resource-level apply_hints with merge/replace dispositions, resource_defaults configuration, and direct hints application to individual resources.

Poem

🐰 Hop, hop! The defaults take their rest,
Write dispositions find their nest,
Resource by resource, we align,
The pipeline's flow is now so fine!

Pre-merge checks

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically describes the main change: adding write_dispositions to each resource across warehouse extract-and-load scripts.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI Agents

In
@warehouses/accelerator/extract_load/accelerator_sharepoint/extract_and_load.py:
- Around line 58-59: The write_disposition entries are using dicts instead of
simple strings; change the two occurrences where
write_disposition={"disposition": "replace"} to write_disposition="replace"
(e.g., in the load calls for edr_equipment and edr_equipment_mapping inside
extract_and_load.py) so they match the simple-string dlt syntax used elsewhere
and reserve dict form only for dispositions with extra params like
merge/strategy.

🧹 Nitpick comments (2)

warehouses/accelerator/extract_load/accelerator_sharepoint/extract_and_load.py (1)

73-81: Simplify write_disposition to string format in apply_hints.

The write_disposition parameter uses a dictionary format {"disposition": "replace"}, but since there is no additional configuration (such as merge strategy), the simpler string format "replace" is clearer and more consistent with other simple write disposition usages in the codebase.

Suggested fix

         .with_name("edr_equipment_mapping")
         .apply_hints(
-            write_disposition={"disposition": "replace"},
+            write_disposition="replace",
         )

warehouses/accelerator/extract_load/electricity_sharepoint/extract_and_load.py (1)

158-160: Simplify write_disposition to use string format.

The write_disposition parameter accepts both formats: a simple string ("merge") or a structured dict ({"disposition": "merge", "strategy": "upsert"}). The dict format is intended for specifying additional merge strategies; since no strategy is specified here, the simpler string format is more appropriate.

-    yield from reader.apply_hints(
-        write_disposition={"disposition": "merge"},
-    )
+    yield from reader.apply_hints(
+        write_disposition="merge",
+    )

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between cc89dcc and 961ba6f.

📒 Files selected for processing (4)

• elt-common/src/elt_common/cli.py
• warehouses/accelerator/extract_load/accelerator_sharepoint/extract_and_load.py
• warehouses/accelerator/extract_load/electricity_sharepoint/extract_and_load.py
• warehouses/accelerator/extract_load/statusdisplay/extract_and_load.py

💤 Files with no reviewable changes (1)

• elt-common/src/elt_common/cli.py

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: elt-common end-to-end tests
• GitHub Check: warehouses end-to-end tests

🔇 Additional comments (1)

warehouses/accelerator/extract_load/statusdisplay/extract_and_load.py (1)

28-30: Correct approach for setting resource defaults.

The resource_defaults configuration properly sets the write disposition for all resources from this REST API source. The string format "replace" is correct.