Skip to content

Commit 1fa237f

Browse files
LEANDERANTONYclaude
andcommitted
Lever: drop limit=100 — was silently capping Mistral and Palantir
Lever's public postings API returns the full site list in a single GET; passing `limit=100` was a soft hint that turned out to be a hard truncation. Verified live: Mistral: upstream 162, cached 100 (-62) Palantir: upstream 226, cached 100 (-126) Dnb: upstream 86, cached 86 (full, only because small) Greenhouse and Ashby use the same "single GET, take everything" pattern with no `limit` param — Databricks lands at 813 jobs fine. The `limit=100` here was just an arbitrary value from when the Lever provider was added; no specific reason. Drop it and trust the API. Memory is fine: the largest public Lever board we've measured is in the low hundreds, and the worker chunks DB writes at 30 either way. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 082c8cb commit 1fa237f

2 files changed

Lines changed: 9 additions & 2 deletions

File tree

src/job_sources/lever.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -323,9 +323,16 @@ def search(self, query: JobSearchQuery) -> JobSourceSearchResponse:
323323
)
324324

325325
def _fetch_site_payload(self, site_name: str) -> list[dict]:
326+
# No `limit`: Lever's public postings API returns the full
327+
# site list in a single response, so passing `limit=100`
328+
# silently truncated boards like Mistral (162) and Palantir
329+
# (226) down to 100. Greenhouse and Ashby take the same
330+
# "single GET, take everything" approach. The largest
331+
# public Lever board we've measured is in the low hundreds,
332+
# so memory is not a concern.
326333
response = self._http_session.get(
327334
self._list_url(site_name),
328-
params={"mode": "json", "limit": 100},
335+
params={"mode": "json"},
329336
timeout=self._timeout_seconds,
330337
)
331338
response.raise_for_status()

tests/test_lever_job_source.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def test_lever_adapter_returns_normalized_results():
6565
assert response.results[0].metadata["site_name"] == "example"
6666
assert response.results[0].metadata["salary_text"].startswith("INR 3,000,000 - 4,500,000")
6767
assert response.source_details["example"] == "matched"
68-
assert fake_session.calls[0]["params"] == {"mode": "json", "limit": 100}
68+
assert fake_session.calls[0]["params"] == {"mode": "json"}
6969

7070

7171
def test_lever_adapter_matches_bangalore_query_to_bengaluru_location():

0 commit comments

Comments
 (0)