Feat/saved jobs fix and progress v2#338
Open
IfThingsThenStuff wants to merge 1348 commits into
Open
Conversation
…ependencies chore(deps): update ci dependencies
Updated the prerequisites section to include a link for installing uv.
…-patch-1 docs(README): installation link for uv in README
<!-- greptile_comment -->
<h3>Greptile Summary</h3>
This PR introduces a `SequentialToolExecutionMiddleware` that wraps every MCP tool call in an `asyncio.Lock`, ensuring only one LinkedIn scraping operation runs at a time within a single server process. This protects the shared Patchright browser session from race conditions when the server is used in `streamable-http` mode with concurrent clients. The middleware is registered in `create_mcp_server()`, provides optional in-flight progress notifications to queued callers, and emits debug-level timing logs for lock wait/hold durations. Documentation is updated consistently across `README.md`, `AGENTS.md`, and `docs/docker-hub.md`.
- **New file `linkedin_mcp_server/sequential_tool_middleware.py`**: clean `asyncio.Lock`-based middleware with progress reporting and structured debug logging.
- **`server.py`**: one-line addition of `mcp.add_middleware(SequentialToolExecutionMiddleware())` at server creation.
- **`tests/test_server.py`**: four async tests covering middleware registration, serialization under concurrency (`asyncio.gather`), result preservation, and progress-reporting behaviour.
- **Docs**: `README.md`, `AGENTS.md`, `docs/docker-hub.md` all updated with notes about the serialization behaviour and how to observe it via debug logging.
<h3>Confidence Score: 5/5</h3>
- This PR is safe to merge — the change is well-scoped, correctly implemented, and backed by targeted tests.
- The core mechanism (`asyncio.Lock` held for the duration of each tool call) is the correct primitive for serializing work within a single asyncio event loop, which is exactly what uvicorn/FastMCP uses per worker. The lock is created inside the middleware instance (not at class level), so each server instance gets its own independent lock. Python 3.12+ is required, so there are no concerns about `asyncio.Lock` behaviour outside a running event loop. The finally block unconditionally releases the lock even on exceptions, and cancellation is handled correctly by the `async with` context manager. Tests cover all meaningful scenarios including concurrent execution, result integrity, and progress notifications.
- No files require special attention.
<h3>Important Files Changed</h3>
| Filename | Overview |
|----------|----------|
| linkedin_mcp_server/sequential_tool_middleware.py | New middleware that serializes MCP tool calls using asyncio.Lock; clean implementation with correct lock acquisition/release and optional progress reporting. |
| linkedin_mcp_server/server.py | Adds SequentialToolExecutionMiddleware to the server at creation time; one-line integration that is straightforward and correct. |
| tests/test_server.py | New test file covering middleware registration, serialization under concurrency, result preservation, and progress reporting; tests are correct and well-scoped but the concurrency test has a minor potential for improvement. |
| AGENTS.md | Adds a note about tool-call serialization under the Transport Modes section; accurate and consistent with the implementation. |
| README.md | Adds a paragraph about serialization behavior and how to observe it via debug logging; documentation is clear and correct. |
| docs/docker-hub.md | Appends a note about tool serialization to the Docker notes section; consistent with README and AGENTS.md changes. |
</details>
<h3>Sequence Diagram</h3>
```mermaid
sequenceDiagram
participant C1 as Client 1
participant C2 as Client 2
participant MW as SequentialToolExecutionMiddleware
participant Lock as asyncio.Lock
participant Tool as LinkedIn Tool
C1->>MW: on_call_tool(tool_name)
MW-->>C1: report_progress("Queued waiting for scraper lock")
C2->>MW: on_call_tool(tool_name)
MW-->>C2: report_progress("Queued waiting for scraper lock")
MW->>Lock: acquire() [Client 1 wins]
Lock-->>MW: acquired
MW-->>C1: report_progress("Scraper lock acquired, starting tool")
MW->>Tool: call_next(context) [Client 1 executes]
Note over Lock: Client 2 waits here
Tool-->>MW: ToolResult
MW->>Lock: release()
MW-->>C1: return ToolResult
Lock-->>MW: acquired [Client 2 unblocked]
MW-->>C2: report_progress("Scraper lock acquired, starting tool")
MW->>Tool: call_next(context) [Client 2 executes]
Tool-->>MW: ToolResult
MW->>Lock: release()
MW-->>C2: return ToolResult
```
<sub>Last reviewed commit: 2df6e04</sub>
<!-- /greptile_comment -->
…raping_add_max_scrolls_parameter_to_get_person_profile feat(scraping): Add max_scrolls parameter to get_person_profile
…ump_version_to_4.9.1 chore: Bump version to 4.9.1
Author
|
@stickerdaniel - anything stopping this getting merged? |
- Drop Spanish-only aria-label from compose focus selector; generic contenteditable fallback already covers all locales - Patch asyncio.sleep in TestSendMessageComposerInteraction helper so tests no longer burn 1.4s of real wall time per run - Set wait_for mocks before the call in test_returns_locator_when_count_positive and assert_not_called() after, so the early-return invariant is actually verified
Owner
|
Thanks for the PR, give me some time to review |
…-compose-box-actionability Fix/344 bypass patchright compose box actionability in send_message
chore: Bump version to 4.9.2
LinkedIn's invitation modal (role="dialog") was not being detected after clicking Connect, causing connect_with_person to report success without actually sending the request. Changes: - Wait for [role="dialog"] to become visible after clicking Connect - Increase _dialog_is_open timeout from 1s to 3s for modal render time - Fix note flow: use nth(count-2) for secondary button instead of first, which was incorrectly clicking the dismiss/X button - Add explicit wait for textarea visibility after clicking "Add a note" - All selectors are structural (role="dialog", button, textarea) with no hardcoded LinkedIn class names or aria-labels
- P1: add elif guard to return note_not_supported when note is requested but modal has ≤2 buttons (no secondary action available), preventing silent note drop - P2: revert _dialog_is_open default timeout to 1000ms (original), pass timeout=3000 only at the connectable call site to avoid adding 2s latency to Accept operations
…me-tool-status-sync docs(readme): sync tool status table
- Wrap modal handling in existing state == "connectable" guard so Accept flow no longer pays the 3s _dialog_is_open timeout for a modal that never appears - Correct outer comment to match the three-button reality of the invitation modal (was "two buttons", contradicting the inner comment) Addresses Greptile P2 review feedback on stickerdaniel#368.
…t-artdeco-modal-handling fix: handle invitation modal in connect_with_person using structural selectors
…ump_version_to_4.9.3 chore: Bump version to 4.9.3
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Contributor
|
Too many files changed for review. ( |
Author
|
relates to #364 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
replaces #167
I wanted the ability to read out my saved jobs - so, I added it. It will handle multiple pages.
Let me know if this is aligned to what you would like to include. Let me know of any changes you think are needed.
Summary
scrape_saved_jobstoLinkedInExtractor— scrapes the LinkedIn jobs tracker page, extracts job IDs from link hrefs, and paginates through results using numbered page buttonsget_saved_jobsMCP tool with progress reporting viaon_progresscallbacktotal_pageswithmax_pagesfor accurate progress percentagesSetfor O(1) job ID deduplication in the DOM polling function