feat: Complete API overhaul matching Scrappey documentation#63
Conversation
This major update brings complete API coverage: Session Management: - sessions.create - Create persistent browser sessions - sessions.destroy - Destroy sessions - sessions.list - List active sessions - sessions.active - Check session status - websocket.create - Create WebSocket connections Browser Actions (20+ types): - click, type, goto, wait, scroll, hover - wait_for_selector, wait_for_function, wait_for_load_state - wait_for_cookie, execute_js, keyboard, dropdown - switch_iframe, set_viewport, if/while conditionals - solve_captcha (15 captcha types), discord_login, remove_iframes Antibot Bypass Options: - cloudflareBypass, datadomeBypass, kasadaBypass - disableAntiBot, detectIncapsula, spsnspidChallenge Captcha Solving: - automaticallySolveCaptchas, alwaysLoad (multi-select) - Support for turnstile, recaptcha, hcaptcha, funcaptcha, etc. Response Options: - screenshot, screenshotUpload, screenshotWidth/Height - pdf, base64, base64Response, binary - innerText, includeImages, includeLinks - regex extraction, filter fields, listAllRedirects Request Interception: - abortOnDetection, abortOnPostRequest - blackListedDomains, neverCacheDomains - dontLoadMainSite, dontLoadFirstRequest Browser Configuration: - browser type/version, userAgent, locales - forceUniqueFingerprint, webrtcIpv4/Ipv6 Advanced Options: - localStorage, cookiejar, fullPageLoad - waitForUrl, removeIframes, blockCookieBanners - AI parsing with autoparse, structure, model Error Handling: - Added missing CODE-0038 (FingerprintJS challenge) BREAKING CHANGE: Major API expansion with new operations
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
| baseAction.then = JSON.parse(action.thenActions || '[]'); | ||
| } catch { | ||
| baseAction.then = []; |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Previously, invalid JSON in thenActions/orActions fields for 'if' and 'while' browser actions was silently caught and replaced with empty arrays. This caused workflows to fail silently without any error feedback to the user. Now throws descriptive Error with the specific JSON parsing issue, consistent with jsonHeaders error handling in requestBodyBuilder.ts. Fixes silent workflow failure when users provide malformed JSON in conditional browser action configurations.
| } | ||
|
|
||
| const customProxy = this.getNodeParameter('custom_proxy', itemIndex, false) as boolean; | ||
| if (customProxy && credentials?.proxyUrl) { | ||
| body.proxy = credentials.proxyUrl as string; | ||
| } | ||
| } | ||
|
|
||
| const response = await genericHttpRequest.call(this, 'POST', '', { body }); | ||
| return response; | ||
| }; |
There was a problem hiding this comment.
Bug: The sessionCreate and websocketCreate methods ignore the "No Proxy" option, causing the user's selection to have no effect and potentially use a proxy unexpectedly.
Severity: HIGH | Confidence: High
🔍 Detailed Analysis
The UI provides a "No Proxy" option for sessionCreate and websocketCreate operations. However, the implementation in methods.ts for these functions only handles 'proxyFromCredentials' and 'proxyFromScrappey'. It lacks a case for 'noProxy'. Consequently, when a user selects "No Proxy", the noProxy: true flag is not added to the request body, and the selection is silently ignored. This can lead to the request unintentionally using a default proxy instead of being sent directly, which contradicts the user's explicit configuration.
💡 Suggested Fix
In methods.ts, update the SessionCreate and WebSocketCreate methods to handle the 'noProxy' case. Add a condition similar to the one in requestBodyBuilder.ts: if whichProxyToUse is 'noProxy', set body.noProxy = true. This will ensure the user's selection is respected.
🤖 Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: nodes/Scrappey/methods.ts#L136-L182
Potential issue: The UI provides a "No Proxy" option for `sessionCreate` and
`websocketCreate` operations. However, the implementation in `methods.ts` for these
functions only handles `'proxyFromCredentials'` and `'proxyFromScrappey'`. It lacks a
case for `'noProxy'`. Consequently, when a user selects "No Proxy", the `noProxy: true`
flag is not added to the request body, and the selection is silently ignored. This can
lead to the request unintentionally using a default proxy instead of being sent
directly, which contradicts the user's explicit configuration.
Did we get this right? 👍 / 👎 to inform future reviews.
Reference ID: 527205
1 participant
This major update brings complete API coverage:
Session Management:
Browser Actions (20+ types):
Antibot Bypass Options:
Captcha Solving:
Response Options:
Request Interception:
Browser Configuration:
Advanced Options:
Error Handling:
BREAKING CHANGE: Major API expansion with new operations