Merged
Conversation
## [1.60.0](v1.59.0...v1.60.0) (2025-06-26) ### Features * update the readme ([939e170](939e170)) ### CI * **release:** 1.60.0-beta.1 [skip ci] ([9fb5f7c](9fb5f7c))
## [1.61.0](v1.60.0...v1.61.0) (2025-07-03) ### Features * update doc ([2dc6b9b](2dc6b9b))
- Fixed typo in docstring (trasfrom -> transforms) - Added comprehensive error handling for missing schema keys - Added fallback values for malformed array items and missing references - Improved logging in SmartScraperGraph (replaced print with logger) - Added proper validation for pydantic schema structure These fixes prevent KeyError exceptions and improve production reliability.
docs: removed duplicated line
…form-bugs Fix critical schema transformation bugs and improve logging
## [1.62.0](v1.61.0...v1.62.0) (2025-08-13) ### Features * update pr ([c07b3c0](c07b3c0)) ### Docs * removed duplicated line ([c2abb9f](c2abb9f))
## [1.63.0](v1.62.0...v1.63.0) (2025-10-22) ### Features * update model tokens ([79db9b9](79db9b9))
## [1.63.1](v1.63.0...v1.63.1) (2025-10-24) ### Bug Fixes * url redirect ([8f0433c](8f0433c))
- Add timeout parameter to FetchNode (default: 30 seconds)
- Apply timeout to requests.get() calls to prevent indefinite hangs
- Implement timeout for PDF parsing using ThreadPoolExecutor
- Propagate timeout to ChromiumLoader via loader_kwargs
- Add comprehensive unit tests for timeout functionality
- Fully backward compatible (timeout can be disabled with None)
Fixes issue with requests.get() and PDF parsing blocking indefinitely
on slow/unresponsive servers or large documents.
Usage:
node_config={'timeout': 30} # Custom timeout
node_config={'timeout': None} # Disable timeout
node_config={} # Use default 30s timeout
…ation feat: Add configurable timeout to FetchNode
## [1.64.0](v1.63.1...v1.64.0) (2025-11-06) ### Features * Add configurable timeout to FetchNode ([e81a4ed](e81a4ed))
docs: update korean readme
Co-authored-by: VinciGit00 <88108002+VinciGit00@users.noreply.github.com>
Co-authored-by: VinciGit00 <88108002+VinciGit00@users.noreply.github.com>
Rewrote commit history to follow Conventional Commits format for semantic-release: - fix(imports): langchain imports update - docs(timeout): timeout configuration guide Addresses feedback from @VinciGit00 to use semantic release commit format. Co-authored-by: VinciGit00 <88108002+VinciGit00@users.noreply.github.com>
Add SEMANTIC_COMMITS.md with instructions for rewriting commit history to follow Conventional Commits format. Includes the exact commit messages needed and steps for manual rebase. The commits need to be rewritten as: - fix(imports): for the langchain import fixes - docs(timeout): for the timeout documentation Automated tools cannot force-push, so maintainer needs to apply manually. Co-authored-by: VinciGit00 <88108002+VinciGit00@users.noreply.github.com>
Update SEMANTIC_COMMITS.md to use feat(timeout) instead of docs(timeout) as requested. The timeout feature documentation exposes user-facing functionality and warrants a feature-level semantic version bump. Changed commit 323f26a recommendation from: - docs(timeout): add comprehensive timeout configuration guide To: - feat(timeout): add configurable timeout support for FetchNode Addresses feedback from @VinciGit00. Co-authored-by: VinciGit00 <88108002+VinciGit00@users.noreply.github.com>
## [1.72.0](v1.71.0...v1.72.0) (2026-01-20) ### Features * add new tests ([f315f3a](f315f3a))
## [1.73.0](v1.72.0...v1.73.0) (2026-01-30) ### Features * update model tokens ([9c24ecc](9c24ecc))
…point use custom api for tracing
## [1.73.1](v1.73.0...v1.73.1) (2026-02-16) ### Bug Fixes * handle list content in telemetry event validation ([b17b154](b17b154))
MiniMax provides an OpenAI-compatible API, making integration straightforward. This adds: - MiniMax model wrapper class (OpenAI-compatible) - Model token mappings for MiniMax-M1, M2, and M2.5 models - Provider routing in abstract_graph factory - README update listing MiniMax as a supported provider
## [1.74.0](v1.73.1...v1.74.0) (2026-03-15) ### Features * add MiniMax as a supported LLM provider ([6a2f8ec](6a2f8ec))
- Add MiniMax-M2.7 and MiniMax-M2.7-highspeed to model list - Set MiniMax-M2.7 as default model (first in list) - Keep all previous models as alternatives - Add unit tests for MiniMax model configuration
feat: upgrade MiniMax default model to M2.7
## [1.75.0](v1.74.0...v1.75.0) (2026-03-18) ### Features * upgrade MiniMax default model to M2.7 ([f47be50](f47be50))
Library code should never write directly to stdout. Migrated all 13 print() calls to use the existing get_logger() infrastructure with appropriate log levels (debug/info/warning).
…logging fix: replace print() statements with proper logging across codebase
## [1.75.1](v1.75.0...v1.75.1) (2026-03-24) ### Bug Fixes * replace print() statements with proper logging across codebase ([1d9551a](1d9551a))
- Limit OS matrix to ubuntu-only on PRs (macOS/Windows on push only) - Reduce Python matrix from 3.10/3.11/3.12 to 3.10/3.12 - Run benchmarks only on push to main - Run code-quality checks only on push events - Remove Playwright install from benchmark job - Delete duplicate code-quality.yml workflow Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ci: reduce GitHub Actions costs by ~85% on PRs
dosubot
bot
added
the
size:XXL
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+12
to
+54
| name: Unit Tests (Python ${{ matrix.python-version }}) | ||
| runs-on: ${{ matrix.os }} | ||
|
|
||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest", "macos-latest", "windows-latest"]') }} | ||
| python-version: ['3.10', '3.12'] | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v4 | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| uv sync | ||
|
|
||
| - name: Install Playwright browsers | ||
| run: | | ||
| uv run playwright install chromium | ||
|
|
||
| - name: Run unit tests | ||
| run: | | ||
| uv run pytest tests/ -m "unit or not integration" --cov --cov-report=xml --cov-report=term | ||
|
|
||
| - name: Upload coverage to Codecov | ||
| uses: codecov/codecov-action@v4 | ||
| with: | ||
| file: ./coverage.xml | ||
| flags: unittests | ||
| name: codecov-${{ matrix.os }}-py${{ matrix.python-version }} | ||
| token: ${{ secrets.CODECOV_TOKEN }} | ||
| if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12' | ||
|
|
||
| integration-tests: |
Comment on lines
+55
to
+100
| name: Integration Tests | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| test-group: [smart-scraper, multi-graph, file-formats] | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v4 | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| uv sync | ||
|
|
||
| - name: Install Playwright browsers | ||
| run: | | ||
| uv run playwright install chromium | ||
|
|
||
| - name: Run integration tests | ||
| env: | ||
| OPENAI_APIKEY: ${{ secrets.OPENAI_APIKEY }} | ||
| ANTHROPIC_APIKEY: ${{ secrets.ANTHROPIC_APIKEY }} | ||
| GROQ_APIKEY: ${{ secrets.GROQ_APIKEY }} | ||
| run: | | ||
| uv run pytest tests/integration/ -m integration --integration -v | ||
|
|
||
| - name: Upload test results | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: integration-test-results-${{ matrix.test-group }} | ||
| path: | | ||
| htmlcov/ | ||
| benchmark_results/ | ||
|
|
||
| benchmark-tests: |
Comment on lines
+101
to
+140
| name: Performance Benchmarks | ||
| runs-on: ubuntu-latest | ||
| if: github.event_name == 'push' && github.ref == 'refs/heads/main' | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v4 | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| uv sync | ||
|
|
||
| - name: Run performance benchmarks | ||
| env: | ||
| OPENAI_APIKEY: ${{ secrets.OPENAI_APIKEY }} | ||
| run: | | ||
| uv run pytest tests/ -m benchmark --benchmark -v | ||
|
|
||
| - name: Upload benchmark results | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: benchmark-results | ||
| path: benchmark_results/ | ||
|
|
||
| - name: Compare with baseline | ||
| if: github.event_name == 'pull_request' | ||
| run: | | ||
| # Download baseline from main branch | ||
| # Compare and comment on PR if regression detected | ||
| echo "Benchmark comparison would run here" | ||
|
|
||
| code-quality: |
Comment on lines
+141
to
+178
| name: Code Quality Checks | ||
| runs-on: ubuntu-latest | ||
| if: github.event_name == 'push' | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v4 | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| uv sync | ||
|
|
||
| - name: Run Ruff linting | ||
| run: | | ||
| uv run ruff check scrapegraphai/ tests/ | ||
|
|
||
| - name: Run Black formatting check | ||
| run: | | ||
| uv run black --check scrapegraphai/ tests/ | ||
|
|
||
| - name: Run isort check | ||
| run: | | ||
| uv run isort --check-only scrapegraphai/ tests/ | ||
|
|
||
| - name: Run type checking with mypy | ||
| run: | | ||
| uv run mypy scrapegraphai/ | ||
| continue-on-error: true | ||
|
|
||
| test-coverage-report: |
Comment on lines
+179
to
+201
| name: Test Coverage Report | ||
| needs: [unit-tests, integration-tests] | ||
| runs-on: ubuntu-latest | ||
| if: always() | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Download coverage artifacts | ||
| uses: actions/download-artifact@v4 | ||
|
|
||
| - name: Generate coverage report | ||
| run: | | ||
| echo "Coverage report generation would run here" | ||
|
|
||
| - name: Comment coverage on PR | ||
| if: github.event_name == 'pull_request' | ||
| uses: py-cov-action/python-coverage-comment-action@v3 | ||
| with: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
|
|
||
| test-summary: |
Comment on lines
+202
to
+213
| name: Test Summary | ||
| needs: [unit-tests, integration-tests, code-quality] | ||
| runs-on: ubuntu-latest | ||
| if: always() | ||
|
|
||
| steps: | ||
| - name: Check test results | ||
| run: | | ||
| echo "All test jobs completed" | ||
| echo "Unit tests: ${{ needs.unit-tests.result }}" | ||
| echo "Integration tests: ${{ needs.integration-tests.result }}" | ||
| echo "Code quality: ${{ needs.code-quality.result }}" |
dosubot
bot
added
documentation
|
🎉 This PR is included in version 1.76.0-beta.1 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.