Skip to content

Commit f2a326d

Browse files
committed
capture more error code
Signed-off-by: Zhiyuan Chen <this@zyc.ai>
1 parent 2821832 commit f2a326d

3 files changed

Lines changed: 4 additions & 2 deletions

File tree

README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Using built-in crawlers is very simple. A minimal example is shown as follows.
5252
google_crawler = GoogleImageCrawler(storage={'root_dir': 'your_image_dir'})
5353
google_crawler.crawl(keyword='cat', max_num=100)
5454
55-
You can also configurate number of threads and apply advanced search options.
55+
You can also configure number of threads and apply advanced search options.
5656
(Note: compatible with 0.6.0 and later versions)
5757

5858
.. code:: python

icrawler/downloader.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ def download(self, task, default_ext, timeout=5, max_retry=3, overwrite=False, *
129129
if self.reach_max_num():
130130
self.signal.set(reach_max_num=True)
131131
break
132-
elif response.status_code != 200:
132+
elif 200 <= response.status_code < 300:
133133
self.logger.error("Response status code %d, file %s", response.status_code, file_url)
134134
break
135135
elif not self.keep_file(task, response, **kwargs):

pyproject.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ classifiers = [
3636
"Programming Language :: Python :: 3.10",
3737
"Programming Language :: Python :: 3.11",
3838
"Programming Language :: Python :: 3.12",
39+
"Programming Language :: Python :: 3.13",
40+
"Programming Language :: Python :: 3.14",
3941
"Topic :: Internet :: WWW/HTTP",
4042
"Topic :: Software Development :: Libraries :: Python Modules",
4143
"Topic :: Utilities",

0 commit comments

Comments
 (0)