Skip to content

Sleep mechanism for github ban #46

@luyaor

Description

@luyaor

When I was using github-flask to crawl all the fork list of one repo on multi-thread:
request('GET', 'repos/%s/forks' % project_name, True)

It wil be ban by github.

image

Is there any way I can avoid this if I use all_pages=True in
request(method, resource, all_pages=False, **kwargs) ?

My idea is that give user a option for not crawling so fast, like sleep(1) after each API call.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions