Feature - Stop long running scrapers.

[knmn2000]

Introduced a button on the jobs page of scrapydweb, which sends a stop signal to all the currently running scrapers that have been running longer than a certain amount of time, which can be specified in the settings.

This will be especially useful in cases where the project contains hundreds of running spiders.

[pep8speaks]

Hello @knmn2000! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file __init__.py:

Line 87:9: E116 unexpected indentation (comment)

• In the file common.py:

Line 93:13: E722 do not use bare 'except'

• In the file default_settings.py:

Line 68:121: E501 line too long (158 > 120 characters)

• In the file models.py:

Line 127:17: E126 continuation line over-indented for hanging indent
Line 144:17: E126 continuation line over-indented for hanging indent

• In the file scrapydweb_settings_v10.py:

Line 68:121: E501 line too long (158 > 120 characters)

• In the file utils/check_app_config.py:

Line 83:13: W503 line break before binary operator
Line 86:13: W503 line break before binary operator
Line 87:13: W503 line break before binary operator
Line 88:13: W503 line break before binary operator
Line 401:13: W503 line break before binary operator
Line 617:1: W391 blank line at end of file

• In the file utils/poll.py:

Line 61:13: E116 unexpected indentation (comment)
Line 139:101: E226 missing whitespace around arithmetic operator
Line 217:53: E226 missing whitespace around arithmetic operator
Line 224:48: E226 missing whitespace around arithmetic operator

• In the file utils/scheduler.py:

Line 52:5: E116 unexpected indentation (comment)
Line 53:5: E116 unexpected indentation (comment)
Line 54:5: E116 unexpected indentation (comment)

• In the file utils/send_email.py:

Line 63:9: E722 do not use bare 'except'
Line 66:13: E722 do not use bare 'except'
Line 82:13: E722 do not use bare 'except'

• In the file utils/setup_database.py:

Line 158:9: E722 do not use bare 'except'

• In the file views/baseview.py:

Line 137:13: W503 line break before binary operator
Line 144:13: W503 line break before binary operator
Line 512:17: W503 line break before binary operator
Line 516:17: W503 line break before binary operator

• In the file views/dashboard/jobs.py:

Line 150:13: E722 do not use bare 'except'
Line 320:25: W503 line break before binary operator

• In the file views/files/log.py:

Line 68:110: E226 missing whitespace around arithmetic operator
Line 146:13: W503 line break before binary operator
Line 147:13: W503 line break before binary operator
Line 148:13: W503 line break before binary operator
Line 148:13: E129 visually indented line with same indent as next logical line
Line 269:13: W503 line break before binary operator
Line 269:13: E129 visually indented line with same indent as next logical line
Line 312:13: E722 do not use bare 'except'
Line 371:16: W503 line break before binary operator
Line 372:16: W503 line break before binary operator
Line 466:20: W503 line break before binary operator
Line 494:13: W503 line break before binary operator
Line 495:13: W503 line break before binary operator
Line 496:9: E124 closing bracket does not match visual indentation

• In the file views/files/projects.py:

Line 103:13: E722 do not use bare 'except'

• In the file views/operations/schedule.py:

Line 179:121: E501 line too long (126 > 120 characters)
Line 180:121: E501 line too long (129 > 120 characters)
Line 181:121: E501 line too long (141 > 120 characters)
Line 182:121: E501 line too long (126 > 120 characters)

• In the file views/operations/scrapyd_deploy.py:

Line 51:4: E114 indentation is not a multiple of four (comment)
Line 52:4: E114 indentation is not a multiple of four (comment)
Line 53:4: E114 indentation is not a multiple of four (comment)
Line 72:5: E722 do not use bare 'except'

• In the file views/overview/tasks.py:

Line 157:121: E501 line too long (131 > 120 characters)
Line 159:121: E501 line too long (122 > 120 characters)
Line 160:121: E501 line too long (149 > 120 characters)
Line 273:121: E501 line too long (123 > 120 characters)

• In the file views/system/settings.py:

Line 39:47: E228 missing whitespace around modulo operator

• In the file views/utilities/send_text.py:

Line 44:44: W503 line break before binary operator
Line 45:44: W503 line break before binary operator
Line 52:44: W503 line break before binary operator
Line 53:44: W503 line break before binary operator
Line 56:44: W503 line break before binary operator
Line 57:44: W503 line break before binary operator

[laggardkernel]

I don't think this should be handled by ScrapydWeb. Scrapy has a extension scrapy.extensions.closespider.CloseSpider to close spiders on specific condition. Setting CLOSESPIDER_TIMEOUT is what you want.