Commit a000038
committed
src/robots.txt.ejs: remove blank line comments
Remove comment about blank lines not being allowed. This may have
been true at one point, but recent verification with Google's own
robotstxt tool shows no difference in parsing with or without the
blank lines. Also, the RFC for the Robots Exclusion Protocol does
not mention such a limitation.
See: https://github.com/google/robotstxt
See: https://www.rfc-editor.org/rfc/rfc9309.html1 parent 082924c commit a000038
1 file changed
Lines changed: 2 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | 8 | | |
10 | 9 | | |
11 | 10 | | |
| |||
26 | 25 | | |
27 | 26 | | |
28 | 27 | | |
29 | | - | |
| 28 | + | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
33 | | - | |
| 32 | + | |
34 | 33 | | |
35 | 34 | | |
36 | 35 | | |
| |||
0 commit comments