Skip to content

Commit 42287a8

Browse files
committed
LIBDRUM-1018. Ensure UMD customization is part of default access group
Ensures that the UMD customization enabling the rule forbidding crawling of `/browse/*` endpoints is part of the default access group by removing the blank line that separates the rule from the rest of the group. In a "robots.txt" file, any blank line ends a group, so this was done to ensure that this rule is part of the ruleset used by all crawlers. https://umd-dit.atlassian.net/browse/LIBDRUM-1018
1 parent be20a5f commit 42287a8

1 file changed

Lines changed: 0 additions & 1 deletion

File tree

src/robots.txt.ejs

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@ Disallow: /profile
1818
Disallow: /workflowitems
1919
# Crawlers should be able to access entity pages, but not the facet search links present on entity pages
2020
Disallow: /entities/*?f
21-
2221
# Optionally uncomment the following line ONLY if sitemaps are working
2322
# and you have verified that your site is being indexed correctly.
2423
# UMD Customization

0 commit comments

Comments
 (0)