When running the actor with Apify Proxy enabled for some domains (for example https://www.ekklisiaonline.gr/), the run fails early with No valid sitemaps were discovered from the provided startUrls. even though the site may have valid sitemap endpoints.
The likely cause is that sitemap discovery is performed through a proxy URL and some sites bloc proxy IPs, HEAD requests, or access to common sitemap paths from that network path. In that scenario, discovery returns an empty result and the actor fails immediately. If I try without proxy then run passes.
Suggested fix: if discovery through proxy returns empty (or times out due to network/proxy-related conditions), retry sitemap discovery once without proxy before failing.
When running the actor with Apify Proxy enabled for some domains (for example https://www.ekklisiaonline.gr/), the run fails early with No valid sitemaps were discovered from the provided startUrls. even though the site may have valid sitemap endpoints.
The likely cause is that sitemap discovery is performed through a proxy URL and some sites bloc proxy IPs, HEAD requests, or access to common sitemap paths from that network path. In that scenario, discovery returns an empty result and the actor fails immediately. If I try without proxy then run passes.
Suggested fix: if discovery through proxy returns empty (or times out due to network/proxy-related conditions), retry sitemap discovery once without proxy before failing.