You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+21-20Lines changed: 21 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,15 @@
1
-
# Microservice for prerendering web pages.
1
+
# Microservice for prerendering web pages
2
2
3
-
Web pages that are rendered on the browser side and made without SSR are not seen by bots. Because bots can't render pages. This microservice allows all bots to see the already rendered page.
3
+
Web pages rendered on the client side without SSR are not visible to bots, because bots can't render pages. This microservice allows all bots to see the already rendered page.
4
4
5
5
Features:
6
6
7
7
- Allows bots to see content on web pages without SSR
8
-
- Works on any framework, ex: JQuery, Angular 1, StencilJS and others
9
-
- Works on any technologies, ex: WebComponents, Microfrontends, Dynamic Content and others
10
-
- Fast speed and low memory usage, inside only web engine without heavy browser
11
-
-The prerender can work not only for bots but also for all clients if the application supports rerendering
12
-
- Includes exporter of metrics for Prometheus
8
+
- Works with any framework, e.g. JQuery, Angular 1, StencilJS, and others
9
+
- Works with any technology, e.g. WebComponents, Microfrontends, Dynamic Content, and others
10
+
- Fast and low memory usage, uses only a web engine without a heavy browser
11
+
-Prerendering can work not only for bots but also for all clients if the application supports rerendering
12
+
- Includes a Prometheus metrics exporter
13
13
14
14
## Try
15
15
@@ -19,13 +19,13 @@ To try the microservice features, run the container with the command:
19
19
docker run -it --rm -p 3000:3000 mtsrus/botview
20
20
```
21
21
22
-
Now you can open the browser and check the work with the command:
22
+
Now you can open the browser and check the result with the command:
23
23
24
24
```sh
25
25
http://localhost:3000/render/https://mts.ru/
26
26
```
27
27
28
-
The fully rendered page should display, including all content.
28
+
The fully rendered page should be displayed, including all content.
-`-e BOTVIEW_NAV_TIMEOUT=30000` - [This setting will change the default maximum navigation time](https://playwright.dev/docs/api/class-page#page-set-default-navigation-timeout),
@@ -71,25 +72,25 @@ The server will return a response with the specified status code.
71
72
72
73
-`-e BOTVIEW_BLOCK_MEDIA=true` - Block loading of media files to improve performance, default true.
73
74
74
-
-`-e BOTVIEW_BLOCK_URLS="https://an.yandex.ru, https://mc.yandex.ru"` - Comma or space separated list of URL prefixes to block (uses startsWith matching), default blocks Yandex analytics.
75
+
-`-e BOTVIEW_BLOCK_URLS="https://an.yandex.ru, https://mc.yandex.ru"` - Comma or space separated list of URL prefixes to block (uses startsWith matching), by default blocks Yandex analytics.
75
76
Example: `"https://google-analytics.com,https://facebook.com/tr"` will block all requests starting with these URLs.
76
-
Set to empty string `""` to disable default blocking.
77
+
Set to an empty string `""` to disable default blocking.
77
78
78
-
-`-e BOTVIEW_BLOCK_URLS_REGEX=""` - Comma or space separated list of regular expressions to block URLs, default empty.
79
+
-`-e BOTVIEW_BLOCK_URLS_REGEX=""` - Comma or space separated list of regular expressions to block URLs, default is empty.
79
80
Example: `".*\.ads\..*,.*tracking.*"` will block URLs containing ".ads." or "tracking" anywhere in the URL.
80
81
81
82
-`-e BOTVIEW_LOG_LEVEL=info` - Log level for the application, default "info".
82
83
Available levels: `trace`, `debug`, `info`, `warn`, `error`, `fatal`.
83
84
84
85
## Troubleshooting Timeout Issues
85
86
86
-
If you encounter timeout errors during prerendering, it's usually caused by "leaked requests" - network requests that don't complete properly and prevent the page from finishing loading.
87
+
If you encounter timeout errors during prerendering, it is usually caused by "leaked requests" - network requests that do not complete properly and prevent the page from finishing loading.
87
88
88
89
To fix this:
89
90
90
91
1.**Check the logs** for entries containing `"Leaked requests"` - these will show which URLs are causing the timeout
91
-
2.**Add problematic URLs to BOTVIEW_BLOCK_URLS** to prevent them from being loaded during prerendering
92
-
3.**Use the container parameter**`-e BOTVIEW_BLOCK_URLS="url1,url2,url3"` to block specific URLs
92
+
2.**Add problematic URLs to BOTVIEW_BLOCK_URLS** to prevent them from being loaded during prerendering.
93
+
3.**Use the container parameter**`-e BOTVIEW_BLOCK_URLS="url1,url2,url3"` to block specific URLs.
93
94
94
95
**Example:**
95
96
@@ -102,13 +103,13 @@ By default, `https://an.yandex.ru` and `https://mc.yandex.ru` are blocked due to
102
103
103
104
## Metrics Prometheus
104
105
105
-
The microservice has built-in Prometheus monitoring and is located on the endpoint `/metrics`.
106
+
The microservice has built-in Prometheus monitoring and is available at the endpoint `/metrics`.
106
107
107
108
Block this endpoint on the proxy if you do not need to provide access to metrics from outside your network.
108
109
109
110
## Proxy server setup
110
111
111
-
In order to catch bots and send them to the prerendering microservice, you need to configure a proxy server.
112
+
To catch bots and send them to the prerendering microservice, you need to configure a proxy server.
0 commit comments