Skip to content

Commit d6bc15b

Browse files
petersmythepetersmythe
committed
Make seeder thread pool sizes configurable via system properties / env vars
Allow GWC_SEEDER_CORE_POOL_SIZE and GWC_SEEDER_MAX_POOL_SIZE to be specified as Java system properties or OS environment variables, with system properties taking precedence. Falls back to the Spring XML constructor arguments (16/32) when not set or invalid. Invalid values (non-numeric, zero, negative) are logged as warnings and the defaults are used. If corePoolSize exceeds maxPoolSize after resolution, maxPoolSize is automatically adjusted upward to match. Includes unit tests and documentation updates. Closes #1429
1 parent 5d1c9e5 commit d6bc15b

8 files changed

Lines changed: 364 additions & 3 deletions

File tree

‎documentation/en/user/source/production/index.rst‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,43 @@ These applicaiton properties can be established by any of the following ways, in
9797
- As a System environment variable: `export GWC_SEED_ABORT_LIMIT=2000; <your usual command to run GWC here>` (or for Tomcat, use the Tomcat's `CATALINA_OPTS` in Tomcat's `bin/catalina.sh` as this: `CATALINA_OPTS="GWC_SEED_ABORT_LIMIT=2000 GWC_SEED_RETRY_COUNT=2`
9898

9999

100+
Seeder Thread Pool Size
101+
+++++++++++++++++++++++
102+
103+
The seeder thread pool controls how many seeding threads can run concurrently. By default, the core pool size is ``16`` and the maximum pool size is ``32``. These can be configured in ``geowebcache.xml``:
104+
105+
.. code-block:: xml
106+
107+
<gwcConfiguration>
108+
...
109+
<seederCorePoolSize>24</seederCorePoolSize>
110+
<seederMaxPoolSize>64</seederMaxPoolSize>
111+
...
112+
</gwcConfiguration>
113+
114+
* ``seederCorePoolSize`` : the number of threads to keep in the pool, even if they are idle. Defaults to ``16``.
115+
* ``seederMaxPoolSize`` : the maximum number of threads allowed in the pool. Defaults to ``32``.
116+
117+
The configuration can be overridden at runtime using Java system properties or OS environment variables (useful for Docker and cloud deployments):
118+
119+
* ``GWC_SEEDER_CORE_POOL_SIZE`` : overrides ``seederCorePoolSize`` from the XML configuration.
120+
* ``GWC_SEEDER_MAX_POOL_SIZE`` : overrides ``seederMaxPoolSize`` from the XML configuration.
121+
122+
These overrides are resolved in the following order of precedence:
123+
124+
- As a Java system property: for example ``java -DGWC_SEEDER_CORE_POOL_SIZE=24 -DGWC_SEEDER_MAX_POOL_SIZE=64 ...``
125+
- As a System environment variable: ``export GWC_SEEDER_CORE_POOL_SIZE=24``
126+
127+
If the value is not set or is not a valid positive integer, the default is used. Invalid values are logged as warnings.
128+
129+
If ``seederCorePoolSize`` (or its override) is set to a value greater than ``seederMaxPoolSize``, the maximum pool size will be automatically adjusted upward to match the core pool size.
130+
131+
.. note::
132+
Unlike the seed failure tolerance settings above, the environment variable overrides are resolved directly by the Java code and do not support servlet context parameters in ``WEB-INF/web.xml``. Only Java system properties (``-D``) and OS environment variables are supported as overrides.
133+
134+
Increasing these values is useful when seeding many layers in parallel, especially in combination with a larger HTTP connection pool to the backend WMS.
135+
136+
100137
Resource Allocation
101138
-------------------
102139

‎geowebcache/core/src/main/java/org/geowebcache/config/GeoWebCacheConfiguration.java‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,10 @@ public class GeoWebCacheConfiguration {
4646
/* Default values */
4747
private Integer backendTimeout;
4848

49+
private Integer seederCorePoolSize;
50+
51+
private Integer seederMaxPoolSize;
52+
4953
private String lockProvider;
5054

5155
private transient LockProvider lockProviderInstance;
@@ -128,6 +132,26 @@ public void setBackendTimeout(Integer backendTimeout) {
128132
this.backendTimeout = backendTimeout;
129133
}
130134

135+
/** @see ServerConfiguration#getSeederCorePoolSize() */
136+
public Integer getSeederCorePoolSize() {
137+
return seederCorePoolSize;
138+
}
139+
140+
/** @param seederCorePoolSize the core pool size for the seeder thread pool */
141+
public void setSeederCorePoolSize(Integer seederCorePoolSize) {
142+
this.seederCorePoolSize = seederCorePoolSize;
143+
}
144+
145+
/** @see ServerConfiguration#getSeederMaxPoolSize() */
146+
public Integer getSeederMaxPoolSize() {
147+
return seederMaxPoolSize;
148+
}
149+
150+
/** @param seederMaxPoolSize the maximum pool size for the seeder thread pool */
151+
public void setSeederMaxPoolSize(Integer seederMaxPoolSize) {
152+
this.seederMaxPoolSize = seederMaxPoolSize;
153+
}
154+
131155
/** @return the cacheBypassAllowed */
132156
public Boolean getCacheBypassAllowed() {
133157
return cacheBypassAllowed;

‎geowebcache/core/src/main/java/org/geowebcache/config/ServerConfiguration.java‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,4 +103,23 @@ public interface ServerConfiguration extends BaseConfiguration {
103103

104104
/** The version number should match the XSD namespace and the version of GWC */
105105
String getVersion();
106+
107+
/**
108+
* The core pool size for the seeder thread pool. This is the number of threads to keep in the pool, even if they
109+
* are idle.
110+
*
111+
* @return the configured core pool size, or {@code null} if not set (defaults to 16)
112+
*/
113+
default Integer getSeederCorePoolSize() {
114+
return null;
115+
}
116+
117+
/**
118+
* The maximum pool size for the seeder thread pool. This is the maximum number of threads allowed in the pool.
119+
*
120+
* @return the configured max pool size, or {@code null} if not set (defaults to 32)
121+
*/
122+
default Integer getSeederMaxPoolSize() {
123+
return null;
124+
}
106125
}

‎geowebcache/core/src/main/java/org/geowebcache/seed/SeederThreadPoolExecutor.java‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,108 @@ public class SeederThreadPoolExecutor extends ThreadPoolExecutor implements Disp
2828

2929
private static final ThreadFactory tf = new CustomizableThreadFactory("GWC Seeder Thread-");
3030

31+
/**
32+
* Environment variable / system property name for configuring the core pool size. Looked up from Java system
33+
* properties first, then from OS environment variables. If neither is set or the value is not a valid positive
34+
* integer, the {@code corePoolSize} constructor argument is used as the default.
35+
*/
36+
public static final String GWC_SEEDER_CORE_POOL_SIZE = "GWC_SEEDER_CORE_POOL_SIZE";
37+
38+
/**
39+
* Environment variable / system property name for configuring the maximum pool size. Looked up from Java system
40+
* properties first, then from OS environment variables. If neither is set or the value is not a valid positive
41+
* integer, the {@code maxPoolSize} constructor argument is used as the default.
42+
*/
43+
public static final String GWC_SEEDER_MAX_POOL_SIZE = "GWC_SEEDER_MAX_POOL_SIZE";
44+
45+
/**
46+
* Holds the resolved pool sizes computed once during construction. This avoids resolving each property multiple
47+
* times (which would cause duplicate log messages) while working within the constraint that {@code super()} must be
48+
* the first statement in the constructor.
49+
*/
50+
private static final ThreadLocal<int[]> RESOLVED_SIZES = new ThreadLocal<>();
51+
3152
public SeederThreadPoolExecutor(int corePoolSize, int maxPoolSize) {
32-
super(corePoolSize, maxPoolSize, 60, TimeUnit.SECONDS, new LinkedBlockingQueue<>(), tf);
53+
super(
54+
resolveAndCacheSizes(corePoolSize, maxPoolSize)[0],
55+
getCachedMaxPoolSize(),
56+
60,
57+
TimeUnit.SECONDS,
58+
new LinkedBlockingQueue<>(),
59+
tf);
60+
RESOLVED_SIZES.remove();
61+
log.info("Seeder thread pool initialized with corePoolSize="
62+
+ getCorePoolSize()
63+
+ ", maxPoolSize="
64+
+ getMaximumPoolSize());
65+
}
66+
67+
/**
68+
* Resolves both pool sizes once, validates the core <= max constraint, caches the result in a ThreadLocal for the
69+
* second argument to {@code super()}, and returns the array.
70+
*/
71+
private static int[] resolveAndCacheSizes(int defaultCore, int defaultMax) {
72+
int core = resolvePoolSize(GWC_SEEDER_CORE_POOL_SIZE, defaultCore);
73+
int max = resolvePoolSize(GWC_SEEDER_MAX_POOL_SIZE, defaultMax);
74+
if (core > max) {
75+
log.warning(GWC_SEEDER_CORE_POOL_SIZE
76+
+ " ("
77+
+ core
78+
+ ") is greater than "
79+
+ GWC_SEEDER_MAX_POOL_SIZE
80+
+ " ("
81+
+ max
82+
+ "), adjusting maxPoolSize to match corePoolSize");
83+
max = core;
84+
}
85+
int[] sizes = {core, max};
86+
RESOLVED_SIZES.set(sizes);
87+
return sizes;
88+
}
89+
90+
/** Retrieves the cached max pool size computed by {@link #resolveAndCacheSizes}. */
91+
private static int getCachedMaxPoolSize() {
92+
return RESOLVED_SIZES.get()[1];
93+
}
94+
95+
/**
96+
* Resolves a pool size configuration value by looking up the given property name first as a Java system property,
97+
* then as an OS environment variable. Falls back to the provided default if neither is set or the value is not a
98+
* valid positive integer.
99+
*
100+
* @param propertyName the system property / environment variable name to look up
101+
* @param defaultValue the fallback value if the property is not set or invalid
102+
* @return the resolved pool size
103+
*/
104+
static int resolvePoolSize(String propertyName, int defaultValue) {
105+
String value = System.getProperty(propertyName);
106+
if (value == null || value.isBlank()) {
107+
value = System.getenv(propertyName);
108+
}
109+
if (value != null && !value.isBlank()) {
110+
try {
111+
int parsed = Integer.parseInt(value.trim());
112+
if (parsed > 0) {
113+
log.info("Using configured value for " + propertyName + "=" + parsed);
114+
return parsed;
115+
} else {
116+
log.warning("Invalid value for "
117+
+ propertyName
118+
+ "="
119+
+ value
120+
+ " (must be a positive integer), using default "
121+
+ defaultValue);
122+
}
123+
} catch (NumberFormatException e) {
124+
log.warning("Invalid value for "
125+
+ propertyName
126+
+ "="
127+
+ value
128+
+ " (not a valid integer), using default "
129+
+ defaultValue);
130+
}
131+
}
132+
return defaultValue;
33133
}
34134

35135
/**

‎geowebcache/core/src/main/resources/geowebcache.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@
44
xsi:schemaLocation="http://geowebcache.org/schema/1.28.0 http://geowebcache.org/schema/1.28.0/geowebcache.xsd">
55
<version>1.8.0</version>
66
<backendTimeout>120</backendTimeout>
7+
<!-- Seeder thread pool configuration (optional, can be overridden by
8+
GWC_SEEDER_CORE_POOL_SIZE and GWC_SEEDER_MAX_POOL_SIZE env vars) -->
9+
<!-- <seederCorePoolSize>16</seederCorePoolSize> -->
10+
<!-- <seederMaxPoolSize>32</seederMaxPoolSize> -->
711
<serviceInformation>
812
<title>GeoWebCache</title>
913
<description>GeoWebCache is an advanced tile cache for WMS servers. It supports a large variety of protocols and

‎geowebcache/core/src/main/resources/org/geowebcache/config/geowebcache.xsd‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,26 @@
3838
</xs:documentation>
3939
</xs:annotation>
4040
</xs:element>
41+
<xs:element name="seederCorePoolSize" type="xs:positiveInteger" minOccurs="0">
42+
<xs:annotation>
43+
<xs:documentation xml:lang="en">
44+
The number of threads to keep in the seeder thread pool,
45+
even if they are idle. Defaults to 16.
46+
Can be overridden by the GWC_SEEDER_CORE_POOL_SIZE
47+
system property or environment variable.
48+
</xs:documentation>
49+
</xs:annotation>
50+
</xs:element>
51+
<xs:element name="seederMaxPoolSize" type="xs:positiveInteger" minOccurs="0">
52+
<xs:annotation>
53+
<xs:documentation xml:lang="en">
54+
The maximum number of threads allowed in the seeder
55+
thread pool. Defaults to 32.
56+
Can be overridden by the GWC_SEEDER_MAX_POOL_SIZE
57+
system property or environment variable.
58+
</xs:documentation>
59+
</xs:annotation>
60+
</xs:element>
4161
<xs:element name="lockProvider" type="xs:string" minOccurs="0">
4262
<xs:annotation>
4363
<xs:documentation xml:lang="en">

0 commit comments

Comments
 (0)