Skip to content

Commit 6007aa4

Browse files
authored
Run Solid Queue in async mode on Render free tier (#23)
* Run Solid Queue in async mode on Render free tier Companion to #22 (WEB_CONCURRENCY=0). Free tier (512MB) can't fit SQ's default fork mode, which spawns supervisor + worker + dispatcher + scheduler subprocesses for ~460MB of RSS overhead on top of Puma. The container OOM-cycles every ~7-15 min — the deploy still looks "live" but HTTP returns 502s during each restart window. `solid_queue_mode :async` (documented switch in lib/puma/plugin/solid_queue.rb of the gem) tells the plugin to run worker/dispatcher/scheduler as threads inside the Puma master process. The 4 SQ subprocesses collapse into ~50MB of thread overhead in Puma. Container RSS drops from ~530MB (cycling into OOM) to a flat ~300MB. Trade-off per SQ's own docs ("Only use async if you know what you're doing and have strong reasons to"): - Less isolation between SQ and Puma. A leaky/hung SQ thread affects request serving. For low-traffic student projects this is the right call; revisit if upgrading to a paid plan. - No inter-process parallelism for jobs. Free tier is already 0.1 vCPU, so this isn't real lost throughput. - Recurring tasks and concurrency controls still work unchanged. The DB pool bump (5 → 8) is non-optional: previously the pool was sized for Puma's 3 request threads only; now the same process also runs 3 SQ worker threads + dispatcher + scheduler. Under any load the old pool=5 would throw ConnectionTimeoutError. The 5 -> 8 default still respects DB_POOL env override for projects that need finer control. Diagnosed and validated on raghubetina/aplace: raghubetina/aplace#58 — RSS metric showed the predicted plateau at ~300MB with zero growth after deploy. * Pin solid_queue ~> 1.3 for solid_queue_mode :async support This branch's puma.rb adds `solid_queue_mode :async if ENV["SOLID_QUEUE_IN_PUMA"]`, but the lockfile resolved to solid_queue 1.1.4, which has no such DSL method. Booting in production crashes immediately with: config/puma.rb:47:in 'Puma::DSL#_load_from': undefined method 'solid_queue_mode' for an instance of Puma::DSL (NoMethodError) Async supervisor mode was re-introduced in solid_queue 1.3.0 (rails/solid_queue#644); the earlier 0.4 implementation was removed in 0.7. Pinning to ~> 1.3 documents the minimum that matches this branch's puma.rb and updates the lockfile to 1.4.0. Caught by deploying a marketplace smoke-test app built from this branch + #22 + #24 to Render free tier.
1 parent 8899422 commit 6007aa4

4 files changed

Lines changed: 52 additions & 37 deletions

File tree

Gemfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ gem "importmap-rails" # JavaScript with ESM import maps
88
gem "turbo-rails" # Hotwire page acceleration (SPA-like)
99
gem "stimulus-rails" # Hotwire JavaScript framework
1010
gem "solid_cache" # Database-backed Rails.cache
11-
gem "solid_queue" # Database-backed Active Job
11+
gem "solid_queue", "~> 1.3" # Database-backed Active Job — 1.3+ required for solid_queue_mode :async in config/puma.rb
1212
gem "solid_cable" # Database-backed Action Cable
1313
gem "bootsnap", require: false # Faster boot times via caching
1414
gem "thruster", require: false # HTTP caching/compression for Puma

Gemfile.lock

Lines changed: 35 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ GEM
8686
erubi (>= 1.0.0)
8787
rack (>= 0.9.0)
8888
rouge (>= 1.0.0)
89-
bigdecimal (4.1.0)
89+
bigdecimal (4.1.2)
9090
bindex (0.8.1)
9191
binding_of_caller (2.0.0)
9292
debug_inspector (>= 1.2.0)
@@ -131,8 +131,9 @@ GEM
131131
devise
132132
indefinite_article
133133
drb (2.2.3)
134+
erb (6.0.4)
134135
erubi (1.13.1)
135-
et-orbi (1.2.11)
136+
et-orbi (1.4.0)
136137
tzinfo
137138
faraday (2.12.2)
138139
faraday-net_http (>= 2.0, < 3.5)
@@ -154,10 +155,10 @@ GEM
154155
ffi-compiler (1.3.2)
155156
ffi (>= 1.15.5)
156157
rake
157-
fugit (1.11.1)
158-
et-orbi (~> 1, >= 1.2.11)
158+
fugit (1.12.1)
159+
et-orbi (~> 1.4)
159160
raabro (~> 1.4)
160-
globalid (1.2.1)
161+
globalid (1.3.0)
161162
activesupport (>= 6.1)
162163
grade_runner (0.0.16)
163164
activesupport (>= 2.3.5)
@@ -188,8 +189,9 @@ GEM
188189
indefinite_article (0.2.5)
189190
activesupport
190191
io-console (0.8.2)
191-
irb (1.15.1)
192+
irb (1.18.0)
192193
pp (>= 0.6.0)
194+
prism (>= 1.3.0)
193195
rdoc (>= 4.0.0)
194196
reline (>= 0.4.2)
195197
json (2.19.3)
@@ -211,7 +213,7 @@ GEM
211213
ffi-compiler (~> 1.0)
212214
rake (~> 13.0)
213215
logger (1.7.0)
214-
loofah (2.24.0)
216+
loofah (2.25.1)
215217
crass (~> 1.0.2)
216218
nokogiri (>= 1.12.0)
217219
mail (2.8.1)
@@ -223,8 +225,7 @@ GEM
223225
matrix (0.4.2)
224226
method_source (1.1.0)
225227
mini_mime (1.1.5)
226-
mini_portile2 (2.8.9)
227-
minitest (6.0.2)
228+
minitest (6.0.6)
228229
drb (~> 2.0)
229230
prism (~> 1.5)
230231
msgpack (1.8.0)
@@ -240,24 +241,21 @@ GEM
240241
net-smtp (0.5.1)
241242
net-protocol
242243
nio4r (2.7.4)
243-
nokogiri (1.18.4)
244-
mini_portile2 (~> 2.8.2)
244+
nokogiri (1.19.3-aarch64-linux-gnu)
245245
racc (~> 1.4)
246-
nokogiri (1.18.4-aarch64-linux-gnu)
246+
nokogiri (1.19.3-aarch64-linux-musl)
247247
racc (~> 1.4)
248-
nokogiri (1.18.4-aarch64-linux-musl)
248+
nokogiri (1.19.3-arm-linux-gnu)
249249
racc (~> 1.4)
250-
nokogiri (1.18.4-arm-linux-gnu)
250+
nokogiri (1.19.3-arm-linux-musl)
251251
racc (~> 1.4)
252-
nokogiri (1.18.4-arm-linux-musl)
252+
nokogiri (1.19.3-arm64-darwin)
253253
racc (~> 1.4)
254-
nokogiri (1.18.4-arm64-darwin)
254+
nokogiri (1.19.3-x86_64-darwin)
255255
racc (~> 1.4)
256-
nokogiri (1.18.4-x86_64-darwin)
256+
nokogiri (1.19.3-x86_64-linux-gnu)
257257
racc (~> 1.4)
258-
nokogiri (1.18.4-x86_64-linux-gnu)
259-
racc (~> 1.4)
260-
nokogiri (1.18.4-x86_64-linux-musl)
258+
nokogiri (1.19.3-x86_64-linux-musl)
261259
racc (~> 1.4)
262260
octokit (5.6.1)
263261
faraday (>= 1, < 3)
@@ -293,13 +291,13 @@ GEM
293291
nio4r (~> 2.0)
294292
raabro (1.4.0)
295293
racc (1.8.1)
296-
rack (3.2.5)
297-
rack-session (2.1.0)
294+
rack (3.2.6)
295+
rack-session (2.1.2)
298296
base64 (>= 0.1.0)
299297
rack (>= 3.0.0)
300298
rack-test (2.2.0)
301299
rack (>= 1.3)
302-
rackup (2.2.1)
300+
rackup (2.3.1)
303301
rack (>= 3)
304302
rails (8.0.2)
305303
actioncable (= 8.0.2)
@@ -319,12 +317,12 @@ GEM
319317
actionpack (>= 5.0.1.rc1)
320318
actionview (>= 5.0.1.rc1)
321319
activesupport (>= 5.0.1.rc1)
322-
rails-dom-testing (2.2.0)
320+
rails-dom-testing (2.3.0)
323321
activesupport (>= 5.0.0)
324322
minitest
325323
nokogiri (>= 1.6)
326-
rails-html-sanitizer (1.6.2)
327-
loofah (~> 2.21)
324+
rails-html-sanitizer (1.7.0)
325+
loofah (~> 2.25)
328326
nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
329327
rails_db (2.5.0)
330328
activerecord
@@ -343,15 +341,17 @@ GEM
343341
thor (~> 1.0, >= 1.2.2)
344342
zeitwerk (~> 2.6)
345343
rainbow (3.1.1)
346-
rake (13.3.1)
344+
rake (13.4.2)
347345
ransack (4.3.0)
348346
activerecord (>= 6.1.5)
349347
activesupport (>= 6.1.5)
350348
i18n
351-
rdoc (6.12.0)
349+
rdoc (7.2.0)
350+
erb
352351
psych (>= 4.0.0)
352+
tsort
353353
regexp_parser (2.11.3)
354-
reline (0.6.0)
354+
reline (0.6.3)
355355
io-console (~> 0.5)
356356
responders (3.1.1)
357357
actionpack (>= 5.2)
@@ -438,13 +438,13 @@ GEM
438438
activejob (>= 7.2)
439439
activerecord (>= 7.2)
440440
railties (>= 7.2)
441-
solid_queue (1.1.4)
441+
solid_queue (1.4.0)
442442
activejob (>= 7.1)
443443
activerecord (>= 7.1)
444444
concurrent-ruby (>= 1.3.1)
445-
fugit (~> 1.11.0)
445+
fugit (~> 1.11)
446446
railties (>= 7.1)
447-
thor (~> 1.3.1)
447+
thor (>= 1.3.1)
448448
stimulus-rails (1.3.4)
449449
railties (>= 6.0.0)
450450
stringio (3.2.0)
@@ -453,7 +453,7 @@ GEM
453453
unicode-display_width (~> 3.0)
454454
terminal-table (4.0.0)
455455
unicode-display_width (>= 1.1.1, < 4)
456-
thor (1.3.2)
456+
thor (1.5.0)
457457
thruster (0.1.12)
458458
thruster (0.1.12-aarch64-linux)
459459
thruster (0.1.12-arm64-darwin)
@@ -490,7 +490,7 @@ GEM
490490
websocket-extensions (0.1.5)
491491
xpath (3.2.0)
492492
nokogiri (~> 1.8)
493-
zeitwerk (2.7.2)
493+
zeitwerk (2.8.0)
494494
zip (2.0.2)
495495

496496
PLATFORMS
@@ -541,7 +541,7 @@ DEPENDENCIES
541541
shoulda-matchers (~> 7.0)
542542
solid_cable
543543
solid_cache
544-
solid_queue
544+
solid_queue (~> 1.3)
545545
stimulus-rails
546546
thruster
547547
tsort

config/database.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,12 @@ default: &default
1717
encoding: unicode
1818
# For details on connection pooling, see Rails configuration guide
1919
# https://guides.rubyonrails.org/configuring.html#database-pooling
20-
pool: <%= ENV.fetch("DB_POOL") { 5 } %>
20+
#
21+
# Sized for Puma's 3 request threads + Solid Queue's async-mode threads
22+
# (3 worker + dispatcher + scheduler) sharing this pool. The previous
23+
# default of 5 was right at the edge and produced ConnectionTimeoutError
24+
# under any load once SQ threads moved into the Puma process.
25+
pool: <%= ENV.fetch("DB_POOL") { 8 } %>
2126

2227

2328
development:

config/puma.rb

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,16 @@
3636
# Run the Solid Queue supervisor inside of Puma for single-server deployments
3737
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"]
3838

39+
# Run Solid Queue in async mode: worker/dispatcher/scheduler run as threads
40+
# inside the Puma master process instead of being forked into 4 subprocesses.
41+
# Required on Render's 512MB free plan — fork mode adds ~460MB of process
42+
# overhead (supervisor + 3 children) and pushes the container into OOM-loop.
43+
# Async mode collapses that to ~50MB of additional thread overhead in Puma
44+
# master. Trade-off: less isolation between SQ threads and Puma — a hung or
45+
# leaky SQ thread affects request serving. For low-traffic student projects
46+
# the trade-off is right; revisit if upgrading to a paid plan with more RAM.
47+
solid_queue_mode :async if ENV["SOLID_QUEUE_IN_PUMA"]
48+
3949
# Specify the PID file. Defaults to tmp/pids/server.pid in development.
4050
# In other environments, only set the PID file if requested.
4151
pidfile ENV["PIDFILE"] if ENV["PIDFILE"]

0 commit comments

Comments
 (0)