Use JSON-format structured logging in production (#882)

## Status

- Related to
RaspberryPiFoundation/digital-editor-issues#1455

## What's changed?

- Adds the Semantic Logging gem and uses it to make log messages more
useful. See a comparison
[here](https://logger.rocketjob.io/rails.html#configuration).
    - This includes controller and action names by default.
- In the production environment only, logs are structured in JSON-format
to make filtering easier in BetterStack.
- Example: [filtering down to GET requests on the
FeaturesController](https://telemetry.betterstack.com/team/t62404/tail?s=l314881&q=message.payload.controller%3D%22Api%3A%3AFeaturesController%22%20message.payload.method%3D%22GET%22&a=1781695637116522.107740926).
- _The change in `application.rb` looks like it also enables JSON
logging in non-prod environments, however this only applies to the log
files written to disk, not the logging output in standard out._
- Disables Flipper logging, which was at DEBUG level with no clear way
of changing the level.
 
## Steps to perform after deploying to production

None.

Author: mwtrew

Gemfile

@@ -43,6 +43,7 @@ gem 'puma', '~> 8.0'
 gem 'rack_content_type_default', '~> 1.1'
 gem 'rack-cors'
 gem 'rails', '~> 8.1.3'
+gem 'rails_semantic_logger', '~> 4.20'
 gem 'ruby-progressbar', '~> 1.13', require: false
 gem 'ruby-vips'
 gem 'sentry-rails'

Gemfile.lock

@@ -403,6 +403,10 @@ GEM
     rails-html-sanitizer (1.7.0)
       loofah (~> 2.25)
       nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
+    rails_semantic_logger (4.20.0)
+      rack
+      railties (>= 5.1)
+      semantic_logger (~> 4.16)
     railties (8.1.3)
       actionpack (= 8.1.3)
       activesupport (= 8.1.3)
@@ -513,6 +517,8 @@ GEM
       childprocess (>= 0.5, < 5.0)
       rexml (~> 3.2, >= 3.2.5)
       rubyzip (>= 1.2.2)
+    semantic_logger (4.18.0)
+      concurrent-ruby (~> 1.0)
     sentry-rails (6.6.0)
       railties (>= 5.2.0)
       sentry-ruby (~> 6.6.0)
@@ -618,6 +624,7 @@ DEPENDENCIES
   rack_content_type_default (~> 1.1)
   rails (~> 8.1.3)
   rails-erd
+  rails_semantic_logger (~> 4.20)
   rspec
   rspec-rails
   rspec_junit_formatter

config/application.rb

@@ -73,5 +73,8 @@ class Application < Rails::Application
 
     config.x.cloudflare_turnstile.secret_key = ENV.fetch('CLOUDFLARE_TURNSTILE_SECRET_KEY', nil)
     config.x.cloudflare_turnstile.enabled = ENV['CLOUDFLARE_TURNSTILE_SECRET_KEY'].present?
+
+    config.rails_semantic_logger.format = :json
+    config.semantic_logger.application = 'editor-api'
   end
 end

config/environments/production.rb

@@ -45,10 +45,13 @@
   # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
   # config.force_ssl = true
 
-  # Log to STDOUT by default
-  config.logger = ActiveSupport::Logger.new($stdout)
-                                       .tap  { |logger| logger.formatter = Logger::Formatter.new }
-                                       .then { |logger| ActiveSupport::TaggedLogging.new(logger) }
+  # Log to STDOUT on Heroku in JSON format, where this variable is set automatically.
+  if ENV['RAILS_LOG_TO_STDOUT'].present?
+    $stdout.sync = true
+    config.rails_semantic_logger.add_file_appender = false
+    config.semantic_logger.add_appender(io: $stdout, formatter: :json)
+    config.semantic_logger.application = "editor-api@#{ENV['HEROKU_SLUG_COMMIT'] || 'unknown'}"
+  end
 
   # Prepend all log lines with the following tags.
   config.log_tags = [:request_id]

config/initializers/flipper.rb

@@ -18,7 +18,7 @@
   # config.flipper.strict = Rails.env.development? && :warn
 
   ## Show Flipper checks in logs
-  # config.flipper.log = true
+  config.flipper.log = false
 
   ## Reconfigure Flipper to use the Memory adapter and disable Cloud in tests
   # config.flipper.test_help = true