Improve Spring preloading (#4802)

- Load the "init_block" of spec_helper pre-fork when using Spring
  - Previously the init_block was only used by Spork.
  - Spring would run it on every fork. This made tests load slower and
    reduced the benefit of using Spring, since less was being preloaded
    than could be.
- I blindly copied over the current init/run blocks, without
  investigating in any depth. It's possible that more can be moved into
  the "init" method, to further reduce test load times when using
  Spring. This could be a worthwhile follow-up, especially if/when Spork
  is removed.
- Extract out a SpecHelperHelper module so the "init_block" can be
  shared between spec_helper and Spring
- Enforce that SpecHelperHelper.init only runs once. That way it doesn't
  run post-fork, which is a minor performance improvement and means we
  don't need to worry about everything in there being idempotent.
- Add documentation to spec/README

Author: Gerg

config/spring.rb

@@ -0,0 +1,2 @@
+require_relative '../spec/spec_helper_helper'
+SpecHelperHelper.init

spec/README.md

@@ -114,13 +114,55 @@ user        2.968       0.046       2.856       2.991       3.015
 sys         1.636       0.042       1.569       1.640       1.720
 ```
 
-## Running Tests In Preloaded (Fast) Mode:
+### Running Tests In Preloaded (Fast) Mode:
 
 Running unit tests is a good thing, but it can be annoying waiting for
 the ruby interpreter to load and then initialize `rspec` every single
 time you make a change. Fortunately, many other people have run into
-this same frustration and published their solutions to the problem. We
-use the `spork` library to speed up the `edit-run-fix` cycle.
+this same frustration and published their solutions to the problem. 
+
+#### Spring
+
+Rails Spring (not to be confused with the Java project with the same name) runs
+CC in a background process and then forks it every time you run tests. That
+means that everything loaded prior to the fork doesn't need to be re-loaded
+every time you run tests. This can speed up tests substantially.
+
+To use spring, run `./bin/rspec` in place of `rspec`. Spring should
+automatically watch and reload files, but you can manually stop it with
+`./bin/spring stop`. It will automatically start again the next time you run
+`./bin/rspec`.
+
+Example performance improvement:
+```sh
+❯ multitime -n 10 bundle exec rspec spec/unit/actions/app_create_spec.rb
+
+...
+
+===> multitime results
+1: bundle exec rspec spec/unit/actions/app_create_spec.rb
+            Mean        Std.Dev.    Min         Median      Max
+real        24.471      2.420       20.169      25.499      27.303
+user        4.320       0.255       3.761       4.354       4.642
+sys         2.514       0.158       2.206       2.562       2.698
+```
+
+```sh
+❯ multitime -n 10 bundle exec ./bin/rspec spec/unit/actions/app_create_spec.rb
+
+...
+
+===> multitime results
+1: bundle exec ./bin/rspec spec/unit/actions/app_create_spec.rb
+            Mean        Std.Dev.    Min         Median      Max
+real        18.628      2.077       13.934      19.062      21.821
+user        0.177       0.032       0.129       0.185       0.233
+sys         0.103       0.014       0.078       0.107       0.126
+```
+
+#### Spork (Legacy)
+
+Spork is an older implementation of the same "forking" strategy implemented by Spring.
 
 ### Running Individual Tests
 

spec/spec_helper.rb

@@ -1,6 +1,7 @@
 SPEC_HELPER_LOADED = true
 require 'rubygems'
 require 'mock_redis'
+require 'spec_helper_helper'
 
 begin
   require 'spork'
@@ -12,225 +13,18 @@
   run_spork = false
 end
 
-# --- Instructions ---
-# Sort the contents of this file into a Spork.prefork and a Spork.each_run
-# block.
-#
-# The Spork.prefork block is run only once when the spork server is started.
-# You typically want to place most of your (slow) initializer code in here, in
-# particular, require'ing any 3rd-party gems that you don't normally modify
-# during development.
-#
-# The Spork.each_run block is run each time you run your specs.  In case you
-# need to load files that tend to change during development, require them here.
-# With Rails, your application modules are loaded automatically, so sometimes
-# this block can remain empty.
-#
-# Note: You can modify files loaded *from* the Spork.each_run block without
-# restarting the spork server.  However, this file itself will not be reloaded,
-# so if you change any of the code inside the each_run block, you still need to
-# restart the server.  In general, if you have non-trivial code in this file,
-# it's advisable to move it into a separate file so you can easily edit it
-# without restarting spork.  (For example, with RSpec, you could move
-# non-trivial code into a file spec/support/my_helper.rb, making sure that the
-# spec/support/* files are require'd from inside the each_run block.)
-#
-# Any code that is left outside the two blocks will be run during preforking
-# *and* during each_run -- that's probably not what you want.
-#
-# These instructions should self-destruct in 10 seconds.  If they don't, feel
-# free to delete them.
-
-init_block = proc do
-  $LOAD_PATH.push(File.expand_path(__dir__))
-
-  require File.expand_path('../config/boot', __dir__)
-
-  if ENV['COVERAGE']
-    require 'simplecov'
-    SimpleCov.start do
-      add_filter '/spec/'
-      add_filter '/errors/'
-      add_filter '/docs/'
-    end
-  end
-  ENV['PB_IGNORE_DEPRECATIONS'] = 'true'
-  ENV['RAILS_ENV'] ||= 'test'
-
-  require 'machinist/sequel'
-  require 'machinist/object'
-  require 'rack/test'
-  require 'timecop'
-
-  require 'steno/steno'
-  require 'webmock/rspec'
-
-  require 'pry'
-
-  require 'cloud_controller'
-  require 'allowy/rspec'
-
-  require 'rspec_api_documentation'
-  require 'services'
-
-  require 'support/bootstrap/spec_bootstrap'
-  require 'rspec/collection_matchers'
-  require 'rspec/its'
-  require 'rspec/wait'
-end
-
-each_run_block = proc do
-  # Moving SpecBootstrap.init into the init-block means that changes in code files aren't detected.
-  VCAP::CloudController::SpecBootstrap.init(do_schema_migration: !ENV['NO_DB_MIGRATION'])
-
-  Dir[File.expand_path('support/**/*.rb', File.dirname(__FILE__))].each { |file| require file }
-
-  # each-run here?
-  RSpec.configure do |rspec_config|
-    rspec_config.filter_run_when_matching :focus
-
-    rspec_config.mock_with :rspec do |mocks|
-      mocks.verify_partial_doubles = true
-    end
-    rspec_config.filter_run_excluding :stepper
-    rspec_config.expose_dsl_globally = false
-    rspec_config.backtrace_exclusion_patterns = [%r{/gems/}, %r{/bin/rspec}]
-
-    rspec_config.expect_with(:rspec) do |config|
-      config.syntax = :expect
-      config.max_formatted_output_length = 1000
-    end
-    rspec_config.extend DeprecationHelpers
-    rspec_config.include Rack::Test::Methods
-    rspec_config.include ModelCreation
-    rspec_config.include TimeHelpers
-    rspec_config.include LinkHelpers
-    rspec_config.include BackgroundJobHelpers
-    rspec_config.include LogHelpers
-
-    rspec_config.include ServiceBrokerHelpers
-    rspec_config.include UserHelpers
-    rspec_config.include UserHeaderHelpers
-    rspec_config.include ControllerHelpers, type: :v2_controller, file_path: EscapedPath.join(%w[spec unit controllers])
-    rspec_config.include ControllerHelpers, type: :api
-    rspec_config.include ControllerHelpers, file_path: EscapedPath.join(%w[spec acceptance])
-    rspec_config.include RequestSpecHelper, file_path: EscapedPath.join(%w[spec acceptance])
-    rspec_config.include ControllerHelpers, file_path: EscapedPath.join(%w[spec request])
-    rspec_config.include RequestSpecHelper, file_path: EscapedPath.join(%w[spec request])
-    rspec_config.include LifecycleSpecHelper, file_path: EscapedPath.join(%w[spec request lifecycle])
-    rspec_config.include ApiDsl, type: :api
-    rspec_config.include LegacyApiDsl, type: :legacy_api
-
-    rspec_config.include IntegrationHelpers, type: :integration
-    rspec_config.include IntegrationHttp, type: :integration
-    rspec_config.include IntegrationSetupHelpers, type: :integration
-    rspec_config.include IntegrationSetup, type: :integration
-
-    rspec_config.include SpaceRestrictedResponseGenerators
-
-    rspec_config.before(:all) do
-      WebMock.disable_net_connect!(allow: %w[codeclimate.com fake.bbs])
-    end
-    rspec_config.before(:all, type: :integration) do
-      WebMock.allow_net_connect!
-      @uaa_server = FakeUAAServer.new(6789)
-      @uaa_server.start
-    end
-    rspec_config.before(:all, type: :migration) do
-      skip 'Skipped due to NO_DB_MIGRATION env variable being set' if ENV['NO_DB_MIGRATION']
-    end
-    rspec_config.after(:all, type: :integration) do
-      WebMock.disable_net_connect!(allow: %w[codeclimate.com fake.bbs])
-      @uaa_server.stop
-    end
-
-    rspec_config.before(:example, :log_db) do
-      db = DbConfig.new.connection
-      db.loggers << Logger.new($stdout)
-      db.sql_log_level = :info
-    end
-
-    rspec_config.example_status_persistence_file_path = 'spec/examples.txt'
-    rspec_config.expose_current_running_example_as :example # Can be removed when we upgrade to rspec 3
-
-    rspec_config.before :suite do
-      VCAP::CloudController::SpecBootstrap.seed
-      # We only want to load rake tasks once:
-      # calling this more than once will load tasks again and 'invoke' or 'execute' calls
-      # will call rake tasks multiple times
-      Application.load_tasks
-    end
-
-    rspec_config.before do
-      Delayed::Worker.destroy_failed_jobs = false
-      Sequel::Deprecation.output = StringIO.new
-      Sequel::Deprecation.backtrace_filter = 5
-
-      TestConfig.context = example.metadata[:job_context] || :api
-      TestConfig.reset
-
-      Fog::Mock.reset
-
-      if Fog.mock?
-        CloudController::DependencyLocator.instance.droplet_blobstore.ensure_bucket_exists
-        CloudController::DependencyLocator.instance.package_blobstore.ensure_bucket_exists
-        CloudController::DependencyLocator.instance.global_app_bits_cache.ensure_bucket_exists
-        CloudController::DependencyLocator.instance.buildpack_blobstore.ensure_bucket_exists
-      end
-
-      VCAP::CloudController::SecurityContext.clear
-      VCAP::Request.current_id = nil
-      allow_any_instance_of(VCAP::CloudController::UaaTokenDecoder).to receive(:uaa_issuer).and_return(UAAIssuer::ISSUER)
-
-      mock_redis = MockRedis.new
-      allow(Redis).to receive(:new).and_return(mock_redis)
-    end
-
-    rspec_config.around do |example|
-      isolation = DatabaseIsolation.choose(example.metadata[:isolation], DbConfig.new.connection)
-      isolation.cleanly { example.run }
-    end
-
-    rspec_config.after do
-      raise "Sequel Deprecation String found: #{Sequel::Deprecation.output.string}" unless Sequel::Deprecation.output.string == ''
-
-      Sequel::Deprecation.output.close unless Sequel::Deprecation.output.closed?
-    end
-
-    rspec_config.after :all do
-      TmpdirCleaner.clean
-    end
-
-    rspec_config.after do
-      Timecop.return
-    end
-
-    rspec_config.after(:each, type: :legacy_api) { add_deprecation_warning }
-
-    RspecApiDocumentation.configure do |c|
-      c.app = VCAP::CloudController::RackAppBuilder.new.build(TestConfig.config_instance,
-                                                              VCAP::CloudController::Metrics::RequestMetrics.new,
-                                                              VCAP::CloudController::Logs::RequestLogs.new(Steno.logger('request.logs')))
-      c.format = %i[html json]
-      c.api_name = 'Cloud Foundry API'
-      c.template_path = 'spec/api/documentation/templates'
-      c.curl_host = 'https://api.[your-domain.com]'
-    end
-  end
-end
-
 if run_spork
   Spork.prefork do
     # Loading more in this block will cause your tests to run faster. However,
     # if you change any configuration or code from libraries loaded here, you'll
     # need to restart spork for it to take effect.
-    init_block.call
+    SpecHelperHelper.init
   end
   Spork.each_run do
     # This code will be run each time you run your specs.
-    each_run_block.call
+    SpecHelperHelper.each_run
   end
 else
-  init_block.call
-  each_run_block.call
+  SpecHelperHelper.init
+  SpecHelperHelper.each_run
 end