Simeplcov on CircleCI 2.0

[jpparsons]

Hi,

I have been using simeplecov-parallel with CircleCI 1.0 parallel nodes for a while and it works great. Iv'e recently started looking at CircleCI 2.0. Not sure if this is supported yet. In any case, when merging the result sets between nodes (docker containers), the ssh is failing.

Merging SimpleCov result into the master node...
Syncing CircleCI nodes...
Waiting for slave nodes to be ready for download...
ssh: Could not resolve hostname node1: Name or service not known

In CircleCI 1.0, ssh keypairs would setup by CircleCI container startup:
Adding per build SSH keypair
Add containers to ~/.ssh/config as node0, node1, node2, node3, node4, node5, node6, node7

No similar configuration setup in Circle 2.0.

Would performing a similar SSH keypair setup (manually in my circleci config) work? Or any other suggestions.

I can provide more detail if needed.

Thanks in advance,

John

[yujinakayama]

CircleCI 2.0 is not yet supported, though we'll support it in the future.

[mohjay]

Would love Circle 2.0 support too

[usmanasif]

CircleCI 2.0 is a default preference for people now. So please add its support on priority.
Thanks.

[jpparsons]

In case this helps anyone, here is how I got simplecov to work in CircleCI 2.0. Iv'e been using this for a few months now.

I basically have a rake task that is called from the circle config to merge the result set. Ive left out details that are not specific to this setup.

In config.yml there is a job step run after rspec tests that manually copies each result set file to a common folder on each parallel node and then saves it to a workspace so a downstream job can load it up. The coverage job calls the rake task that merges the result set and then saves it to an artifacts folder if you want to look at the files later after a build.

In config.yml

      steps:
        - checkout
        - run:
            name: Run rspec tests
 ...etc
        - run:
            name: Stash Coverage Results
            command: |
              mkdir coverage_results
              cp -R coverage/.resultset.json coverage_results/.resultset-${CIRCLE_NODE_INDEX}.json
        - persist_to_workspace:
            root: .
            paths:
              - coverage_results

coverage:
      steps:
        - checkout
   
        - attach_workspace:
            at: .

        - run:
            name: Merge and check coverage
            command: |
              RUN_COVERAGE=true bundle exec rake simplecov:report_coverage

        - store_artifacts:
            path: ~/productplan/coverage
            destination: coverage

workflows:
  version: 2
  build_and_test:
    jobs:
        - rspec
        - coverage:
             requires:
               - rspec

In circleci config.yml I use workflows, so coverage does not run until rspec successfully runs

I have a rake task called simplecov.rake

if Rails.env.test?
  require_relative "../../spec/simple_cov_helpers"
  namespace :simplecov do
    desc "merge_results"
    task report_coverage: :environment do
      SimpleCovHelpers.report_coverage
    end
  end
end

Here is simple_cov_helpers

require "simplecov"

class SimpleCovHelpers
  SKIPPED_FILES = %w(/app/controllers....etc.) # MODIFY WITH LIST OF FILES YOU WANT EXCLUDED FROM COVERAGE

  class << self
    def report_coverage(base_dir: "./coverage_results")
      setup_coverage
      SimpleCov.start 'rails'
      new(base_dir: base_dir).merge_results
    end

    def setup_coverage
      check_coverage = ENV.fetch("SKIP_COVERAGE_CHECK", "false") == "false"
      SimpleCov.minimum_coverage(86) if check_coverage
      SimpleCov.merge_timeout(3600)

      SKIPPED_FILES.each do |file|
        SimpleCov.add_filter(file)
      end

      Dir.glob(File.join(".", "app", "*")).select { |f| File.directory?(f) && (f !~ /(admin|assets)\z/) }.each do |dirname|
        item = File.basename(dirname)
        next if item == "admin"
        SimpleCov.add_group item.capitalize, "app/#{item}"
      end
    end
  end

  attr_reader :base_dir

  def initialize(base_dir:)
    @base_dir = base_dir
  end

  def all_results
    Dir["#{base_dir}/.resultset*.json"]
  end

  def merge_results
    all_results.each do |result_file_name|
      Rails.logger.info "Processing #{result_file_name}"
      result = SimpleCov::Result.from_hash(JSON.parse(File.read(result_file_name)))
      SimpleCov::ResultMerger.store_result(result)
    end
  end
end

[trev]

Thanks @jpparsons - it didn't work for me but it set me in the right direction. For anybody who's interested in a working config as of today: https://gist.github.com/trev/9cc964a54c8d5b62f4def891eba6b976

[ziaulrehman40]

Thanks @jpparsons and @trev i used your guys's solution and have got it working.
For anyone who want to use this, i should mention that in workflows there does not seem to be any option to rerun without cache. Which is needed sometimes. Beware of this before using workflows

[ziaulrehman40]

For anyone looking at this, here is a detailed gist for this issue, covering two approaches i was able to get working with help from different resources.

[kylejginavan]

Any reason these solutions can not be merged into the gem?

[rahul342]

Need to rename the command_name to make this work (per the simplecov README to avoid clobbering). The following worked for me -

require 'simplecov'

class SimpleCovHelper
  class << self
    def report_coverage(base_dir: './coverage_results')
      SimpleCov.start 'rails' do
        add_filter '/bin/'
        add_filter '/db/'
        add_filter '/spec/' # for rspec
        add_filter '/test/' # for minitest
        add_filter '/proto/' # for protobuff
        merge_timeout(3600)
      end
      new(base_dir: base_dir).merge_results
    end
  end

  attr_reader :base_dir

  def initialize(base_dir:)
    @base_dir = base_dir
  end

  def all_results
    Dir["#{base_dir}/.resultset*.json"]
  end

  def merge_results
    results = all_results.each_with_index.map do |file, i|
      puts "processing #{file}"
      r = SimpleCov::Result.from_hash(JSON.parse(File.read(file)))
      r.command_name = "command-#{i}"
      r
    end
    merged_result = SimpleCov::ResultMerger.merge_results(*results)
    merged_result.command_name = 'circlci-rspec'
    SimpleCov::ResultMerger.store_result(merged_result)
  end
end

SimpleCovHelper.report_coverage

Also you don't need rake task or evenbundle install to run this. Simply -

 coverage:
    docker:
      - image: circleci/ruby:2.6.2
    working_directory: ~/repo
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run:
          name: Merge and upload coverage
          command: |
            gem install simplecov
            ruby spec/simple_cov_helper.rb
      - store_artifacts:
          path: coverage

should do and is pretty fast.