Add Version types tags and branches

[taylorsilva]

This is a massive change to the git resource. I am expanding the scope and workflows supported by this resource to support:

• Returning a list of branches that exist in a repo
• Returning tags that exist in a repo

This change is backwards-compatible. Existing usage of this resource is NOT effected by this change. Your pipelines do not need to change and this should not break anything for anyone.
See the updated README.md for details.

TLDR: added a version_type key with values commits,tags,branches. Default is commits and is existing behaviour of the resource.

Why version_type as the key?: I wanted to avoid confusion with the type key that's already present in the resource.

Motivation

This issue kicked me off down this rabbit hole: concourse/examples#28

There have been resource types that solve these problems already. For getting branches of a repo https://github.com/aoldershaw/git-branches-resource was a resource type made by a previous maintainer. This resource is basically a copy of this resource's auth features with some other scripts bolted on.

Pretty much any resource that does things with git would start with THIS resource's scripts and build off from there. Of course those resources would go stale and drift behind this resource.

I decided to consolidate the community's effort around getting tags and branches into this resource. I've been on teams that would have really benefited from this being built into the resource.

Getting tags and branches are both light-weight operations. With branches we use git ls-remote and with tags we create an empty repo and only pull down refs/tags/*.

Examples

Tracking tags:

resource_types:
  - name: git
    type: registry-image
    source:
      repository: docker.io/taylorsilva/git-resource
      tag: branches

resources:
  - name: tags
    type: git
    source:
      version_type: tags
      uri: https://github.com/golang/go.git
      tag_sort: semver
      tag_regex: "go1\\.[0-9]+\\.[0-9]+"

jobs:
  - name: tags
    plan:
      - get: tags

Tracking branches:

resource_types:
  - name: git
    type: registry-image
    source:
      repository: docker.io/taylorsilva/git-resource
      tag: branches

resources:
  - name: branches
    type: git
    source:
      version_type: branches
      uri: https://github.com/golang/go.git
      branch_filters:
        - "release/*"

jobs:
  - name: branches
    plan:
      - get: branches

About tags...

Technically you can already pull down tags with the current release of the resource. There's already tag_filter and tag_regex to support filtering tags. I have personally found this feature to not work as well as I would hope.

Take this resource as an example where I want to track tags published to the golang/go repo for new go releases:

- name: tags
  type: git
  source:
    uri: https://github.com/golang/go.git
    tag_regex: "go1\\.[0-9]+\\.[0-9]+"

First, the initial cloning takes MINUTES because the resource pulls down a bunch of info that isn't related to finding tags.
Once the repo is cloned in the check step, it doesn't find all the tags! The first check returns only one tag:

This happens because 1) large repo, too much commit history for the resource to go through, so it only finds the most recent tag. Smaller repos will return more tags, 2) because we search a single branch we might miss tags. A common practice I've seen is for projects to create the tag on a commit that exists on a non-main branch. This resource struggles to find tags like that.

Overall I've found the tagging finding experience of this resource to just suck. It's slow and inaccurate.

Now, if you set version_type: tags, we actually do find all the tags and it's basically instant!

  - name: tags-new
    type: git
    source:
      version_type: tags
      uri: https://github.com/golang/go.git
      tag_sort: semver
      tag_regex: "go1\\.[0-9]+\\.[0-9]+"

Runs in seconds and returns all the tags:

I currently have it coded to checkout the tag in a detached HEAD state. Hopefully this is fine for others use-cases.

[taylorsilva]

I'm going to leave this open for a bit for folks to give feedback.