|
| 1 | +--- |
| 2 | +layout: single |
| 3 | +title: "Integrate your CI" |
| 4 | +sidebar: |
| 5 | + nav: guides |
| 6 | +--- |
| 7 | + |
| 8 | +{% include toc %} |
| 9 | + |
| 10 | +At Netflix, we built CI integrations to enable features like the Builds view in Spinnaker, fetching artifact metadata and more. You can see the full list of CI features [here](/guides/user/managed-delivery/CI-features). |
| 11 | + |
| 12 | +This guide will explain how to add an integration to other CI providers, so that users leveraging those providers can enjoy the new features we provide. |
| 13 | + |
| 14 | +## Viewing CI details in Spinnaker |
| 15 | + |
| 16 | +We recently added the option to see CI details in Spinnaker, in a new "Builds" tab. |
| 17 | +It looks like this: |
| 18 | +{% |
| 19 | + include |
| 20 | + figure |
| 21 | + image_path="./ci-view.png" |
| 22 | +%} |
| 23 | + |
| 24 | +This is a new tab that can be accessed from the main Spinnaker UI. |
| 25 | +The UI code (in `deck`) is calling the `gate` service to fetch information, and `gate` is just proxying the call to `igor`. |
| 26 | + |
| 27 | +If you wish to use it, this is what you'll need to do: |
| 28 | +0. Make sure you are using [gate](https://github.com/spinnaker/gate). Here's the [CiController](https://github.com/spinnaker/gate/blob/master/gate-web/src/main/groovy/com/netflix/spinnaker/gate/controllers/CiController.java) the UI is calling (no need to do any actions here). |
| 29 | +1. Fork [igor](https://github.com/spinnaker/igor). |
| 30 | +2. Create a new service in `igor` which implements the [`CiBuildService`](https://github.com/spinnaker/igor/blob/master/igor-web/src/main/java/com/netflix/spinnaker/igor/ci/CiBuildService.java) interface, and contains two endpoints: |
| 31 | +- `ci/builds` which returns the `GenericBuild` object. |
| 32 | +- `ci/builds/{buildId}/output` which returns the build's log by providing a build number, in a map format which looks like: |
| 33 | +``` |
| 34 | +{"log": "...."} |
| 35 | +``` |
| 36 | + |
| 37 | +- Example: your service implementation should be something like: |
| 38 | +``` |
| 39 | + public class SomeCiService implements CiBuildService[..] |
| 40 | +``` |
| 41 | +These endpoints are called by the [CiController](https://github.com/spinnaker/igor/blob/master/igor-web/src/main/java/com/netflix/spinnaker/igor/ci/CiController.java). |
| 42 | + |
| 43 | +Once both of these endpoints are implemented, it should be possible for the installations using that implementation to show CI information in Spinnaker! |
| 44 | +If you contribute a new implementation to the project, make sure to include documentation for operators explaining how to enable and configure it in their installation. |
| 45 | + |
| 46 | + |
| 47 | +## Managed Delivery artifacts |
| 48 | + |
| 49 | +You can read about how the CI integration enriches the Managed Delivery experience with artifacts [here](/guides/user/managed-delivery/CI-features). |
| 50 | + |
| 51 | +### Detecting a new published artifact |
| 52 | + |
| 53 | +When a new artifact is published, `keel` gets notified by `echo` (for Debian packages) or by `igor` (for Docker images). Each artifact has a `metadata` section, which is populated by the corresponding artifact supplier implementation in `keel`. |
| 54 | +For example, for `docker` artifacts, the `metadata` field is populated by querying `clouddriver`, which stores information from the docker registries configured in the Spinnaker installation. |
| 55 | + |
| 56 | +Here is an example of a Docker artifact event tracked by `keel` (this is a piece of a log statement): |
| 57 | +``` |
| 58 | +Received artifact published event: ArtifactPublishedEvent(artifacts=[PublishedArtifact(name=mce/agent, type=DOCKER, |
| 59 | + reference=dockerRegistry:v2:testregistry:mce/agent, version=h2012.2626f5f, customKind=false, location=testregistry, artifactAccount=null, provenance=null, uuid=null, metadata={date=1607111907158, registry=testregistry, fullname=mce/agent, tag=tag, commitId=2626f5f, buildNumber=2012, branch=tags/v1.2.0^0} |
| 60 | +``` |
| 61 | + |
| 62 | +In order to get additional metadata for each tracked artifact (such as build and git details), we've created a convention that the `buildNumber` and `commitId` will be part of the artifact metadata object, regardless of the artifact type. |
| 63 | +In this example, we will query our CI provider to get metadata for build number `2012` and commit id `2626f5f`. |
| 64 | + |
| 65 | +### Getting artifact metadata |
| 66 | + |
| 67 | +Detailed metadata (like commit message, author, timestamp) is visible in the Environments view in the UI, by clicking on an artifact version. |
| 68 | + |
| 69 | +In order to get the metadata information, you'll have to plug in your CI provider. |
| 70 | +`keel`'s [buildService](https://github.com/spinnaker/keel/blob/master/keel-igor/src/main/kotlin/com/netflix/spinnaker/keel/igor/BuildService.kt) calls `igor` to fetch this information. It uses the endpoint `<igorBaseURL>/ci/builds`, which looks like: |
| 71 | +```kotlin |
| 72 | + @GET("/ci/builds") |
| 73 | + @Headers("Accept: application/json") |
| 74 | + suspend fun getArtifactMetadata( |
| 75 | + @Query("commitId") commitId: String, |
| 76 | + @Query("buildNumber") buildNumber: String, |
| 77 | + @Query("projectKey") projectKey: String? = null, |
| 78 | + @Query("repoSlug") repoSlug: String? = null, |
| 79 | + @Query("completionStatus") completionStatus: String? = null |
| 80 | + ): List<Build>? |
| 81 | +``` |
| 82 | + |
| 83 | +At Netflix, we pass as input the `commitId` and `buildNumber`. Those are mandatory to uniquely identify an artifact across repositories. |
| 84 | +The response contains a list of [builds](https://github.com/spinnaker/keel/blob/master/keel-igor/src/main/kotlin/com/netflix/spinnaker/keel/igor/model/Build.kt) (we pick the first from the list), which look like: |
| 85 | +```kotlin |
| 86 | +data class Build( |
| 87 | + val building: Boolean = false, |
| 88 | + val fullDisplayName: String? = null, |
| 89 | + val name: String? = null, |
| 90 | + val number: Int = 0, |
| 91 | + val duration: Long? = null, |
| 92 | + /** String representation of time in nanoseconds since Unix epoch */ |
| 93 | + val timestamp: String? = null, |
| 94 | + |
| 95 | + val result: Result? = null, |
| 96 | + val url: String? = null, //url to get build details from the CI provider (like jenkins) |
| 97 | + val id: String? = null, //build uid |
| 98 | + |
| 99 | + val scm: List<GenericGitRevision>? = null, |
| 100 | + val properties: Map<String, Any?>? = null |
| 101 | +) |
| 102 | +``` |
| 103 | +The [`Build`](https://github.com/spinnaker/keel/blob/master/keel-igor/src/main/kotlin/com/netflix/spinnaker/keel/igor/model/Build.kt) object is a mirror of [`GenericBuild`](https://github.com/spinnaker/igor/blob/master/igor-core/src/main/java/com/netflix/spinnaker/igor/build/model/GenericBuild.java) in `igor`. |
| 104 | + |
| 105 | +Note: if you already implemented the `/ci/builds` endpoint as described above, jump to step 3 and you are done! |
| 106 | + |
| 107 | +If you are interested in plugging your CI provider, you'll need to do the following: |
| 108 | +1. Fork [igor](https://github.com/spinnaker/igor). |
| 109 | +2. Implement the `ci/builds` endpoint, and return a `GenericBuild` response (which is converted to `Build` object above). |
| 110 | +- [Controller endpoints](https://github.com/spinnaker/igor/blob/master/igor-web/src/main/java/com/netflix/spinnaker/igor/ci/CiController.java) |
| 111 | +- [Generic build service](https://github.com/spinnaker/igor/blob/master/igor-web/src/main/java/com/netflix/spinnaker/igor/ci/CiBuildService.java) |
| 112 | +- Example: your service implementation should be somthing like: |
| 113 | +``` |
| 114 | + public class SomeCiService implements CiBuildService[...] |
| 115 | +``` |
| 116 | +3. Make sure the artifacts being sent to `keel` contains the `buildNumber` and `commitId`. |
| 117 | + |
| 118 | + |
| 119 | +### See code changes between deployments |
| 120 | + |
| 121 | +The logic to generate a link to compare commits is done in `keel`: [ArtifactVersionLinks](https://github.com/spinnaker/keel/blob/master/keel-core/src/main/kotlin/com/netflix/spinnaker/keel/artifacts/ArtifactVersionLinks.kt). |
| 122 | + |
| 123 | +Note: this supports only Stash/BitBucket at the moment. We'll appreciate contributions for other SCMs! |
| 124 | + |
| 125 | +### Surface build information in the UI |
| 126 | + |
| 127 | +A new section called "Pre-deployment" is now available in the UI. This section will surface pre-deployment steps like baking (for Debian packages only) or building. |
| 128 | +By clicking on "See details", you'll be taken to the CI-detailed view (see above), or a default job log which you can provide as a part of your implementation (in the `url` field in [`Build`](https://github.com/spinnaker/keel/blob/master/keel-igor/src/main/kotlin/com/netflix/spinnaker/keel/igor/model/Build.kt) object, see above). |
0 commit comments