Skip to content

Commit 579eaea

Browse files
marekzanmarekzan
authored
Merge pull request #10 from cloudogu/feature/pluggable_scm_provider
Feature/pluggable scm provider
2 parents df274b2 + 49aa5d1 commit 579eaea

14 files changed

Lines changed: 233 additions & 87 deletions

File tree

‎README.md‎

Lines changed: 69 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -7,21 +7,21 @@ working example bundled with the complete infrastructure for a gitops deep dive.
77
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
88
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
99

10-
1110
- [Use Case](#use-case)
1211
- [Features](#features)
1312
- [Examples](#examples)
1413
- [Minimal](#minimal)
1514
- [More options](#more-options)
1615
- [Real life examples](#real-life-examples)
1716
- [Usage](#usage)
18-
- [Default Structure](#default-structure)
17+
- [Default Folder Structure](#default-folder-structure)
1918
- [FluxV1](#fluxv1)
2019
- [Plain-k8s](#plain-k8s)
2120
- [Helm](#helm)
2221
- [FluxV2](#fluxv2)
2322
- [ArgoCD](#argocd)
2423
- [GitOps-Config](#gitops-config)
24+
- [SCM-Provider](#scm-provider)
2525
- [Stages](#stages)
2626
- [Namespaces](#namespaces)
2727
- [Conventions for stages](#conventions-for-stages)
@@ -82,10 +82,12 @@ This will simply [validate](#validators) and deploy the resources from the sourc
8282

8383
```groovy
8484
def gitopsConfig = [
85-
scmmCredentialsId : 'scmm-user',
86-
scmmConfigRepoUrl : 'http://scmm-scm-manager/scm/repo/fluxv1/gitops',
87-
scmmPullRequestBaseUrl: 'http://scmm-scm-manager/scm'',
88-
scmmPullRequestRepo: 'fluxv1/gitops',
85+
scm: [
86+
provider: 'SCMManager',
87+
credentialsId: 'scmm-user',
88+
baseUrl: 'http://scmm-scm-manager/scm',
89+
repositoryUrl: 'fluxv1/gitops'
90+
],
8991
application: 'spring-petclinic',
9092
stages: [
9193
staging: [
@@ -108,10 +110,12 @@ For production it will open a PR with the changes.
108110

109111
```groovy
110112
def gitopsConfig = [
111-
scmmCredentialsId : 'scmm-user',
112-
scmmConfigRepoUrl : 'http://scmm-scm-manager/scm/repo/fluxv1/gitops',
113-
scmmPullRequestBaseUrl: 'http://scmm-scm-manager/scm'',
114-
scmmPullRequestRepo: 'fluxv1/gitops',
113+
scm: [
114+
provider: 'SCMManager',
115+
credentialsId: 'scmm-user',
116+
baseUrl: 'http://scmm-scm-manager/scm',
117+
repositoryUrl: 'fluxv1/gitops'
118+
],
115119
cesBuildLibRepo: <cesBuildLibRepo> /* Default: 'https://github.com/cloudogu/ces-build-lib' */ ,
116120
cesBuildLibVersion: <cesBuildLibVersion> /* Default: a recent cesBuildLibVersion see deployViaGitops.groovy */ ,
117121
cesBuildLibCredentialsId: <cesBuildLibCredentialsId> /* Default: '', empty due to default public github repo */,
@@ -253,10 +257,6 @@ You can find a complete yet simple example [here](#examples).
253257
First of all there are some mandatory properties e.g. the information about your gitops repository and the application repository.
254258

255259
```
256-
scmmCredentialsId: 'scmm-user', // ID of credentials defined in Jenkins used to authenticated with SCMM
257-
scmmConfigRepoUrl: 'http://scmm-scm-manager/scm/repo/fluxv1/gitops', // this is your full gitops repo url e.g.
258-
scmmPullRequestBaseUrl: 'http://scmm-scm-manager/scm', // this is your gitops base url
259-
scmmPullRequestRepo: 'fluxv1/gitops', // this is the gitops repo
260260
application: 'spring-petclinic' // Name of the application. Used as a folder in GitOps repo
261261
```
262262

@@ -268,6 +268,61 @@ mainBranch: 'main'
268268
```
269269
---
270270

271+
## SCM-Provider
272+
273+
The scm-section defines where your 'gitops-repository' resides (url, provider) and how to access it (credentials):
274+
275+
```groovy
276+
def gitopsConfig = [
277+
...
278+
scm: [
279+
provider: 'SCMManager', // This is the name of the scm-provider in use, for a list of supported providers watch below!
280+
credentialsId: 'scmm-user', // ID of credentials defined in Jenkins used to authenticated with SCMM
281+
baseUrl: 'http://scmm-scm-manager/scm', // this is your gitops base url
282+
repositoryUrl: 'fluxv1/gitops' // this is the gitops repo
283+
],
284+
...
285+
]
286+
```
287+
It currently supports the following scm-provider:
288+
289+
- [SCMManager](https://www.scm-manager.org/)
290+
291+
To empower people to participate and encourage the integration of other scm-software (e.g. github), we decided to implement an abstraction for the
292+
scm-provider. By extending the SCM-Provider class you can integrate your own provider! Please feel free to contribute!
293+
294+
Example:
295+
296+
```groovy
297+
import com.cloudogu.gitopsbuildlib.scm.SCMProvider
298+
299+
class GitHub extends SCMProvider {
300+
@Override
301+
void setBaseUrl(String baseUrl) { this.baseUrl = baseUrl }
302+
303+
@Override
304+
void setRepositoryUrl(String repositoryUrl) { this.repositoryUrl = repositoryUrl }
305+
306+
@Override
307+
void setCredentials(String credentialsId) { this.credentials = credentialsId }
308+
309+
@Override
310+
String getRepositoryUrl() { return "${this.baseUrl}/${repositoryUrl}"}
311+
312+
@Override
313+
void createOrUpdatePullRequest(String stageBranch, String mainBranch, String title, String description) {
314+
// TODO: this is a specific implementation for github
315+
// 1. creating a pr on the given repo with the given details
316+
// 2. update a pr on the given repo if it already exists
317+
//
318+
// Note: Credentials given are credentialsId from Jenkins!
319+
}
320+
}
321+
322+
```
323+
324+
---
325+
271326
## Stages
272327
The GitOps-build-lib supports builds on multiple stages. A stage is defined by a name and contains a namespace (used to
273328
generate the resources) and a deployment-flag. If no stages is passed into the gitops-config by the user, the default

‎pom.xml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -148,6 +148,12 @@
148148
<groupId>org.apache.maven.plugins</groupId>
149149
<artifactId>maven-surefire-plugin</artifactId>
150150
<version>2.22.2</version>
151+
<!-- further debug possibilities
152+
<configuration>
153+
<redirectTestOutputToFile>true</redirectTestOutputToFile>
154+
<trimStackTrace>false</trimStackTrace>
155+
</configuration>
156+
<- end -->
151157
</plugin>
152158

153159
<plugin>

‎src/com/cloudogu/gitopsbuildlib/scm/SCMManager.groovy‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
package com.cloudogu.gitopsbuildlib.scm
2+
3+
class SCMManager extends SCMProvider {
4+
protected String credentials
5+
protected String baseUrl
6+
protected String repositoryUrl
7+
8+
9+
SCMManager(def script) {
10+
super(script)
11+
}
12+
13+
@Override
14+
void setBaseUrl(String baseUrl) { this.baseUrl = baseUrl }
15+
16+
@Override
17+
void setRepositoryUrl(String repositoryUrl) { this.repositoryUrl = repositoryUrl }
18+
19+
@Override
20+
void setCredentials(String credentialsId) { this.credentials = credentialsId }
21+
22+
@Override
23+
String getRepositoryUrl() {
24+
return "${this.baseUrl}/repo/${this.repositoryUrl}"
25+
}
26+
27+
@Override
28+
void createOrUpdatePullRequest(String stageBranch, String mainBranch, String title, String description) {
29+
script.cesBuildLib.SCMManager.new(script, this.baseUrl, this.credentials).createOrUpdatePullRequest(this.repositoryUrl, stageBranch, mainBranch, title, description)
30+
}
31+
}

‎src/com/cloudogu/gitopsbuildlib/scm/SCMProvider.groovy‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
package com.cloudogu.gitopsbuildlib.scm
2+
3+
abstract class SCMProvider {
4+
5+
protected script
6+
7+
SCMProvider(def script) {
8+
this.script = script
9+
}
10+
11+
abstract void setBaseUrl(String baseUrl)
12+
13+
abstract void setRepositoryUrl(String repositoryUrl)
14+
15+
abstract void setCredentials(String credentialsId)
16+
17+
abstract String getRepositoryUrl()
18+
19+
abstract void createOrUpdatePullRequest(String stageBranch, String mainBranch, String title, String description)
20+
}

‎test/com/cloudogu/gitopsbuildlib/DeployViaGitopsTest.groovy‎

Lines changed: 30 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,6 @@ package com.cloudogu.gitopsbuildlib
22

33
import com.cloudogu.ces.cesbuildlib.DockerMock
44
import com.cloudogu.ces.cesbuildlib.Git
5-
import com.cloudogu.ces.cesbuildlib.SCMManager
6-
75
import com.cloudogu.gitopsbuildlib.validation.Kubeval
86
import com.cloudogu.gitopsbuildlib.validation.Yamllint
97
import com.lesfurets.jenkins.unit.BasePipelineTest
@@ -42,10 +40,12 @@ class DeployViaGitopsTest extends BasePipelineTest {
4240

4341
Map gitopsConfig(Map stages, Map deployments) {
4442
return [
45-
scmmCredentialsId : 'scmManagerCredentials',
46-
scmmConfigRepoUrl : 'configRepositoryUrl',
47-
scmmPullRequestBaseUrl : 'http://scmm-scm-manager/scm',
48-
scmmPullRequestRepo : 'fluxv1/gitops',
43+
scm : [
44+
provider : 'SCMManager',
45+
credentialsId: 'scmManagerCredentials',
46+
baseUrl : 'http://scmm-scm-manager/scm',
47+
repositoryUrl : 'fluxv1/gitops',
48+
],
4949
cesBuildLibRepo : 'cesBuildLibRepo',
5050
cesBuildLibVersion : 'cesBuildLibVersion',
5151
cesBuildLibCredentialsId: 'cesBuildLibCredentialsId',
@@ -114,7 +114,7 @@ class DeployViaGitopsTest extends BasePipelineTest {
114114
cesBuildLibMock = new CesBuildLibMock()
115115
git = mock(Git.class)
116116
docker = new DockerMock().createMock()
117-
scmm = mock(SCMManager.class)
117+
scmm = mock(com.cloudogu.ces.cesbuildlib.SCMManager.class)
118118

119119
cesBuildLibMock.Docker.new = {
120120
return docker
@@ -291,7 +291,7 @@ spec:
291291

292292
ArgumentCaptor<Map> argumentCaptor = ArgumentCaptor.forClass(Map.class)
293293
verify(git).call(argumentCaptor.capture())
294-
assertThat(argumentCaptor.getValue().url).isEqualTo('configRepositoryUrl')
294+
assertThat(argumentCaptor.getValue().url).isEqualTo('http://scmm-scm-manager/scm/repo/fluxv1/gitops')
295295
assertThat(argumentCaptor.getValue().branch).isEqualTo('main')
296296
verify(git, times(1)).fetch()
297297

@@ -332,7 +332,7 @@ spec:
332332

333333
ArgumentCaptor<Map> argumentCaptor = ArgumentCaptor.forClass(Map.class)
334334
verify(git).call(argumentCaptor.capture())
335-
assertThat(argumentCaptor.getValue().url).isEqualTo('configRepositoryUrl')
335+
assertThat(argumentCaptor.getValue().url).isEqualTo('http://scmm-scm-manager/scm/repo/fluxv1/gitops')
336336
assertThat(argumentCaptor.getValue().branch).isEqualTo('main')
337337
verify(git, times(1)).fetch()
338338

@@ -448,10 +448,10 @@ spec:
448448
void 'error on single missing mandatory field'() {
449449

450450
def gitopsConfigMissingMandatoryField = [
451-
scmmConfigRepoUrl : 'configRepositoryUrl',
452-
scmmPullRequestBaseUrl: 'configRepositoryPRBaseUrl',
453-
scmmPullRequestRepo : 'scmmPullRequestRepo',
454-
application : 'application',
451+
scm : [
452+
baseUrl : 'http://scmm-scm-manager/scm',
453+
repositoryUrl: 'fluxv1/gitops',
454+
],
455455
gitopsTool : 'FLUX_V1',
456456
deployments : [
457457
sourcePath: 'k8s',
@@ -463,7 +463,7 @@ spec:
463463
]
464464
]
465465
],
466-
stages : [
466+
stages : [
467467
staging : [deployDirectly: true],
468468
production: [deployDirectly: false],
469469
qa : []
@@ -476,20 +476,20 @@ spec:
476476

477477
assertThat(
478478
helper.callStack.findAll { call -> call.methodName == "error" }.any { call ->
479-
callArgsToString(call).contains("[scmmCredentialsId]")
479+
callArgsToString(call).contains("[scm.provider]")
480480
}).isTrue()
481481
}
482482

483483
@Test
484484
void 'error on single non valid mandatory field'() {
485485

486486
def gitopsConfigMissingMandatoryField = [
487-
scmmCredentialsId : 'scmManagerCredentials',
488-
scmmConfigRepoUrl : 'configRepositoryUrl',
489-
scmmPullRequestBaseUrl: '',
490-
scmmPullRequestRepo : 'scmmPullRequestRepo',
491-
scmmPullRequestUrl : 'configRepositoryPRUrl',
492-
application : 'application',
487+
scm : [
488+
provider : 'SCMManager',
489+
baseUrl : 'http://scmm-scm-manager/scm',
490+
repositoryUrl: 'fluxv1/gitops',
491+
],
492+
application : '',
493493
gitopsTool : 'FLUX_V1',
494494
deployments : [
495495
sourcePath: 'k8s',
@@ -501,7 +501,7 @@ spec:
501501
]
502502
]
503503
],
504-
stages : [
504+
stages : [
505505
staging : [deployDirectly: true],
506506
production: [deployDirectly: false],
507507
qa : []
@@ -514,16 +514,20 @@ spec:
514514

515515
assertThat(
516516
helper.callStack.findAll { call -> call.methodName == "error" }.any { call ->
517-
callArgsToString(call).contains("[scmmPullRequestBaseUrl]")
517+
callArgsToString(call).contains("[application]")
518518
}).isTrue()
519519
}
520520

521521
@Test
522522
void 'error on missing or non valid values on mandatory fields'() {
523523

524524
def gitopsConfigMissingMandatoryField = [
525-
scmmPullRequestBaseUrl: null,
526-
application : '',
525+
scm : [
526+
credentialsId: 'scmManagerCredentials',
527+
baseUrl : 'http://scmm-scm-manager/scm',
528+
repositoryUrl : '',
529+
],
530+
application: '',
527531
gitopsTool : 'FLUX_V1',
528532
stages : []
529533
]
@@ -534,7 +538,7 @@ spec:
534538

535539
assertThat(
536540
helper.callStack.findAll { call -> call.methodName == "error" }.any { call ->
537-
callArgsToString(call).contains("[scmmCredentialsId, scmmConfigRepoUrl, scmmPullRequestBaseUrl, scmmPullRequestRepo, application, stages]")
541+
callArgsToString(call).contains("[scm.provider, scm.repositoryUrl, application, stages]")
538542
}).isTrue()
539543
}
540544

‎test/com/cloudogu/gitopsbuildlib/GitMock.groovy‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,6 @@
11
package com.cloudogu.ces.cesbuildlib
22

3-
import org.mockito.invocation.InvocationOnMock
4-
import org.mockito.stubbing.Answer
5-
6-
import static org.mockito.ArgumentMatchers.*
73
import static org.mockito.Mockito.mock
8-
import static org.mockito.Mockito.when
94

105
class GitMock {
116
List<String> actualArgs = new LinkedList<>()

‎test/com/cloudogu/gitopsbuildlib/ScriptMock.groovy‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ class ScriptMock {
1212
List<String> actualShArgs = new LinkedList<>()
1313
List<String> actualEchoArgs = new LinkedList<>()
1414
List<String> actualReadYamlArgs = new LinkedList<>()
15+
List<String> actualGitArgs = new LinkedList<>()
1516
String actualDir
1617
def configYaml = '''\
1718
---
@@ -43,7 +44,7 @@ to:
4344
],
4445
],
4546
docker: dockerMock.createMock(),
46-
git: gitMock.createMock(),
47+
git: { args -> actualGitArgs += args.toString() },
4748
pwd : { 'pwd' },
4849
sh : { args -> actualShArgs += args.toString() },
4950
echo : { args -> actualEchoArgs += args.toString() },

‎test/com/cloudogu/gitopsbuildlib/deployment/DeploymentTest.groovy‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import com.cloudogu.gitopsbuildlib.ScriptMock
44
import com.cloudogu.gitopsbuildlib.validation.HelmKubeval
55
import com.cloudogu.gitopsbuildlib.validation.Kubeval
66
import com.cloudogu.gitopsbuildlib.validation.Yamllint
7-
import org.junit.Test
7+
import org.junit.jupiter.api.*
88
import static org.assertj.core.api.Assertions.assertThat
99

1010
class DeploymentTest {

0 commit comments

Comments
 (0)