Skip to content

development.adoc

Latest commit

 

History

History
429 lines (320 loc) · 18.2 KB

development.adoc

File metadata and controls

429 lines (320 loc) · 18.2 KB
Raw

Development Quick Start Guide

In order to start working with the Couchbase Operator you first need to install Kubernetes. Kubernetes-IN-Docker (KIND) is used for all CI and generally recommended during development. To install KIND follow the official instructions. It has the benefit of allowing running multiple clusters and each can be configured independently with different versions of K8S or other settings.

Build Prerequsites

You will need the following from your favorite package manager:

  • go (1.22.0+)

  • git

  • git-review

  • make

  • docker

Ensure docker is allocated a decent amount of RAM as it defaults to 2GB for some reason, suggest 8GB+: https://kind.sigs.k8s.io/docs/user/quick-start#settings-for-docker-desktop

Note

Although not needed, we recommend 4 VCPUs and 8 GiB of RAM. When running acceptance tests it’s not uncommon to see up to 8 containers running and Couchbase Server does use a lot of processing power to rebalance.

Preparing the Environment

Here is an example of using the Brew package manager for macOS. This also sets up an IDE (Visual Studio Code in this case) along with supporting tooling for linting and using Helm charts. Each line is intended to cover a particular aspect (e.g. code then test) to make it clearer in case you want to use other tools.

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew doctor
$ brew install git python repo cmake ccache git-review go@1.22 make
$ brew install --cask docker
$ brew install kind kubectl helm
$ brew install --cask visual-studio-code
$ brew install asciidoc aspell node
$ python3 -m pip install  -r scripts/requirements.txt

Make sure to explicitly start the graphical elements above (docker particularly) to cover all credentials and permissions.

Ensure docker is allocated a decent amount of RAM as it defaults to 2GiB for some reason currently which will not be enough, suggest 8GiB+: https://kind.sigs.k8s.io/docs/user/quick-start#settings-for-docker-desktop

Preparing to Submit Code

Ensure you have set up your SSH keys, the GitHub documentation covers it reasonably well. Since we’ll primarily be working with gerrit, ensure your local ssh agent’s config file has the following in addition to github.com host.

Host review.couchbase.org
  AddKeysToAgent yes
  IdentityFile ~/.ssh/git_ssh_couchbase

To work with the code, you will need to do the following:

  • Create ssh keys (used for Gerrit authentication) and add to Gerrit.

  • Ensure Gerrit is set up with your email address used by git configuration.

  • Get an account on Gerrit for the couchbase-operator repository (raise a CBIT or contact Couchbase) - this requires addition of the user to the NorthScale group.

  • Have git and git-review installed.

Once complete you can clone the repository and ready it for code review.

Important

Your <gerrit username> in following steps can be found by looking at your profile on Gerrit under the HTTP credentials.

Depending on your username vs your Couchbase/machine login/git id the git review step may need some additional input. For example if your local username is firstnamelastname but your Gerrit username is firstname-lastname then you need to provide this otherwise it will fail to login initially. This can also be provided by the git config command:

$ git config --global user.name "<firstname> <lastname>"
$ git config --global user.email "<email address>"
$ git config --global --add gitreview.username "<gerrit username>"

Be aware that to build the code you need a certain path structure for the code generator make rules to function.

$ mkdir -p github.com/couchbase
$ git clone ssh://<gerrit username>@review.couchbase.org:29418/couchbase-operator.git github.com/couchbase/couchbase-operator
$ cd github.com/couchbase/couchbase-operator
$ gitdir=$(git rev-parse --git-dir); scp -p -P 29418 -O <gerrit username>@review.couchbase.org:hooks/commit-msg ${gitdir}/hooks/
$ git review -s

Your work flow is entirely up to you however we recommend the following process.

  • Before doing anything create an issue in Jira so we have an ID for simple tracking e.g. K8S-836.

  • Create a feature branch git checkout -b k8s_836_my_change master.

  • Do your work and test it.

  • Commit the code with a message in the form:

    K8S-836: What My Change Does

A detailed description of WHAT you are changing and WHY.  The width of
lines should be kept to below 80 characters as Gerrit won't display it
very nicely otherwise.  Some clever editors will automatically do word
wrap for you e.g. vim.

  • Submit your change to Gerrit git review (use -R to prevent automatic rebase).

  • To update your patch set do your changes and amend git commit --amend then review with git review preserving the Change-ID which was automatically added by the prior commit.

Building the Operator

Run make command to build the container images.

$ make images

The various make targets are documented below.

Table 1. Make targets
Command Description

make

Build all binaries (required to run tests, must pass).

make images

Build docker images (couchbase/couchbase-operator:v1, couchbase/couchbase-operator-admission:v1, and couchbase/couchbase-operator-certification:v1).

make test-unit

Runs unit tests (must pass).

make lint

Checks for code standards, style and bugs (must pass).

make docs

Detects doc changes and auto-generates Asciidoc from code.

make docs-lint

Command used by CI to ensure you’ve auto-generated, checked in, spellchecked and the links aren’t broken (must pass).

make lint-python

Checks all python scripts are standard, not broken and not dubious (must pass).

You should see something similar to the following with the docker images command. The make file will automatically tag the image with something sane.

REPOSITORY                                     TAG                                        IMAGE ID       CREATED        SIZE
couchbase/couchbase-operator-certification     v1                                         c825c3d17788   13 days ago    479MB
couchbase/couchbase-operator-admission         v1                                         375316d73a6e   13 days ago    45.8MB
couchbase/couchbase-operator                   v1                                         90a8c4e7a26a   13 days ago    50.4MB
Note

Make sure you tag your container image with something other than ‘latest’ and use that tag while you pull the image. Otherwise, if you do not specify version of your image, it will be assumed as :latest, with pull image policy of Always correspondingly. This may eventually result in ErrImagePull as you may not have any versions of your container image out there in the default container registry (usually DockerHub) yet.

Running Tests

Install the following:

  • kind

  • kubectl

Create a cluster: kind create cluster

Install some images

You need to poke local docker images into kind, if the images are already in the public internet (e.g. DockerHub) you can skip this although it will speed things up and prevent hitting rate limits for pulling from those remote registries.

$ make # create the tools
$ make kind-images # create and install the container images

Not strictly required but improves caching performance - replace with the server version you want to test.

$ docker pull couchbase/server:7.0.3
$ kind load docker-image couchbase/server:7.0.3

Running a test

It may be helpful to make an alias in your bashrc (or similar profile script - default shell on macOS seems to be ZSH so in the .zshrc) file:

Note

Depending on your OS, you will need to change the cao path and image tags for the alias. The cao path will start with ./build/bin/…​ (e.g. for a Mac, this path is ./build/bin/kubernetes-darwin-arm64/cao). To find the correct tag, if you have built the images locally you can use docker images (see the TAG column).

 
$ alias test-couchbase='./build/bin/<cao path>/cao certify --image couchbase/couchbase-operator-certification:<certification tag> --clean --parallel=1 -- -operator-image couchbase/couchbase-operator:<operator tag> -admission-image couchbase/couchbase-admission-controller:<admission controller tag>'

Then you can run some common commands to get you started:

$ test-couchbase -test TestCreateCluster
$ test-couchbase -suite sanity

Note that arguments can be passed to the certification binary (cao) before the --, and arguments following it are passed to the underlying test framework.

All available arguments for the cao binary can be found in pkg/certification/certify.go. All available arguments for the test framework can be found in test/e2e/framework/framework.go.

Test requirements

  1. When you submit code, it must be tested sufficiently, either with a unit test or a full-on integration test.

  2. When you write a test it must always pass, if it doesn’t and you think it’s a bug, talk to development and submit a fix with the test!

  3. Tests should be stable, tests should pass, if they don’t then it just wastes time across the team analyzing the same issues over and over again.

Analyzing logs

To gather logs from tests, add the --collect-logs flag to your test command.

$ test-couchbase -suite p0 --collect-logs

Test runs will place the results of the test run in the base directory, in an archive called couchbase-operator-certification-<TIMESTAMP>.tar.bz2. Inside there is an artifacts folder, containing the test results.xml and logs folder with logs for every test, plus a cbopinfo archive for any failed test. You can unzip this with tar -xf <FILE>, and view an individual instances logs via ./namespace/pod/<PODNAME>/

Logify can be used to pull apart the logs collected via cbopinfo.

Targeting other branches

The assumption is that you have branched from the right upstream branch. Note if you branch from master then this is unlikely to be correct for the specific version you are targeting. You may need to re-base from the version you want and also trigger Gerrit to do it:

$ git rebase --onto origin/2.1.x HEAD~ <your branch name>
$ git review 2.1.x

This will target the 2.1.x release.

Similarly to cherry pick changes already submitted from another change set:

$ git checkout -b test origin/2.1.x
$ git cherry-pick <hash>
$ git review 2.1.x

Verify documentation

To verify documentation updates a special container can be used which will process it all and serve it locally for you to verify.

$ docker run --rm -ti \
  --publish 8080:80 \
  --mount type=bind,source=${PWD},target=/src,readonly \
  spjmurray/couchbase-antora-preview:latest
2021-11-11 13:50:01.513 INFO Using default repo path /src
2021-11-11 13:50:01.768 INFO Using branch k8s_2520_add_contribution_documentation
2021-11-11 13:50:03.522 INFO Using path ./docs/user
2021-11-11 13:50:03.526 INFO Cloning latest document repository ...
2021-11-11 13:50:07.495 INFO Creating site playbook ...
2021-11-11 13:50:07.552 INFO Generating static content ...
2021-11-11 13:50:16.679 INFO Serving content on http://localhost:8080
2021-11-11 13:50:16.686 INFO Serving repo content on http://localhost:8080/operator/2.3/overview.html
2021-11-11 13:50:16.686 INFO Starting webserver ...
2021-11-11 13:50:17.094 INFO Press Ctl+C to exit ...

As you can see it shows you where to go to view the generated documentation. Make sure to change port 8080 if it is already in use locally.

Relation chains

You can set up a set of related changes in Gerrit all under the same topic like so.

  • Create a new branch from where you want the next changeset in the relation chain to start from

  • Make your changes

  • Commit but when triggering a review specify the same topic to use, otherwise it will use the branch name.

Code reviews

When it hits Gerrit, your change will be automatically reviewed by a robot.

Important
 ITS OPINION IS FINAL, DO NOT REMOVE -1 VOTES.

Using Gerrit is not always intuitive so some useful links are here:

As of 2.1 EVERY commit must go through validation/sanity to ensure you haven’t broken the code for anyone else.

For documentation changes, there is no need to run the tests but for everything else you must.

If you are adding tests to P0 or P1, then run them too. You may choose to run an explicit set of targeted individual tests, this is described in the UI. Note that running tests is a good thing but target the effort at the areas changed - consuming the cluster for the whole test suite unnecessarily will slow down other builds and therefore productivity.

To run tests:

  1. In Gerrit click Download and copy the git reference that looks like refs/changes/88/136688/2.

  2. Navigate to Jenkins CICD job

  3. Select build with parameters and replace the refspec with the one you have just copied from Gerrit.

  4. Add any other parameters (especially suites), and click build.

At the end you should navigate to the test results page, see if you have broken anything (and then fix it repeating the above), if not post the link to Gerrit.

Finally reviewers should:

  • Check the code is decent!

  • Ensure that validation hasn’t been overridden

  • Ensure that the code has decent test coverage where applicable

  • Ensure the test has been run and passes

  • Only then may you bless with a +2 and submit the change.

  • Once reviewed, the author then submits.

Debugging

  1. Create a kind cluster

  2. kubectl apply -f example/crd.yaml

  3. make debug

    • build all containers

    • load them into kind

    • select the debug image as the operator image and set the command to run dlv

  4. Port forward 30123 from the operator pod i.e kubectl port-forward couchbase-operator-8bbc65c4f-q5l8d 30123:30123

  5. Attach a delve client, e.g. Visual Studio Code configuration is launch.json:

    {
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
       {
                    "name": "Remote debug in Kubernetes",
                    "type": "go",
                    "request": "attach",
                    "mode":"remote",
                    "remotePath": "/go/src/github.com/couchbase/couchbase-operator",
                    "port": 30123,
                    "host": "127.0.0.1",
                    "showLog": true
                }
    ]
}

If you pause on a breakpoint for longer than 15 seconds, the operator lease will time expire. This can be extended by kubectl edit lease and changing leaseDurationSeconds to something more appropriate.

Next Steps

Please consult the user documentation for instructions on how to deploy the Operator and a Couchbase cluster in Kubernetes, view cluster status and logs etc.

Important

The main documentation refers to .yaml paths which are distributed in the main package. Typically in development we will use the examples in the examples directory of the Operator source tree.

Selecting Couchbase Server Version

The containers used by couchbase-operator are hosted on Docker Hub. You can modify the version to install by editing example/couchbase-cluster.yaml and altering the /spec/version value. Valid container version tags are available on the Couchbase Server Docker Hub.

MiniKube

This guide assumes the usage of KIND but there are other options available, most notably MiniKube which we will cover here. MiniKube handles deploying a single node Kubernetes cluster by using VirtualBox. To install minikube follow the official instructions for various operating systems.

Note

The default version of Kubernetes installed is hard coded in Minikube. You can always use the --kubernetes-version flag to set the version you require. At the time of writing we support 1.17.0+ onwards, but the latest version is recommended given the fast paced nature of things.

Once MiniKube is installed you can run the command below to start MiniKube.

$ minikube start --cpus 4 --memory 8192

Running on MiniKube

To run this operator on minikube you first need to build a container image which can be deployed into the Kubernetes environment. Normally you would need to push your container image to a container registry (e.g. DockerHub) and then pull it into Kubernetes, but since you are using MiniKube you can build the image using the same container host as the Minikube VM. That way the images are automatically present. To do this, make sure you are using the Minikube Docker daemon:

$ eval $(minikube docker-env)

Later when you no longer want to use the Docker host in the MiniKube VM you can undo this change by running the command below.

$ eval $(minikube docker-env -u)

Cross Architecture Image Compilation

By default all make actions assume you are working for the same target as your host machine.

To build a different image target override IMAGE_TARGET in the Makefile. e.g. - To build AMD64 images on an ARM64 machine, run make images -e IMAGE_TARGET=kubernetes-linux-amd64. - To build ARM64 images on an AMD64 machine, run make images -e IMAGE_TARGET=kubernetes-linux-arm64.

For more information on what variables exist and overwriting them, the Makefile is fairly well documented.