diff --git a/ci/docs/publish.md b/ci/docs/publish.md deleted file mode 100644 index e97de1047a..0000000000 --- a/ci/docs/publish.md +++ /dev/null @@ -1,15 +0,0 @@ -# Publish a stemcell. -## Verify stemcell builds -Whenever a USN (Ubuntu Security Notice) is published, a ticket is automaticly created in the community-stemcell board (https://github.com/orgs/cloudfoundry/projects/4). -There would also be a notification in the #bosh-private channel of the Cloud Foundry slack. - -A USN triggers the stemcell build pipeline, when everything is successfully built, we can publish the new candidate stemcell version. -Please check the latest build of the aggregate-candidate-stemcells job of the stemcell line you need to release (aggregate-candidate-stemcells-1.x), -you should see a successful build which was triggered by the USN notice. -Then, in the stemcells-publisher pipeline, check if google and aws light stemcells were successfully built as a result of the USN. - -## Publish stemcells -In order to publish new stemcells, trigger a build of the publish-ubuntu-jammy-1 job (e.g. https://bosh.ci.cloudfoundry.org/teams/main/pipelines/stemcells-publisher/jobs/publish-ubuntu-jammy-1). -When this job is finished, new stemcell versions will be availeble on bosh.io and a github draft release created on -https://github.com/cloudfoundry/bosh-linux-stemcell-builder/releases. -Check the draft release if it has the correct usns and cves and add extra information if necessary in the release notes and publish the github release. \ No newline at end of file diff --git a/ci/pipelines/vars.yml b/ci/pipelines/vars.yml index c3c20cc44b..958510a9a5 100644 --- a/ci/pipelines/vars.yml +++ b/ci/pipelines/vars.yml @@ -1,6 +1,5 @@ #@data/values --- - stemcell_details: branch: ubuntu-jammy major_version: 1 diff --git a/docs/develop.md b/docs/develop.md deleted file mode 100644 index 24ed473d1c..0000000000 --- a/docs/develop.md +++ /dev/null @@ -1,39 +0,0 @@ -Current development on new stemcells. - -requirements: -- [bucc](https://github.com/starkandwayne/bucc) -- [virtualbox](https://www.virtualbox.org/) 6.1 => - -## create stemcell -run container -``` -cd ~/workspace/bosh-linux-stemcell-builder/ci/docker -./run os-image-stemcell-builder-impish -bundle exec rake stemcell:build_os_image[ubuntu,impish,$PWD/tmp/ubuntu_base_image.tgz] -``` - -## deploy a bosh director with the new stemcell on virtualbox -``` -mkdir -p ~/workspace -git clone https://github.com/starkandwayne/bucc -cd bucc -mkdir operators -``` -``` -echo -e ' -- name: stemcell - path: /resource_pools/name=vms/stemcell? - type: replace - value: - url: file://~/workspace/bosh/bosh-linux-stemcell-builder/tmp/bosh-stemcell-1.23-vsphere-esxi-ubuntu-impish-go_agent.tgz - -#user = vcap -#password = c1oudc0w -- name: stemcell - path: /resource_pools/0/env/bosh/password? - type: replace - value: "$6$3RO2Vvl4EXS2TMRD$IaNjbMHYCSBiQLQr0PKK8AdfDHTsNunqh3kO7USouNS/tWAvH0JmtDfrhLlHwN0XUCUrBVpQ02hoHYgTdaaeY1" -' > operators/stemcell.yml -``` - -run `bucc up` \ No newline at end of file diff --git a/ci/docs/fips.md b/docs/fips.md similarity index 58% rename from ci/docs/fips.md rename to docs/fips.md index 54c7b9cb9d..b38ae1afe7 100644 --- a/ci/docs/fips.md +++ b/docs/fips.md @@ -1,45 +1,53 @@ # FIPS stemcells ## access to the fips stemcell buckets + fips stemcells when published in the pipeline will be put in a private bucket called `bosh-core-stemcells-fips` if a working groups needs these fips stemcell the can retrieve them with the -[bosh-io-stemcell](https://github.com/concourse/bosh-io-stemcell-resource) concourse resource =>1.2.1 +[bosh-io-stemcell](https://github.com/concourse/bosh-io-stemcell-resource) concourse resource => 1.2.1 by setting -``` + +```yaml resources: - name: stemcell type: bosh-io-stemcell source: name: bosh-aws-xen-hvm-ubuntu-jammy-fips-go_agent auth: - access_key: hmac-accesskey - secret_key: hmac-secretkey + access_key: ((access_key-credhub_ref)) + secret_key: ((secret_key-credhub_ref)) ``` + for this you need a service account setup with hmac keys https://cloud.google.com/storage/docs/authentication/hmackeys ## setup access -to setup access permissions for the `bosh-core-stemcells-fips` bucket + +setup access permissions for the `bosh-core-stemcells-fips` bucket ### working group actions -a service accunt should be setup in the working group that want to access the fips stemcells. + +a service account should be setup in the working group that want to access the fips stemcells. this account should then be enabled with [hmac keys](https://cloud.google.com/storage/docs/authentication/hmackeys) #### bucket owner actions + requirements: - [gcloud](https://cloud.google.com/sdk/docs/install) - [gsutil](https://cloud.google.com/storage/docs/gsutil_install) -login to the cloud-foundry-310819 project ` gcloud auth login` +Login to the `cloud-foundry-310819` GCP project `gcloud auth login` and setup +access for cross-project buckets. In the example below replace PLACEHOLDER with +the service account that is created in the previous steps. +Ex: `test-dev@myproject.iam.gserviceaccount.com` -setup access for cross project cloud buckets. reference: https://cloud.google.com/dataprep/docs/concepts/gcs-buckets -replace PLACEHOLDER with the service account that is created in the previous steps for example test-dev@myproject.iam.gserviceaccount.com -``` -gsutil defacl ch -u PLACEHOLDER gs://bosh-core-stemcells-fips +```shell +gsutil defacl ch -u PLACEHOLDER:READER gs://bosh-core-stemcells-fips gsutil acl ch -u PLACEHOLDER:READER gs://bosh-core-stemcells-fips gsutil -m acl ch -r -u PLACEHOLDER:READER gs://bosh-core-stemcells-fips ``` +Reference: https://cloud.google.com/dataprep/docs/concepts/gcs-buckets diff --git a/docs/new_stemcell_line.md b/docs/new_stemcell_line.md index 055c07a02d..74b64e006b 100644 --- a/docs/new_stemcell_line.md +++ b/docs/new_stemcell_line.md @@ -1,30 +1,40 @@ # Creating a new stemcell line -1. Create a new branch from the passing commit you want to release from. Use `{os_name}-{os_version}/{major}.x` format for branch name (e.g. `ubuntu-jammy/1.x`). - - `git checkout -b <> {commit}` - -1. Add, commit, and push the new branch. - +1. Create a new branch from the passing commit you want to release from. Use `ubuntu-${short_name}` format for branch name. + + ```shell + export short_name="jammy" + + git switch -c ubuntu-${short_name} {commit} + ``` + +2. Update `ci/pipelines/vars.yml` with the appropriate values + + ```yaml + #@data/values + stemcell_details: + branch: ubuntu-jammy + # ... + blobstore_types: + - dav + # ... ``` - git push origin <> - ``` - -1. Switch back to master branch - ``` - git checkout master +3. Update `STEMCELL_LINE` in `ci/configure.sh`: + + ```shell + STEMCELL_LINE="ubuntu-${short_name}" ``` -1. On master, update `ci/{os_name}-{os_version}/configure-aggregated-pipeline.sh` with the new branch details using the previous release branch as an example. Specifically, be sure to update the interpolated variables for the correct branch. For `initial_version`, use the same value of the stemcell produced by the commit in the `master` pipeline (e.g. `2.0.0`). +4. Add, commit, and push the new branch. + ```shell + git push --set-upstream origin HEAD ``` - ./ci/{os_name}-{os_version}/configure-aggregated-pipeline.sh - ``` - -1. Once configured, the stemcell should automatically trigger and create the next minor version of the stemcell (e.g. `2.1.0`). +5. Configure the new pipeline: + ```shell + ./ci/configure.sh + ``` -# References - -* [Stemcell Support Matrix](https://docs.google.com/spreadsheets/d/11LgvmuR-XxXpKB-UVi91FL0nkITGhoB-G1NHPwfnweo/edit) (internal only) +6. Once configured, the stemcell pipeline should automatically trigger. diff --git a/docs/publish.md b/docs/publish.md new file mode 100644 index 0000000000..5bd889290b --- /dev/null +++ b/docs/publish.md @@ -0,0 +1,53 @@ +# Publish a stemcell + +## Verify stemcells have built + +The following CI build groups process USN(s), and post a message to the +`#bosh-private` channel in the +[Cloud Foundry slack](https://cloudfoundry.slack.com) +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-jammy-builder?group=automatic-triggers +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-noble-builder?group=automatic-triggers + +A USN triggers the stemcell build pipeline to pick up newer packages which +address the USN(s). Before publishing a new stemcell verify the latest build of +`aggregate-candidate-stemcells` job for the stemcell line in question: +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-jammy-builder/jobs/aggregate-candidate-stemcells +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-noble-builder/jobs/aggregate-candidate-stemcells + +Look for a successful build triggered by the USN(s). + +In the publisher pipelines: +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-jammy-publisher +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-noble-publisher + +verify that the lite stemcell builds for Google and AWS have been successfully +triggered by the USN(s). + +## Publish stemcells + +In order to publish new stemcells, trigger a build of the +`publish-ubuntu-${short_name}-1` job: +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-jammy-publisher/jobs/publish-ubuntu-jammy-1 +- https://bosh.ci.cloudfoundry.org/teams/stemcell/pipelines/ubuntu-noble-publisher/jobs/publish-ubuntu-noble-1 + +This job will take around 1h to run. When this job is finished, new stemcell +versions will be available on bosh.io and a GitHub draft release will have been +created at +- https://github.com/cloudfoundry/bosh-linux-stemcell-builder/releases. + +Check to see if the draft release has the correct USNs and CVEs, add any extra +information (see below) to the release notes, and publish the GitHub release. + +Then [Finalize the draft release on GitHub](https://github.com/cloudfoundry/bosh-linux-stemcell-builder/releases) +- Select the appropriate `Previous Tag` and click "Generate release notes" - + - Do NOT leave it set to `Auto` as that will result in incorrect release notes. +- Scroll to the bottom; cut-and-paste the "What's Changed" section and paste it + at the top +- Check the items for correctness; they may have already been published in a + previous release. Branch merges can confuse GitHub's auto-generated notes. +- Reword the bullet items to convey what was fixed: + +| old (bad) | new (good) | +|----------------------------------------------|-----------------------------------------------------------------------------------------------------------------------| +| 🐞 Fix “Cannot connect to the monit daemon.” | Fixed issue with BOSH service-broker tasks failing under heavy usage with error "Cannot connect to the monit daemon." | +| Installing ethtool in Jammy by @ in | Stemcells now include the networking utility, ethtool, useful for troubleshooting & resolving networking issues. |