Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,633 changes: 833 additions & 800 deletions Cargo.lock

Large diffs are not rendered by default.

15 changes: 7 additions & 8 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,23 +42,23 @@ nonzero_ext = "0.3"
# these package versions should be moved in lockstep to match upstream
actix-web = { version = "4.9", features = ["rustls-0_23"] }
tracing-actix-web = "0.7"
opentelemetry-instrumentation-actix-web = { version = "0.22", features = ["metrics"] }
opentelemetry-instrumentation-actix-web = { version = "0.23", features = ["metrics"] }

Comment on lines -45 to 46

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be worth querying the metric endpoint to see that they are properly exposed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like prometheus metrics are being properly exposed.

# opentelemetry dependencies
opentelemetry = { version = "0.29", features = ["metrics"]}
opentelemetry_sdk = { version = "0.29", features = ["rt-tokio-current-thread", "metrics"]}
opentelemetry-prometheus = "0.29"
opentelemetry = { version = "0.31", features = ["metrics"]}
opentelemetry_sdk = { version = "0.31", features = ["rt-tokio-current-thread", "metrics"]}
opentelemetry-prometheus = "0.31"
prometheus = "0.14"

# k8s-openapi must match the version required by kube and enable a k8s version feature
k8s-openapi = { version = "0.21", default-features = false, features = ["v1_24"] }
kube = { version = "0.88", default-features = false, features = [ "derive", "runtime", "rustls-tls" ] }
k8s-openapi = { version = "0.26", default-features = false, features = ["v1_30"] }

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this is set to v1_30 (the oldest supported by k8s-openapi@0.26) I didn't have any issues updating a k8s 1.29 cluster.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubernetes has a +/- 1 support story for the API, so 1.30 should work for 1.29.

We use very stable elements of the k8s API - theoretically very new versions of k8s could make changes that we don't support, but it is unlikely.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The https://kubernetes.io/releases/version-skew-policy/ says slightly different - the controller/client should not be newer than the api-server. So technically there is no guaranteed compatibility between controllers under version 1.30 and api-server under 1.29.

Is it possible to pin it to 1_29 until we drop the 1.29 variants?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's necessary since we've confirmed the ability to update on a 1.29 cluster. Extended support does drop soon, but I don't think we should hold these updates back when we've validated the operator functionality.

kube = { version = "2", default-features = false, features = [ "derive", "runtime", "rustls-tls" ] }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kube crate has a 3.0 now. Any concern if we further bump to that?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to maintain compatibility with v1_30. Jumping to kube@3.0 only guarantees API compatibility with a minimum of v1_31.


regex = "1"
reqwest = { version = "0.12", default-features = false, features = [ "json", "rustls-tls" ] }
rustls = { version = "0.23", default-features = false, features = ["aws_lc_rs", "logging", "std", "tls12"] }
rustls-pemfile = { version = "2" }
schemars = "0.8"
schemars = "1"
semver = "1"
serde = "1"
serde_json = "1"
Expand All @@ -72,4 +72,3 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["registry", "env-filter", "json"] }
uuid = { version = "1", default-features = false, features = ["serde", "v4"] }
validator = { version = "0.20", features = ["derive"] }
webpki = { version = "0.22", features = ["std"] }
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ DESTDIR ?= .
# tarball.
DISTFILE ?= $(DESTDIR:/=)/$(subst /,_,$(IMAGE_NAME)).tar.gz

BOTTLEROCKET_SDK_VERSION = v0.65.0
BOTTLEROCKET_SDK_VERSION = v0.70.0

# Tools used during the chart release lifecycle
export KUBECONFORM_VERSION = v0.6.3
Expand Down
8 changes: 6 additions & 2 deletions agent/src/agentclient.rs
Original file line number Diff line number Diff line change
Expand Up @@ -713,7 +713,9 @@ pub mod agentclient_error {
},

#[snafu(display("Unable to operate on BottlerocketShadow: '{}'", error))]
BottlerocketShadowError { error: BottlerocketShadowRWError },
BottlerocketShadowError {
error: Box<BottlerocketShadowRWError>,
},

#[snafu(display("Agent client failed due to internal assertion issue: '{}'", message))]
Assertion { message: String },
Expand All @@ -737,7 +739,9 @@ pub mod agentclient_error {

impl From<BottlerocketShadowRWError> for Error {
fn from(err: BottlerocketShadowRWError) -> Self {
Self::BottlerocketShadowError { error: err }
Self::BottlerocketShadowError {
error: Box::new(err),
}
}
}

Expand Down
3 changes: 1 addition & 2 deletions apiserver/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ awc = { workspace = true }
opentelemetry-instrumentation-actix-web = { workspace = true }
rustls = { workspace = true }
rustls-pemfile = { workspace = true }
webpki = { workspace = true }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What prompts this removal? Is it just marked as unused?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just unused. We could leave it but it adds an unnecessary dependency and we have to keep clarify.toml and deny.toml up to date to accommodate it.

opentelemetry = { workspace = true }
opentelemetry_sdk = { workspace = true }
opentelemetry-prometheus = { workspace = true }
Expand All @@ -40,7 +39,7 @@ log = { workspace = true }
mockall = { workspace = true, optional = true }
nonzero_ext = { workspace = true }
reqwest = { workspace = true }
schemars = "0.8.11"
schemars = { workspace = true }
serde = { workspace = true, features = [ "derive" ] }
serde_json = { workspace = true }
snafu = { workspace = true }
Expand Down
6 changes: 5 additions & 1 deletion apiserver/src/auth/authorizor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,11 @@ pub(crate) mod test {
) -> K8STokenAuthorizor<MockTokenReviewer> {
let mut pod_store = reflector::store::Writer::<Pod>::default();
let pod_reader = pod_store.as_reader();
pod_store.apply_watcher_event(&Event::Restarted(pods));
pod_store.apply_watcher_event(&Event::Init);
for pod in pods {
pod_store.apply_watcher_event(&Event::InitApply(pod));
}
pod_store.apply_watcher_event(&Event::InitDone);
Comment on lines -278 to +282

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what the context on this change is?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was an API change in the kube-runtime reflector store. Basically Event::Restarted was broken up into three phases.


K8STokenAuthorizor::new(reviewer, namespace.to_string(), pod_reader, audiences)
}
Expand Down
2 changes: 1 addition & 1 deletion apiserver/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ pub mod apiserver_error {

#[snafu(display("Error creating prometheus registry: '{}'", source))]
PrometheusRegsitry {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we have some pre-existing typo...

Suggested change
PrometheusRegsitry {
PrometheusRegistry {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is an already big change I'd like to save this for a separate PR.

source: opentelemetry_sdk::metrics::MetricError,
source: opentelemetry_sdk::error::OTelSdkError,
},
}
}
8 changes: 1 addition & 7 deletions clarify.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ license-files = [
expression = "ISC AND (Apache-2.0 OR ISC) AND OpenSSL AND MIT"
license-files = [
{ path = "LICENSE", hash = 0xf308ccd7 },
{ path = "aws-lc/LICENSE", hash = 0xba56ed2c },
{ path = "aws-lc/LICENSE", hash = 0x2ff829bc },
{ path = "aws-lc/third_party/fiat/LICENSE", hash = 0x75829ee2 },
]

Expand Down Expand Up @@ -142,9 +142,3 @@ license-files = [
{ path = "LICENSE-MIT", hash = 0x386ca1bc },
{ path = "LICENSE.md", hash = 0xfe66d806 },
]

[clarify.rustls-webpki]
expression = "ISC"
license-files = [
{ path = "LICENSE", hash = 0x001c7e6c },
]
2 changes: 1 addition & 1 deletion controller/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ pub mod controller_error {

#[snafu(display("Error creating prometheus registry: '{}'", source))]
PrometheusRegsitry {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we have pre-existing typo

Suggested change
PrometheusRegsitry {
PrometheusRegistry {

source: opentelemetry_sdk::metrics::MetricError,
source: opentelemetry_sdk::error::OTelSdkError,
},
}
}
26 changes: 0 additions & 26 deletions deny.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,29 +38,3 @@ skip-tree = [
# Deny crates from unknown registries or git repositories.
unknown-registry = "deny"
unknown-git = "deny"

[[licenses.clarify]]
name = "ring"
expression = "Apache-2.0 AND ISC AND MIT AND OpenSSL"
license-files = [
{ path = "LICENSE", hash = 0xd99693a },
{ path = "LICENSE-BoringSSL", hash = 0x8f795a3 },
{ path = "LICENSE-other-bits", hash = 0x8b770de2 },
{ path = "third_party/fiat/LICENSE", hash = 0x931a8dd4 },
{ path = "src/polyfill/once_cell/LICENSE-APACHE", hash = 0x24b54f4b },
{ path = "src/polyfill/once_cell/LICENSE-MIT", hash = 0x69371061 },
]

[[licenses.clarify]]
name = "webpki"
expression = "ISC"
license-files = [
{ path = "LICENSE", hash = 0x001c7e6c }
]

[[licenses.clarify]]
name = "rustls-webpki"
expression = "ISC"
license-files = [
{ path = "LICENSE", hash = 0x001c7e6c },
]
18 changes: 14 additions & 4 deletions deploy/tests/golden/custom-resource-definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,10 @@ spec:
description: Auto-generated derived type for BottlerocketShadowSpec via `CustomResource`
properties:
spec:
description: The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state, and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
description: |-
The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node
linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state,
and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
properties:
state:
description: Records the desired state of the `BottlerocketShadow`
Expand All @@ -72,7 +75,9 @@ spec:
- state
type: object
status:
description: '`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent, while the spec is updated by the brupop controller.'
description: |-
`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent,
while the spec is updated by the brupop controller.
nullable: true
properties:
crash_count:
Expand Down Expand Up @@ -130,7 +135,10 @@ spec:
description: Auto-generated derived type for BottlerocketShadowSpec via `CustomResource`
properties:
spec:
description: The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state, and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
description: |-
The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node
linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state,
and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
properties:
state:
description: Records the desired state of the `BottlerocketShadow`
Expand All @@ -154,7 +162,9 @@ spec:
- state
type: object
status:
description: '`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent, while the spec is updated by the brupop controller.'
description: |-
`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent,
while the spec is updated by the brupop controller.
nullable: true
properties:
current_state:
Expand Down
20 changes: 15 additions & 5 deletions deploy/tests/snapshots/insta_tests__generated_crds.snap
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
source: deploy/tests/insta_tests.rs
assertion_line: 30
expression: crds
---
apiVersion: apiextensions.k8s.io/v1
Expand Down Expand Up @@ -52,7 +53,10 @@ spec:
description: Auto-generated derived type for BottlerocketShadowSpec via `CustomResource`
properties:
spec:
description: The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state, and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
description: |-
The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node
linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state,
and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
properties:
state:
description: Records the desired state of the `BottlerocketShadow`
Expand All @@ -76,7 +80,9 @@ spec:
- state
type: object
status:
description: '`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent, while the spec is updated by the brupop controller.'
description: |-
`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent,
while the spec is updated by the brupop controller.
nullable: true
properties:
crash_count:
Expand Down Expand Up @@ -134,7 +140,10 @@ spec:
description: Auto-generated derived type for BottlerocketShadowSpec via `CustomResource`
properties:
spec:
description: The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state, and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
description: |-
The `BottlerocketShadowSpec` can be used to drive a node through the update state machine. A node
linearly drives towards the desired state. The brupop controller updates the spec to specify a node's desired state,
and the host agent drives state changes forward and updates the `BottlerocketShadowStatus`.
properties:
state:
description: Records the desired state of the `BottlerocketShadow`
Expand All @@ -158,7 +167,9 @@ spec:
- state
type: object
status:
description: '`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent, while the spec is updated by the brupop controller.'
description: |-
`BottlerocketShadowStatus` surfaces the current state of a bottlerocket node. The status is updated by the host agent,
while the spec is updated by the brupop controller.
nullable: true
properties:
current_state:
Expand Down Expand Up @@ -189,4 +200,3 @@ spec:
storage: false
subresources:
status: {}