Skip to content

Commit ce84252

Browse files
authored
Kubernetes deployment instructions updates (#50)
* update instructions for deploying prometheus on k8s * add explorer k8s instruction * add ruleset upgrade instructions * update alert manager config
1 parent 281b141 commit ce84252

7 files changed

Lines changed: 239 additions & 12 deletions

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
{
22
"docker": "Docker",
3+
"kubernetes": "Kubernetes"
34
}

src/pages/deploying-alertmanager/docker.mdx

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ short: Docker
44
---
55

66
import { Steps } from "nextra/components";
7+
import CreateSlackApplication from "@/snippets/create-slack-application.mdx"
78

89
# Deploying Alertmanager on Docker
910

@@ -17,7 +18,7 @@ Alertmanager is available as a Docker image from the official Prometheus organiz
1718

1819
Make sure you have Prometheus running in the environment you're hoping to connect the Alertmanager to. See [Deploying Prometheus to Docker](deploying-prometheus/docker).
1920

20-
### Autometrics-instrumented application with defined SLOs
21+
### Autometrics-instrumented application with SLOs
2122

2223
Autometrics enables you to set up smarter alerts using the Service Level Objectives (SLO) approach. You can read more about SLOs in Autometrics [here](/slo) and follow the guides for SLO-instrumentation under each supported language:
2324

@@ -34,11 +35,7 @@ In the following guide we'll deploy a basic Alertmanager instance configured to
3435

3536
### Create a Slack application
3637

37-
First, we need to create a Slack application that will be used to send notifications to Slack. You can create a new Slack application [here](https://api.slack.com/apps). Make sure to install it to the workspace you want to send notifications to.
38-
39-
Once you've created and installed the application, you'll need to get the bot token that will be used to send notifications. You can find it under the `OAuth & Permissions` tab in the application settings.
40-
41-
It will look something like this: `xoxb-*****`.
38+
<CreateSlackApplication />
4239

4340
### Create a configuration file for Alertmanager
4441

@@ -98,4 +95,4 @@ alerting:
9895
- "alertmanager:9093" # or the host address of the Alertmanager instance
9996
```
10097

101-
</Steps>
98+
</Steps>
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
title: Deploying Alertmanager in Kubernetes
3+
short: Kubernetes
4+
---
5+
6+
import { Steps, Tabs, Tab } from "nextra/components"
7+
import CreateSlackApplication from "@/snippets/create-slack-application.mdx"
8+
9+
# Deploying Alertmanager in Kubernetes
10+
11+
Alertmanager is a utility service deployed alongside Prometheus to help route, group, and prioritize alerts. Alertmanager does not trigger alerts - alerting *rules* are set in Prometheus itself, Alertmanager only takes care of the lifecycle of the alert *after* it has been triggered (you can think of it as Zapier but for Prometheus alerts).
12+
13+
Alertmanager is part of the Prometheus stack, the most popular open-source monitoring solution for Kubernetes, therefore deploying and configuring it is pretty straightforward.
14+
15+
## Requirements
16+
- Kubernetes cluster running with `kubectl` configured to access it
17+
- [Helm](https://helm.sh/docs/intro/install) installed
18+
19+
### Autometrics-instrumented application with SLOs
20+
21+
Autometrics enables you to set up smarter alerts using the Service Level Objectives (SLO) approach. You can read more about SLOs in Autometrics [here](/slo) and follow the guides for SLO-instrumentation under each supported language:
22+
23+
- [Rust](/rust/adding-alerts-and-slos)
24+
- [TypeScript](/typescript/adding-alerts-and-slos)
25+
- [Go](/go/adding-alerts-and-slos)
26+
- [Python](/python/adding-alerts-and-slos)
27+
28+
## Instructions
29+
30+
<Steps>
31+
32+
### Create a `monitoring` namespace
33+
34+
It is a good practice to keep all monitoring resources in a separate namespace. To create a namespace run the following command (if you already have a namespace for monitoring tools, you can skip this step):
35+
36+
```bash
37+
kubectl create namespace monitoring
38+
```
39+
40+
### Deploy Alertmanager to Kubernetes
41+
42+
Run the following command to install Alertmanager using the Helm chart. If you've already installed Prometheus using Helm, you can skip this step.
43+
44+
```bash
45+
helm install prometheus prometheus-community/kube-prometheus-stack --namespace monitoring
46+
```
47+
48+
This will install Alertmanager in your cluster along with Prometheus, Node Exporter, and a few other components. As these components are installed together they are configured to work with each other out of the box.
49+
50+
</Steps>
51+
52+
You can now configure your Alertmanager to group, organize, and send alerts to whichever channel you're using for incident response. See [Alertmanager documentation](https://prometheus.io/docs/alerting/latest/configuration/) for details.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"docker": "Docker",
3+
"kubernetes": "Kubernetes",
34
"northflank": "Northflank",
45
"railway": "Railway"
56
}
Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
---
2+
title: Deploying Explorer in Kubernetes
3+
short: Kubernetes
4+
---
5+
6+
import { Steps, Tabs, Tab } from "nextra/components"
7+
8+
# Deploying Explorer in Kubernetes
9+
10+
Explorer can be deployed in Kubernetes using the Docker image of the `am` CLI tool.
11+
12+
#### Requirements
13+
- Kubernetes cluster running with `kubectl` configured to access it
14+
15+
<Steps>
16+
17+
### Create a `monitoring` namespace
18+
19+
It is a good practice to keep all monitoring resources in a separate namespace. To create a namespace run the following command:
20+
21+
```bash
22+
kubectl create namespace monitoring
23+
```
24+
25+
(If you already have a namespace for monitoring, you can skip this step)
26+
27+
### Create a deployment and service configuration
28+
29+
In order to deploy Explorer, we need it to successfully connect to Prometheus and be made accessible from outside the cluster. To do that we will create a deployment and a service configuration.
30+
31+
As the configuration is quite simple we keep it in a single file, but it is also possible to split it into two files.
32+
33+
Create a file named `explorer.yaml` with the following content:
34+
35+
```yaml
36+
apiVersion: apps/v1
37+
kind: Deployment
38+
metadata:
39+
name: explorer
40+
namespace: monitoring
41+
labels:
42+
app: explorer
43+
spec:
44+
selector:
45+
matchLabels:
46+
app: explorer
47+
replicas: 1
48+
template:
49+
metadata:
50+
labels:
51+
app: explorer
52+
spec:
53+
containers:
54+
- name: explorer
55+
image: autometrics/am-proxy:latest
56+
imagePullPolicy: IfNotPresent
57+
env:
58+
- name: PROMETHEUS_URL
59+
value: http://<prometheus_service_name>.<namespace>.svc.cluster.local
60+
---
61+
apiVersion: v1
62+
kind: Service
63+
metadata:
64+
name: explorer
65+
namespace: monitoring
66+
spec:
67+
selector:
68+
app: explorer
69+
type: LoadBalancer
70+
ports:
71+
- port: 6789
72+
targetPort: 6789 # Explorer default port
73+
```
74+
75+
Note: the `LoadBalancer` configuration will vary depending on the Kubernetes provider and will need to be adapted accordingly. The above configuration is for a Kubernetes cluster running locally with [minikube](https://minikube.sigs.k8s.io/docs/).
76+
77+
### Apply the configuration
78+
79+
To apply the configuration run the following command:
80+
81+
```bash
82+
kubectl apply -f explorer.yaml
83+
```
84+
85+
### Verify the deployment
86+
87+
To verify that the deployment was successful, navigate to the URL of the service. If the deployment was successful, you should see the Explorer UI.
88+
89+
</Steps>

src/pages/deploying-prometheus/kubernetes.mdx

Lines changed: 87 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,51 @@
22
title: Deploying Prometheus in Kubernetes
33
short: Kubernetes
44
---
5+
6+
import { Steps, Tabs, Tab } from "nextra/components"
7+
58
# Deploying Prometheus in Kubernetes
69

10+
Prometheus is the most popular open-source monitoring solution for Kubernetes, therefore deploying and configuring it is pretty straightforward.
11+
712
Prometheus supports service discovery, so it can automatically find Kubernetes
813
pods to scrape.
914

10-
If you have Prometheus installed using the [helm
11-
chart](https://github.com/prometheus-community/helm-charts), service discovery
12-
should be enabled by default.
15+
## Requirements
16+
17+
- Kubernetes cluster running with `kubectl` configured to access it
18+
- Make sure you have [Helm](https://helm.sh/docs/intro/install) installed
19+
20+
21+
## Instructions
22+
23+
<Steps>
24+
25+
### Create a `monitoring` namespace
26+
27+
It is a good practice to keep all monitoring resources in a separate namespace.
28+
29+
```bash
30+
kubectl create namespace monitoring
31+
```
32+
33+
### Deploy Prometheus to Kubernetes
34+
35+
36+
Run the following command to install Prometheus using the Helm chart:
37+
38+
```bash
39+
helm install prometheus prometheus-community/kube-prometheus-stack --namespace monitoring
40+
```
41+
42+
This will install Prometheus in your cluster along with Alertmanager, Node Exporter, and a few other components.
43+
44+
45+
### Make Autometrics discoverable
1346

14-
Add the following annotations to your app's deployment:
47+
If you have installed Prometheus using the [helm chart](https://github.com/prometheus-community/helm-charts), service discovery should be enabled by default.
48+
49+
Service discovery uses Kubernetes labels to tell Prometheus which ports to look for. In order to make sure that your application metrics are available add the following annotations to your app's deployment:
1550

1651
```yaml filename="deployment.yaml"
1752
apiVersion: apps/v1
@@ -22,5 +57,52 @@ spec:
2257
annotations:
2358
prometheus.io/scrape: "true"
2459
prometheus.io/path: "/metrics"
25-
prometheus.io/port: "9091"
60+
prometheus.io/port: "9464" # this can be different depending on which library you have used for instrumentation
61+
```
62+
63+
### (Optional) Add Autometrics ruleset to Prometheus
64+
65+
If you want to make use of the SLO alerting features in Autometrics, you need to add the Autometrics alerting rules to your Prometheus instance.
66+
67+
You can find the ruleset in the [Autometrics repository](https://github.com/autometrics-dev/autometrics-shared/blob/main/autometrics.rules.yml).
68+
69+
#### Save the ruleset to a file
70+
71+
```bash
72+
curl https://raw.githubusercontent.com/autometrics-dev/autometrics-shared/main/autometrics.rules.yml > autometrics.rules.yml
73+
```
74+
75+
#### Create a ConfigMap from the ruleset
76+
77+
```bash
78+
kubectl create configmap autometrics-rules --from-file=autometrics.rules.yml --namespace monitoring
2679
```
80+
81+
#### Create a `values.yaml` file for the Prometheus Helm chart
82+
83+
The `values.yaml` file is used to add extra configuration to the Prometheus Helm chart. In this case we will mount the extra configuration as a ConfigMap and modify Prometheus configuration to include the Autometrics ruleset.
84+
85+
```yaml filename="values.yaml"
86+
server:
87+
extraConfigmapMounts:
88+
- name: autometrics-rules
89+
mountPath: /autometrics-alerts/
90+
configMap: autometrics-rules
91+
readOnly: true
92+
serverFiles:
93+
prometheus.yml:
94+
rule_files:
95+
- /etc/config/recording_rules.yml # leave whichever rules you have already
96+
- /etc/config/alerting_rules.yml #
97+
- /autometrics-alerts/autometrics.rules.yml # <- add the autometrics ruleset here
98+
```
99+
100+
#### Upgrade Prometheus using Helm
101+
102+
Upgrade the Prometheus installation using the `helm upgrade` command, passing in the `values.yaml` file.
103+
104+
```bash
105+
helm upgrade prometheus prometheus-community/kube-prometheus-stack --namespace monitoring -f values.yaml
106+
```
107+
108+
</Steps>
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
First, we need to create a Slack application that will be used to send notifications to Slack. You can create a new Slack application [here](https://api.slack.com/apps). Make sure to install it to the workspace you want to send notifications to.
2+
3+
Once you've created and installed the application, you'll need to get the bot token that will be used to send notifications. You can find it under the `OAuth & Permissions` tab in the application settings.
4+
5+
It will look something like this: `xoxb-*****`.

0 commit comments

Comments
 (0)