Add Deploying otel collector + small improvements (#53)

* Include Deployment guide for Otel Collector
- Railway (complete)
- Northflank (draft)

Update ReadMe: include instructions how to run docs locally

Add: include the card for northflank to the deploying explorer page

* include deployment for otel collector on northflank

* Hide Prometheus deployment for local and fly.io

* Remove the word "deploying" from side bar

* adding links to autometrics-shared deployment resources

---------

Co-authored-by: Nele Lea Uhlemann <nlea@Neles-MBP.home>
Co-authored-by: Nele Lea Uhlemann <nlea@Neles-MBP.fritz.box>
Co-authored-by: Laurynas Keturakis <laurynas.keturakis@gmail.com>

Author: 4 people

README.md

@@ -3,3 +3,8 @@
 Uses Nextra to generate a static site from markdown files.
 
 The `_app.jsx` file contains some application setup logic.
+
+## Run the docs locally
+- Ensure you have [yarn](https://yarnpkg.com/) installed
+- In the repo run `yarn` to install the dependencies
+- Run `yarn dev` to spin up the side locally

public/images/northflank-otel-collector-networking.png

@@ -30,9 +30,10 @@
 		"type": "separator",
 		"title": "Deployment"
 	},
-	"deploying-prometheus": "Deploying Prometheus",
-	"deploying-explorer": "Deploying Explorer",
-	"deploying-alertmanager": "Deploying Alertmanager",
+	"deploying-prometheus": "Prometheus",
+	"deploying-explorer": "Explorer",
+	"deploying-alertmanager": "Alertmanager",
+	"deploying-otel-collector": "OTel Collector",
 	"other": {
 		"type": "separator",
 		"title": ""

public/images/railway-otel-collector-port-variable.png

@@ -1,5 +1,5 @@
 ---
-title: Deploying Alertmanager on Docker
+title: Alertmanager on Docker
 short: Docker
 ---
 

src/pages/_meta.json

@@ -1,6 +1,6 @@
 import { Cards, Card } from "nextra-theme-docs"
 
-# Deploying Explorer
+# Explorer
 
 Explorer is a special Autometrics-specific interface for Prometheus data. It is a simple client-based application that connects to a running Prometheus instance. Once connected it automatically recognizes Autometrics-labeled metrics and visualizes them in a simple and easy to use interface.
 
@@ -12,11 +12,13 @@ See [Local Development](/docs/local-development) for more information.
 
 ## Deploying Explorer
 
-The `am` CLI can be run as a simple service app and is available as a Docker image. See guides below for platform-specific instructions.
+The `am` CLI can be run as a simple service app and is available as a Docker image. See and guides below for platform-specific instructions.
 
 <Cards num={1}>
 	<Card
 	children icon title = "Deploying Explorer with Docker" href="/deploying-explorer/docker"/>
 	<Card
+	children icon title = "Deploying Explorer on Northflank" href="/deploying-explorer/northflank"/>
+	<Card
 	children icon title = "Deploying Explorer on Railway" href="/deploying-explorer/railway"/>
 </Cards>

src/pages/deploying-alertmanager/docker.mdx

@@ -0,0 +1,18 @@
+import { Cards, Card } from "nextra-theme-docs"
+
+#  OTel Collector 
+
+The [OpenTelemetry (OTel) Collector](https://opentelemetry.io/docs/collector/) functions as a vendor-neutral proxy in the OpenTelemetry ecosystem. It efficiently receives, processes, and exports telemetry data, offering versatility in handling various sources, formats, and protocols.
+The OTel Collector supports both scraping and pushing of telemetry data, providing flexibility for diverse data collection needs
+
+
+## Deploying OTel Collector
+
+The OTel Collector is available as a Docker image. See [resources](https://github.com/autometrics-dev/autometrics-shared/tree/main/deployment-guides-resources/otel-collector) and guides below for platform-specific instructions.
+
+<Cards num={1}>
+    <Card
+	children icon title = "Deploying OTel Collector on Northflank" href="/deploying-explorer/northflank"/>
+	<Card
+	children icon title = "Deploying OTel Collector on Railway" href="/deploying-explorer/railway"/>
+</Cards>

src/pages/deploying-explorer.mdx

@@ -0,0 +1,4 @@
+{
+	"northflank": "Northflank",
+	"railway": "Railway"
+}

src/pages/deploying-otel-collector.mdx

@@ -0,0 +1,76 @@
+---
+title: Deploying OTel Collector on Northflank
+short: Northflank
+---
+import { Steps } from "nextra/components";
+
+# Deploying OTel Collector on Northflank
+OTel Collector is available as an official [Docker image](https://hub.docker.com/r/otel/opentelemetry-collector) on Docker Hub. We can use it as a base image and layer in our configuration before deploying it on Railway.
+You find the resources for the Deployment [here](https://github.com/autometrics-dev/autometrics-shared/tree/main/deployment-guides-resources/otel-collector).
+
+<Steps>
+
+### Create a GitHub repository and a Otel Collector configuration file
+We will use Northflank's ability to deploy from a repository using a [Dockerfile](https://northflank.com/docs/v1/application/build/build-with-a-dockerfile) to set up our Prometheus instance.
+
+Create a file named `otel-collector-config.yaml` to specify a receiver endpoint for sending metrics from the instrumented application.
+
+
+```yaml {filename="otel-collector-config.yml"} showLineNumbers
+receivers:
+  otlp:
+    protocols:
+      http:
+        endpoint: 0.0.0.0:4318
+
+  pipelines:
+    metrics:
+      receivers: [otlp]
+```
+
+#### Optional: Export metrics
+As the OTel Collector acts as a proxy, we can define an exporter to send metrics. 
+In this example, metrics are pushed to [Prometheus's remote write receiver](https://prometheus.io/docs/prometheus/latest/querying/api/#remote-write-receiver). 
+Note that while Prometheus advises against this method for efficient sample ingestion, it is chosen here because Autometrics relies on Prometheus for superior querying and visualization capabilities.
+
+```yaml {filename="otel-collector-config.yml"} showLineNumbers
+exporters:
+  prometheusremotewrite:
+    endpoint: http://prometheus:9090/api/v1/write
+    resource_to_telemetry_conversion:
+      enabled: true # Convert resource attributes to
+
+  pipelines:
+    metrics:
+      exporters: [prometheusremotewrite]
+```
+You can find an example otel-config.yaml [here](https://github.com/autometrics-dev/autometrics-shared/blob/main/deployment-guides-resources/otel-collector/otel-collector-config.yaml).
+
+### Add a Dockerfile in the root of the repository
+The below [Dockerfile](https://github.com/autometrics-dev/autometrics-shared/blob/main/deployment-guides-resources/otel-collector/Dockerfiles) uses the base Otel Collector image and loads our configuration 
+
+```Dockerfile 
+FROM otel/opentelemetry-collector:latest
+ADD otel-collector-config.yaml /etc/otelcol/config.yaml
+```
+
+### Create a Northflank service from a GitHub repo
+Create a Northflank service from our newly created GitHub repo, it should pick up the configuration from the Dockerfile.
+
+### Add Network configurations
+In order to make Prometheus accessible to other services in your Northflank network or externally configure the PORT & DNS settings under the service network section in the Northflank UI. 
+
+![Northflank Network Settings for Otel Collector](/images/northflank-otel-collector-networking.png)
+
+
+### Configure application code
+ Utilize the 'push' capabilities in your chosen Autometrics language library, detailed in each language's quickstart section. 
+ When initializing the OTLP exporter, specify the endpoint for your deployed OTel Collector. 
+ Note that the OTel Collector expects metrics at **/v1/metrics**.
+ 
+
+ If you have deployed the OTel collector with the port set to 4318 your internal Northflank endpoint should look something like this: 
+
+ `http://name-of-collector-service:4318/v1/metrics`
+
+</Steps>

src/pages/deploying-otel-collector/_meta.json

@@ -0,0 +1,75 @@
+---
+title: Deploying OTel Collector on Railway
+short: Railway
+---
+import { Steps } from "nextra/components";
+
+# Deploying OTel Collector on Railway
+OTel Collector is available as an official [Docker image](https://hub.docker.com/r/otel/opentelemetry-collector) on Docker Hub. We can use it as a base image and layer in our configuration before deploying it on Railway
+You find the resources for the Deployment [here](https://github.com/autometrics-dev/autometrics-shared/tree/main/deployment-guides-resources/otel-collector).
+
+<Steps>
+
+### Create a GitHub repository and a Otel Collector configuration file
+To set up the OTel Collector instance, utilize Railway's repository deployment and leverage [Dockerfile features](https://docs.railway.app/deploy/dockerfiles). 
+Create a file named `otel-collector-config.yaml` to specify a receiver endpoint for sending metrics from the instrumented application
+
+```yaml {filename="otel-collector-config.yml"} showLineNumbers
+receivers:
+  otlp:
+    protocols:
+      http:
+        endpoint: 0.0.0.0:4318
+
+  pipelines:
+    metrics:
+      receivers: [otlp]
+```
+#### Optional: Export metrics
+As the OTel Collector acts as a proxy, we can define an exporter to send metrics. 
+In this example, metrics are pushed to [Prometheus's remote write receiver](https://prometheus.io/docs/prometheus/latest/querying/api/#remote-write-receiver). 
+Note that while Prometheus advises against this method for efficient sample ingestion, it is chosen here because Autometrics relies on Prometheus for superior querying and visualization capabilities.
+
+```yaml {filename="otel-collector-config.yml"} showLineNumbers
+exporters:
+  prometheusremotewrite:
+    endpoint: http://prometheus:9090/api/v1/write
+    resource_to_telemetry_conversion:
+      enabled: true # Convert resource attributes to
+
+  pipelines:
+    metrics:
+      exporters: [prometheusremotewrite]
+```
+You can find the full otel-config.yaml [here](https://github.com/autometrics-dev/autometrics-shared/blob/main/deployment-guides-resources/otel-collector/otel-collector-config.yaml).
+
+### Add a Dockerfile in the root of the repository
+The below [Dockerfile](https://github.com/autometrics-dev/autometrics-shared/blob/main/deployment-guides-resources/otel-collector/Dockerfiles) uses the base Otel Collector image and loads our configuration 
+
+```Dockerfile 
+FROM otel/opentelemetry-collector:latest
+
+ADD otel-collector-config.yaml /etc/otelcol/config.yaml
+```
+
+### Create a Railway service from a GitHub repo
+Create a Railway service from our newly created GitHub repo, it should pick up the configuration from the Dockerfile.
+
+### Add a PORT environment variable
+In order to make the OTel Collector accessible to other services in your Railway project or externally (optional) - you need to tell Railway which port to listen on using the "magic" PORT environment variable.
+
+To add it go to Service > Variables and add just the port number.
+![Railway add PORT variable for Otel Collector](/images/railway-otel-collector-port-variable.png)
+
+
+### Configure application code
+ Utilize the 'push' capabilities in your chosen Autometrics language library, detailed in each language's quickstart section. 
+ When initializing the OTLP exporter, specify the endpoint for your deployed OTel Collector. 
+ Note that the OTel Collector expects metrics at **/v1/metrics**.
+ 
+
+ If you have deployed the OTel collector with the port set to 4318 your internal Railway endpoint should look something like this: 
+
+ `http://name-of-collector-service:4318/v1/metrics`
+
+</Steps>