Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions oeps/architectural-decisions/oep-0045-arch-ops-and-config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ OEP-45: Configuring and Operating Open edX
* - Title
- Configuring and Operating Open edX
* - Last Modified
- 2024-05-17
- 2026-07-03
* - Authors
- Bill DeRusha <bill@edx.org>
* - Arbiter
Expand Down Expand Up @@ -129,7 +129,7 @@ Additionally it is not clear which settings are required to be overridden and wh

The settings defined in ``required.py`` and ``defaults.py`` files are mutually exclusive, representing all IDA specific settings as well as installed library settings whose values either must be provided or whose defaults are not considered production-ready.

``required.py`` variables must be overridden by operators. The application will check that operators provided these values, and will not start unless they are set. This allows operators to fail fast rather than finding out about an unset value when users exercise those breaking codepaths. Application developers are encouraged to keep the list of required settings to a minimum.
``required.py`` variables must be overridden by operators. The application will check that operators provided these values, and will not start unless they are set. This allows operators to fail fast rather than finding out about an unset value when users exercise those breaking code paths. Application developers are encouraged to keep the list of required settings to a minimum.

This new settings structure obviates the need for any other python files in the settings directory (such as ``devstack.py``, ``test.py``, etc). The values currently set in those files should be moved to a corresponding ``devstack.yml``, ``test.yml``, etc in the same settings directory. This gives developers and operators more consistency across environments since the same code paths are being executed with different values.

Expand Down Expand Up @@ -198,7 +198,7 @@ There are many django project which configure their applications by grabbing the
Implementation Strategy
***********************

Discussion of implentation of this OEP will happen in a `separate Pull Request`_ .
Discussion of implementation of this OEP will happen in a `separate Pull Request`_ .

.. _separate Pull Request: https://github.com/openedx/openedx-proposals/pull/144

Expand All @@ -217,6 +217,11 @@ The following related decisions modify or enhance this OEP, but have not yet bee
Change History
**************

2026-07-03
==========

* Addressing typos in the documentation

2024-05-16
==========
* Change status to "Needs Revision"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Deploying Open edX on Kubernetes Using Helm
###########################################
Provisioning Open edX Infrastructure on Kubernetes
##################################################


Status
Expand All @@ -26,40 +26,53 @@ Despite this, many major commercial Open edX hosting providers have migrated tow
Decision
********

For Open edX operators who wish to deploy Open edX on Kubernetes, community-maintained `Helm Charts`_ will be used to simplify the process of deploying a kubernetes cluster capable of loading one or more Open edX instances.
For Open edX operators who wish to deploy Open edX on Kubernetes, community-maintained `Terraform`_ modules and `Helm Charts`_ will be used to simplify the process of provisioning a Kubernetes cluster capable of loading one or more Open edX instances.

Building on top of the `decision 0001`_ those single instances will in turn be managed by Tutor and the Tutor plugin ecosystem.
Building on top of the `decision 0001`_, individual instances will be configured by Tutor and the Tutor plugin ecosystem.

As the needs and deployment scale of Open edX operators can vary significantly, the Helm charts will be designed to be flexible, incorporating sensible defaults but allowing customization of almost every aspect of the deployment. This includes supporting various cloud providers and different underlying technologies for Kubernetes resources. E.g. Ingress controllers.
As the needs and deployment scale of Open edX operators can vary significantly, the Terraform modules and Helm charts will be designed to be flexible, incorporating sensible defaults but allowing customization of almost every aspect of the deployment. This includes supporting various cloud providers and different underlying technologies for Kubernetes resources. E.g. Ingress controllers.

The goal is for these Helm charts to be developed and maintained collaboratively, by the Open edX Operators that are using them. All relevant best practices from both the Open edX and Helm projects should be followed from the start, such as `OEP-55 Project Maintainers`_.
The goal is for these Terraform modules and Helm charts to be developed and maintained collaboratively, by the Open edX Operators that are using them. All relevant best practices from Open edX, popular Terraform modules, and Helm projects should be followed from the start, such as `OEP-55 Project Maintainers`_.


.. _OEP-55 Project Maintainers: https://docs.openedx.org/projects/openedx-proposals/en/latest/processes/oep-0055-proc-project-maintainers.html
.. _Terraform: https://developer.hashicorp.com/terraform
.. _Helm Charts: https://helm.sh/


Consequences
************

Terraform Modules
=================

Although several IaC solutions are available, Terraform is the most stable and widely recognized option. Its extensive module registry provides APIs and integrations for most cloud providers, SaaS platforms, and even local utilities. Terraform makes it predictable and convenient to set up infrastructure that Helm can then deploy to and operate on.

Terraform modules are written in HCL (HashiCorp Configuration Language), which makes them easy to understand. However, Terraform also adds another layer of complexity to the already complex Open edX stack.


Helm Considerations
===================

Helm is the best known package manager for Kubernetes. It's status as a graduated project from the Cloud Native Computing Foundation is a clear indication of the thriving adoption, open governance and commitment to the open community.

Helm makes it possible to package and publish charts, which can be rendered as kubernetes manifests. These charts provide robust support for versioning, deployment and rollback. At the same time Helm makes it easy for operators to customize application configurations during deployment and for developers to provide sensible defaults.

Charts are built with the Go language templating engine. This is a drawback, since it means adding a new language to the ecosystem with a new learning curve. Nevertheless, this is considered acceptable due to the large adoption of Helm in the devops community.
Charts are built with the Go language templating engine. This is a drawback, since it means adding a new language to the ecosystem with a new learning curve. Nevertheless, this is considered acceptable due to the large adoption of Helm in the devops community.


Best Practices
==============

**Cloud Providers**

As much as possible, the Helm chart should aim to be agnostic toward the underlying cloud provider used (e.g. AWS, Azure, Google Cloud Platform, etc.). In cases where provider-specific code is required, the Charts should be written in such a way that they can be configured to work with multiple providers, and not e.g. only support AWS.
As much as possible, the Terraform modules and Helm chart should aim to be agnostic toward the underlying cloud provider used (e.g. AWS, Azure, Google Cloud Platform, etc.). In cases where provider-specific code is required, the Charts should be written in such a way that they can be configured to work with multiple providers, and not e.g. only support AWS.

**Terraform Modularity**

**Helm subcharts and modularity**
The community-maintained Terraform modules should aim for being standalone and provider agnostic as much as possible. In such cases when it is not possible (e.g., setting up an EKS cluster on AWS), the Terraform module boundaries should be clear enough that they can be used independently. For example, an operator may reuse the community-maintained Terraform module for setting up an EKS cluster on AWS, but that should not pull in dependencies on the RDS cluster module.

**Helm Subcharts and Modularity**

The project should aim to make the charts available in a way that is composable and granular. Extending a single subchart and using it in the composition should be a better and faster alternative than forking the whole project.

Expand Down Expand Up @@ -113,7 +126,13 @@ A `working proof of concept`_ that was written as part of the research for this
Change History
**************

2023--30
2026-07-03
==========

* Extend the decision with Terraform which is integral part of the repository.
* Rename the decision to reflect its purpose better.

2023-01-30
==========

* Document updated to include community feedback.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
Deploying Open edX on Instances Kubernetes
##########################################

Status
******

Draft


Context
*******

Hosting Open edX instances on Kubernetes requires a complex infrastructure setup, especially when multiple instances are involved. Decisions made during cluster setup can directly affect the performance, latency, and reliability of those instances

The `decision 0002`_ elaborates on the need for community-maintained components to reduce this complexity. Although the Terraform modules and Helm chart support a modular setup already removing a significant part of the complexity, they do not fully abstract that away. Furthermore, none of these components are addressing the significant challenge of deploying instances on Kubernetes.

Open edX commercial providers are addressing this challenge with purpose-made solutions that are mostly not compatible with each other, although they are built using some community-maintained and provided components.

.. _decision 0002: https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.html


Decision
********

We recommend that all large Open edX deployments use the community-maintained Terraform modules and Helm chart described in decision 0002, existing solutions in the Open edX ecosystem (`GitHub Actions`_), provider-maintained open-source tooling (`Drydock`_ and `Picasso`_), and cloud-native, provider-agnostic deployment tools (`ArgoCD`_, and `Argo Workflows`_). We encourage providers to share any additional deployment code/tooling they use as open source repos or templates that other providers can use as a reference. If any particular template such as OpenCraft's `Launchpad`_ becomes adopted by multiple providers, it should be considered for adoption as a community standard, and this OEP should be updated to recommend its use.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In decision 2, the framing is:

For , the community maintains , and will make it compatible with

For Open edX operators who wish to deploy Open edX on Kubernetes, community-maintained Terraform modules and Helm Charts will be used to simplify the process of provisioning a Kubernetes cluster capable of loading one or more Open edX instances.

Building on top of the decision 0001, individual instances will be configured by Tutor and the Tutor plugin ecosystem

I like that framing better than to offer a straight up recommendation. Part of the appeal of OpenedX is the chameleon approach.

How do you feel about using something like:

A coalition of commercial providers maintains the Terraform modules and Helm chart described in decision 0002, as well as a growing ecosystem of open-source deployment tools. For operators deploying Open edX on Kubernetes, these resources (Drydock, Picasso, ArgoCD, Argo Workflows and GitHub Actions) provide flexible reference architectures.
We recognize that Open edX deployments vary widely in scale and infrastructure. Operators are free to blend these community tools with their own existing stack, and we encourage providers to share new deployment code, modules, or templates with the community.

Because Open edX intentionally supports diverse infrastructure needs, none of these tools are mandatory. Instead, the community focuses its efforts on ensuring these reference implementations and provider templates like OpenCraft's Launchpad, are maintained well enough to serve as reliable blueprints.


.. _GitHub Actions: https://github.com/features/actions
.. _Drydock: https://github.com/eduNEXT/drydock
.. _Picasso: https://github.com/eduNEXT/picasso
.. _ArgoCD: https://argo-cd.readthedocs.io/en/stable/
.. _Argo Workflows: https://argoproj.github.io/workflows/
.. _Launchpad: https://github.com/open-craft/launchpad-cluster-template


Consequences
************

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add an introductory session, that explains these consequences and recommendations apply to those that decide to take the joint path of harmony+tools(argo, drydock,...)

Build Pipeline Considerations
=============================

In order to build Open edX instance images with Tutor, the CI/CD pipelines should be executed. The cluster template is going to use Picasso to simplify the image building process, though this binds the rendered clusters to GitHub Actions.


Deployment Pipeline Considerations
==================================

The deployment of instances are going to be handled by ArgoCD that reads the Kubernetes manifests rendered by Tutor and pushed to the cluster repository by GitHub Actions. Although this keeps continuous deployment convenient, easy to oversee, and ensures that every change can be audited, secrets are ending up in the version control system.

This is the result of some limitations coming from Tutor-rendered Kubernetes manifests.


Best Practices
==============

In order to keep a healthy state for the cluster template, these best practices should be followed:

* All infrastructure dependency should be coming from the community-maintained and provided Terraform modules and Helm charts, laid out in decision 0002.
* Per-instance resource provisioning (e.g., database users) should be as much provider-agnostic as possible.
* No company- or Open edX provider-specific logic should live in the cluster template or rendered cluster.
* The instance deployment should be handled by cloud provider-agnostic tooling.
* The CI/CD pipelines should do only the bare-minimum needed for instance setup.
* No fragile or dangerous operations should be performed by the CI/CD pipelines except the instance deprovisioning flow.
* The operators should handle the infrastructure modifications from their computer rather than from CI/CD pipelines.
* Infrastructure provisioning should be done by the 3rd-party CLI tooling (such as `argo`, `tofu`/`terraform`, etc.) or by the scripts provided by Launchpad that is wrapping these 3rd-party tools.


Change History
**************

2026-07-03
==========

* Document created
1 change: 1 addition & 0 deletions oeps/redirects.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@
"processes/oep-0056-proc-architectural-advisory-process.rst" "archived/oep-0056-proc-architectural-advisory-process.rst"
"architectural-decisions/oep-0006-arch-context-xblock-fields.rst" "archived/oep-0006-arch-context-xblock-fields.rst"
"best-practices/oep-0018-bp-python-dependencies.rst" "archived/oep-0018-bp-python-dependencies.rst"
"architectural-decisions/oep-0045/decisions/0002-helm-for-multi-instance-k8s.rst" "architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.rst"