diff --git a/oeps/architectural-decisions/oep-0045-arch-ops-and-config.rst b/oeps/architectural-decisions/oep-0045-arch-ops-and-config.rst index 1a998d978..6dc3c35c7 100644 --- a/oeps/architectural-decisions/oep-0045-arch-ops-and-config.rst +++ b/oeps/architectural-decisions/oep-0045-arch-ops-and-config.rst @@ -11,7 +11,7 @@ OEP-45: Configuring and Operating Open edX * - Title - Configuring and Operating Open edX * - Last Modified - - 2024-05-17 + - 2026-07-03 * - Authors - Bill DeRusha * - Arbiter @@ -129,7 +129,7 @@ Additionally it is not clear which settings are required to be overridden and wh The settings defined in ``required.py`` and ``defaults.py`` files are mutually exclusive, representing all IDA specific settings as well as installed library settings whose values either must be provided or whose defaults are not considered production-ready. -``required.py`` variables must be overridden by operators. The application will check that operators provided these values, and will not start unless they are set. This allows operators to fail fast rather than finding out about an unset value when users exercise those breaking codepaths. Application developers are encouraged to keep the list of required settings to a minimum. +``required.py`` variables must be overridden by operators. The application will check that operators provided these values, and will not start unless they are set. This allows operators to fail fast rather than finding out about an unset value when users exercise those breaking code paths. Application developers are encouraged to keep the list of required settings to a minimum. This new settings structure obviates the need for any other python files in the settings directory (such as ``devstack.py``, ``test.py``, etc). The values currently set in those files should be moved to a corresponding ``devstack.yml``, ``test.yml``, etc in the same settings directory. This gives developers and operators more consistency across environments since the same code paths are being executed with different values. @@ -198,7 +198,7 @@ There are many django project which configure their applications by grabbing the Implementation Strategy *********************** -Discussion of implentation of this OEP will happen in a `separate Pull Request`_ . +Discussion of implementation of this OEP will happen in a `separate Pull Request`_ . .. _separate Pull Request: https://github.com/openedx/openedx-proposals/pull/144 @@ -217,6 +217,11 @@ The following related decisions modify or enhance this OEP, but have not yet bee Change History ************** +2026-07-03 +========== + +* Addressing typos in the documentation + 2024-05-16 ========== * Change status to "Needs Revision" diff --git a/oeps/architectural-decisions/oep-0045/decisions/0002-helm-for-multi-instance-k8s.rst b/oeps/architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.rst similarity index 64% rename from oeps/architectural-decisions/oep-0045/decisions/0002-helm-for-multi-instance-k8s.rst rename to oeps/architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.rst index 10e13bbc7..d9bb66d97 100644 --- a/oeps/architectural-decisions/oep-0045/decisions/0002-helm-for-multi-instance-k8s.rst +++ b/oeps/architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.rst @@ -1,5 +1,5 @@ -Deploying Open edX on Kubernetes Using Helm -########################################### +Provisioning Open edX Infrastructure on Kubernetes +################################################## Status @@ -26,22 +26,31 @@ Despite this, many major commercial Open edX hosting providers have migrated tow Decision ******** -For Open edX operators who wish to deploy Open edX on Kubernetes, community-maintained `Helm Charts`_ will be used to simplify the process of deploying a kubernetes cluster capable of loading one or more Open edX instances. +For Open edX operators who wish to deploy Open edX on Kubernetes, community-maintained `Terraform`_ modules and `Helm Charts`_ will be used to simplify the process of provisioning a Kubernetes cluster capable of loading one or more Open edX instances. -Building on top of the `decision 0001`_ those single instances will in turn be managed by Tutor and the Tutor plugin ecosystem. +Building on top of the `decision 0001`_, individual instances will be configured by Tutor and the Tutor plugin ecosystem. -As the needs and deployment scale of Open edX operators can vary significantly, the Helm charts will be designed to be flexible, incorporating sensible defaults but allowing customization of almost every aspect of the deployment. This includes supporting various cloud providers and different underlying technologies for Kubernetes resources. E.g. Ingress controllers. +As the needs and deployment scale of Open edX operators can vary significantly, the Terraform modules and Helm charts will be designed to be flexible, incorporating sensible defaults but allowing customization of almost every aspect of the deployment. This includes supporting various cloud providers and different underlying technologies for Kubernetes resources. E.g. Ingress controllers. -The goal is for these Helm charts to be developed and maintained collaboratively, by the Open edX Operators that are using them. All relevant best practices from both the Open edX and Helm projects should be followed from the start, such as `OEP-55 Project Maintainers`_. +The goal is for these Terraform modules and Helm charts to be developed and maintained collaboratively, by the Open edX Operators that are using them. All relevant best practices from Open edX, popular Terraform modules, and Helm projects should be followed from the start, such as `OEP-55 Project Maintainers`_. .. _OEP-55 Project Maintainers: https://docs.openedx.org/projects/openedx-proposals/en/latest/processes/oep-0055-proc-project-maintainers.html +.. _Terraform: https://developer.hashicorp.com/terraform .. _Helm Charts: https://helm.sh/ Consequences ************ +Terraform Modules +================= + +Although several IaC solutions are available, Terraform is the most stable and widely recognized option. Its extensive module registry provides APIs and integrations for most cloud providers, SaaS platforms, and even local utilities. Terraform makes it predictable and convenient to set up infrastructure that Helm can then deploy to and operate on. + +Terraform modules are written in HCL (HashiCorp Configuration Language), which makes them easy to understand. However, Terraform also adds another layer of complexity to the already complex Open edX stack. + + Helm Considerations =================== @@ -49,7 +58,7 @@ Helm is the best known package manager for Kubernetes. It's status as a graduate Helm makes it possible to package and publish charts, which can be rendered as kubernetes manifests. These charts provide robust support for versioning, deployment and rollback. At the same time Helm makes it easy for operators to customize application configurations during deployment and for developers to provide sensible defaults. -Charts are built with the Go language templating engine. This is a drawback, since it means adding a new language to the ecosystem with a new learning curve. Nevertheless, this is considered acceptable due to the large adoption of Helm in the devops community. +Charts are built with the Go language templating engine. This is a drawback, since it means adding a new language to the ecosystem with a new learning curve. Nevertheless, this is considered acceptable due to the large adoption of Helm in the devops community. Best Practices @@ -57,9 +66,13 @@ Best Practices **Cloud Providers** -As much as possible, the Helm chart should aim to be agnostic toward the underlying cloud provider used (e.g. AWS, Azure, Google Cloud Platform, etc.). In cases where provider-specific code is required, the Charts should be written in such a way that they can be configured to work with multiple providers, and not e.g. only support AWS. +As much as possible, the Terraform modules and Helm chart should aim to be agnostic toward the underlying cloud provider used (e.g. AWS, Azure, Google Cloud Platform, etc.). In cases where provider-specific code is required, the Charts should be written in such a way that they can be configured to work with multiple providers, and not e.g. only support AWS. + +**Terraform Modularity** -**Helm subcharts and modularity** +The community-maintained Terraform modules should aim for being standalone and provider agnostic as much as possible. In such cases when it is not possible (e.g., setting up an EKS cluster on AWS), the Terraform module boundaries should be clear enough that they can be used independently. For example, an operator may reuse the community-maintained Terraform module for setting up an EKS cluster on AWS, but that should not pull in dependencies on the RDS cluster module. + +**Helm Subcharts and Modularity** The project should aim to make the charts available in a way that is composable and granular. Extending a single subchart and using it in the composition should be a better and faster alternative than forking the whole project. @@ -113,7 +126,13 @@ A `working proof of concept`_ that was written as part of the research for this Change History ************** -2023--30 +2026-07-03 +========== + +* Extend the decision with Terraform which is integral part of the repository. +* Rename the decision to reflect its purpose better. + +2023-01-30 ========== * Document updated to include community feedback. diff --git a/oeps/architectural-decisions/oep-0045/decisions/0003-deploying-instances-on-k8s.rst b/oeps/architectural-decisions/oep-0045/decisions/0003-deploying-instances-on-k8s.rst new file mode 100644 index 000000000..fca889fc8 --- /dev/null +++ b/oeps/architectural-decisions/oep-0045/decisions/0003-deploying-instances-on-k8s.rst @@ -0,0 +1,73 @@ +Deploying Open edX on Instances Kubernetes +########################################## + +Status +****** + +Draft + + +Context +******* + +Hosting Open edX instances on Kubernetes requires a complex infrastructure setup, especially when multiple instances are involved. Decisions made during cluster setup can directly affect the performance, latency, and reliability of those instances + +The `decision 0002`_ elaborates on the need for community-maintained components to reduce this complexity. Although the Terraform modules and Helm chart support a modular setup already removing a significant part of the complexity, they do not fully abstract that away. Furthermore, none of these components are addressing the significant challenge of deploying instances on Kubernetes. + +Open edX commercial providers are addressing this challenge with purpose-made solutions that are mostly not compatible with each other, although they are built using some community-maintained and provided components. + +.. _decision 0002: https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.html + + +Decision +******** + +We recommend that all large Open edX deployments use the community-maintained Terraform modules and Helm chart described in decision 0002, existing solutions in the Open edX ecosystem (`GitHub Actions`_), provider-maintained open-source tooling (`Drydock`_ and `Picasso`_), and cloud-native, provider-agnostic deployment tools (`ArgoCD`_, and `Argo Workflows`_). We encourage providers to share any additional deployment code/tooling they use as open source repos or templates that other providers can use as a reference. If any particular template such as OpenCraft's `Launchpad`_ becomes adopted by multiple providers, it should be considered for adoption as a community standard, and this OEP should be updated to recommend its use. + +.. _GitHub Actions: https://github.com/features/actions +.. _Drydock: https://github.com/eduNEXT/drydock +.. _Picasso: https://github.com/eduNEXT/picasso +.. _ArgoCD: https://argo-cd.readthedocs.io/en/stable/ +.. _Argo Workflows: https://argoproj.github.io/workflows/ +.. _Launchpad: https://github.com/open-craft/launchpad-cluster-template + + +Consequences +************ + +Build Pipeline Considerations +============================= + +In order to build Open edX instance images with Tutor, the CI/CD pipelines should be executed. The cluster template is going to use Picasso to simplify the image building process, though this binds the rendered clusters to GitHub Actions. + + +Deployment Pipeline Considerations +================================== + +The deployment of instances are going to be handled by ArgoCD that reads the Kubernetes manifests rendered by Tutor and pushed to the cluster repository by GitHub Actions. Although this keeps continuous deployment convenient, easy to oversee, and ensures that every change can be audited, secrets are ending up in the version control system. + +This is the result of some limitations coming from Tutor-rendered Kubernetes manifests. + + +Best Practices +============== + +In order to keep a healthy state for the cluster template, these best practices should be followed: + +* All infrastructure dependency should be coming from the community-maintained and provided Terraform modules and Helm charts, laid out in decision 0002. +* Per-instance resource provisioning (e.g., database users) should be as much provider-agnostic as possible. +* No company- or Open edX provider-specific logic should live in the cluster template or rendered cluster. +* The instance deployment should be handled by cloud provider-agnostic tooling. +* The CI/CD pipelines should do only the bare-minimum needed for instance setup. +* No fragile or dangerous operations should be performed by the CI/CD pipelines except the instance deprovisioning flow. +* The operators should handle the infrastructure modifications from their computer rather than from CI/CD pipelines. +* Infrastructure provisioning should be done by the 3rd-party CLI tooling (such as `argo`, `tofu`/`terraform`, etc.) or by the scripts provided by Launchpad that is wrapping these 3rd-party tools. + + +Change History +************** + +2026-07-03 +========== + +* Document created diff --git a/oeps/redirects.txt b/oeps/redirects.txt index 3937d8188..7a66953d1 100644 --- a/oeps/redirects.txt +++ b/oeps/redirects.txt @@ -26,3 +26,4 @@ "processes/oep-0056-proc-architectural-advisory-process.rst" "archived/oep-0056-proc-architectural-advisory-process.rst" "architectural-decisions/oep-0006-arch-context-xblock-fields.rst" "archived/oep-0006-arch-context-xblock-fields.rst" "best-practices/oep-0018-bp-python-dependencies.rst" "archived/oep-0018-bp-python-dependencies.rst" +"architectural-decisions/oep-0045/decisions/0002-helm-for-multi-instance-k8s.rst" "architectural-decisions/oep-0045/decisions/0002-openedx-hosting-infrastructure-on-k8s.rst"