[DTOSS-12318] - feat(iac): add ADO pipeline and Terraform for Arc infrastructure

[josielsouzanordcloud]

Description

This PR introduces Azure DevOps (ADO) pipelines and Terraform modules for setting up Arc infrastructure. The changes include:

• Creation of ADO pipeline configuration files for deploying Arc infrastructure to different environments.
• Addition of issue templates and security documentation in the .github directory.
• Removal of unused destroy pipeline and script.
• Update of CI/CD workflows to include a deploy stage.
• Addition of Terraform module for Arc infrastructure, including resource groups, networking, and Bastion configuration.
• Replacement of environment-specific .sh files with a single .tfvars file for the review environment.
• Update of .gitignore to include Terraform state files and modules.

These changes enable the automated deployment and management of Arc infrastructure across different environments, enhancing the CI/CD pipeline and infrastructure as code (IaC) practices.

Type of change

• Bug fix
• New feature
• Breaking change
• Infrastructure change
• Pipeline / CI
• Documentation update
• Refactor

Jira link

DTOSS-12318
https://nhsd-jira.digital.nhs.uk/browse/DTOSS-12318

Review notes

• Pipeline and CI/CD:
  • Review the .azuredevops/pipelines/deploy.yml file to ensure the ADO pipeline triggers correctly and uses the right parameters.
  • Check the .github/workflows/stage-4-deploy.yaml workflow for proper deployment logic and environment handling.
  • Ensure the updated .github/workflows/cicd-1-pull-request.yaml and .github/workflows/cicd-2-main-branch.yaml workflows include the new deploy stage.
• Terraform Modules:
  • Verify the infrastructure/modules/arc-infra/ module files for correct resource definitions and variable usage.
  • Ensure the infrastructure/terraform/ main files include the new Arc infrastructure module and propagate variables correctly.
  • Confirm the removal of environment-specific .sh files and replacement with review/variables.tfvars is complete.
• Documentation and Issue Templates:
  • Review the added .github/ISSUE_TEMPLATE/ files for clarity and completeness.
  • Check the .github/SECURITY.md and docs/CODE_OF_CONDUCT.md files for accurate information and alignment with project standards.
• General:
  • Confirm the .tool-versions file is updated with the correct Terraform version.
  • Verify the .gitignore file includes all necessary entries to avoid committing sensitive or unnecessary files.

Breaking changes

• Removal of environment-specific .sh files for dev, preprod, and prod environments.

Impact

• Deployment process for Arc infrastructure is now automated via ADO pipelines.
• Environment-specific configuration is handled through .tfvars files, reducing redundancy.
• Security and governance practices are improved with added issue templates and documentation.
• CI/CD pipeline structure is updated to include a deploy stage for PRs and main branch commits.

Review checklist

• No hardcoded secrets, credentials, or tokens
• PR title follows correct format (with ticket prefix if ticket provided, without if not)

Terraform (*.tf, *.tfvars):

• terraform plan reviewed — no unintended resource destruction
• Mandatory tags on all new resources
• Resource locks on critical resources
• Variable changes propagated to all environment .tfvars files
• State file impact assessed

Pipeline (*.yml, *.yaml):

• No secrets hardcoded in pipeline YAML
• GPG signed commits configured correctly
• Environment approval gates in place for production stages

[steventux]

Who is the intended audience for this and the other ISSUE_TEMPLATE yaml files?

[steventux]

Do we need this? Is it mandated somewhere? It feels onerous (pun intended)

[steventux]

Can this IP address live in a variable? It's repeated a few times, it would mean only one # gitleaks:allow :)

[steventux]

This looks good to me, not that well versed in Powershell but the TF / Deployment side looks good.