Enhance Template Handling with Jinja2 Support and Custom Filters (#4)

#### Overview
This PR introduces significant enhancements to the template processing
capabilities by integrating Jinja2 for more flexible and powerful
templating. Additionally, new custom filters have been added to extend
the functionality, specifically for fetching the latest GitHub release.

#### Changes
- **Jinja2 Integration**: Replaced the existing string templating
mechanism with Jinja2, allowing for more complex templating options.
- **New Template Variables**: Added support for default template
variables like `file_name` and `file_directory`.
- **Custom Filters**: Introduced a new custom Jinja2 filter
`latest_release` that fetches the latest release version from a GitHub
repository.
- **Updated Documentation**: Enhanced the README files (`README.md` and
`README.es.md`) with detailed information on how to use the new
templating features, including variables and custom filters.
- **Configuration Updates**: Extended the `generic-app.yaml` and
`structure.yaml` examples to demonstrate the use of the new templating
features.
- **Dependencies**: Added `jinja2` and `PyGithub` to `requirements.txt`
to support the new templating and GitHub API functionalities.

#### Justification
These changes provide more robust and flexible template processing,
enabling users to create more dynamic and maintainable configurations.
The new features simplify complex templating scenarios and extend the
capability to interact with external APIs, like GitHub, directly within
the templates.

#### Impact
- **Developers**: Improved ability to create and manage complex
templates with dynamic content.
- **Project Maintenance**: Enhanced documentation will make it easier
for new contributors to understand and utilize the templating system.
- **Dependencies**: Additional dependencies (`jinja2` and `PyGithub`)
will be required, which might affect environments where this project is
used.

Author: httpdss

README.es.md

@@ -141,20 +141,51 @@ Aquí tienes un ejemplo de un archivo de configuración YAML:
 structure:
   - README.md:
       content: |
-        # ${project_name}
-        Este es un repositorio de plantilla.
+        # {{@ project_name @}}
+        This is a template repository.
   - script.sh:
       permissions: '0777'
       content: |
         #!/bin/bash
-        echo "¡Hola, ${author_name}!"
+        echo "Hello, {{@ author_name @}}!"
   - LICENSE:
       file: https://raw.githubusercontent.com/nishanths/license/master/LICENSE
   - src/main.py:
       content: |
-        print("¡Hola, Mundo!")
+        print("Hello, World!")
 ```
 
+### Variables de plantilla
+
+Puedes usar variables de plantilla en tu archivo de configuración encerrándolas entre `{{@` y `@}}`. Por ejemplo, `{{@ project_name @}}` será reemplazado con el valor de la variable `project_name` en tiempo de ejecución.
+
+Si necesitas definir bloques, puedes usar la notación de inicio de bloque `{%@` y la notación de final de bloque `%@}`.
+
+Para definir comentarios, puedes usar la notación de inicio de comentario `{#@` y la notación de fin de comentario `@#}`.
+
+#### Variables de plantilla predeterminadas
+
+- `file_name`: El nombre del archivo que se está procesando.
+- `file_directory`: El nombre del directorio del archivo que se está procesando.
+
+#### Filtros personalizados de Jinja2
+
+##### `latest_release`
+
+Este filtro obtiene la versión más reciente de una release en un repositorio de GitHub. Toma el nombre del repositorio como argumento.
+
+```yaml
+structure:
+  - README.md:
+      content: |
+        # MyProject
+        Latest release: {{@ "httpdss/struct" | latest_release @}}
+```
+
+Esto utiliza PyGithub para obtener la última release del repositorio, por lo que configurar la variable de entorno `GITHUB_TOKEN` te dará acceso a repositorios privados.
+
+Si ocurre un error en el proceso, el filtro devolverá `LATEST_RELEASE_ERROR`.
+
 ## 👩‍💻 Desarrollo
 
 Para comenzar con el desarrollo, sigue estos pasos:

README.md

@@ -146,20 +146,51 @@ Here is an example of a YAML configuration file:
 structure:
   - README.md:
       content: |
-        # ${project_name}
+        # {{@ project_name @}}
         This is a template repository.
   - script.sh:
       permissions: '0777'
       content: |
         #!/bin/bash
-        echo "Hello, ${author_name}!"
+        echo "Hello, {{@ author_name @}}!"
   - LICENSE:
       file: https://raw.githubusercontent.com/nishanths/license/master/LICENSE
   - src/main.py:
       content: |
         print("Hello, World!")
 ```
 
+### Template variables
+
+You can use template variables in your configuration file by enclosing them in `{{@` and `@}}`. For example, `{{@ project_name @}}` will be replaced with the value of the `project_name` variable at runtime.
+
+If you need to define blocks you can use starting block notation `{%@` and end block notation `%@}`.
+
+To define comments you can use the comment start notation `{#@` and end comment notation `@#}`.
+
+#### Default template variables
+
+- `file_name`: The name of the file being processed.
+- `file_directory`: The name of the directory of file that is being processed.
+
+#### Custom Jinja2 filters
+
+##### `latest_release`
+
+This filter fetches the latest release version of a GitHub repository. It takes the repository name as an argument.
+
+```yaml
+structure:
+  - README.md:
+      content: |
+        # MyProject
+        Latest release: {{@ "httpdss/struct" | latest_release @}}
+```
+
+This uses PyGithub to fetch the latest release of the repository so setting the `GITHUB_TOKEN` environment variable will give you access to private repositories.
+
+If there is an error in the process, the filter will return `LATEST_RELEASE_ERROR`.
+
 ## 👩‍💻 Development
 
 To get started with development, follow these steps:

contribs/generic-app.yaml

@@ -47,6 +47,39 @@ structure:
   - .gitkeep:
       content: ""
   - .gitmodules:
+      content: |
+        provider "aws" {
+          region = "{{@ aws_region @}}"
+        }
+  - .devops/apps/environments/prod/main.tf:
+      content: "# Terraform configuration for the prod environment"
+  - .devops/apps/environments/prod/providers.tf:
+      content: |
+        provider "aws" {
+          region = "{{@ aws_region @}}"
+        }
+  - .devops/apps/environments/stage/main.tf:
+      content: "# Terraform configuration for the stage environment"
+  - .devops/apps/environments/stage/providers.tf:
+      content: |
+        provider "aws" {
+          region = "{{@ aws_region @}}"
+        }
+  - .devops/apps/environments/qa/main.tf:
+      content: "# Terraform configuration for the qa environment"
+  - .devops/apps/environments/qa/providers.tf:
+      content: |
+        provider "aws" {
+          region = "{{@ aws_region @}}"
+        }
+  - .devops/apps/init/main.tf:
+      content: "# Terraform configuration for the init environment"
+  - .devops/apps/init/providers.tf:
+      content: |
+        provider "aws" {
+          region = "{{@ aws_region @}}"
+        }
+  - .github/workflows/pre-commit.yaml:
       content: |
         # This file defines the submodules used by the repository
         # See https://git-scm.com/docs/gitmodules

example/structure.yaml

@@ -1,13 +1,13 @@
 structure:
   - README.md:
       content: |
-        # ${project_name}
+        # {{@ project_name @}}
         This is a template repository.
   - script.sh:
       permissions: '0777'
       content: |
         #!/bin/bash
-        echo "Hello, ${author_name}!"
+        echo "Hello, {{@ author_name @}}!"
   - LICENSE:
       file: https://raw.githubusercontent.com/nishanths/license/master/LICENSE
   - .github/workflows/ci.yml:

requirements.txt

@@ -2,3 +2,5 @@ PyYAML
 requests
 openai
 python-dotenv
+jinja2
+PyGithub

struct_module/file_item.py

@@ -2,10 +2,11 @@
 import os
 import shutil
 import logging
-from string import Template
+from jinja2 import Environment
 import time
 from openai import OpenAI
 from dotenv import load_dotenv
+from struct_module.filters import get_latest_release
 
 load_dotenv()
 
@@ -17,6 +18,7 @@ class FileItem:
     def __init__(self, properties):
         self.logger = logging.getLogger(__name__)
         self.name = properties.get("name")
+        self.file_directory = self._get_file_directory()
         self.content = properties.get("content")
         self.remote_location = properties.get("file")
         self.permissions = properties.get("permissions")
@@ -34,6 +36,9 @@ def __init__(self, properties):
             self.logger.debug(f"Using OpenAI model: {openai_model}")
             self.openai_model = openai_model
 
+    def _get_file_directory(self):
+        return os.path.dirname(self.name)
+
     def process_prompt(self, dry_run=False):
         if self.user_prompt:
             self.logger.debug(f"Using user prompt: {self.user_prompt}")
@@ -72,11 +77,32 @@ def fetch_content(self):
             self.content = response.text
             self.logger.debug(f"Fetched content: {self.content}")
 
+    def _merge_default_template_vars(self, template_vars):
+        default_vars = {
+            "file_name": self.name,
+            "file_directory": self.file_directory,
+        }
+        if not template_vars:
+            return default_vars
+        return {**default_vars, **template_vars}
+
     def apply_template_variables(self, template_vars):
-        if self.content and template_vars:
-            self.logger.debug(f"Applying template variables: {template_vars}")
-            template = Template(self.content)
-            self.content = template.substitute(template_vars)
+        vars = self._merge_default_template_vars(template_vars)
+        logging.debug(f"Applying template variables: {vars}")
+
+        env = Environment(
+            trim_blocks=True,
+            block_start_string='{%@',
+            block_end_string='@%}',
+            variable_start_string='{{@',
+            variable_end_string='@}}',
+            comment_start_string='{#@',
+            comment_end_string='@#}'
+        )
+        env.filters['latest_release'] = get_latest_release
+        template = env.from_string(self.content)
+
+        self.content = template.render(vars)
 
     def create(self, base_path, dry_run=False, backup_path=None, file_strategy='overwrite'):
         file_path = os.path.join(base_path, self.name)

struct_module/filters.py

@@ -0,0 +1,25 @@
+import os
+from github import Github
+
+def get_latest_release(repo_name):
+    token = os.getenv('GITHUB_TOKEN')
+
+    # Use the token if available, otherwise proceed without authentication
+    if token:
+        g = Github(token)
+    else:
+        g = Github()
+
+    try:
+        # Get the repository object
+        repo = g.get_repo(repo_name)
+        # Get the latest release
+        latest_release = repo.get_latest_release()
+        return latest_release.tag_name
+    except Exception:
+        # If an error occurs, return the default branch name
+        try:
+            default_branch = repo.default_branch
+            return default_branch
+        except Exception as e:
+            return "LATEST_RELEASE_ERROR"