Merge pull request #545 from DefangLabs/kevin/mastra-llm

Author: KevyVo

.github/workflows/deploy-changed-samples.yml

@@ -79,7 +79,6 @@ jobs:
           TEST_DB_POSTGRESDB_SSL_ENABLED: ${{ secrets.TEST_DB_POSTGRESDB_SSL_ENABLED }}
           TEST_DB_POSTGRESDB_SSL_REJECT_UNAUTHORIZED: ${{ secrets.TEST_DB_POSTGRESDB_SSL_REJECT_UNAUTHORIZED }}
           TEST_GITHUB_TOKEN: ${{ secrets.TEST_GITHUB_TOKEN }}
-          TEST_GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.TEST_GOOGLE_GENERATIVE_AI_API_KEY }}
           TEST_HASURA_GRAPHQL_ADMIN_SECRET: ${{ secrets.TEST_HASURA_GRAPHQL_ADMIN_SECRET }}
           TEST_HASURA_GRAPHQL_DATABASE_URL: ${{ secrets.TEST_HASURA_GRAPHQL_DATABASE_URL }}
           TEST_JUPYTER_TOKEN: ${{ secrets.TEST_JUPYTER_TOKEN }}
@@ -93,6 +92,7 @@ jobs:
           TEST_MODEL: ${{ secrets.TEST_MODEL }}
           TEST_MONGO_INITDB_ROOT_USERNAME: ${{ secrets.TEST_MONGO_INITDB_ROOT_USERNAME }}
           TEST_MONGO_INITDB_ROOT_PASSWORD: ${{ secrets.TEST_MONGO_INITDB_ROOT_PASSWORD }}
+          TEST_LLM_MODEL: ${{ secrets.TEST_LLM_MODEL }}
           TEST_N8N_ENCRYPTION_KEY: ${{ secrets.TEST_N8N_ENCRYPTION_KEY }}
           TEST_NC_DB: ${{ secrets.TEST_NC_DB }}
           TEST_NC_S3_ENDPOINT: ${{ secrets.TEST_NC_S3_ENDPOINT }}

samples/mastra-nextjs/.github/workflows/deploy.yaml

@@ -20,8 +20,8 @@ jobs:
       - name: Deploy
         uses: DefangLabs/defang-github-action@v1.2.0
         with:
-          config-env-vars: POSTGRES_PASSWORD GOOGLE_GENERATIVE_AI_API_KEY GITHUB_TOKEN
+          config-env-vars: POSTGRES_PASSWORD LLM_MODEL GITHUB_TOKEN
         env:
           POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
+          LLM_MODEL: ${{ secrets.LLM_MODEL }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GOOGLE_GENERATIVE_AI_API_KEY: ${{ secrets.GOOGLE_GENERATIVE_AI_API_KEY }}

samples/mastra-nextjs/README.md

@@ -26,6 +26,14 @@ To run the application locally for development, use the development compose file
 docker compose -f compose.dev.yaml up
 ```
 
+### `GOOGLE_GENERATIVE_AI_API_KEY`
+
+When running locally with Docker Compose, you will need to set the `GOOGLE_GENERATIVE_AI_API_KEY` environment variable to your GCP API key. You can get the API key from the [Google AI Studio](https://aistudio.google.com/).
+
+### `LLM_MODEL`
+
+When running locally with Docker Compose, you are limited to the models on this list: [Google models](https://mastra.ai/models/providers/google).
+
 This will:
 
 - Start PostgreSQL with volume persistence for local development
@@ -38,26 +46,26 @@ You can access mastra-nextjs at `http://localhost:3000` once the containers are
 
 For this sample, you will need to provide the following [configuration](https://docs.defang.io/docs/concepts/configuration). Note that if you are using the 1-click deploy option, you can set these values as secrets in your GitHub repository and the action will automatically deploy them for you.
 
-### `GOOGLE_GENERATIVE_AI_API_KEY`
-
-Your Google Generative AI API key for accessing the Gemini Flash model. You can get this from the [Google AI Studio](https://aistudio.google.com/).
-
 ### `POSTGRES_PASSWORD`
 
 The password for your Postgres database. You need to set this before deploying for the first time.
 
 _You can easily set this to a random string using `defang config set POSTGRES_PASSWORD --random`_
 
-### `DB_URL`
+### `LLM_MODEL`
+
+_You can easily set this using `defang config set LLM_MODEL=<SELECTED_MODEL>`_
 
-The [PostgreSQL database connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). This will be automatically configured when using BYOC managed database services. It should look something like this: `postgresql://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]`.
+The large language model to use for the AI-powered chat. This can be set to models like `anthropic.claude-3-5-sonnet-20241022-v2:0` for AWS or `gemini-2.5-flash` for Google Cloud. Here is a list of supported models for [GCP](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex#model-capabilities) and [AWS](https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#model-capabilities). For AWS make sure you [request access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) to the model in AWS Bedrock console and for GCP make sure you have [request access](https://docs.cloud.google.com/marketplace/docs/enable-ai-models#get-ai-models) to the model in GCP Vertex AI console.
 
 ### `DB_SSL`
 
 Set to `true` to enable SSL. Set to `false` to disable SSL, which is used for Defang Playground. (Can be set directly in the Docker Compose file.)
 
 ### `GITHUB_TOKEN` (Optional)
 
+_You can easily set this using `defang config set GITHUB_TOKEN=<YOUR_GITHUB_TOKEN>`_
+
 A [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token) to increase API rate limits when fetching repository data. This is optional but recommended for better performance.
 Setting the permissions to public repositories only is sufficient, unless you want to access private repositories that you have access to.
 
@@ -89,6 +97,8 @@ The large context window of Gemini Flash allows the agent to hold more code in m
 
 ### Defang Playground
 
+When deploying, the Playground environment is limited by resource constraints, so only Google Gemini flash or flash-lite models are supported. That said, I recommend using one of the flash models from this [list](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex#model-capabilities). If you want to use other models, please use [Defang BYOC](https://docs.defang.io/docs/tutorials/deploy-to-your-cloud).
+
 Deploy your application to the Defang Playground by opening up your terminal and typing:
 
 ```bash

samples/mastra-nextjs/app/package.json

@@ -10,9 +10,13 @@
     "lint": "next lint"
   },
   "dependencies": {
+    "@ai-sdk/amazon-bedrock": "^3.0.67",
+    "@ai-sdk/google-vertex": "^3.0.86",
+    "@ai-sdk/provider": "^2.0.0",
     "@assistant-ui/react": "^0.11.39",
     "@assistant-ui/react-ai-sdk": "^1.1.11",
     "@assistant-ui/react-markdown": "^0.11.4",
+    "@aws-sdk/credential-providers": "^3.946.0",
     "@mastra/ai-sdk": "^0.2.7",
     "@mastra/core": "^0.24.1",
     "@mastra/memory": "^0.15.11",