-
Notifications
You must be signed in to change notification settings - Fork 15
docs: Add Cloud Run and Node JS connector examples #506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
52398df
docs: Add Cloud Run and Node JS connector examples
kgala2 123e926
chore: fix lint
kgala2 6a4a712
chore: add rate limiter
kgala2 933e3b8
chore: update lint
kgala2 33da62b
chore: rename connection functions
kgala2 2197978
chore: update knex tedious sample
kgala2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Node.js Samples for Cloud Run | ||
|
|
||
| This directory contains samples demonstrating how to connect to Cloud SQL from Cloud Run using various Node.js ORMs and the [Cloud SQL Node.js Connector](https://github.com/GoogleCloudPlatform/cloud-sql-nodejs-connector). | ||
|
|
||
| ## Available Samples | ||
|
|
||
| * [Knex.js](./knex) | ||
| * [Prisma](./prisma) | ||
| * [Sequelize](./sequelize) | ||
| * [TypeORM](./typeorm) | ||
|
|
||
| Each ORM directory contains subdirectories for supported databases (MySQL, PostgreSQL, SQL Server), and each example includes implementations in: | ||
| * CommonJS (`.cjs`) | ||
| * ES Modules (`.mjs`) | ||
| * TypeScript (`.ts`) | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| 1. A Google Cloud Project with billing enabled. | ||
| 2. A Cloud SQL instance. | ||
| 3. A Cloud Run service account with the `Cloud SQL Client` IAM role. | ||
| 4. For IAM Authentication, the service account must be added as a database user. | ||
|
|
||
| ## Deployment | ||
|
|
||
| Refer to the `README.md` in each ORM directory for specific deployment instructions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,160 @@ | ||
| # Connecting Cloud Run to Cloud SQL with the Node.js Connector | ||
|
|
||
| This guide provides a comprehensive walkthrough of how to connect a Cloud Run service to a Cloud SQL instance using the Cloud SQL Node.js Connector. It covers connecting to instances with both public and private IP addresses and demonstrates how to handle database credentials securely. | ||
|
|
||
| ## Develop a Node.js Application | ||
|
|
||
| The following Node.js applications demonstrate how to connect to a Cloud SQL instance using the Cloud SQL Node.js Connector. | ||
|
|
||
| ### `mysql2/index.cjs` and `pg/index.mjs` | ||
|
|
||
| These files contain the core application logic for connecting to a Cloud SQL for MySQL or PostgreSQL instance. They provide two separate authentication methods, each exposed at a different route: | ||
| - `/`: Password-based authentication | ||
| - `/iam`: IAM-based authentication | ||
|
|
||
| ### `tedious/index.ts` | ||
|
|
||
| This file contains the core application logic for connecting to a Cloud SQL for SQL Server instance. It uses the `cloud-sql-nodejs-connector` to create a database connection pool with password-based authentication at the `/` route. | ||
|
|
||
| > [!NOTE] | ||
| > | ||
| > Cloud SQL for SQL Server does not support IAM database authentication. | ||
|
|
||
| ## Lazy Instantiation | ||
|
|
||
| In a Cloud Run service, global variables are initialized when the container instance starts up. The application instance then handles subsequent requests until the container is spun down. | ||
|
|
||
| The `Connector` and `knex` objects are defined as global variables (initially set to `null`) and are lazily instantiated (created only when needed) inside the request handlers. | ||
|
|
||
| This approach offers several benefits: | ||
|
|
||
| 1. **Faster Startup:** By deferring initialization until the first request, the Cloud Run service can start listening for requests almost immediately, reducing cold start latency. | ||
| 2. **Resource Efficiency:** Expensive operations, like establishing background connections or fetching secrets, are only performed when actually required. | ||
| 3. **Connection Reuse:** Once initialized, the global `Connector` and `knex` instances are reused for all subsequent requests to that container instance. This prevents the overhead of creating new connections for every request and avoids hitting connection limits. | ||
|
|
||
| ## IAM Authentication Prerequisites | ||
|
|
||
| For IAM authentication to work, you must ensure two things: | ||
|
|
||
| 1. **The Cloud Run service's service account has the `Cloud SQL Client` role.** You can grant this role with the following command: | ||
| ```bash | ||
| gcloud projects add-iam-policy-binding PROJECT_ID \ | ||
| --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \ | ||
| --role="roles/cloudsql.client" | ||
| ``` | ||
| Replace `PROJECT_ID` with your Google Cloud project ID and `SERVICE_ACCOUNT_EMAIL` with the email of the service account your Cloud Run service is using. | ||
|
|
||
| 2. **The service account is added as a database user to your Cloud SQL instance.** You can do this with the following command: | ||
| ```bash | ||
| gcloud sql users create SERVICE_ACCOUNT_EMAIL \ | ||
| --instance=INSTANCE_NAME \ | ||
| --type=cloud_iam_user | ||
| ``` | ||
| Replace `SERVICE_ACCOUNT_EMAIL` with the same service account email and `INSTANCE_NAME` with your Cloud SQL instance name. | ||
|
|
||
| ## Deploy the Application to Cloud Run | ||
|
|
||
| Follow these steps to deploy the application to Cloud Run. | ||
|
|
||
| ### Build and Push the Docker Image | ||
|
|
||
| 1. **Enable the Artifact Registry API:** | ||
|
|
||
| ```bash | ||
| gcloud services enable artifactregistry.googleapis.com | ||
| ``` | ||
|
|
||
| 2. **Create an Artifact Registry repository:** | ||
|
|
||
| ```bash | ||
| gcloud artifacts repositories create REPO_NAME \ | ||
| --repository-format=docker \ | ||
| --location=REGION | ||
| ``` | ||
|
|
||
| 3. **Configure Docker to authenticate with Artifact Registry:** | ||
|
|
||
| ```bash | ||
| gcloud auth configure-docker REGION-docker.pkg.dev | ||
| ``` | ||
|
|
||
| 4. **Build the Docker image (replace `mysql2` with `pg` or `tedious` as needed):** | ||
|
|
||
| ```bash | ||
| docker build -t REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME mysql2 | ||
| ``` | ||
|
|
||
| 5. **Push the Docker image to Artifact Registry:** | ||
|
|
||
| ```bash | ||
| docker push REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME | ||
| ``` | ||
|
|
||
| ### Deploy to Cloud Run | ||
|
|
||
| Deploy the container image to Cloud Run using the `gcloud run deploy` command. | ||
|
|
||
| **Sample Values:** | ||
| * `SERVICE_NAME`: `my-cloud-run-service` | ||
| * `REGION`: `us-central1` | ||
| * `PROJECT_ID`: `my-gcp-project-id` | ||
| * `REPO_NAME`: `my-artifact-repo` | ||
| * `IMAGE_NAME`: `my-app-image` | ||
| * `INSTANCE_CONNECTION_NAME`: `my-gcp-project-id:us-central1:my-instance-name` | ||
| * `DB_USER`: `my-db-user` (for password-based authentication) | ||
| * `DB_IAM_USER`: `my-service-account@my-gcp-project-id.iam.gserviceaccount.com` (for IAM-based authentication) | ||
| * `DB_NAME`: `my-db-name` | ||
| * `DB_PASSWORD`: `my-user-pass-name` | ||
| * `VPC_NETWORK`: `my-vpc-network` | ||
| * `SUBNET_NAME`: `my-vpc-subnet` | ||
|
|
||
| **For MySQL and PostgreSQL (Public IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_IAM_USER=DB_IAM_USER,DB_NAME=DB_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME \ | ||
| --region=REGION \ | ||
| --update-secrets=DB_PASSWORD=DB_PASSWORD:latest | ||
| ``` | ||
|
|
||
| **For MySQL and PostgreSQL (Private IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_IAM_USER=DB_IAM_USER,DB_NAME=DB_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME,IP_TYPE=PRIVATE \ | ||
| --network=VPC_NETWORK \ | ||
| --subnet=SUBNET_NAME \ | ||
| --vpc-egress=private-ranges-only \ | ||
| --region=REGION \ | ||
| --update-secrets=DB_PASSWORD=DB_PASSWORD:latest | ||
| ``` | ||
|
|
||
| **For SQL Server (Public IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_NAME=DB_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME \ | ||
| --region=REGION \ | ||
| --update-secrets=DB_PASSWORD=DB_PASSWORD:latest | ||
| ``` | ||
|
|
||
| **For SQL Server (Private IP):** | ||
|
|
||
| ```bash | ||
| gcloud run deploy SERVICE_NAME \ | ||
| --image=REGION-docker.pkg.dev/PROJECT_ID/REPO_NAME/IMAGE_NAME \ | ||
| --set-env-vars=DB_USER=DB_USER,DB_NAME=DB_NAME,INSTANCE_CONNECTION_NAME=INSTANCE_CONNECTION_NAME,IP_TYPE=PRIVATE \ | ||
| --network=VPC_NETWORK \ | ||
| --subnet=SUBNET_NAME \ | ||
| --vpc-egress=private-ranges-only \ | ||
| --region=REGION \ | ||
| --update-secrets=DB_PASSWORD=DB_PASSWORD:latest | ||
| ``` | ||
|
|
||
| > [!NOTE] | ||
| > **`For PSC connections`** | ||
| > | ||
| > To connect to the Cloud SQL instance with PSC connection type, create a PSC endpoint, a DNS zone and DNS record for the instance in the same VPC network as the Cloud Run service and replace the `IP_TYPE` in the deploy command with `PSC`. To configure DNS records, refer to [Connect to an instance using Private Service Connect](https://docs.cloud.google.com/sql/docs/mysql/configure-private-service-connect) guide |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Use the official Node.js image. | ||
| # https://hub.docker.com/_/node | ||
| FROM node:25-slim | ||
|
|
||
| # Create and change to the app directory. | ||
| WORKDIR /usr/src/app | ||
|
|
||
| # Copy application dependency manifests to the container image. | ||
| # A wildcard is used to ensure both package.json AND package-lock.json are copied. | ||
| # Copying this separately prevents re-running npm install on every code change. | ||
| COPY package*.json ./ | ||
|
|
||
| # Install production dependencies. | ||
| # If you add a package-lock.json speed your build by switching to 'npm ci'. | ||
| # RUN npm ci --only=production | ||
| RUN npm install --omit=dev | ||
|
|
||
| # Copy local code to the container image. | ||
| COPY . . | ||
|
|
||
| # Run the web service on container startup. | ||
| CMD ["node", "index.cjs"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,160 @@ | ||
| // Copyright 2025 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // https://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| const express = require('express'); | ||
| const {Connector, IpAddressTypes} = require('@google-cloud/cloud-sql-connector'); | ||
| const knex = require('knex'); | ||
|
|
||
| const app = express(); | ||
|
|
||
| // set up rate limiter: maximum of five requests per minute | ||
| const RateLimit = require('express-rate-limit'); | ||
| const limiter = RateLimit({ | ||
| // 15 minutes | ||
| windowMs: 15 * 60 * 1000, | ||
| // max 100 requests per windowMs | ||
| max: 100, | ||
| }); | ||
|
|
||
| // apply rate limiter to all requests | ||
| app.use(limiter); | ||
|
|
||
| // Connector and connection pools are initialized as null to allow for lazy instantiation. | ||
| // Lazy instantiation is a best practice for Cloud Run applications because it allows | ||
| // the application to start faster and only initialize connections when they are needed. | ||
| // This is especially important in serverless environments where applications may be | ||
| // started and stopped frequently. | ||
| let connector = null; | ||
| let passwordPool = null; | ||
| let iamPool = null; | ||
|
|
||
| // Helper to get the IP type enum from string | ||
| function getIpType(ipTypeStr) { | ||
| const ipType = ipTypeStr || 'PUBLIC'; | ||
| if (ipType === 'PRIVATE') { | ||
| return IpAddressTypes.PRIVATE; | ||
| } else if (ipType === 'PSC') { | ||
| return IpAddressTypes.PSC; | ||
| } else { | ||
| return IpAddressTypes.PUBLIC; | ||
| } | ||
| } | ||
|
|
||
| // Function to create a database connection pool using IAM authentication | ||
| async function createIamConnectionPool() { | ||
| const instanceConnectionName = process.env.INSTANCE_CONNECTION_NAME; | ||
| // IAM service account email | ||
| const dbUser = process.env.DB_IAM_USER; | ||
| const dbName = process.env.DB_NAME; | ||
| const ipType = getIpType(process.env.IP_TYPE); | ||
|
|
||
| // Creates a new connector object. | ||
| if (!connector) { | ||
| connector = new Connector(); | ||
| } | ||
|
|
||
| // Get the connection options for the Cloud SQL instance. | ||
| const clientOpts = await connector.getOptions({ | ||
| instanceConnectionName, | ||
| ipType: ipType, | ||
| authType: 'IAM', | ||
| }); | ||
|
|
||
| // Create a new knex connection pool. | ||
| return knex({ | ||
| client: 'mysql2', | ||
| connection: { | ||
| ...clientOpts, | ||
| user: dbUser, | ||
| database: dbName, | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| // Function to create a database connection pool using password authentication | ||
| async function createPasswordConnectionPool() { | ||
| const instanceConnectionName = process.env.INSTANCE_CONNECTION_NAME; | ||
| // Database username | ||
| const dbUser = process.env.DB_USER; | ||
| const dbName = process.env.DB_NAME; | ||
| const dbPassword = process.env.DB_PASSWORD; | ||
| const ipType = getIpType(process.env.IP_TYPE); | ||
|
|
||
| // Creates a new connector object. | ||
| if (!connector) { | ||
| connector = new Connector(); | ||
| } | ||
|
|
||
| // Get the connection options for the Cloud SQL instance. | ||
| const clientOpts = await connector.getOptions({ | ||
| instanceConnectionName, | ||
| ipType: ipType, | ||
| }); | ||
|
|
||
| // Create a new knex connection pool. | ||
| return knex({ | ||
| client: 'mysql2', | ||
| connection: { | ||
| ...clientOpts, | ||
| user: dbUser, | ||
| password: dbPassword, | ||
| database: dbName, | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| // Helper to get or create the password pool | ||
| async function getPasswordConnectionPool() { | ||
| if (!passwordPool) { | ||
| passwordPool = await createPasswordConnectionPool(); | ||
| } | ||
| return passwordPool; | ||
| } | ||
|
|
||
| // Helper to get or create the IAM pool | ||
| async function getIamConnectionPool() { | ||
| if (!iamPool) { | ||
| iamPool = await createIamConnectionPool(); | ||
| } | ||
| return iamPool; | ||
| } | ||
|
|
||
| app.get('/', async (req, res) => { | ||
| try { | ||
| const db = await getPasswordConnectionPool(); | ||
| // Use knex to run a simple query | ||
| const result = await db.raw('SELECT 1'); | ||
| // Knex raw result for mysql2 is [rows, fields] | ||
| res.send(`Database connection successful (password authentication), result: ${JSON.stringify(result[0])}`); | ||
| } catch (err) { | ||
| console.error(err); | ||
| res.status(500).send(`Error connecting to the database (password authentication): ${err.message}`); | ||
| } | ||
| }); | ||
|
|
||
| app.get('/iam', async (req, res) => { | ||
| try { | ||
| const db = await getIamConnectionPool(); | ||
| const result = await db.raw('SELECT 1'); | ||
| res.send(`Database connection successful (IAM authentication), result: ${JSON.stringify(result[0])}`); | ||
| } catch (err) { | ||
| console.error(err); | ||
| res.status(500).send(`Error connecting to the database (IAM authentication): ${err.message}`); | ||
| } | ||
| }); | ||
|
|
||
|
|
||
| const port = parseInt(process.env.PORT) || 8080; | ||
| app.listen(port, () => { | ||
| console.log(`Server running on port ${port}`); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| { | ||
| "name": "knex-mysql-cloudrun", | ||
| "version": "1.0.0", | ||
| "description": "Knex MySQL example for Cloud Run (CommonJS)", | ||
| "main": "index.cjs", | ||
| "type": "commonjs", | ||
| "scripts": { | ||
| "start": "node index.cjs" | ||
| }, | ||
| "dependencies": { | ||
| "@google-cloud/cloud-sql-connector": "^1.8.4", | ||
| "express": "^5.1.0", | ||
| "knex": "^3.1.0", | ||
| "mysql2": "^3.15.2", | ||
| "express-rate-limit": "8.2.1" | ||
| }, | ||
| "devDependencies": { | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.