Skip to content

Commit a470a8e

Browse files
DrisDaryDrisDary
authored
Ci for az sample app tests (#31)
* Test CI pipeline on feature branch * added correct environment * added dependencies 1 * added dependecy files * removed if * addedd docker build job * licence refactor * removed not needed registering cloud part * fixed perm settings * removed az cli disable conn varification * removed comments * added terraform tests only * added script tests only * added start_interception * added pre login to azure cli * Âtest * . * .. * testing disabling az cli auth * reverts * reverts 2 * reverts 3 * reverts 4 * added .net no proxy * --no-build * skip .net test * skip sql test * testing dotnet test * only run failing tests * web app sql certificate set as trusted * trusted cert * added -N -C to all sql cmds * removed install bicep * run all test * run all test * increase runner * clean disk after each test * removed verbose debu * removed unnecessary files * updated readme * only run on merge to main * increase localstack start time to 120 s
1 parent 7a6ae92 commit a470a8e

13 files changed

Lines changed: 385 additions & 16 deletions

File tree

‎.gitattributes‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
*.sh text eol=lf
2+
*.yml text eol=lf
3+
*.bicep text eol=lf
4+
*.tf text eol=lf

‎.github/workflows/run-samples.yml‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
name: Samples CI
2+
3+
# Theory of Operation:
4+
# This workflow automates the testing of Azure sample applications against the LocalStack Azure emulator.
5+
# It follows the best practices from the localstack-pro repository:
6+
# 1. Parallel Testing: Splits the sample suite into shards to reduce execution time.
7+
# 2. Standardized Tooling: Uses a Makefile for environment setup and test orchestration.
8+
# 3. Cloud Emulation: Configures the Azure CLI to target the LocalStack emulator.
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
12+
cancel-in-progress: true
13+
14+
on:
15+
pull_request:
16+
branches: [ main ]
17+
workflow_dispatch:
18+
19+
jobs:
20+
scripts:
21+
name: "Run Test Scripts (amd64) — Part ${{ matrix.shard }} of ${{ matrix.splits }}"
22+
environment: AZURE
23+
strategy:
24+
fail-fast: false
25+
matrix:
26+
shard: [1, 2]
27+
splits: [2]
28+
runs-on: ubuntu-latest
29+
30+
env:
31+
IMAGE_NAME: localstack/localstack-azure-alpha
32+
DEFAULT_TAG: latest
33+
34+
steps:
35+
- name: Checkout repo
36+
uses: actions/checkout@v4
37+
38+
- name: Set up environment
39+
run: echo "AZURE_CONFIG_DIR=${{ runner.temp }}/azure-cli" >> $GITHUB_ENV
40+
41+
- name: Set up Python
42+
uses: actions/setup-python@v5
43+
with:
44+
python-version: '3.12'
45+
cache: 'pip'
46+
47+
- name: Set up .NET
48+
uses: actions/setup-dotnet@v4
49+
with:
50+
dotnet-version: '9.0'
51+
52+
- name: Install System Dependencies
53+
# Essential tools for script execution, app packaging, and database connectivity.
54+
# jq: for parsing JSON responses from Azure CLI.
55+
# zip: for packaging function/web apps.
56+
# unixodbc-dev & libsnappy-dev: required for Python database drivers (pyodbc, pymongo).
57+
run: |
58+
sudo apt-get update
59+
sudo apt-get install -y jq zip unixodbc-dev libsnappy-dev
60+
find . -name "*.sh" -exec chmod +x {} +
61+
62+
- name: Install test dependencies
63+
# Mirroring the localstack-pro approach: install all Python dependencies
64+
# (including the localstack CLI) into a virtual environment to avoid system-level conflicts.
65+
run: make install
66+
67+
- name: Login to Docker Hub
68+
# Mandatory login to Docker Hub to benefit from higher rate limits for authenticated pulls.
69+
# This prevents '429 Too Many Requests' errors during the pull of large emulator images.
70+
uses: docker/login-action@v3
71+
with:
72+
username: ${{ secrets.DOCKERHUB_PULL_USERNAME }}
73+
password: ${{ secrets.DOCKERHUB_PULL_TOKEN }}
74+
75+
- name: Free up disk space
76+
# Azure emulator images are large. Pruning unused Docker objects ensures enough
77+
# disk space is available on the GitHub runner for image pulls and sidecar containers.
78+
run: |
79+
docker system prune -af --volumes
80+
docker builder prune -af
81+
82+
- name: Pull LocalStack Azure Image
83+
# Explicitly pull the image before starting. This mirrors the "Build Docker Image"
84+
# step in localstack-pro and ensures the pull logic is separated from the start logic.
85+
run: docker pull ${{ env.IMAGE_NAME }}:${{ env.DEFAULT_TAG }}
86+
87+
- name: Start LocalStack
88+
# Run the emulator in detached mode using the virtual environment.
89+
# We use 'python -m localstack.cli.main' to ensure the correct CLI version from the venv is used.
90+
run: |
91+
source .venv/bin/activate
92+
python -m localstack.cli.main start -d
93+
python -m localstack.cli.main wait -t 120
94+
env:
95+
IMAGE_NAME: ${{ env.IMAGE_NAME }}:${{ env.DEFAULT_TAG }}
96+
LOCALSTACK_AUTH_TOKEN: ${{ secrets.TEST_LOCALSTACK_AUTH_TOKEN }}
97+
DOCKER_FLAGS: "-e MSSQL_ACCEPT_EULA=Y"
98+
LS_LOG: "DEBUG"
99+
DISABLE_EVENTS: "1"
100+
ACTIVATE_PRO: "1"
101+
DNS_ADDRESS: "0"
102+
103+
- name: Install Azure Functions Core Tools
104+
# Required for publishing function app samples to the emulator.
105+
run: npm install -g azure-functions-core-tools@4 --unsafe-perm true
106+
107+
- name: Install MSSQL ODBC and Tools
108+
# Required for the 'web-app-sql-database' sample which uses 'sqlcmd' to
109+
# initialize and verify the database schema in the local emulator.
110+
run: |
111+
curl https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc
112+
curl https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list
113+
sudo apt-get update
114+
sudo ACCEPT_EULA=Y apt-get install -y msodbcsql18 mssql-tools18
115+
echo "/opt/mssql-tools18/bin" >> $GITHUB_PATH
116+
117+
- name: Run Test Scripts
118+
# Executes the sharded test suite. Each shard runs a subset of samples in parallel.
119+
run: make test SHARD=${{ matrix.shard }} SPLITS=${{ matrix.splits }}
120+
env:
121+
LOCALSTACK_AUTH_TOKEN: ${{ secrets.TEST_LOCALSTACK_AUTH_TOKEN }}
122+
123+
- name: Get LocalStack Logs
124+
# Captured on failure or success to provide a detailed audit trail of the emulator's activity.
125+
if: always()
126+
run: make logs

‎Makefile‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
VENV_BIN = python3 -m venv
2+
VENV_DIR ?= .venv
3+
VENV_ACTIVATE = $(VENV_DIR)/bin/activate
4+
VENV_RUN = . $(VENV_ACTIVATE)
5+
6+
ifeq ($(OS),Windows_NT)
7+
VENV_ACTIVATE = $(VENV_DIR)/Scripts/activate
8+
VENV_RUN = $(VENV_DIR)/Scripts/activate
9+
endif
10+
11+
venv: $(VENV_ACTIVATE)
12+
13+
$(VENV_ACTIVATE): pyproject.toml
14+
test -d $(VENV_DIR) || $(VENV_BIN) $(VENV_DIR)
15+
$(VENV_RUN); pip install --upgrade pip setuptools wheel
16+
touch $(VENV_ACTIVATE)
17+
18+
install: venv ## Install dependencies
19+
$(VENV_RUN); pip install -r requirements-dev.txt
20+
chmod +x run-samples.sh
21+
22+
SHARD ?= 1
23+
SPLITS ?= 1
24+
25+
test: venv ## Run all samples
26+
$(VENV_RUN); bash ./run-samples.sh $(SHARD) $(SPLITS)
27+
28+
logs: venv ## Get LocalStack logs
29+
$(VENV_RUN); localstack logs
30+
31+
.PHONY: venv install test logs

‎README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,29 @@ Each sample project is organized by Azure service and includes:
4242
- Step-by-step deployment guides and tutorials.
4343
- Optionally, testing and validation scripts.
4444

45+
## Local Testing
46+
47+
To validate all samples locally, you can run the same test suite used in the CI. This script will start LocalStack, configure the Azure CLI cloud profile, and execute the deployment and test scripts for each sample.
48+
49+
```bash
50+
cd localstack-azure-samples
51+
52+
# Set your LOCALSTACK_AUTH_TOKEN
53+
export LOCALSTACK_AUTH_TOKEN=<your-token>
54+
55+
# Or create a .env file:
56+
# echo "LOCALSTACK_AUTH_TOKEN=<your-token>" > .env
57+
58+
./run-samples.sh
59+
```
60+
61+
### Troubleshooting: Line Endings
62+
If you encounter errors like `invalid option name` or `: command not found` when running on Linux/WSL, it's likely due to Windows-style line endings (CRLF). You can fix this by running:
63+
```bash
64+
find . -name "*.sh" -exec sed -i 's/\r$//' {} +
65+
```
66+
Or by installing and using `dos2unix`.
67+
4568
## Configuration
4669

4770
Follow the comprehensive setup guide in [LocalStack for Azure Quick Start](./docs/LOCALSTACK.md) to configure your LocalStack for Azure development environment.

‎pyproject.toml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
[build-system]
2+
requires = ['setuptools>=64', 'wheel']
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "localstack-azure-samples"
7+
version = "0.1.0"
8+
description = "Samples for LocalStack Azure"
9+
requires-python = ">=3.10"
10+
dependencies = [
11+
"flask",
12+
"pyodbc",
13+
"pymongo",
14+
"azure-functions",
15+
"azure-identity",
16+
"azure-storage-blob",
17+
"azure-mgmt-cosmosdb",
18+
"azure-core",
19+
"python-dotenv",
20+
"localstack",
21+
"azlocal",
22+
"terraform-local",
23+
]
24+
25+
[project.optional-dependencies]
26+
dev = [
27+
"pytest>=7.0",
28+
"pytest-xdist",
29+
"pytest-timeout",
30+
"pytest-rerunfailures",
31+
]

‎requirements-dev.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
-r requirements-runtime.txt
2+
pytest
3+
pytest-xdist
4+
pytest-timeout
5+
pytest-rerunfailures

‎requirements-runtime.txt‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
flask
2+
pyodbc
3+
pymongo
4+
azure-functions
5+
azure-identity
6+
azure-storage-blob
7+
azure-mgmt-cosmosdb
8+
azure-core
9+
python-dotenv
10+
localstack
11+
azlocal
12+
terraform-local

‎run-samples.sh‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# Helper script to run all sample tests locally, replicating the CI environment.
5+
# Requirements:
6+
# - Docker
7+
# - Python 3.12+
8+
# - .NET 9.0+
9+
# - Node.js & npm
10+
# - Azure CLI (az)
11+
# - LocalStack CLI
12+
# - azlocal & terraform-local (pip install azlocal terraform-local)
13+
# - funclocal (pip install funclocal)
14+
# - Azure Functions Core Tools (func)
15+
# - jq & zip (sudo apt-get install jq zip)
16+
# - MSSQL Tools (sqlcmd)
17+
# - LOCALSTACK_AUTH_TOKEN environment variable
18+
19+
# 0. Load environment variables from .env file if it exists
20+
if [ -f .env ]; then
21+
echo "Loading environment variables from .env file..."
22+
# Use a subshell to avoid exporting everything if not needed,
23+
# but here we actually want them in the environment.
24+
set -a
25+
source .env
26+
set +a
27+
fi
28+
29+
# 1. Check for required tools
30+
command -v localstack >/dev/null 2>&1 || { echo >&2 "localstack CLI is required but not installed. Aborting."; exit 1; }
31+
command -v az >/dev/null 2>&1 || { echo >&2 "az CLI is required but not installed. Aborting."; exit 1; }
32+
command -v azlocal >/dev/null 2>&1 || { echo >&2 "azlocal is required but not installed. Run 'pip install azlocal'. Aborting."; exit 1; }
33+
command -v funclocal >/dev/null 2>&1 || { echo >&2 "funclocal is required but not installed. Run 'pip install azlocal'. Aborting."; exit 1; }
34+
command -v tflocal >/dev/null 2>&1 || { echo >&2 "tflocal is required but not installed. Run 'pip install terraform-local'. Aborting."; exit 1; }
35+
command -v func >/dev/null 2>&1 || { echo >&2 "Azure Functions Core Tools (func) is required but not installed. Aborting."; exit 1; }
36+
37+
if [ -z "${LOCALSTACK_AUTH_TOKEN:-}" ]; then
38+
echo "Error: LOCALSTACK_AUTH_TOKEN is not set. It is required for the Azure emulator."
39+
exit 1
40+
fi
41+
42+
# 1. Start LocalStack
43+
if ! localstack status | grep -q "running"; then
44+
echo "Starting LocalStack Azure emulator..."
45+
IMAGE_NAME=localstack/localstack-azure-alpha localstack start -d
46+
localstack wait -t 60
47+
else
48+
echo "LocalStack is already running."
49+
fi
50+
51+
# 2. Configure Azure CLI for LocalStack
52+
echo "Configuring Azure CLI for LocalStack..."
53+
if [ -n "${AZURE_CONFIG_DIR:-}" ]; then
54+
mkdir -p "$AZURE_CONFIG_DIR"
55+
fi
56+
57+
if command -v azlocal >/dev/null 2>&1; then
58+
azlocal login || true
59+
azlocal start_interception
60+
else
61+
az login --service-principal -u any-app -p any-pass --tenant any-tenant || true
62+
fi
63+
64+
65+
# 3. Define Samples
66+
SAMPLES=(
67+
"samples/function-app-front-door/python|bash scripts/deploy_all.sh --name-prefix testafd --use-localstack|"
68+
"samples/function-app-managed-identity/python|bash scripts/user-managed-identity.sh|bash scripts/validate.sh && bash scripts/test.sh"
69+
"samples/function-app-storage-http/dotnet|bash scripts/deploy.sh|bash scripts/validate.sh && bash scripts/call-http-triggers.sh"
70+
"samples/web-app-cosmosdb-mongodb-api/python|bash scripts/deploy.sh|bash scripts/validate.sh && bash scripts/call-web-app.sh"
71+
"samples/web-app-managed-identity/python|bash scripts/user-assigned.sh|bash scripts/validate.sh && bash scripts/call-web-app.sh"
72+
"samples/web-app-sql-database/python|bash scripts/deploy.sh|bash scripts/validate.sh && bash scripts/get-web-app-url.sh"
73+
)
74+
75+
# 4. Calculate Shard
76+
TOTAL=${#SAMPLES[@]}
77+
SHARD=${1:-1}
78+
SPLITS=${2:-1}
79+
80+
COUNT=$(( TOTAL / SPLITS ))
81+
START=$(( (SHARD - 1) * COUNT ))
82+
83+
if [ "$SHARD" -eq "$SPLITS" ]; then
84+
COUNT=$(( TOTAL - START ))
85+
fi
86+
87+
echo "Running samples shard $SHARD of $SPLITS (index $START, count $COUNT)"
88+
89+
# 5. Run Samples
90+
for (( i=START; i<START+COUNT; i++ )); do
91+
item="${SAMPLES[$i]}"
92+
IFS='|' read -r path deploy test <<< "$item"
93+
echo "============================================================"
94+
echo "Testing Sample: $path"
95+
echo "============================================================"
96+
97+
pushd "$path" > /dev/null
98+
99+
echo "Deploying..."
100+
eval "$deploy"
101+
102+
if [ -n "$test" ]; then
103+
echo "Testing..."
104+
eval "$test"
105+
fi
106+
107+
popd > /dev/null
108+
echo "Completed: $path"
109+
110+
# Cleanup Docker resources after each test to free up disk space
111+
echo "Cleaning up Docker resources..."
112+
docker system prune -af --volumes || true
113+
echo ""
114+
done
115+
116+
echo "All samples completed successfully!"

‎samples/function-app-front-door/python/scripts/.last_deploy_all.env‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
RESOURCE_GROUP="rg-testafd-30236"
2+
PROFILE_NAME="afd-testafd-30236"
3+
EP_BASIC="ep-testafd-basic-30236"
4+
EP_MULTI="ep-testafd-multi-30236"
5+
EP_SPEC="ep-testafd-spec-30236"
6+
EP_RULES="ep-testafd-rules-30236"
7+
EP_STATE="ep-testafd-state-30236"
8+
FUNC_MAIN="fa-testafd-30236"
9+
FUNC_A="fa-testafda-30236"
10+
FUNC_B="fa-testafdb-30236"

‎samples/function-app-front-door/python/scripts/deploy_all.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ create_function_app() {
212212
STORAGE_KEY=$(az storage account keys list -g "$RESOURCE_GROUP" -n "$storageName" --query "[0].value" -o tsv)
213213
if [[ -z "$STORAGE_KEY" ]]; then echo "Failed to get storage key for $storageName" >&2; exit 1; fi
214214
local STORAGE_CONNECTION_STRING
215-
STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=$storageName;AccountKey=$STORAGE_KEY;BlobEndpoint=https://$storageName.blob.localhost.localstack.cloud:4566;QueueEndpoint=https://$storageName.queue.localhost.localstack.cloud:4566;TableEndpoint=https://$storageName.table.localhost.localstack.cloud:4566;FileEndpoint=https://$storageName.file.localhost.localstack.cloud:4566"
215+
STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=http;AccountName=$storageName;AccountKey=$STORAGE_KEY;BlobEndpoint=http://$storageName.blob.localhost.localstack.cloud:4566;QueueEndpoint=http://$storageName.queue.localhost.localstack.cloud:4566;TableEndpoint=http://$storageName.table.localhost.localstack.cloud:4566;FileEndpoint=http://$storageName.file.localhost.localstack.cloud:4566"
216216
az functionapp config appsettings set -g "$RESOURCE_GROUP" -n "$funcName" \
217217
--settings AzureWebJobsStorage="$STORAGE_CONNECTION_STRING" WEBSITE_CONTENTAZUREFILECONNECTIONSTRING="$STORAGE_CONNECTION_STRING" SCM_RUN_FROM_PACKAGE= -o none
218218
fi
@@ -228,7 +228,7 @@ publish_function_code() {
228228
echo "Error: Azure Functions Core Tools ('func') not found in PATH." >&2; exit 1
229229
fi
230230
pushd "$FUNCTION_SRC" >/dev/null
231-
funclocal azure functionapp publish "$funcName" --python --build local --verbose --debug
231+
funclocal azure functionapp publish "$funcName" --python --build local #--verbose --debug
232232
popd >/dev/null
233233
else
234234
rm -f "$zipPath"; ( cd "$FUNCTION_SRC" && zip -rq "$zipPath" . )

0 commit comments

Comments
 (0)