Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 23 additions & 9 deletions docs/quota_check.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
## Check Quota Availability Before Deployment

Before deploying the accelerator, **ensure sufficient quota availability** for the required model.
> **We recommend increasing the capacity to 100k tokens for optimal performance.**

### Login if you have not done so already
```
azd auth login
```


### 📌 Default Models & Capacities:
```
Expand All @@ -15,27 +22,34 @@ eastus, uksouth, eastus2, northcentralus, swedencentral, westus, westus2, southc
- Only model(s) provided → The script will check for those models in the default regions.
- Only region(s) provided → The script will check default models in the specified regions.
- Both models and regions provided → The script will check those models in the specified regions.
- `--verbose` passed → Enables detailed logging output for debugging and traceability.

### **Input Formats**
✔️ Run without parameters to check default models & regions:
> Use the --models, --regions, and --verbose options for parameter handling:

✔️ Run without parameters to check default models & regions without verbose logging:
```
./quota_check_params.sh
```
✔️ Model name and required capacity in the format:
✔️ Enable verbose logging:
```
./quota_check_params.sh --verbose
```
✔️ Check specific model(s) in default regions:
```
./quota_check_params.sh gpt-4o:30
./quota_check_params.sh --models gpt-4o:30,text-embedding-ada-002:80
```
✔️ Multiple models can be passed, separated by commas:
✔️ Check default models in specific region(s):
```
./quota_check_params.sh gpt-4o:30,text-embedding-ada-002:80
./quota_check_params.sh --regions eastus,westus
```
✔️ Passing Both models and regions:
```
./quota_check_params.sh gpt-4o:30 eastus,westus2
./quota_check_params.sh --models gpt-4o:30 --regions eastus,westus2
```
✔️ Check default models in specific regions:
✔️ All parameters combined:
```
./quota_check_params.sh "" eastus,westus2
./quota_check_params.sh --models gpt-4:30,text-embedding-ada-002:80 --regions eastus,westus --verbose
```

### **Sample Output**
Expand All @@ -62,7 +76,7 @@ The final table lists regions with available quota. You can select any of these
### **If using VS Code or Codespaces**
1. Open the terminal in VS Code or Codespaces.
2. If you're using VS Code, click the dropdown on the right side of the terminal window, and select `Git Bash`.
![git_bash](images/git_bash.png)
![git_bash](Images/git_bash.png)
3. Navigate to the `scripts` folder where the script files are located and make the script as executable:
```sh
cd infra/scripts
Expand Down
4 changes: 2 additions & 2 deletions infra/scripts/checkquota_km.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ echo "✅ Azure subscription set successfully."

# Define models and their minimum required capacities
declare -A MIN_CAPACITY=(
["OpenAI.Standard.gpt-4o-mini"]=$GPT_MIN_CAPACITY #km generic
["OpenAI.Standard.text-embedding-ada-002"]=$TEXT_EMBEDDING_MIN_CAPACITY #km generic
["OpenAI.GlobalStandard.gpt-4o-mini"]=$GPT_MIN_CAPACITY #km generic
["OpenAI.GlobalStandard.text-embedding-ada-002"]=$TEXT_EMBEDDING_MIN_CAPACITY #km generic
)

VALID_REGION=""
Expand Down
69 changes: 58 additions & 11 deletions infra/scripts/quota_check_params.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,50 @@
#!/bin/bash
# VERBOSE=false

MODELS=""
REGIONS=""
VERBOSE=false

while [[ $# -gt 0 ]]; do
case "$1" in
--models)
MODELS="$2"
shift 2
;;
--regions)
REGIONS="$2"
shift 2
;;
--verbose)
VERBOSE=true
shift
;;
*)
echo "Unknown option: $1"
exit 1
;;
esac
done

# Fallback to defaults if not provided
[[ -z "$MODELS" ]]
[[ -z "$REGIONS" ]]

echo "Models: $MODELS"
echo "Regions: $REGIONS"
echo "Verbose: $VERBOSE"

for arg in "$@"; do
if [ "$arg" = "--verbose" ]; then
VERBOSE=true
fi
done

log_verbose() {
if [ "$VERBOSE" = true ]; then
echo "$1"
fi
}

# Default Models and Capacities (Comma-separated in "model:capacity" format)
DEFAULT_MODEL_CAPACITY="gpt-4o:30,gpt-4o-mini:30,gpt-4:30,text-embedding-ada-002:80"
Expand Down Expand Up @@ -51,8 +97,8 @@ DEFAULT_REGIONS="eastus,uksouth,eastus2,northcentralus,swedencentral,westus,west
IFS=',' read -r -a DEFAULT_REGION_ARRAY <<< "$DEFAULT_REGIONS"

# Read parameters (if any)
IFS=',' read -r -a USER_PROVIDED_PAIRS <<< "$1"
USER_REGION="$2"
IFS=',' read -r -a USER_PROVIDED_PAIRS <<< "$MODELS"
USER_REGION="$REGIONS"

IS_USER_PROVIDED_PAIRS=false

Expand Down Expand Up @@ -100,12 +146,12 @@ INDEX=1

VALID_REGIONS=()
for REGION in "${REGIONS[@]}"; do
echo "----------------------------------------"
echo "🔍 Checking region: $REGION"
log_verbose "----------------------------------------"
log_verbose "🔍 Checking region: $REGION"

QUOTA_INFO=$(az cognitiveservices usage list --location "$REGION" --output json | tr '[:upper:]' '[:lower:]')
if [ -z "$QUOTA_INFO" ]; then
echo "⚠️ WARNING: Failed to retrieve quota for region $REGION. Skipping."
log_verbose "⚠️ WARNING: Failed to retrieve quota for region $REGION. Skipping."
continue
fi

Expand All @@ -128,7 +174,7 @@ for REGION in "${REGIONS[@]}"; do
for MODEL_TYPE in "${MODEL_TYPES[@]}"; do
FOUND=false
INSUFFICIENT_QUOTA=false
echo "🔍 Checking model: $MODEL_NAME with required capacity: $REQUIRED_CAPACITY ($MODEL_TYPE)"
log_verbose "🔍 Checking model: $MODEL_NAME with required capacity: $REQUIRED_CAPACITY ($MODEL_TYPE)"

MODEL_INFO=$(echo "$QUOTA_INFO" | awk -v model="\"value\": \"$MODEL_TYPE\"" '
BEGIN { RS="},"; FS="," }
Expand All @@ -137,7 +183,7 @@ for REGION in "${REGIONS[@]}"; do

if [ -z "$MODEL_INFO" ]; then
FOUND=false
echo "⚠️ WARNING: No quota information found for model: $MODEL_NAME in region: $REGION for model type: $MODEL_TYPE."
log_verbose "⚠️ WARNING: No quota information found for model: $MODEL_NAME in region: $REGION for model type: $MODEL_TYPE."
continue
fi

Expand All @@ -153,7 +199,7 @@ for REGION in "${REGIONS[@]}"; do
LIMIT=$(echo "$LIMIT" | cut -d'.' -f1)

AVAILABLE=$((LIMIT - CURRENT_VALUE))
echo "✅ Model: $MODEL_TYPE | Used: $CURRENT_VALUE | Limit: $LIMIT | Available: $AVAILABLE"
log_verbose "✅ Model: $MODEL_TYPE | Used: $CURRENT_VALUE | Limit: $LIMIT | Available: $AVAILABLE"

if [ "$AVAILABLE" -ge "$REQUIRED_CAPACITY" ]; then
FOUND=true
Expand All @@ -168,9 +214,10 @@ for REGION in "${REGIONS[@]}"; do
fi

if [ "$FOUND" = false ]; then
echo "❌ No models found for model: $MODEL_NAME in region: $REGION (${MODEL_TYPES[*]})"
log_verbose "❌ No models found for model: $MODEL_NAME in region: $REGION (${MODEL_TYPES[*]})"

elif [ "$INSUFFICIENT_QUOTA" = true ]; then
echo "⚠️ Model $MODEL_NAME in region: $REGION has insufficient quota (${MODEL_TYPES[*]})."
log_verbose "⚠️ Model $MODEL_NAME in region: $REGION has insufficient quota (${MODEL_TYPES[*]})."
fi
done
done
Expand Down Expand Up @@ -200,4 +247,4 @@ else
echo "➡️ To request a quota increase, visit: https://aka.ms/oai/stuquotarequest"
fi

echo "✅ Script completed."
echo "✅ Script completed."
Loading