Skip to content

Commit f2e906b

Browse files
vineeth-thummaBavneet Singhatchutbarlimcnealm13matthewmcneal
committed
[Azure RBAC] Deprecate 3P mode flags, fix Azure RBAC enablement bug, add E2E coverage and improve logging (#20)
* add pester tests for connectedk8s cli extension * Pass the force delete param to the API call (#4) * forcedelete * format * add code owner * mypy * Parameterize for airgapped clouds (#5) * Add parameterization for the airgapped clouds * Fix azdev style * MCR path function * azdev, ruff, and mypy --------- Co-authored-by: Matthew McNeal (from Dev Box) <mmcneal@microsoft.com> * Oras client fix to work with different MCRs (#6) Co-authored-by: mmcneal <mmcneal@microsoft.com> * fix CI testcases for nodepool image issues (#8) * update errors for the config and connectivity issues (#11) * update errors * format * style * update python version to 3.13 (#12) * Update cluster diagnostics image to 1.29.3 (#7) * Update cluster diagnostics helm chart to 1.29.3 * Fix lint issues --------- Co-authored-by: bgriddaluru <bharath.griddaluru@microsoft.com> * RBAC deprecation & fix the issue * typo * fix comments * update tests * add pester tests for connectedk8s cli extension * Pass the force delete param to the API call (#4) * forcedelete * format * add code owner * mypy * fix CI testcases for nodepool image issues (#8) * update errors for the config and connectivity issues (#11) * update errors * format * style * update python version to 3.13 (#12) * rebase * fix tests * fix version * fix mypy, lint * fix test * fix test * fix test * fix test * fix test * rename test * deprecate flags * rebase * rebase * bump version for release --------- Co-authored-by: Bavneet Singh <bavneetsingh@microsoft.com> Co-authored-by: Atchut Kumar Barli <atchut@gmail.com> Co-authored-by: mcnealm13 <57726243+mcnealm13@users.noreply.github.com> Co-authored-by: Matthew McNeal (from Dev Box) <mmcneal@microsoft.com> Co-authored-by: Bavneet Singh <33008256+bavneetsingh16@users.noreply.github.com> Co-authored-by: bgriddaluru <117554445+bgriddaluru@users.noreply.github.com> Co-authored-by: bgriddaluru <bharath.griddaluru@microsoft.com> Co-authored-by: vithumma <vithumma@microsoft.com>
1 parent b102428 commit f2e906b

File tree

10 files changed

+151
-20
lines changed

10 files changed

+151
-20
lines changed

‎src/connectedk8s/HISTORY.rst‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22
33
Release History
44
===============
5+
1.10.10
6+
+++++
7+
* Deprecated '--app-id' and '--app-secret' RBAC parameters from the extension by adding them to _breaking_change.py.
8+
* Bug fix for https://github.com/Azure/azure-cli-extensions/issues/8498.
9+
* Update warning to use the latest kubelogin version which has support for generating PoP token.
10+
511
1.10.9
612
++++++
713
* Added support for associating and disassociating gateways in CLI and updated SDK version to '2025-08-01-preview'.
@@ -78,7 +84,7 @@ Release History
7884
++++++
7985
* New api version 2024-07-1-preview added
8086
* Adding functionality for workload identity feature.
81-
* Cluster create and update waits for agent state
87+
* Cluster create and update waits for agent state
8288

8389
1.7.3
8490
++++++

‎src/connectedk8s/azext_connectedk8s/_breaking_change.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
from azure.cli.core.breaking_change import register_argument_deprecate
6+
7+
register_argument_deprecate("connectedk8s enable-features", "--app-id")
8+
register_argument_deprecate("connectedk8s enable-features", "--app-secret")

‎src/connectedk8s/azext_connectedk8s/_utils.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1056,10 +1056,10 @@ def arm_exception_handler(
10561056
status_code = ex.status_code
10571057
if status_code == 404 and return_if_not_found:
10581058
return
1059-
if status_code is not None and status_code // 100 == 4:
1059+
if status_code and status_code // 100 == 4:
10601060
telemetry.set_user_fault()
10611061
telemetry.set_exception(exception=ex, fault_type=fault_type, summary=summary)
1062-
if status_code is not None and status_code // 100 == 5:
1062+
if status_code and status_code // 100 == 5:
10631063
raise AzureInternalError(
10641064
"Http response error occured while making ARM request: "
10651065
+ str(ex)

‎src/connectedk8s/azext_connectedk8s/custom.py‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2993,6 +2993,10 @@ def enable_features(
29932993
utils.check_features_to_update(features)
29942994
)
29952995

2996+
# Initialize these variables to ensure they are always defined, preventing UnboundLocalError if only a subset of features is enabled.
2997+
final_enable_cl = False
2998+
custom_locations_oid = None
2999+
29963000
# Check if cluster is private link enabled
29973001
connected_cluster = client.get(resource_group_name, cluster_name)
29983002

@@ -3152,8 +3156,9 @@ def enable_features(
31523156
# apps for authN/authZ.
31533157
cmd_helm_upgrade.extend(["--set", "systemDefaultValues.guard.authnMode=arc"])
31543158
logger.warning(
3155-
"Please use the kubelogin version v0.0.32 or higher which has support for generating PoP token(s). "
3156-
"This is needed by guard running in 'arc' authN mode."
3159+
"[Azure RBAC] For secure authentication, ensure you have the latest kubelogin installed which supports PoP tokens. "
3160+
"This is required for Azure RBAC. Download or upgrade at: https://github.com/Azure/kubelogin/releases. "
3161+
"If you encounter authentication errors, please verify your kubelogin version and refer to the documentation: https://learn.microsoft.com/en-us/azure/azure-arc/kubernetes/azure-rbac"
31573162
)
31583163
cmd_helm_upgrade.extend(
31593164
[

‎src/connectedk8s/setup.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# TODO: Confirm this is the right version number you want and it matches your
1414
# HISTORY.rst entry.
1515

16-
VERSION = "1.10.9"
16+
VERSION = "1.10.10"
1717

1818
# The full list of classifiers is available at
1919
# https://pypi.python.org/pypi?%3Aaction=list_classifiers

‎testing/pipeline/k8s-custom-pipelines.yml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ stages:
2828
parameters:
2929
jobName: BasicOnboardingTest
3030
path: ./test/configurations/BasicOnboarding.Tests.ps1
31+
- template: ./templates/run-test.yml
32+
parameters:
33+
jobName: EnableDisableFeaturesTest
34+
path: ./test/configurations/EnableDisableFeatures.Tests.ps1
3135
- template: ./templates/run-test.yml
3236
parameters:
3337
jobName: AutoUpdateTest
@@ -177,13 +181,12 @@ stages:
177181
pip install pytest
178182
cd /home/vsts/work/1/s/src/connectedk8s/azext_connectedk8s/tests/unittests
179183
pytest --junitxml=test-results.xml
180-
181184
displayName: 'Run UnitTests test'
182185
- task: PublishTestResults@2
183186
inputs:
184187
testResultsFormat: 'JUnit'
185188
testResultsFiles: '**/test-results.xml'
186-
failTaskOnFailedTests: true
189+
failTaskOnFailedTests: true
187190
- job: SourceTests
188191
displayName: "Integration Tests, Build Tests"
189192
pool:

‎testing/pipeline/templates/run-test.yml‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ jobs:
4141
azdev extension build $(EXTENSION_NAME)
4242
workingDirectory: $(CLI_REPO_PATH)
4343
displayName: "Setup and Build Extension with azdev"
44-
44+
4545
- bash: |
4646
K8S_CONFIG_VERSION=$(ls ${EXTENSION_FILE_NAME}* | cut -d "-" -f2)
4747
echo "##vso[task.setvariable variable=K8S_CONFIG_VERSION]$K8S_CONFIG_VERSION"
@@ -60,7 +60,7 @@ jobs:
6060
--arg AKS_CLUSTER_NAME "$AKS_CLUSTER_NAME" \
6161
--arg ARC_CLUSTER_NAME "$ARC_CLUSTER_NAME" \
6262
--arg K8S_CONFIG_VERSION "$K8S_CONFIG_VERSION" \
63-
'{subscriptionId: $SUB_ID, resourceGroup: $RG, aksClusterName: $AKS_CLUSTER_NAME, arcClusterName: $ARC_CLUSTER_NAME, extensionVersion: {"connectedk8s": $K8S_CONFIG_VERSION}}')
63+
'{subscriptionId: $SUB_ID, resourceGroup: $RG, aksClusterName: $AKS_CLUSTER_NAME, arcClusterName: $ARC_CLUSTER_NAME, extensionVersion: {"connectedk8s": $K8S_CONFIG_VERSION}, customLocationsOid: "51dfe1e8-70c6-4de5-a08e-e18aff23d815"}')
6464
echo $JSON_STRING > settings.json
6565
cat settings.json
6666
workingDirectory: $(TEST_PATH)
@@ -74,7 +74,6 @@ jobs:
7474
chmod +x ./kind
7575
./kind create cluster
7676
displayName: "Create and Start the Kind cluster"
77-
7877
- bash: |
7978
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
8079
displayName: "Upgrade az to latest version"
@@ -94,7 +93,6 @@ jobs:
9493
inlineScript: |
9594
.\Bootstrap.ps1 -CI
9695
workingDirectory: $(TEST_PATH)
97-
9896
- task: AzureCLI@2
9997
displayName: Run the Test Suite for ${{ parameters.path }}
10098
inputs:

‎testing/test/configurations/BasicOnboarding.Tests.ps1‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Describe 'Basic Onboarding Scenario' {
1010

1111
# Loop and retry until the configuration installs
1212
$n = 0
13-
do
13+
do
1414
{
1515
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
1616
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
@@ -34,7 +34,7 @@ Describe 'Basic Onboarding Scenario' {
3434

3535
# Loop and retry until the configuration installs
3636
$n = 0
37-
do
37+
do
3838
{
3939
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
4040
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)

‎testing/test/configurations/EnableDisableFeatures.Tests.ps1‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
Describe 'ConnectedK8s Enable Disable Features Scenario' {
2+
BeforeAll {
3+
. $PSScriptRoot/../helper/Constants.ps1
4+
5+
function Invoke-AzCommand {
6+
param (
7+
[string]$Command
8+
)
9+
Write-Host "Executing: $Command" -ForegroundColor Yellow
10+
$result = Invoke-Expression $Command
11+
return $result
12+
}
13+
14+
function Wait-ForProvisioning {
15+
param (
16+
[string]$expectedProvisioningState,
17+
[string]$expectedAutoUpdate
18+
)
19+
$n = 0
20+
do {
21+
$output = Invoke-AzCommand "az connectedk8s show -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup)"
22+
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
23+
$provisioningState = ($output | ConvertFrom-Json).provisioningState
24+
$autoUpdate = $jsonOutput.RootElement.GetProperty("arcAgentProfile").GetProperty("agentAutoUpgrade").GetString()
25+
Write-Host "Provisioning State: $provisioningState"
26+
Write-Host "Auto Update: $autoUpdate"
27+
if ($provisioningState -eq $expectedProvisioningState -and $autoUpdate -eq $expectedAutoUpdate) {
28+
break
29+
}
30+
Start-Sleep -Seconds 10
31+
$n += 1
32+
} while ($n -le $MAX_RETRY_ATTEMPTS)
33+
$n | Should -BeLessOrEqual $MAX_RETRY_ATTEMPTS
34+
}
35+
}
36+
37+
It 'Onboard Connected cluster with no features enabled' {
38+
Invoke-AzCommand "az connectedk8s connect -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) -l $ARC_LOCATION --no-wait"
39+
$? | Should -BeTrue
40+
Start-Sleep -Seconds 10
41+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
42+
}
43+
44+
It 'Enable azure-rbac feature' {
45+
Invoke-AzCommand "az connectedk8s enable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features azure-rbac"
46+
$? | Should -BeTrue
47+
Start-Sleep -Seconds 10
48+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
49+
}
50+
51+
It 'Disable azure-rbac feature' {
52+
Invoke-AzCommand "az connectedk8s disable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features azure-rbac --yes"
53+
$? | Should -BeTrue
54+
Start-Sleep -Seconds 10
55+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
56+
}
57+
58+
It 'Enable cluster-connect feature' {
59+
Invoke-AzCommand "az connectedk8s enable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features cluster-connect"
60+
$? | Should -BeTrue
61+
Start-Sleep -Seconds 10
62+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
63+
}
64+
65+
It 'Disable cluster-connect feature' {
66+
Invoke-AzCommand "az connectedk8s disable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features cluster-connect --yes"
67+
$? | Should -BeTrue
68+
Start-Sleep -Seconds 10
69+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
70+
}
71+
72+
It 'Enable custom-locations feature' {
73+
Invoke-AzCommand "az connectedk8s enable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features custom-locations --custom-locations-oid $($ENVCONFIG.customLocationsOid)"
74+
$? | Should -BeTrue
75+
Start-Sleep -Seconds 10
76+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
77+
}
78+
79+
It 'Disable custom-locations feature' {
80+
Invoke-AzCommand "az connectedk8s disable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features custom-locations --yes"
81+
$? | Should -BeTrue
82+
Start-Sleep -Seconds 10
83+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
84+
}
85+
86+
It 'Enable all features (cluster-connect, custom-locations, azure-rbac) together' {
87+
Invoke-AzCommand "az connectedk8s enable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features cluster-connect custom-locations azure-rbac --custom-locations-oid $($ENVCONFIG.customLocationsOid)"
88+
$? | Should -BeTrue
89+
Start-Sleep -Seconds 10
90+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
91+
}
92+
93+
It 'Disable all features (cluster-connect, custom-locations, azure-rbac) together' {
94+
Invoke-AzCommand "az connectedk8s disable-features -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) --features cluster-connect custom-locations azure-rbac --yes"
95+
$? | Should -BeTrue
96+
Start-Sleep -Seconds 10
97+
Wait-ForProvisioning -expectedProvisioningState $SUCCEEDED -expectedAutoUpdate "Enabled"
98+
}
99+
100+
It "Delete the connected instance" {
101+
Invoke-AzCommand "az connectedk8s delete -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup) -y"
102+
$? | Should -BeTrue
103+
104+
# Wait for deletion to propagate through the resource model
105+
Start-Sleep -Seconds 30
106+
107+
# Configuration should be removed from the resource model - expect ResourceNotFound error
108+
$output = Invoke-AzCommand "az connectedk8s show -n $($ENVCONFIG.arcClusterName) -g $($ENVCONFIG.resourceGroup)" 2>&1
109+
$output | Should -Match "ResourceNotFound"
110+
}
111+
}

‎testing/test/configurations/Gateway.Tests.ps1‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Describe 'Onboarding with Gateway Scenario' {
1212

1313
# Loop and retry until the configuration installs
1414
$n = 0
15-
do
15+
do
1616
{
1717
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
1818
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
@@ -36,12 +36,12 @@ Describe 'Onboarding with Gateway Scenario' {
3636

3737
# Loop and retry until the configuration installs
3838
$n = 0
39-
do
39+
do
4040
{
4141
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
4242
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
4343
$provisioningState = ($output | ConvertFrom-Json).provisioningState
44-
$gatewayStatus = $jsonOutput.RootElement.GetProperty("gateway").GetProperty("enabled").GetBoolean()
44+
$gatewayStatus = $jsonOutput.RootElement.GetProperty("gateway").GetProperty("enabled").GetBoolean()
4545
Write-Host "Provisioning State: $provisioningState"
4646
Write-Host "Gateway Status: $gatewayStatus"
4747
if ($provisioningState -eq $SUCCEEDED -and $gatewayStatus -eq $false) {
@@ -60,7 +60,7 @@ Describe 'Onboarding with Gateway Scenario' {
6060

6161
# Loop and retry until the configuration installs
6262
$n = 0
63-
do
63+
do
6464
{
6565
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
6666
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
@@ -84,12 +84,12 @@ Describe 'Onboarding with Gateway Scenario' {
8484

8585
# Loop and retry until the configuration installs
8686
$n = 0
87-
do
87+
do
8888
{
8989
$output = az connectedk8s show -n $ENVCONFIG.arcClusterName -g $ENVCONFIG.resourceGroup
9090
$jsonOutput = [System.Text.Json.JsonDocument]::Parse($output)
9191
$provisioningState = ($output | ConvertFrom-Json).provisioningState
92-
$gatewayStatus = $jsonOutput.RootElement.GetProperty("gateway").GetProperty("enabled").GetBoolean()
92+
$gatewayStatus = $jsonOutput.RootElement.GetProperty("gateway").GetProperty("enabled").GetBoolean()
9393
Write-Host "Provisioning State: $provisioningState"
9494
Write-Host "Gateway Status: $gatewayStatus"
9595
if ($provisioningState -eq $SUCCEEDED -and $gatewayStatus -eq $false) {

0 commit comments

Comments
 (0)