Skip to content

fix(ml): handle workspace stuck in soft-deleted state for permanently-delete and create commands#10127

Draft
a0x1ab with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-azure-ml-workspace-deletion
Draft

fix(ml): handle workspace stuck in soft-deleted state for permanently-delete and create commands#10127
a0x1ab with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-azure-ml-workspace-deletion

Conversation

Copilot AI commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Related command

az ml workspace delete --permanently-delete / az ml workspace create

Problem

When a workspace is soft-deleted, the ARM GET endpoint returns 404 for it. The SDK's begin_delete() calls self.get(name) first — so az ml workspace delete --permanently-delete on an already-soft-deleted workspace always fails with "workspace not found" before the actual purge REST call is made. This also blocks az ml workspace create with the same name, with no guidance on how to unblock it.

Changes

workspace.py — core fix:

  • ml_workspace_delete: catch ResourceNotFoundError from begin_delete separately. When --permanently-delete is set and the workspace is not reachable via get() (already soft-deleted), fall back to calling the underlying REST operation directly with force_to_purge=True, bypassing the SDK's pre-flight get() check:
# Before: SDK's begin_delete() called self.get(name) first → 404 → failed
# After: falls through to direct REST call when workspace is already soft-deleted
del_result = ml_client.workspaces._operation.begin_delete(
    resource_group_name=resource_group_name,
    workspace_name=name,
    force_to_purge=True,
)
  • ml_workspace_create: on failure, emit a targeted WARNING with the exact purge command when the error resembles a soft-delete conflict (keywords: soft, deleted, conflict, already exists):
WARNING: If a workspace named 'my-ws' already exists in a soft-deleted state and is blocking
this creation, you can permanently delete (purge) it with:
  az ml workspace delete -g my-rg -n my-ws --permanently-delete

_help/_workspace_help.py / _params/_workspace_params.py — updated help text for ml workspace delete and --permanently-delete / -p to document the soft-deleted workspace purge scenario.


This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines

  • Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
  • Have you run python scripts/ci/test_index.py -q locally? (pip install azdev required)
  • My extension version conforms to the Extension version schema

For new extensions:

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

@azure-client-tools-bot-prd

Copy link
Copy Markdown

Hi @copilot,
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in setup.py as well.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@yonzhan

yonzhan commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

ML

…te commands

- In ml_workspace_delete: when permanently_delete=True and the workspace
  is not found via the SDK's get() (i.e. already soft-deleted), fall back
  to calling the underlying REST API directly with force_to_purge=True.
  This allows users to purge an already-soft-deleted workspace that was
  blocking operations like workspace creation with the same name.

- In ml_workspace_create: emit a helpful warning when the creation failure
  looks like it was caused by a soft-deleted workspace with the same name,
  pointing the user to 'az ml workspace delete --permanently-delete'.

- Update help text for 'ml workspace delete' and the --permanently-delete
  flag to document the soft-deleted workspace purge scenario.

Fixes: Azure ML workspace stuck in soft-deleted state (#10126)
Copilot AI changed the title [WIP] Fix Azure ML workspace soft-deleted state issue fix(ml): handle workspace stuck in soft-deleted state for permanently-delete and create commands Jul 21, 2026
Copilot AI requested a review from a0x1ab July 21, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[machinelearningservices] Azure ML workspace stuck in soft-deleted state: create blocked, permanently-delete cannot find workspace

4 participants