Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,10 @@ In case multi-cluster support is enabled (default) and you have access to multip
- `name` (`string`) **(required)** - The name of the virtual machine
- `namespace` (`string`) **(required)** - The namespace of the virtual machine

- **vm_troubleshoot** - Diagnose KubeVirt VirtualMachine issues with automated root-cause detection. Collects VM status, VMI status, volumes, DataVolume/PVC state, cloud-init configuration, pod state, logs, and events, then runs heuristic checks to identify specific problems and suggest fixes. Returns a 'Detected Issues' section with CRITICAL/WARNING findings and actionable remediation steps, followed by raw diagnostic data. Use this tool FIRST whenever a user asks why a VM is not starting, stuck in Provisioning, crashlooping, failing to migrate, or exhibiting unexpected behavior. Automatically detects: missing StorageClasses, invalid PVC specs, dangerous cloud-init commands (shutdown/halt), nodeSelector migration blockers, failed migrations, and pod crashloops.
- `name` (`string`) **(required)** - The name of the VirtualMachine to troubleshoot
- `namespace` (`string`) **(required)** - The namespace of the VirtualMachine to troubleshoot

</details>

<details>
Expand Down
100 changes: 100 additions & 0 deletions evals/tasks/kubevirt/troubleshoot-vm-cloudinit-shutdown/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
kind: Task
apiVersion: mcpchecker/v1alpha2
metadata:
labels:
suite: kubevirt
requires: kubevirt
name: "troubleshoot-vm-cloudinit-shutdown"
difficulty: medium
description: "Diagnose a VM crashlooping due to a shutdown command in cloud-init"
spec:
requires:
- extension: kubernetes
as: k8s
setup:
- k8s.delete:
apiVersion: v1
kind: Namespace
metadata:
name: vm-test-troubleshoot-ci
ignoreNotFound: true
- k8s.create:
apiVersion: v1
kind: Namespace
metadata:
name: vm-test-troubleshoot-ci
- k8s.create:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: crashloop-vm
namespace: vm-test-troubleshoot-ci
spec:
runStrategy: Always
template:
spec:
domain:
devices:
disks:
- name: containerdisk
disk:
bus: virtio
- name: cloudinit
disk:
bus: virtio
resources:
requests:
memory: 2Gi
terminationGracePeriodSeconds: 0
volumes:
- name: containerdisk
containerDisk:
image: quay.io/containerdisks/fedora:latest
- name: cloudinit
cloudInitNoCloud:
userData: |
#cloud-config
runcmd:
- shutdown -h now
- script:
inline: |-
#!/usr/bin/env bash
echo "Waiting for VM to attempt boot and crash..."
sleep 60
verify:
- script:
inline: |-
#!/usr/bin/env bash
NS="vm-test-troubleshoot-ci"

echo "=== Verification: Agent identified cloud-init issue ==="

if ! kubectl get virtualmachine crashloop-vm -n "$NS" > /dev/null 2>&1; then
echo "✗ VirtualMachine crashloop-vm no longer exists"
exit 1
fi
echo "✓ VirtualMachine crashloop-vm still exists"
exit 0
- llmJudge:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: contains: "shutdown" is a fairly weak check — the agent could pass by saying something generic like "the VM appears to shut down" without actually diagnosing the cloud-init cause.

Consider tightening to contains: "shutdown -h now" or adding a second contains for "cloud-init" to confirm the agent identified the specific command and its source.

For comparison, the missing-storageclass task's contains: "non-existent-sc-xyz" is much stronger since it requires the exact StorageClass name.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

contains: "shutdown -h now"
cleanup:
- k8s.delete:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: crashloop-vm
namespace: vm-test-troubleshoot-ci
ignoreNotFound: true
- k8s.delete:
apiVersion: v1
kind: Namespace
metadata:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prompt explicitly names the tool to use (Use the vm_troubleshoot tool). This is an anti-pattern for evals — it tests whether the agent can follow an instruction rather than whether it can select the right tool for the job.

Consider rephrasing to present the problem and let the agent figure out which tools to use, e.g.:

A VirtualMachine named "crashloop-vm" in the vm-test-troubleshoot-ci namespace keeps restarting.
Diagnose why this VM is crashlooping. Report:
- The root cause of the crashloop
- Which cloud-init command is causing the problem
- How to fix it

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

name: vm-test-troubleshoot-ci
ignoreNotFound: true
prompt:
inline: |-
A VirtualMachine named "crashloop-vm" in the vm-test-troubleshoot-ci namespace keeps restarting.
Diagnose why this VM is crashlooping. Report:
- The root cause of the crashloop
- Which cloud-init command is causing the problem
- How to fix it
105 changes: 105 additions & 0 deletions evals/tasks/kubevirt/troubleshoot-vm-missing-storageclass/task.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
kind: Task
apiVersion: mcpchecker/v1alpha2
metadata:
labels:
suite: kubevirt
requires: kubevirt
name: "troubleshoot-vm-missing-storageclass"
difficulty: medium
description: "Diagnose a VM stuck in Provisioning due to a non-existent StorageClass"
spec:
requires:
- extension: kubernetes
as: k8s
setup:
- k8s.delete:
apiVersion: v1
kind: Namespace
metadata:
name: vm-test-troubleshoot-sc
ignoreNotFound: true
- k8s.create:
apiVersion: v1
kind: Namespace
metadata:
name: vm-test-troubleshoot-sc
- k8s.create:
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: broken-sc-vm
namespace: vm-test-troubleshoot-sc
spec:
runStrategy: Always
dataVolumeTemplates:
- metadata:
name: broken-sc-vm-rootdisk
spec:
storage:
storageClassName: non-existent-sc-xyz
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
source:
registry:
url: docker://quay.io/containerdisks/fedora:latest
template:
spec:
domain:
devices:
disks:
- name: rootdisk
disk:
bus: virtio
resources:
requests:
memory: 2Gi
terminationGracePeriodSeconds: 0
volumes:
- name: rootdisk
dataVolume:
name: broken-sc-vm-rootdisk
- script:
inline: |-
#!/usr/bin/env bash
echo "Waiting for VM to enter Provisioning state..."
sleep 15
verify:
- script:
inline: |-
#!/usr/bin/env bash
NS="vm-test-troubleshoot-sc"

echo "=== Verification: Agent identified missing StorageClass ==="

if ! kubectl get virtualmachine broken-sc-vm -n "$NS" > /dev/null 2>&1; then
echo "✗ VirtualMachine broken-sc-vm no longer exists"
exit 1
fi
echo "✓ VirtualMachine broken-sc-vm still exists"
exit 0
- llmJudge:
contains: "non-existent-sc-xyz"
cleanup:
- k8s.delete:
apiVersion: kubevirt.io/v1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as the cloud-init task — the prompt explicitly names vm_troubleshoot which makes the eval test instruction-following rather than tool selection. Consider removing the tool name and letting the agent discover the right tool on its own, e.g.:

A VirtualMachine named "broken-sc-vm" in the vm-test-troubleshoot-sc namespace is not starting.
Diagnose why this VM is stuck. Report:
- The root cause of the issue
- What StorageClass is missing
- What alternative StorageClasses are available on the cluster

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

kind: VirtualMachine
metadata:
name: broken-sc-vm
namespace: vm-test-troubleshoot-sc
ignoreNotFound: true
- k8s.delete:
apiVersion: v1
kind: Namespace
metadata:
name: vm-test-troubleshoot-sc
ignoreNotFound: true
prompt:
inline: |-
A VirtualMachine named "broken-sc-vm" in the vm-test-troubleshoot-sc namespace is not starting.
Diagnose why this VM is stuck. Report:
- The root cause of the issue
- What StorageClass is missing
- What alternative StorageClasses are available on the cluster
17 changes: 17 additions & 0 deletions pkg/kubevirt/gvr.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,16 @@ var (
}
)

// Migration resources
var (
// VirtualMachineInstanceMigrationGVR is the GroupVersionResource for VirtualMachineInstanceMigration resources
VirtualMachineInstanceMigrationGVR = schema.GroupVersionResource{
Group: "kubevirt.io",
Version: "v1",
Resource: "virtualmachineinstancemigrations",
}
)

// HasVirtualMachine returns a TargetCompatibilityFilter that checks whether any
// target cluster has the VirtualMachine GVK registered.
func HasVirtualMachine(p api.FilteringProvider) func() bool {
Expand All @@ -126,4 +136,11 @@ var (
Version: "v1",
Resource: "pods",
}

// StorageClassGVR is the GroupVersionResource for StorageClass resources
StorageClassGVR = schema.GroupVersionResource{
Group: "storage.k8s.io",
Version: "v1",
Resource: "storageclasses",
}
)
91 changes: 91 additions & 0 deletions pkg/mcp/kubevirt_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,97 @@ func (s *KubevirtSuite) TestVMTroubleshootPrompt() {
})
}

func (s *KubevirtSuite) TestVMTroubleshoot() {
s.Run("vm_troubleshoot missing required params", func() {
testCases := []string{"namespace", "name"}
for _, param := range testCases {
s.Run("missing "+param, func() {
params := map[string]interface{}{
"namespace": "default",
"name": "test-vm",
}
delete(params, param)
toolResult, err := s.CallTool("vm_troubleshoot", params)
s.Require().Nilf(err, "call tool failed %v", err)
s.Truef(toolResult.IsError, "expected call tool to fail due to missing %s", param)
s.Equal(toolResult.Content[0].(*mcp.TextContent).Text, param+" parameter required")
})
}
})

s.Run("vm_troubleshoot returns diagnostic report for existing VM", func() {
dynamicClient := dynamic.NewForConfigOrDie(test.EnvTestRestConfig())
vm := &unstructured.Unstructured{}
vm.SetUnstructuredContent(map[string]interface{}{
"apiVersion": "kubevirt.io/v1",
"kind": "VirtualMachine",
"metadata": map[string]interface{}{
"name": "troubleshoot-vm",
"namespace": "default",
},
"spec": map[string]interface{}{
"runStrategy": "Always",
"template": map[string]interface{}{
"spec": map[string]interface{}{
"volumes": []interface{}{
map[string]interface{}{
"name": "rootdisk",
"containerDisk": map[string]interface{}{
"image": "quay.io/containerdisks/fedora:latest",
},
},
},
},
},
},
})
_, err := dynamicClient.Resource(schema.GroupVersionResource{
Group: "kubevirt.io",
Version: "v1",
Resource: "virtualmachines",
}).Namespace("default").Create(s.T().Context(), vm, metav1.CreateOptions{})
s.Require().NoError(err, "failed to create test VM")

toolResult, err := s.CallTool("vm_troubleshoot", map[string]interface{}{
"namespace": "default",
"name": "troubleshoot-vm",
})
s.Run("no error", func() {
s.Nilf(err, "call tool failed %v", err)
s.Falsef(toolResult.IsError, "call tool failed: %v", toolResult.Content)
})

s.Run("returns diagnostic report with correct VM details", func() {
text := toolResult.Content[0].(*mcp.TextContent).Text
s.Truef(strings.HasPrefix(text, "# VirtualMachine Diagnostic Report: default/troubleshoot-vm"),
"Expected diagnostic report header, got %v", text)
s.Contains(text, "## VirtualMachine Status")
s.Contains(text, "## VirtualMachineInstance")
s.Contains(text, "troubleshoot-vm")
})

// Cleanup
_ = dynamicClient.Resource(schema.GroupVersionResource{
Group: "kubevirt.io",
Version: "v1",
Resource: "virtualmachines",
}).Namespace("default").Delete(s.T().Context(), "troubleshoot-vm", metav1.DeleteOptions{})
})

s.Run("vm_troubleshoot handles non-existent VM gracefully", func() {
toolResult, err := s.CallTool("vm_troubleshoot", map[string]interface{}{
"namespace": "default",
"name": "non-existent-vm",
})
s.Nilf(err, "call tool failed %v", err)
s.Falsef(toolResult.IsError, "tool should not return error for non-existent VM")
text := toolResult.Content[0].(*mcp.TextContent).Text
s.Truef(strings.HasPrefix(text, "# VirtualMachine Diagnostic Report: default/non-existent-vm"),
"Expected diagnostic report header, got %v", text)
s.Contains(text, "not found")
})
}

func (s *KubevirtSuite) TestVMGuestInfo() {
s.Run("vm_guest_info missing required params", func() {
testCases := []string{"namespace", "name"}
Expand Down
29 changes: 29 additions & 0 deletions pkg/mcp/testdata/toolsets-kubevirt-tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -233,5 +233,34 @@
},
"name": "vm_lifecycle",
"title": "Virtual Machine: Lifecycle"
},
{
"annotations": {
"destructiveHint": false,
"idempotentHint": true,
"openWorldHint": true,
"readOnlyHint": true,
"title": "Virtual Machine: Troubleshoot"
},
"description": "Diagnose KubeVirt VirtualMachine issues with automated root-cause detection. Collects VM status, VMI status, volumes, DataVolume/PVC state, cloud-init configuration, pod state, logs, and events, then runs heuristic checks to identify specific problems and suggest fixes. Returns a 'Detected Issues' section with CRITICAL/WARNING findings and actionable remediation steps, followed by raw diagnostic data. Use this tool FIRST whenever a user asks why a VM is not starting, stuck in Provisioning, crashlooping, failing to migrate, or exhibiting unexpected behavior. Automatically detects: missing StorageClasses, invalid PVC specs, dangerous cloud-init commands (shutdown/halt), nodeSelector migration blockers, failed migrations, and pod crashloops.",
"inputSchema": {
"properties": {
"name": {
"description": "The name of the VirtualMachine to troubleshoot",
"type": "string"
},
"namespace": {
"description": "The namespace of the VirtualMachine to troubleshoot",
"type": "string"
}
},
"required": [
"namespace",
"name"
],
"type": "object"
},
"name": "vm_troubleshoot",
"title": "Virtual Machine: Troubleshoot"
}
]
2 changes: 2 additions & 0 deletions pkg/toolsets/kubevirt/toolset.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
vm_create "github.com/containers/kubernetes-mcp-server/pkg/toolsets/kubevirt/vm/create"
vm_guestagent "github.com/containers/kubernetes-mcp-server/pkg/toolsets/kubevirt/vm/guestagent"
vm_lifecycle "github.com/containers/kubernetes-mcp-server/pkg/toolsets/kubevirt/vm/lifecycle"
vm_troubleshoot "github.com/containers/kubernetes-mcp-server/pkg/toolsets/kubevirt/vm/troubleshoot"
)

type Toolset struct{}
Expand All @@ -30,6 +31,7 @@ func (t *Toolset) GetTools(p api.FilteringProvider) []api.ServerTool {
vm_create.Tools(p),
vm_guestagent.Tools(p),
vm_lifecycle.Tools(p),
vm_troubleshoot.Tools(p),
)
}

Expand Down
Loading