KVM: x86/mmu: Drop/zap existing present SPTE even when creating an MMIO SPTE

jira VULN-180395
cve CVE-2026-23401
commit-author Sean Christopherson <seanjc@google.com>
commit aad885e
upstream-diff Used linux-6.1.y backport
  ed5909992f344a7d3f4024261e9f751d9618a27d for the clean pick. Strictly
  speaking, the upstream's `kvm_flush_remote_tlbs_gfn(vcpu->kvm, gfn,
  level)' call corresponds to LTS 9.2-native
  `kvm_flush_remote_tlbs_with_address(vcpu->kvm, gfn &
  -KVM_PAGES_PER_HPAGE(level), KVM_PAGES_PER_HPAGE(level))'. See the
  non-backported commits:
  - 9ffe926 ("KVM: x86/mmu: Fix wrong gfn
    range of tlb flushing in kvm_set_pte_rmapp()"),
  - 8c63e8c ("KVM: x86/mmu: Rename
    kvm_flush_remote_tlbs_with_address()"),
  - c667a3b ("KVM: x86/mmu: Move
    round_gfn_for_level() helper into mmu_internal.h").
  Nevertheless preserved linux-6.1.y's
  `kvm_flush_remote_tlbs_with_address()' call without `gfn' rounding, as
  that's exactly how it's used in the same function a few lines later,
  strongly suggesting that the issues raised in commit 9ffe926 don't
  apply here.

When installing an emulated MMIO SPTE, do so *after* dropping/zapping the
existing SPTE (if it's shadow-present).  While commit a54aa15 was
right about it being impossible to convert a shadow-present SPTE to an
MMIO SPTE due to a _guest_ write, it failed to account for writes to guest
memory that are outside the scope of KVM.

E.g. if host userspace modifies a shadowed gPTE to switch from a memslot
to emulted MMIO and then the guest hits a relevant page fault, KVM will
install the MMIO SPTE without first zapping the shadow-present SPTE.

  ------------[ cut here ]------------
  is_shadow_present_pte(*sptep)
  WARNING: arch/x86/kvm/mmu/mmu.c:484 at mark_mmio_spte+0xb2/0xc0 [kvm], CPU#0: vmx_ept_stale_r/4292
  Modules linked in: kvm_intel kvm irqbypass
  CPU: 0 UID: 1000 PID: 4292 Comm: vmx_ept_stale_r Not tainted 7.0.0-rc2-eafebd2d2ab0-sink-vm #319 PREEMPT
  Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
  RIP: 0010:mark_mmio_spte+0xb2/0xc0 [kvm]
  Call Trace:
   <TASK>
   mmu_set_spte+0x237/0x440 [kvm]
   ept_page_fault+0x535/0x7f0 [kvm]
   kvm_mmu_do_page_fault+0xee/0x1f0 [kvm]
   kvm_mmu_page_fault+0x8d/0x620 [kvm]
   vmx_handle_exit+0x18c/0x5a0 [kvm_intel]
   kvm_arch_vcpu_ioctl_run+0xc55/0x1c20 [kvm]
   kvm_vcpu_ioctl+0x2d5/0x980 [kvm]
   __x64_sys_ioctl+0x8a/0xd0
   do_syscall_64+0xb5/0x730
   entry_SYSCALL_64_after_hwframe+0x4b/0x53
  RIP: 0033:0x47fa3f
   </TASK>
  ---[ end trace 0000000000000000 ]---

	Reported-by: Alexander Bulekov <bkov@amazon.com>
	Debugged-by: Alexander Bulekov <bkov@amazon.com>
	Suggested-by: Fred Griffoul <fgriffo@amazon.co.uk>
Fixes: a54aa15 ("KVM: x86/mmu: Handle MMIO SPTEs directly in mmu_set_spte()")
	Cc: stable@vger.kernel.org
	Signed-off-by: Sean Christopherson <seanjc@google.com>
(cherry picked from commit ed5909992f344a7d3f4024261e9f751d9618a27d)
	Signed-off-by: Marcin Wcisło <marcin.wcislo@conclusive.pl>

Author: pvts-mat

arch/x86/kvm/mmu/mmu.c

@@ -2792,12 +2792,6 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot,
 	pgprintk("%s: spte %llx write_fault %d gfn %llx\n", __func__,
 		 *sptep, write_fault, gfn);
 
-	if (unlikely(is_noslot_pfn(pfn))) {
-		vcpu->stat.pf_mmio_spte_created++;
-		mark_mmio_spte(vcpu, sptep, gfn, pte_access);
-		return RET_PF_EMULATE;
-	}
-
 	if (is_shadow_present_pte(*sptep)) {
 		/*
 		 * If we overwrite a PTE page pointer with a 2MB PMD, unlink
@@ -2819,6 +2813,15 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot,
 			was_rmapped = 1;
 	}
 
+	if (unlikely(is_noslot_pfn(pfn))) {
+		vcpu->stat.pf_mmio_spte_created++;
+		mark_mmio_spte(vcpu, sptep, gfn, pte_access);
+		if (flush)
+			kvm_flush_remote_tlbs_with_address(vcpu->kvm, gfn,
+					KVM_PAGES_PER_HPAGE(level));
+		return RET_PF_EMULATE;
+	}
+
 	wrprot = make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefetch,
 			   true, host_writable, &spte);