Skip to content

Conversation

@adityamaru
Copy link

@adityamaru adityamaru commented Nov 22, 2025

This patch enhances Hyper-V enlightenment support to enable nested virtualization scenarios, specifically allowing Windows guests to run nested Hyper-V VMs and WSL2 on AMD and Intel platforms.

Problem:
When running Windows Server 2025 as a guest with Hyper-V role enabled, nested VM creation fails with 'Hyper-V component not running' errors, and WSL2 installation fails with HCS_E_HYPERV_NOT_INSTALLED. This occurs despite Hyper-V services (vmms, vmcompute) starting successfully and Windows detecting SLAT support.

Root Cause:
Cloud Hypervisor was exposing only 4 out of 13 critical Hyper-V partition privilege flags in CPUID leaf 0x40000003. Windows performs privilege checks before allowing nested VM operations, and without flags like HV_HYPERCALL_AVAILABLE and HV_ACCESS_FREQUENCY_MSRS, it refuses to start nested VMs.

Solution:
This patch implements the following changes based on QEMU's Hyper-V implementation (target/i386/kvm/hyperv-proto.h) and Microsoft's Hypervisor Top-Level Functional Specification (TLFS):

  1. Nested Virtualization Detection:

    • Detects AMD SVM (CPUID 0x8000_0001.ECX bit 2)
    • Detects Intel VMX (CPUID 0x1.ECX bit 5)
    • Only exposes nested features when host supports nested virt
  2. Enhanced CPUID Leaf 0x40000003 (Partition Privileges):

    • Always exposed (base): * HV_TIME_REF_COUNT_AVAILABLE (bit 1) * HV_SYNIC_AVAILABLE (bit 2) * HV_SYNTIMERS_AVAILABLE (bit 3) * HV_REFERENCE_TSC_AVAILABLE (bit 9)
    • Added for nested (when supported):
      • HV_VP_RUNTIME_AVAILABLE (bit 0)
      • HV_APIC_ACCESS_AVAILABLE (bit 4) * HV_HYPERCALL_AVAILABLE (bit 5) - Critical * HV_VP_INDEX_AVAILABLE (bit 6) * HV_RESET_AVAILABLE (bit 7) * HV_ACCESS_FREQUENCY_MSRS (bit 11) - Critical * HV_ACCESS_REENLIGHTENMENTS_CONTROL (bit 13)
  3. Enhanced CPUID Leaf 0x40000003 (Implementation Features, EDX):

    • Added for nested: * HV_MWAIT_AVAILABLE (bit 0) * HV_GUEST_IDLE_STATE_AVAILABLE (bit 5) * HV_FREQUENCY_MSRS_AVAILABLE (bit 8) * HV_GUEST_CRASH_MSR_AVAILABLE (bit 10) * HV_STIMER_DIRECT_MODE_AVAILABLE (bit 19)
  4. New CPUID Leaf 0x4000000A (Nested Features):

    • HV_NESTED_DIRECT_FLUSH (bit 17)
    • HV_NESTED_MSR_BITMAP (bit 19)
  5. Enhanced CPUID Leaf 0x40000004 (Recommendations):

    • Added for nested: * HV_APIC_ACCESS_RECOMMENDED (bit 3) * HV_SYSTEM_RESET_RECOMMENDED (bit 4) * HV_CLUSTER_IPI_RECOMMENDED (bit 10) * HV_EX_PROCESSOR_MASKS_RECOMMENDED (bit 11)
  6. Comprehensive Logging:

    • Logs nested virt detection
    • Logs all CPUID values being exposed
    • Aids in debugging and validation

Testing:

  • Builds without errors
  • Maintains backward compatibility (features only added when nested=true)
  • Validated against QEMU's hyperv-proto.h bit definitions

Expected Results:

  • 'wsl --install -d Ubuntu' succeeds (not HCS_E_HYPERV_NOT_INSTALLED)
  • 'Start-VM NestedTest' succeeds (not 'component not running')
  • Existing non-nested Windows guests continue working
  • Linux guests unaffected

@adityamaru adityamaru closed this Nov 22, 2025
@adityamaru adityamaru reopened this Nov 22, 2025
@adityamaru adityamaru force-pushed the fix/nested-hyperv-amd-issue-7500 branch 3 times, most recently from 71c60b3 to bdd830a Compare November 22, 2025 15:43
…lization

This patch enhances Hyper-V enlightenment support to enable nested
virtualization scenarios, specifically allowing Windows guests to run
nested Hyper-V VMs and WSL2 on AMD and Intel platforms.

Problem:
When running Windows Server 2025 as a guest with Hyper-V role enabled,
nested VM creation fails with 'Hyper-V component not running' errors,
and WSL2 installation fails with HCS_E_HYPERV_NOT_INSTALLED. This
occurs despite Hyper-V services (vmms, vmcompute) starting successfully
and Windows detecting SLAT support.

Root Cause:
Cloud Hypervisor was exposing only 4 out of 13 critical Hyper-V
partition privilege flags in CPUID leaf 0x40000003. Windows performs
privilege checks before allowing nested VM operations, and without
flags like HV_HYPERCALL_AVAILABLE and HV_ACCESS_FREQUENCY_MSRS, it
refuses to start nested VMs.

Solution:
This patch implements the following changes based on QEMU's Hyper-V
implementation (target/i386/kvm/hyperv-proto.h) and Microsoft's
Hypervisor Top-Level Functional Specification (TLFS):

1. Nested Virtualization Detection:
   - Detects AMD SVM (CPUID 0x8000_0001.ECX bit 2)
   - Detects Intel VMX (CPUID 0x1.ECX bit 5)
   - Only exposes nested features when host supports nested virt

2. Enhanced CPUID Leaf 0x40000003 (Partition Privileges):
   - Always exposed (base):
     * HV_TIME_REF_COUNT_AVAILABLE (bit 1)
     * HV_SYNIC_AVAILABLE (bit 2)
     * HV_SYNTIMERS_AVAILABLE (bit 3)
     * HV_REFERENCE_TSC_AVAILABLE (bit 9)
   - Added for nested (when supported):
     * HV_VP_RUNTIME_AVAILABLE (bit 0)
     * HV_APIC_ACCESS_AVAILABLE (bit 4)
     * HV_HYPERCALL_AVAILABLE (bit 5) - Critical
     * HV_VP_INDEX_AVAILABLE (bit 6)
     * HV_RESET_AVAILABLE (bit 7)
     * HV_ACCESS_FREQUENCY_MSRS (bit 11) - Critical
     * HV_ACCESS_REENLIGHTENMENTS_CONTROL (bit 13)

3. Enhanced CPUID Leaf 0x40000003 (Implementation Features, EDX):
   - Added for nested:
     * HV_MWAIT_AVAILABLE (bit 0)
     * HV_GUEST_IDLE_STATE_AVAILABLE (bit 5)
     * HV_FREQUENCY_MSRS_AVAILABLE (bit 8)
     * HV_GUEST_CRASH_MSR_AVAILABLE (bit 10)
     * HV_STIMER_DIRECT_MODE_AVAILABLE (bit 19)

4. New CPUID Leaf 0x4000000A (Nested Features):
   - HV_NESTED_DIRECT_FLUSH (bit 17)
   - HV_NESTED_MSR_BITMAP (bit 19)

5. Enhanced CPUID Leaf 0x40000004 (Recommendations):
   - Added for nested:
     * HV_APIC_ACCESS_RECOMMENDED (bit 3)
     * HV_SYSTEM_RESET_RECOMMENDED (bit 4)
     * HV_CLUSTER_IPI_RECOMMENDED (bit 10)
     * HV_EX_PROCESSOR_MASKS_RECOMMENDED (bit 11)

6. Comprehensive Logging:
   - Logs nested virt detection
   - Logs all CPUID values being exposed
   - Aids in debugging and validation

Testing:
- Builds without errors
- Maintains backward compatibility (features only added when nested=true)
- Validated against QEMU's hyperv-proto.h bit definitions

Expected Results:
- 'wsl --install -d Ubuntu' succeeds (not HCS_E_HYPERV_NOT_INSTALLED)
- 'Start-VM NestedTest' succeeds (not 'component not running')
- Existing non-nested Windows guests continue working
- Linux guests unaffected

Fixes: cloud-hypervisor#7500 (if upstream issue exists)
Signed-off-by: Cloud Hypervisor AI Assistant <[email protected]>
@adityamaru adityamaru force-pushed the fix/nested-hyperv-amd-issue-7500 branch from bdd830a to e475095 Compare November 22, 2025 18:11
@rbradford
Copy link

@adityamaru Thanks for your PR. Please review the CONTRIBUTING.md - in particular the DCO section.

@adityamaru adityamaru marked this pull request as draft November 22, 2025 21:19
@adityamaru
Copy link
Author

Hey @rbradford this is very much an experiemental draft still, it isn't working as I'd expect. If I do get it working and contribute this upstream I will make sure to follow the guidelines!

@rbradford
Copy link

Hey @rbradford this is very much an experiemental draft still, it isn't working as I'd expect. If I do get it working and contribute this upstream I will make sure to follow the guidelines!

My mistake - I didn't realise this was against a fork - i'd clicked through from the the issue :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants