Ubuntu
qemu package

qemu-system-riscv64: crash when starting nested VM

Bug #2039700 reported by Heinrich Schuchardt
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
qemu (Ubuntu)
Fix Released
Undecided
Sergio Durigan Junior

Bug Description

I have started a virtual machine on amd64 with:

/usr/bin/qemu-system-riscv64 -machine virt -nographic -m 8192 -smp 8 -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.bin -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf -device virtio-net-device,netdev=eth0 -netdev user,id=eth0 -device virtio-rng-pci -drive file=ubuntu-23.10-preinstalled-server-riscv64.img,format=raw,if=virtio

That VM provides the hypervisor extension needed for KVM. I try to launch a nested VM with:

qemu-system-riscv64 -machine virt -nographic -m 768 -smp 4 -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf -device virtio-net-device,netdev=eth0 -netdev user,id=eth0 -device virtio-rng-pci -drive file=ubuntu-23.10-preinstalled-server-riscv64.img,format=raw,if=virtio -accel kvm -cpu host

This leads to a crash

ERROR:target/riscv/cpu.c:313:satp_mode_str: code should not be reached
Bail out! ERROR:target/riscv/cpu.c:313:satp_mode_str: code should not be reached
Aborted (core dumped)
ubuntu@ubuntu:~$ ls /var/crash/
_usr_bin_qemu-system-riscv64.1000.crash

The error only occurs when specifying -cpu host (which is enumerated with -cpu list).

Nested emulation was working on kinetic with -cpu host.

-bios must not be specified for the secondary VM running on KVM as kvm provides the SBI implementation.

ProblemType: Bug
DistroRelease: Ubuntu 23.10
Package: qemu-system-misc 1:8.0.4+dfsg-1ubuntu3
ProcVersionSignature: Ubuntu 6.5.0-9.9.1-generic 6.5.3
Uname: Linux 6.5.0-9-generic riscv64
ApportVersion: 2.27.0-0ubuntu5
Architecture: riscv64
CasperMD5CheckResult: unknown
CloudArchitecture: riscv64
CloudBuildName: server
CloudID: nocloud
CloudName: unknown
CloudPlatform: nocloud
CloudSerial: 20231010
CloudSubPlatform: seed-dir (/var/lib/cloud/seed/nocloud-net)
Date: Wed Oct 18 14:56:16 2023
KvmCmdLine: COMMAND STAT EUID RUID PID PPID %CPU COMMAND
Lspci-vt:
 -[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge
            +-01.0 Red Hat, Inc. Virtio RNG
            \-02.0 Red Hat, Inc. Virtio block device
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t:

Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
ProcEnviron:
 LANG=C.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=vt220
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-6.5.0-9-generic root=LABEL=cloudimg-rootfs ro efi=debug earlycon
SourcePackage: qemu
UpgradeStatus: No upgrade log present (probably fresh install)
acpidump:

See original description

Related branches

~sergiodj/ubuntu/+source/qemu:merge-8.1.3-noble
git-ubuntu bot: Approve
Athos Ribeiro (community): Approve
Canonical Server Reporter: Pending requested
Canonical Server Core Reviewers: Pending requested
Diff: 8268 lines (+7660/-10)
18 files modified
debian/changelog (+5155/-3)
debian/control (+54/-7)
debian/control-in (+3/-0)
debian/patches/series (+10/-0)
debian/patches/ubuntu/define-ubuntu-machine-types.patch (+1008/-0)
debian/patches/ubuntu/enable-svm-by-default.patch (+34/-0)
debian/patches/ubuntu/lp2003673-s390x-ap-passthrough-for-pv-guests.patch (+178/-0)
debian/patches/ubuntu/lp2003673-s390x-refactor-ap-functionalities.patch (+100/-0)
debian/patches/ubuntu/lp2003673-s390x-system-reset-before-unprotect-on-reboot.patch (+51/-0)
debian/patches/ubuntu/lp2003673-update-linux-headers-6.6rc1.patch (+782/-0)
debian/patches/ubuntu/pre-bionic-256k-ipxe-efi-roms.patch (+64/-0)
debian/patches/ubuntu/qboot-Disable-LTO-for-ELF-binary-build-step.patch (+44/-0)
debian/qemu-block-extra.postinst (+59/-0)
debian/qemu-kvm-init (+89/-0)
debian/qemu-system-common.install (+1/-0)
debian/qemu-system-common.qemu-kvm.default (+8/-0)
debian/qemu-system-common.qemu-kvm.service (+16/-0)
debian/rules (+4/-0)
Revision history for this message
Heinrich Schuchardt (xypron) wrote :
Revision history for this message
Heinrich Schuchardt (xypron) wrote :
Heinrich Schuchardt (xypron)
Changed in qemu (Ubuntu):
assignee: nobody → Heinrich Schuchardt (xypron)
description: updated
description: updated
Revision history for this message
Heinrich Schuchardt (xypron) wrote :

Running with -cpu host,sv39=on or -cpu host,sv48=on works but writes a lot of warnings

qemu-system-riscv64: warning: disabling h extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zicbom extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zicboz extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zawrs extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zba extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbb extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbc extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbs extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling sstc extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling svadu extension for hart 0x0000000000000000 because privilege spec version does not match
qemu-system-riscv64: warning: disabling h extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zicbom extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zicboz extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zawrs extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zba extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbb extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbc extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling zbs extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling sstc extension for hart 0x0000000000000001 because privilege spec version does not match
qemu-system-riscv64: warning: disabling svadu extension for hart 0x0000000000000001 because privilege spec version does not match

The issue is reported to be fixed by upstream commit
eddabb6b888f ("target/riscv: skip features setup for KVM CPUs")

Changed in qemu (Ubuntu):
assignee: Heinrich Schuchardt (xypron) → nobody
Revision history for this message
Paride Legovini (paride) wrote :

According to the upstream git history, this is fixed in v8.1.0, which is the first non-RC version containing eddabb6b888f. We don't have qemu >= 8.1.0 in Ubuntu yet, but Debian has 8.1.2 already, so Ubuntu will get a fix for this with the fist merge of the NN cycle.

Changed in qemu (Ubuntu):
status: New → Triaged
Sergio Durigan Junior (sergiodj)
Changed in qemu (Ubuntu):
assignee: nobody → Sergio Durigan Junior (sergiodj)
Sergio Durigan Junior (sergiodj)
tags: added: server-todo
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

As discussed on the Sprint there's extra benefit having this in noble despite eventually going for 8.2.

Revision history for this message
Heinrich Schuchardt (xypron) wrote :

@seriodj

When picking up 8.1.2 you may want to consider the following:

There is a bug in 8.1.2 (upstream and Debian) concerning emulating the Zkr RISC-V extension.

The following patch has been accepted by the upstream maintainer:

[PATCH v2 1/1] target/riscv: correct csr_ops[CSR_MSECCFG]
https://lore.<email address hidden>/

The applicable patch version for 8.1.2 is

[PATCH 1/1] target/riscv: correct csr_ops[CSR_MSECCFG]
https://lore.<email address hidden>/

Best regards

Heinrich

Revision history for this message
Sergio Durigan Junior (sergiodj) wrote :

Thanks, xypron.

Just for completeness: as Christian mentioned, this will be fixed in Noble when we merge 8.1.x from Debian. Arguably this should also be considered for SRU in Mantic, but that's a decision I'll postpone for now (until after the sprint).

Revision history for this message
Sergio Durigan Junior (sergiodj) wrote :

@xypron,

I'm working on updating qemu to 8.1.3 on noble, which should address this issue. I tried reproducing it locally but failed during my first attempt, so I thought it'd be easier to ask you to test the package instead. It's here:

https://launchpad.net/~sergiodj/+archive/ubuntu/qemu

Could you give it a try and let me know if it works as expected? Thanks.

Revision history for this message
Heinrich Schuchardt (xypron) wrote :

Hello Sergio,

for testing I used the Ubuntu 24.04 image
https://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/current/noble-preinstalled-server-riscv64.img.xz

Booting a nested VM with KVM using qemu 1:8.1.3+ds-1ubuntu1~ppa1 worked fine.

Best regards

Heinrich

Revision history for this message
Sergio Durigan Junior (sergiodj) wrote : Re: [Bug 2039700] Re: qemu-system-riscv64: crash when starting nested VM

On Tuesday, November 28 2023, Heinrich Schuchardt wrote:

> Hello Sergio,
>
> for testing I used the Ubuntu 24.04 image
> https://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/current/noble-preinstalled-server-riscv64.img.xz
>
> Booting a nested VM with KVM using qemu 1:8.1.3+ds-1ubuntu1~ppa1 worked
> fine.

Thanks for confirming!

--
Sergio
GPG key ID: E92F D0B3 6B14 F1F4 D8E0 EB2F 106D A1C8 C3CB BF14

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package qemu - 1:8.1.3+ds-1ubuntu2

---------------
qemu (1:8.1.3+ds-1ubuntu2) noble; urgency=medium

  * d/p/u/define-ubuntu-machine-types.patch: Remove -hpb Noble machine
    types, as they are not needed by OpenStack anymore. (LP: #2045592)

 -- Sergio Durigan Junior <email address hidden> Mon, 04 Dec 2023 16:44:44 -0500

Changed in qemu (Ubuntu):
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.