Ubuntu
qemu package

hart0: trap handler failed (error -2) (Needs cherry-pick ab3d207f)

Bug #1914883 reported by Nathan Chancellor on 2021-02-06

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	qemu (Ubuntu)	Fix Released	Undecided	Unassigned
	Focal	Fix Released	High	Marc Deslauriers

Bug Description

Commit 5d971f9e67 ("memory: Revert "memory: accept mismatching sizes in memory_region_access_valid"") was cherry-picked to deal with CVE-2020-13754 but the follow up fix in commit ab3d207fe8 ("riscv: sifive_test: Allow 16-bit writes to memory region") was not, resulting in the bug described in the commit message in 1:4.2-3ubuntu6.11: https://github.com/nathanchance/continuous-integration2/runs/1834110909

Please consider adding ab3d207fe8 to the next release so that there is no more regression.

[Changelog]

qemu (1:4.2-3ubuntu6.11) focal; urgency=medium

* d/p/ubuntu/lp-1907656-s390x-s390-virtio-ccw-Reset-PCI-devices-during-subsy:
avoid PCI devices to become unavailable on reset (LP: #1907656)

...

qemu (1:4.2-3ubuntu6.4) focal-security; urgency=medium
  ...
  * SECURITY UPDATE: out-of-bounds access via msi-x mmio operation
    - debian/patches/ubuntu/CVE-2020-13754-1.patch: revert accepting
      mismatching sizes in memory_region_access_valid in memory.c.
    - debian/patches/ubuntu/CVE-2020-13754-2.patch: accept byte and word
      access to core ACPI registers in hw/acpi/core.c.
    - CVE-2020-13754

[CI Log]

Requesting system poweroff
[ 4.312781] reboot: Power down
sbi_trap_error: hart0: trap handler failed (error -2)
sbi_trap_error: hart0: mcause=0x0000000000000007 mtval=0x0000000000100000
sbi_trap_error: hart0: mepc=0x000000008000d4b0 mstatus=0x0000000000001822
sbi_trap_error: hart0: ra=0x00000000800098de sp=0x0000000080015c78
...

Traceback (most recent call last):
  File "./check_logs.py", line 106, in <module>
    boot_test(build)
  File "./check_logs.py", line 94, in boot_test
    run_boot()
  File "./check_logs.py", line 82, in run_boot
    raise e
  File "./check_logs.py", line 78, in run_boot
    subprocess.run(boot_qemu, check=True)
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['./boot-utils/boot-qemu.sh', '-a', 'riscv', '-k', '/home/runner/work/continuous-integration2/continuous-integration2/Image']' returned non-zero exit status 124.
Error: Process completed with exit code 1.

See original description

Tags:

CVE References

2020-13754

Revision history for this message

Bryce Harrington (bryce) wrote on 2021-02-08:

Patch mentioned in bug report Edit (3.1 KiB, text/plain)

https://github.com/qemu/qemu/commit/5c49f7ee3b98316850de6a33952a4ac47701c118

description:	updated
Changed in qemu (Ubuntu Focal):
status:	New → Triaged
tags:	added: server-next
Changed in qemu (Ubuntu Focal):
importance:	Undecided → High

Bryce Harrington (bryce) on 2021-02-08

summary:

- Please cherry-pick ab3d207fe89bc0c63739db19e177af49179aa457 into Focal
- package
+ hart0: trap handler failed (error -2) (Needs cherry-pick ab3d207f)

Revision history for this message

Bryce Harrington (bryce) wrote on 2021-02-08:

Thanks for flagging this issue. I have tagged it for an engineer to follow up.

Ubuntu Foundations Team Bug Bot (crichton) on 2021-02-09

tags:

added: patch

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2021-02-09:

Hi,
thanks for the report, this seems to keep on giving.

Initially we had:

128 qemu (1:4.2-3ubuntu6.4) focal-security; urgency=medium

177 * SECURITY UPDATE: out-of-bounds access via msi-x mmio operation
178 - debian/patches/ubuntu/CVE-2020-13754-1.patch: revert accepting
179 mismatching sizes in memory_region_access_valid in memory.c.
180 - debian/patches/ubuntu/CVE-2020-13754-2.patch: accept byte and word
181 access to core ACPI registers in hw/acpi/core.c.
182 - CVE-2020-13754

But something close to the issue you mentioned was spotted quickly and resolved in

113 qemu (1:4.2-3ubuntu6.5) focal; urgency=medium

118 - as part of the stabilization this also fixes an
119 riscv emulation issue due to the CVE-2020-13754 fixes via
120 d/p/ubuntu/hw-riscv-Allow-64-bit-access-to-SiFive-CLINT.patch

Yet your hint made me wonder "what else" this might have been found to be broken and that is quite a list.

So we have already (in Focal):
https://github.com/qemu/qemu/commit/70b78d4e71494c90d2ccb40381336bc9b9a22f79

But in the meantime there also is this list::
https://github.com/qemu/qemu/commit/5c49f7ee3b98316850de6a33952a4ac47701c118 (== ab3d207fe8 but one is from a stable branch)
https://github.com/qemu/qemu/commit/62a9b228b5fefe0f9e364dfeaf3c65022c63cdb9
https://github.com/qemu/qemu/commit/3059344f01e1bf9625570ef2e8396fa011e9431d
https://github.com/qemu/qemu/commit/e0cf02ce680f11893aca9642e76d6ae68b9375af
https://github.com/qemu/qemu/commit/dba04c3488c4699f5afe96f66e448b1d447cf3fb
https://github.com/qemu/qemu/commit/8e67fda2dd6202ccec093fda561107ba14830a17

Related but not strictly needed:
https://github.com/qemu/qemu/commit/21786c7e59847b1612406ff394958f22e5b323f8

Qemu 5.2 has all the known fixes that exist so far, thereby this is fixed in Hirsute.
The CVE-2020-13754 was released to X/B/F (and G, but before G released).
So IMHO X,B,G seem to need the fixups mentioned above, F needs the same minus the one I already added.

This will eventually need to be pushed to -security, also there is a chance that Mark (doing the security update) and/pr the security community had context/discussions about this.
For now I'll assign this to Mark for his input on this.

While we wait for Marks awnser @nathan - could you outline steps to reproduce an issue related to this. For the SRU one would want commands that fail without the fix and work once applied.
I assume you'd have some RiscV Emulation steps we could use for that?

Hi,
thanks for the report, this seems to keep on giving.

Initially we had:

128 qemu (1:4.2-3ubuntu6.4) focal-security; urgency=medium

177   * SECURITY UPDATE: out-of-bounds access via msi-x mmio operation               
 178     - debian/patches/ubuntu/CVE-2020-13754-1.patch: revert accepting             
 179       mismatching sizes in memory_region_access_valid in memory.c.               
 180     - debian/patches/ubuntu/CVE-2020-13754-2.patch: accept byte and word         
 181       access to core ACPI registers in hw/acpi/core.c.                           
 182     - CVE-2020-13754

But something close to the issue you mentioned was spotted quickly and resolved in

113 qemu (1:4.2-3ubuntu6.5) focal; urgency=medium

118     - as part of the stabilization this also fixes an                            
 119       riscv emulation issue due to the CVE-2020-13754 fixes via                  
 120       d/p/ubuntu/hw-riscv-Allow-64-bit-access-to-SiFive-CLINT.patch

Yet your hint made me wonder "what else" this might have been found to be broken and that is quite a list.

So we have already (in Focal):
https://github.com/qemu/qemu/commit/70b78d4e71494c90d2ccb40381336bc9b9a22f79

Related but not strictly needed:
https://github.com/qemu/qemu/commit/21786c7e59847b1612406ff394958f22e5b323f8

While we wait for Marks awnser  @nathan - could you outline steps to reproduce an issue related to this. For the SRU one would want commands that fail without the fix and work once applied.
I assume you'd have some RiscV Emulation steps we could use for that?

Changed in qemu (Ubuntu Focal):
assignee:	nobody → Marc Deslauriers (mdeslaur)

Revision history for this message

Nathan Chancellor (nathanchance) wrote on 2021-02-09:

Image Edit (18.9 MiB, application/octet-stream)

Sure. Attached is a kernel image and I will post the rootfs in the next comment.

$ timeout --foreground 15s unbuffer \
qemu-system-riscv64 \
-bios /usr/lib/riscv64-linux-gnu/opensbi/qemu/virt/fw_jump.elf \
-display none \
-initrd rootfs.cpio \
-kernel Image \
-m 512m \
-M virt \
-nodefaults \
-serial mon:stdio
...
sbi_trap_error: hart0: trap handler failed (error -2)
sbi_trap_error: hart0: mcause=0x0000000000000007 mtval=0x0000000000100000
sbi_trap_error: hart0: mepc=0x000000008000d4b0 mstatus=0x0000000000001822
sbi_trap_error: hart0: ra=0x00000000800098de sp=0x0000000080015c78
sbi_trap_error: hart0: gp=0xffffffe0012e95e8 tp=0xffffffe0081c1e00
sbi_trap_error: hart0: s0=0x0000000080015c88 s1=0x0000000000000040
sbi_trap_error: hart0: a0=0x0000000000000000 a1=0x0000000080003f66
sbi_trap_error: hart0: a2=0x0000000080003f66 a3=0x0000000080003f66
sbi_trap_error: hart0: a4=0x0000000000100000 a5=0x0000000000005555
sbi_trap_error: hart0: a6=0x0000000000003f66 a7=0x0000000080011158
sbi_trap_error: hart0: s2=0x0000000000000000 s3=0x0000000080016000
sbi_trap_error: hart0: s4=0x0000000000000000 s5=0x0000000000000000
sbi_trap_error: hart0: s6=0x0000000000000001 s7=0x0000000000000000
sbi_trap_error: hart0: s8=0x0000000000000000 s9=0x0000000000000000
sbi_trap_error: hart0: s10=0x0000000000000000 s11=0x0000000000000008
sbi_trap_error: hart0: t0=0x0000000000000000 t1=0x0000000000000000
sbi_trap_error: hart0: t2=0x0000000000000000 t3=0x0000000000000000
sbi_trap_error: hart0: t4=0x0000000000000000 t5=0x0000000000000000
sbi_trap_error: hart0: t6=0x0000000000000000

$ echo ${?}
124

If it works, there should sbi_trap_error output and QEMU should exit cleanly.

Revision history for this message

Nathan Chancellor (nathanchance) wrote on 2021-02-09:

rootfs.cpio Edit (3.3 MiB, application/x-cpio)

Revision history for this message

Christian Ehrhardt  (paelzer) wrote on 2021-02-11:

Thanks Nathan, Mark is working on the set of backports (for the whole collection of fixes that I've found) and we will then test your workload against those preliminary package updates.

Revision history for this message

Launchpad Janitor (janitor) wrote on 2021-02-22:

This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu7.36

---------------
qemu (1:2.11+dfsg-1ubuntu7.36) bionic-security; urgency=medium

  * SECURITY REGRESSION: fix multiple regressions caused by CVE-2020-13754
    security update (LP: #1914883)
    - debian/patches/CVE-2020-13754-3.patch: log invalid memory accesses in
      memory.c.
    - debian/patches/CVE-2020-13754-5.patch: allow 64-bit accesses in
      hw/timer/slavio_timer.c.
    - debian/patches/CVE-2020-13754-6.patch: allow less than 32-bit
      accesses in hw/char/bcm2835_aux.c.
    - debian/patches/CVE-2020-13754-9.patch: fix valid.max_access_size to
      access address registers in hw/usb/hcd-xhci.c.

-- Marc Deslauriers <email address hidden> Wed, 10 Feb 2021 08:37:38 -0500

Changed in qemu (Ubuntu):
status:	New → Fix Released

Revision history for this message

Launchpad Janitor (janitor) wrote on 2021-02-22:

This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.51

---------------
qemu (1:2.5+dfsg-5ubuntu10.51) xenial-security; urgency=medium

  * SECURITY REGRESSION: fix multiple regressions caused by CVE-2020-13754
    security update (LP: #1914883)
    - debian/patches/CVE-2020-13754-5.patch: allow 64-bit accesses in
      hw/timer/slavio_timer.c.
    - debian/patches/CVE-2020-13754-9.patch: fix valid.max_access_size to
      access address registers in hw/usb/hcd-xhci.c.

-- Marc Deslauriers <email address hidden> Wed, 10 Feb 2021 08:40:41 -0500

Changed in qemu (Ubuntu):
status:	New → Fix Released

Revision history for this message

Launchpad Janitor (janitor) wrote on 2021-02-22:

This bug was fixed in the package qemu - 1:4.2-3ubuntu6.14

---------------
qemu (1:4.2-3ubuntu6.14) focal-security; urgency=medium

  * SECURITY REGRESSION: fix multiple regressions caused by CVE-2020-13754
    security update (LP: #1914883)
    - debian/patches/ubuntu/CVE-2020-13754-3.patch: log invalid memory
      accesses in memory.c.
    - debian/patches/ubuntu/CVE-2020-13754-4.patch: allow 16-bit writes to
      memory region in hw/riscv/sifive_test.c.
    - debian/patches/ubuntu/CVE-2020-13754-5.patch: allow 64-bit accesses
      in hw/timer/slavio_timer.c.
    - debian/patches/ubuntu/CVE-2020-13754-6.patch: allow less than 32-bit
      accesses in hw/char/bcm2835_aux.c.
    - debian/patches/ubuntu/CVE-2020-13754-9.patch: fix
      valid.max_access_size to access address registers in
      hw/usb/hcd-xhci.c.