Hello,
Here is an update on the situation.
I completed a kernel bisection between 4.7.8 and 4.8-rc1 (15 kernel
builds) and identified the kernel commit that causes the problem :
commit 021182e52fe01c1f7b126f97fd6ba048dc4234fd
Author: Thomas Garnier <thgar...@google.com>
Date: Tue Jun 21 17:47:03 2016 -0700
x86/mm: Enable KASLR for physical mapping memory regions
Add the physical mapping in the list of randomized memory regions.
The physical memory mapping holds most allocations from boot and heap
allocators. Knowing the base address and physical memory size, an attacker
can deduce the PDE virtual address for the vDSO memory page. This attack
was demonstrated at CanSecWest 2016, in the following presentation:
"Getting Physical: Extreme Abuse of Intel Based Paged Systems":
https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/blob/master/Presentation/CanSec2016_Presentation.pdf
(See second part of the presentation).
The exploits used against Linux worked successfully against 4.6+ but
fail with KASLR memory enabled:
https://github.com/n3k/CansecWest2016_Getting_Physical_Extreme_Abuse_of_Intel_Based_Paging_Systems/tree/master/Demos/Linux/exploits
Similar research was done at Google leading to this patch proposal.
Variants exists to overwrite /proc or /sys objects ACLs leading to
elevation of privileges. These variants were tested against 4.6+.
The page offset used by the compressed kernel retains the static value
since it is not yet randomized during this boot stage.
Signed-off-by: Thomas Garnier <thgar...@google.com>
Signed-off-by: Kees Cook <keesc...@chromium.org>
As you can see, this modification is specific to the x86 architecture
which is why you don't see the issue.
In parallel to that, a patch to fix this situation has been published
yesterday to the makedumpfile mailing list and is currently under review
by the main developper :
https://www.mail-archive.com/kexec@lists.infradead.org/msg16628.html
There is a debian bug opened for this issue :
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842019
While I understand your concern, I cannot push a modification that will
trigger makedumpfile to generate dozen of error messages and a
potentially unusable vmcore to all of the x86 world in order to satisfy
only one architecture.
It is important to remember that, while makedumpfile fails, the kernel
dump capture mechanism reverts to using 'cp' to copy /proc/vmcore
uncompressed so this vmcore file is still usable for analysis.
Hopefully, we should get an ACK for the upstream bug soon, so I can push
a new package to Debian. I will then proceed with the synchronization to
Zesty and SRU to the other stable releases.
I hope that this clarifies the situation.
Kind regards
** Bug watch added: Debian Bug tracker #842019
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842019
** Changed in: makedumpfile (Ubuntu)
Status: Confirmed => In Progress
** Changed in: makedumpfile (Ubuntu)
Importance: High => Medium
** Changed in: makedumpfile (Ubuntu Yakkety)
Importance: High => Medium
** Changed in: makedumpfile (Ubuntu Trusty)
Status: New => Confirmed
** Changed in: makedumpfile (Ubuntu Xenial)
Status: New => Confirmed
** Changed in: makedumpfile (Ubuntu Trusty)
Importance: Undecided => Medium
** Changed in: makedumpfile (Ubuntu Xenial)
Importance: Undecided => Medium
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to makedumpfile in Ubuntu.
https://bugs.launchpad.net/bugs/1626269
Title:
Ubuntu 16.10: kdump is not working in 4.8 kernel.
Status in makedumpfile package in Ubuntu:
In Progress
Status in makedumpfile source package in Trusty:
Confirmed
Status in makedumpfile source package in Xenial:
Confirmed
Status in makedumpfile source package in Yakkety:
Confirmed
Bug description:
== Comment: #0 - PAVITHRA R. PRAKASH - 2016-09-21 00:50:22 ==
---Problem Description---
Ubuntu 16.10: kdump is not working in 4.8 kernel.
---Steps to Reproduce---
1) apt-get install linux-crashdump
2) increase crashdump size:
sudo vim /etc/default/grub.d/kexec-tools.cfg
GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT crashkernel
=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M"
3) sudo update-grub ; reboot the machine
4) sudo sed -i 's/USE_KDUMP=0/USE_KDUMP=1/g' /etc/default/kdump-tools
5) kdump-config show
6) echo "c" > /proc/sysrq-trigger
Logs
====
root@ubuntu:/home/ubuntu# uname -a
Linux ubuntu 4.8.0-11-generic #12-Ubuntu SMP Sat Sep 17 19:58:16 UTC 2016
ppc64le ppc64le ppc64le GNU/Linux
root@ubuntu:/home/ubuntu# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr:
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinux-4.8.0-11-generic
kdump initrd:
/var/lib/kdump/initrd.img: symbolic link to
/var/lib/kdump/initrd.img-4.8.0-11-generic
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinux-4.8.0-11-generic
root=UUID=7ea3831b-f4c3-4f69-8f77-79aefcda70e3 ro splash quiet irqpoll
nr_cpus=1 nousb systemd.unit=kdump-tools.service"
--initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
root@ubuntu:/home/ubuntu# cd
root@ubuntu:~# echo "c" > /proc/sysrq-trigger
[ 50.733424] sysrq: SysRq : Trigger a crash
[ 50.733437] Unable to handle kernel paging request for data at address
0x00000000
[ 50.733441] Faulting instruction address: 0xc0000000005af3f4
[ 50.733444] Oops: Kernel access of bad area, sig: 11 [#1]
[ 50.733446] SMP NR_CPUS=2048 NUMA pSeries
[ 50.733450] Modules linked in: rpadlpar_io rpaphp dccp_diag dccp tcp_diag
udp_diag inet_diag unix_diag af_packet_diag netlink_diag pseries_rng sg
rng_core binfmt_misc ghash_generic gf128mul vmx_crypto ip_tables x_tables
autofs4 ext4 crc16 jbd2 fscrypto mbcache sr_mod cdrom sd_mod bnx2x ibmvscsi
ibmveth scsi_transport_srp ptp pps_core mdio libcrc32c crc32c_generic
crc32c_vpmsum
[ 50.733477] CPU: 2 PID: 1517 Comm: bash Not tainted 4.8.0-11-generic
#12-Ubuntu
[ 50.733480] task: c0000004f7906880 task.stack: c0000004f7898000
[ 50.733482] NIP: c0000000005af3f4 LR: c0000000005b04d8 CTR:
c0000000005af3c0
[ 50.733485] REGS: c0000004f789b990 TRAP: 0300 Not tainted
(4.8.0-11-generic)
[ 50.733488] MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 28242422
XER: 00000001
[ 50.733496] CFAR: c0000000000087d0 DAR: 0000000000000000 DSISR: 42000000
SOFTE: 1
GPR00: c0000000005b04d8 c0000004f789bc10 c000000000f46700 0000000000000063
GPR04: c0000005ef78a9b8 c0000005ef79f7d8 c00000063fe82300 0000000000005970
GPR08: 0000000000000007 0000000000000001 0000000000000000 0000000000000001
GPR12: c0000000005af3c0 c000000007b31200 ffffffffffffffff 0000000022000000
GPR16: 0000000010170dc8 00000100111c0538 0000000010140f58 00000000100c7570
GPR20: 0000000000000000 000000001017dd58 0000000010153618 000000001017b608
GPR24: 00003ffff10b9624 0000000000000001 c000000000e9fd40 0000000000000004
GPR28: c000000000ea0100 0000000000000063 c000000000e528e8 0000000000000000
[ 50.733536] NIP [c0000000005af3f4] sysrq_handle_crash+0x34/0x50
[ 50.733539] LR [c0000000005b04d8] __handle_sysrq+0xe8/0x280
[ 50.733541] Call Trace:
[ 50.733544] [c0000004f789bc10] [c0000000009ff9e8] 0xc0000000009ff9e8
(unreliable)
[ 50.733548] [c0000004f789bc30] [c0000000005b04d8] __handle_sysrq+0xe8/0x280
[ 50.733552] [c0000004f789bcd0] [c0000000005b0c88]
write_sysrq_trigger+0x78/0xa0
[ 50.733556] [c0000004f789bd00] [c0000000003a9a90] proc_reg_write+0xb0/0x110
[ 50.733560] [c0000004f789bd50] [c00000000030c27c] __vfs_write+0x6c/0xe0
[ 50.733564] [c0000004f789bd90] [c00000000030d784] vfs_write+0xd4/0x240
[ 50.733567] [c0000004f789bde0] [c00000000030f49c] SyS_write+0x6c/0x110
[ 50.733571] [c0000004f789be30] [c0000000000095e0] system_call+0x38/0x108
[ 50.733574] Instruction dump:
[ 50.733576] 38427340 7c0802a6 f8010010 f821ffe1 60000000 60000000 3d220019
3949cde0
[ 50.733582] 39200001 912a0000 7c0004ac 39400000 <992a0000> 38210020
e8010010 7c0803a6
[ 50.733589] ---[ end trace f58d72cacaada0df ]---
[ 50.735307]
[ 50.735313] Sending IPI to other CPUs
[ 50.736336] IPI complete
I'm in purgatory
-> smp_release_cpus()
spinning_secondaries = 8
<- smp_release_cpus()
[ 7.250999] sd 0:0:1:0: [sda] Assuming drive cache: write through
/dev/sda2: recovering journal
/dev/sda2: clean, 139510/1884160 files, 1253112/7531008 blocks
[ 8.263985] EXT4-fs (sda2): Cannot load crc32c driver.
mount: mounting /dev/sda2 on /root failed: No such file or directory
mount: mounting /dev on /root/dev failed: No such file or directory
mount: mounting /run on /root/run failed: No such file or directory
run-init: current directory on the same filesystem as the root: error 0
Target filesystem doesn't have requested /sbin/init.
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
run-init: current directory on the same filesystem as the root: error 0
No init found. Try passing init= bootarg.
BusyBox v1.22.1 (Ubuntu 1:1.22.0-19ubuntu2) built-in shell (ash)
Enter 'help' for a list of built-in commands.
(initramfs)
(initramfs)
(initramfs)
(initramfs) exit
mount: mounting /sys on /root/sys failed: No such file or directory
mount: mounting /proc on /root/proc failed: No such file or directory
/init: line 338: can't open /root/dev/console: no such file
[ 381.386973] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x00000200
[ 381.386973]
[ 381.386980] CPU: 0 PID: 1 Comm: init Not tainted 4.8.0-11-generic
#12-Ubuntu
[ 381.386983] Call Trace:
[ 381.386988] [c000000026513c20] [c00000000888bb9c] dump_stack+0xb0/0xf0
(unreliable)
[ 381.386993] [c000000026513c60] [c00000000888832c] panic+0x144/0x308
[ 381.386997] [c000000026513cf0] [c0000000080ce298] do_exit+0xd08/0xd10
[ 381.387001] [c000000026513dc0] [c0000000080ce384] do_group_exit+0x64/0x100
[ 381.387005] [c000000026513e00] [c0000000080ce44c] SyS_exit_group+0x2c/0x30
[ 381.387008] [c000000026513e30] [c0000000080095e0] system_call+0x38/0x108
[ 381.390557] ---[ end Kernel panic - not syncing: Attempted to kill init!
exitcode=0x00000200
[ 381.390557]
== Comment: #5 - Kevin W. Rudd - 2016-09-21 16:26:35 ==
This appears to be similar to the issue noted in the following discussion:
https://lists.debian.org/debian-kernel/2016/04/msg00013.html
(from the kdump dmesg info attached):
[ 9.115577] call_modprobe: crypto-crc32c 2
[ 9.116170] call_modprobe: crypto-crc32c-all 2
[ 9.116709] EXT4-fs (sda2): Cannot load crc32c driver.
Forcing the crc32c module by adding it to /etc/initramfs-tools/modules
got me past the initial EXT4-fs failure, but kdump then ran into a
regression with makedumpfile:
[ 12.331294] kdump-tools[1546]: Starting kdump-tools: * running
makedumpfile -c -d 31 /proc/vmcore /var/crash/201609211609/dump-incomplete
[ 12.391902] kdump-tools[1546]: get_mem_map: Can't distinguish the memory
type.
[ 12.392406] kdump-tools[1546]: The kernel version is not supported.
[ 12.392715] kdump-tools[1546]: The makedumpfile operation may be
incomplete.
[ 12.392997] kdump-tools[1546]: makedumpfile Failed.
[ 12.393309] kdump-tools[1546]: * kdump-tools: makedumpfile failed,
falling back to 'cp'
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/makedumpfile/+bug/1626269/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
