Hi,
so we are testing klp with ORC. And we see
WARNING: CPU: 0 PID: 4574 at ../arch/x86/kernel/stacktrace.c:132
save_stack_trace_tsk_reliable+0x130/0x1b0
Modules linked in: ...
...
Call Trace:
klp_try_switch_task+0x10a/0x2c0
klp_try_complete_transition+0x121/0x1b0
__klp_enable_patch+0xb6/0xe0
klp_enable_patch+0x68/0x70
livepatch_init+0x28/0x50 [klp_tc_3_live_patch_getpid]
Staring into the code, it is caused by save_stack_trace_tsk of the
CPU0's idle task (task = idle_task(0);).
Its stack trace should look like:
ffffffff811083c2 do_idle+0x142/0x1e0
ffffffff8110861d cpu_startup_entry+0x5d/0x60
ffffffff82715f58 start_kernel+0x3ff/0x407
ffffffff827153e8 x86_64_start_kernel+0x14e/0x15d
ffffffff810001bf secondary_startup_64+0x9f/0xa0
ffffffffffffffff 0xffffffffffffffff
1) To have nice secondary_startup_64 there instead of misleading
verify_cpu+0, I had to do:
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -303,6 +305,7 @@ ENTRY(secondary_startup_64)
pushq %rax # target address in negative space
lretq
.Lafter_lret:
+ nop
ENDPROC(secondary_startup_64)
#include "verify_cpu.S"
2) secondary_startup_64 lives at ffffffff810001bf. It is very near
_stext (+0x1bf) and there are no orc entries for that and neither before
that. But orc_find still returns the very first orc entry to me even
though my address is lower:
ffffffff810002d0: sp:sp+8 bp:(und) type:call
The entry is for run_init_process from init/main.c.
Obviously secondary_startup_64 is from head_64.S which has no orc
entries due to OBJECT_FILES_NON_STANDARD_head_$(BITS).o := y.
So doing this had no chance:
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -169,6 +169,7 @@ startup_64:
movq $(early_level4_pgt - __START_KERNEL_map), %rax
jmp 1f
ENTRY(secondary_startup_64)
+ UNWIND_HINT_EMPTY
/*
* At this point the CPU runs in 64bit mode CS.L = 1 CS.D = 0,
* and someone has loaded a mapped page table.
Anyway, I went the way to fix __orc_find anyway. It should return NULL
if the entry is not found (i.e. the found entry has higher IP), so I did:
--- a/arch/x86/kernel/unwind_orc.c
+++ b/arch/x86/kernel/unwind_orc.c
@@ -52,6 +53,8 @@ static struct orc_entry *__orc_find(int
} else
last = mid - 1;
}
+ if (orc_ip(found) > ip)
+ return NULL;
return u_table + (found - ip_table);
}
But now, unwind_init does not succeed, because it cannot find an entry
for LOOKUP_START_IP in the orc_lookup[i] setup loop. Obviously,
head_64.o is at LOOKUP_START_IP with no orc entry in there.
So I did this hack instead of the above:
@@ -52,6 +53,8 @@ static struct orc_entry *__orc_find(int
} else
last = mid - 1;
}
+ if (orc_init && orc_ip(found) > ip)
+ return NULL;
return u_table + (found - ip_table);
}
which produces the nice stack trace on the top.
So now, there are two things:
1) how to fix orc_find & unwind_init properly?
2) how to generate an EMPTY hint for secondary_startup_64 and maybe
later REGS hint after the rsp setup there? The function seems to be a
pretty killer for orc generate (among others in head_64.S):
../orc/arch/x86/kernel/head_64.o: warning: objtool:
secondary_startup_64()+0x25: sibling call from callable instruction with
modified stack frame
../orc/arch/x86/kernel/head_64.o: warning: objtool:
early_idt_handler_array()+0x4: sibling call from callable instruction
with modified stack frame
../orc/arch/x86/kernel/head_64.o: warning: objtool:
early_idt_handler_common()+0x4d: sibling call from callable instruction
with modified stack frame
../orc/arch/x86/kernel/head_64.o: warning: objtool:
secondary_startup_64()+0x9d: unsupported instruction in callable function
thanks,
--
js
suse labs
