We have noticed an issue when a uprobe is attached to a valid file offset, and the content of that file is modified (with no inode change). Executing from a modified file while the uprobe is attached may cause a process to crash. The crash happens if the probed offset is invalid for the new file content.
Here's a simple scenario for an executable leading to consistent crashes (but something similar can happen for DSO too): 1. Create a simple binary B1 that has symbol S1 at offset O1. The binary B1 has inode INO_1. 2. Start a new process P1 and attach an uprobe to this binary at B1:S1:O1 3. Run the binary B1 and the uprobe is fired as expected. 4. Continue to run the process P1 (a.k.a the uprobe is still attached to B1:S1:O1) 5. Create a new binary B2 that no longer has the symbol S1 at offset O1. The binary B2 has inode INO_2 6. Copy the content of binary INO_2 to INO_1 using cp command. As a result the original binary B1 with INO_1 has new contents and no longer has symbol S1 at offset O1. The key here is it’s the same inode. 7. The original process P1 still has an uprobe attached to B1:S1:O1. 8. Re-run the binary B1 and it seg faults immediately. In short any B1 become completely unusable. 9. Detach the uprobe. 10. Re-run the binary B1 and all good Sample gdb output where it crashes (Notice the (bad)) 0x0000555555555154 in _fini () (gdb) x/20i _fini 0x555555555148 <_fini>: repz int3 0x55555555514a <_fini+2>: (bad) 0x55555555514b <_fini+3>: cli 0x55555555514c <_fini+4>: sub $0x8,%rsp 0x555555555150 <_fini+8>: add $0x8,%rsp => 0x555555555154 <_fini+12>: ret Syslogs output: Aug 8 21:33:00 rshahvm2 kernel: [50553.686400] hello_world[30144]: segfault at f7d3e9c8 ip 00007f9701b40144 sp 00000000f7d3e9c8 error 4 in libfoo.so[7f9701b40000+1000] Aug 8 21:33:00 rshahvm2 kernel: [50553.686409] Code: 0f 1e fa 55 48 89 e5 90 90 90 5d c3 f3 0f 1e fa 55 48 89 e5 e8 1d ff ff ff 5d c3 00 00 00 f3 cc 1e fa 48 83 ec 08 48 83 c4 08 <c3> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 What we saw is that when a file (executable or a DSO) is mapped in a process memory, the kernel installs all uprobes registered for that file's inode, no matter if the file has been changed since the initial uprobe registration. If uprobes are installed at a offset that's mo more valid, this behavior can make all new application completely unusable. Just wondering if there is any fix in the kernel for this of if anyone else has observed similar behavior. Thanks, Rahul
