Hi,
I ran into the very same panic multiple times now, i.e. once trying to install
kibana on that node,
but not exactly sure if that's the reason, ther's also lots of stuff running in
the background.
U-Boot TPL 2020.01 (Mar 30 2020 - 19:56:30)
LPDDR3, 800MHz
BW=32 Col=11 Bk=8 CS0 Row=15 CS1 Row=15 CS=2 Die BW=16 Size=4096MB
Trying to boot from BOOTROM
Returning to boot ROM...
U-Boot SPL 2020.01 (Mar 30 2020 - 19:56:30 -0600)
Trying to boot from MMC2
NOTICE: BL31: v2.2(debug):2.2
NOTICE: BL31: Built : 19:47:49, Mar 30 2020
ERROR: over or zero region, nr=4188232, max=10
NOTICE: BL31:Rockchip release version: v1.2
INFO: ARM GICv2 driver initialized
INFO: plat_rockchip_pmu_init: pd status 0xe
INFO: BL31: Initializing runtime services
WARNING: BL31: cortex_a53: CPU workaround for 855873 was missing!
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0x200000
INFO: SPSR = 0x3c9
U-Boot 2020.01 (Mar 30 2020 - 19:56:30 -0600)
Model: Pine64 Rock64
DRAM: 4 GiB
PMIC: RK8050 (on=0x40, off=0x00)
MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0
Loading Environment from MMC... Card did not respond to voltage select!
*** Warning - No block device, using default environment
In: serial@ff130000
Out: serial@ff130000
Err: serial@ff130000
Model: Pine64 Rock64
Net: eth0: ethernet@ff540000
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1...
Found EFI removable media binary efi/boot/bootaa64.efi
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Card did not respond to voltage select!
Scanning disk rksd...@ff500000.blk...
Disk rksd...@ff500000.blk not ready
Scanning disk rksd...@ff520000.blk...
** Unrecognized filesystem type **
Found 3 disks
BootOrder not defined
EFI boot manager: Cannot load any image
169352 bytes read in 10 ms (16.2 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
disks: sd0*
>> OpenBSD/arm64 BOOTAA64 1.2
boot>
NOTE: random seed is being reused.
booting sd0a:/bsd: 8499816+1717104+554040+791336 [1076136+99+628796]=0xee8f30
type 0x2 pa 0x200000 va 0x200000 pages 0x4000 attr 0x8
type 0x7 pa 0x4200000 va 0x4200000 pages 0x3ef3 attr 0x8
type 0x4 pa 0x80f3000 va 0x80f3000 pages 0x1a attr 0x8
type 0x7 pa 0x810d000 va 0x810d000 pages 0xf30b3 attr 0x8
type 0x2 pa 0xfb1c0000 va 0xfb1c0000 pages 0xc27 attr 0x8
type 0x4 pa 0xfbde7000 va 0xfbde7000 pages 0x1 attr 0x8
type 0x2 pa 0xfbde8000 va 0xfbde8000 pages 0x3 attr 0x8
type 0x7 pa 0xfbdeb000 va 0xfbdeb000 pages 0x1 attr 0x8
type 0x2 pa 0xfbdec000 va 0xfbdec000 pages 0x100 attr 0x8
type 0x1 pa 0xfbeec000 va 0xfbeec000 pages 0x2a attr 0x8
type 0x0 pa 0xfbf16000 va 0xfbf16000 pages 0x7 attr 0x8
type 0x4 pa 0xfbf1d000 va 0xfbf1d000 pages 0x1 attr 0x8
type 0x6 pa 0xfbf1e000 va 0x5e70324000 pages 0x1 attr 0x8000000000000008
type 0x4 pa 0xfbf1f000 va 0xfbf1f000 pages 0x2 attr 0x8
type 0x0 pa 0xfbf21000 va 0xfbf21000 pages 0x4 attr 0x8
type 0x4 pa 0xfbf25000 va 0xfbf25000 pages 0x2 attr 0x8
type 0x6 pa 0xfbf27000 va 0x5e7032d000 pages 0x1 attr 0x8000000000000008
type 0x2 pa 0xfbf28000 va 0xfbf28000 pages 0x3008 attr 0x8
type 0x5 pa 0xfef30000 va 0x5e73336000 pages 0x10 attr 0x8000000000000008
type 0x2 pa 0xfef40000 va 0xfef40000 pages 0xc0 attr 0x8
[ using 1705936 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2020 OpenBSD. All rights reserved. https://www.OpenBSD.org
OpenBSD 6.7-current (GENERIC.MP) #696: Wed Jul 1 12:40:57 MDT 2020
dera...@arm64.openbsd.org:/usr/src/sys/arch/arm64/compile/GENERIC.MP
real mem = 4210712576 (4015MB)
avail mem = 4005163008 (3819MB)
random: boothowto does not indicate good seed
mainbus0 at root: Pine64 Rock64
psci0 at mainbus0: PSCI 1.1, SMCCC 1.1
cpu0 at mainbus0 mpidr 0: ARM Cortex-A53 r0p4
cpu0: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu0: 256KB 64b/line 16-way L2 cache
cpu1 at mainbus0 mpidr 1: ARM Cortex-A53 r0p4
cpu1: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu1: 256KB 64b/line 16-way L2 cache
cpu2 at mainbus0 mpidr 2: ARM Cortex-A53 r0p4
cpu2: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu2: 256KB 64b/line 16-way L2 cache
cpu3 at mainbus0 mpidr 3: ARM Cortex-A53 r0p4
cpu3: 32KB 64b/line 2-way L1 VIPT I-cache, 32KB 64b/line 4-way L1 D-cache
cpu3: 256KB 64b/line 16-way L2 cache
efi0 at mainbus0: UEFI 2.8
efi0: Das U-Boot rev 0x20200100
apm0 at mainbus0
syscon0 at mainbus0: "syscon"
"io-domains" at syscon0 not configured
syscon1 at mainbus0: "power-management"
rkclock0 at mainbus0
rkclock_set_frequency: 0x00000145
rkclock_set_frequency: 0x00000045
rkclock_set_frequency: 0x0000003e
rkclock_set_frequency: 0x000000e5
rkclock_set_frequency: 0x00000092
rkclock_set_frequency: 0x000000dc
rkclock_set_frequency: 0x00000061
ampintc0 at mainbus0 nirq 160, ncpu 4 ipi: 0, 1: "interrupt-controller"
rkpinctrl0 at mainbus0: "pinctrl"
rkgpio0 at rkpinctrl0
rkgpio1 at rkpinctrl0
rkgpio2 at rkpinctrl0
rkgpio3 at rkpinctrl0
"fit-images" at mainbus0 not configured
"opp_table0" at mainbus0 not configured
"arm-pmu" at mainbus0 not configured
agtimer0 at mainbus0: tick rate 24000 KHz
"xin24m" at mainbus0 not configured
com0 at mainbus0: ns16550, no working fifo
com0: console
rkiic0 at mainbus0
iic0 at rkiic0
rkpmic0 at iic0 addr 0x18: RK805
"spi" at mainbus0 not configured
simplebus0 at mainbus0: "amba"
"dmac" at simplebus0 not configured
dwmmc0 at mainbus0: 50 MHz base clock
sdmmc0 at dwmmc0: 4-bit, sd high-speed, mmc high-speed, dma
dwmmc1 at mainbus0: 50 MHz base clock
sdmmc1 at dwmmc1: 8-bit, mmc high-speed, dma
dwge0 at mainbus0: address 82:54:08:75:8c:6d
rgephy0 at dwge0 phy 0: RTL8169S/8110S/8211 PHY, rev. 6
ehci0 at mainbus0
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Generic EHCI root hub" rev 2.00/1.00
addr 1
ohci0 at mainbus0: version 1.0
"usb" at mainbus0 not configured
"external-gmac-clock" at mainbus0 not configured
"sdmmc-regulator" at mainbus0 not configured
"vcc-host-5v-regulator" at mainbus0 not configured
"vcc-sys" at mainbus0 not configured
"dmc" at mainbus0 not configured
"usb" at mainbus0 not configured
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 configuration 1 interface 0 "Generic OHCI root hub" rev 1.00/1.00
addr 1
scsibus0 at sdmmc1: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <SD/MMC, SLD32G, 0000> removable
sd0: 29600MB, 512 bytes/sector, 60620800 sectors
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
bootfile: sd0a:/bsd
boot device: sd0
root on sd0a (aeb15307a25f954d.a) swap on sd0b dump on sd0b
cpu0: clock not implemented
Automatic boot in progress: starting file system checks.
/dev/sd0a (aeb15307a25f954d.a): file system is clean; not checking
/dev/sd0h (aeb15307a25f954d.h): file system is clean; not checking
/dev/sd0g (aeb15307a25f954d.g): file system is clean; not checking
/dev/sd0d (aeb15307a25f954d.d): file system is clean; not checking
/dev/sd0f (aeb15307a25f954d.f): file system is clean; not checking
/dev/sd0e (aeb15307a25f954d.e): file system is clean; not checking
/dev/sd0j (aeb15307a25f954d.j): file system is clean; not checking
pf enabled
kern.maxfiles: 7030 -> 65536
kern.bufcachepercent: 20 -> 80
net.inet.ip.forwarding: 0 -> 0
net.inet6.ip6.forwarding: 0 -> 0
starting network
dwge0: no link....
dwge0: 10.0.0.61 lease accepted from 10.0.0.60 (02:ba:3a:e3:12:3c)
reordering libraries: done.
starting early daemons: syslogd pflogd ntpd.
starting RPC daemons:.
savecore: no core dump
checking quotas: done.
clearing /tmp
kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
starting network daemons: sshd snmpd smtpd sndiod.
starting package daemons: puppet filebeat icinga2 nginx mcollectived.
starting local daemons: apmd cron.
Thu Jul 2 23:17:17 CEST 2020
mount
/dev/sd0a on / type ffs (local)
/dev/sd0h on /home type ffs (local, nodev, nosuid, wxallowed)
/dev/sd0g on /tmp type ffs (local, nodev, nosuid)
/dev/sd0d on /usr type ffs (local, nodev)
/dev/sd0f on /usr/X11R6 type ffs (local, nodev)
/dev/sd0e on /usr/local type ffs (local, nodev, wxallowed)
/dev/sd0j on /var type ffs (local, nodev, nosuid)
PKG_PATH=/home pkg_add kibana
uid 0 on /: out of inodes
uid 0 on /: out of inodes
uid 0 on /: file system full
uid 0 on /: file system full
uid 0 on /: out of inodes
uid 0 on /: out of inodes
uid 0 on /: out of inodes
uid 748 on /var: out of inodes
uid 748 on /var: out of inodes
...
uid 784 belongs to _elasticsearch user.
panic: init died (signal 10, exit 0)
Stopped at panic+0x150: TID PID UID PRFLAGS PFLAGS C
PU COMMAND
*128013 1 0 0x802 0x2000 3K init
db_enter() at panic+0x14c
panic() at trapsignal+0x164
trapsignal() at data_abort+0x90
data_abort() at do_el0_sync+0x140
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0x1eab6724c0
ddb{3}> ps
PID TID PPID UID S FLAGS WAIT COMMAND
31538 69205 8860 1000 3 0x100083 ttyin ksh
8860 407736 33843 1000 3 0x90 select sshd
33843 498695 50495 0 3 0x92 poll sshd
83692 340239 1 0 3 0x100098 poll cron
19397 351065 1 0 3 0x80 kqread apmd
87980 165016 12968 67 3 0x90 kqread nginx
73630 77613 12968 67 3 0x90 kqread nginx
6309 129765 12968 67 3 0x90 kqread nginx
9174 347765 12968 67 3 0x90 kqread nginx
12968 249655 1 0 3 0x88 pause nginx
52280 53594 1 99 3 0x100090 poll sndiod
3641 117391 1 110 3 0x100090 poll sndiod
50495 488355 1 0 3 0x80 select sshd
34703 130549 44951 74 3 0x100092 bpf pflogd
44951 168978 1 0 3 0x80 netio pflogd
63492 292899 85800 115 3 0x100092 kqread slaacd
18705 194033 85800 115 3 0x100092 kqread slaacd
85800 159406 1 0 3 0x100080 kqread slaacd
58927 332919 0 0 3 0x14200 bored smr
70183 382734 0 0 3 0x14200 pgzero zerothread
85384 369869 0 0 3 0x14200 aiodoned aiodoned
81872 352159 0 0 3 0x14200 syncer update
9887 25509 0 0 3 0x14200 cleaner cleaner
15634 310727 0 0 3 0x14200 reaper reaper
30651 301266 0 0 3 0x14200 pgdaemon pagedaemon
94627 418391 0 0 3 0x14200 bored crynlk
39098 396675 0 0 3 0x14200 bored crypto
86928 21808 0 0 3 0x14200 usbtsk usbtask
64901 285020 0 0 3 0x14200 usbatsk usbatsk
45962 207407 0 0 3 0x14200 mmctsk sdmmc1
283 214036 0 0 3 0x14200 mmctsk sdmmc0
69427 520025 0 0 3 0x40014200 idle3
26744 33679 0 0 7 0x40014200 idle2
90416 507212 0 0 7 0x40014200 idle1
97471 120227 0 0 3 0x14200 bored softnet
86849 20498 0 0 3 0x14200 bored systqmp
73242 24650 0 0 3 0x14200 bored systq
42355 465745 0 0 3 0x40014200 bored softclock
26152 245965 0 0 7 0x40014200 idle0
21900 216727 0 0 3 0x14200 kmalloc kmthread
* 1 128013 0 0 7 0x2802 init
0 0 -1 0 3 0x10200 scheduler swapper
machine ddbcpu 1
Stopped at ampintc_ipi_ddb+0x1c: db_enter() at ampintc_ipi_ddb+0x18
ampintc_ipi_ddb() at arm_cpu_intr+0x30
arm_cpu_intr() at handle_el1h_irq+0x6c
handle_el1h_irq() at sched_idle+0x220
sched_idle() at proc_trampoline+0x10
ddb{1}> machine ddbcpu 2
Stopped at ampintc_ipi_ddb+0x1c: db_enter() at ampintc_ipi_ddb+0x18
ampintc_ipi_ddb() at arm_cpu_intr+0x30
arm_cpu_intr() at handle_el1h_irq+0x6c
handle_el1h_irq() at sched_idle+0x220
sched_idle() at proc_trampoline+0x10
ddb{2}> machine ddbcpu 3
Stopped at panic+0x150: db_enter() at panic+0x14c
panic() at trapsignal+0x164
trapsignal() at data_abort+0x90
data_abort() at do_el0_sync+0x140
do_el0_sync() at handle_el0_sync+0x74
handle_el0_sync() at 0x1eab6724c0
--- trap ---
ddb{3}> show uvm
Current UVM status:
pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
988716 VM pages: 1584 active, 9729 inactive, 0 wired, 671897 free (84001 zero
)
min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
freemin=32957, free-target=43942, inactive-target=200180, wired-max=329572
faults=5120832, traps=0, intrs=0, ctxswitch=13843278 fpuswitch=0
softint=1055809, syscalls=12491465, kmapent=19
fault counts:
noram=0, noanon=0, noamap=0, pgwait=35, pgrele=0
ok relocks(total)=70330(70790), anget(retries)=1073522(4343), amapcopy=1008
770
neighbor anon/obj pg=163509/726421, gets(lock/unlock)=623190/66333
cases: anon=963489, anoncow=109873, obj=583514, prcopy=39288, przero=342440
7
daemon and swap counts:
woke=242, revs=193, scans=1943598, obscans=42634, anscans=1044671
busy=0, freed=99260, reactivate=856293, deactivate=1010536
pageouts=61791, pending=3665, nswget=4468
nswapdev=1
swpages=524287, swpginuse=1460, swpgonly=1418 paging=0
kernel pointers:
objs(kern)=0xffffff8000ebeb08
ddb{3}> show bcstats
Current Buffer Cache status:
numbufs 60357 busymapped 0, delwri 0
kvaslots 12358 avail kva slots 12358
bufpages 241428, dmapages 241428, dirtypages 0
pendingreads 0, pendingwrites 0
highflips 0, highflops 0, dmaflips 0
ddb{3}> machine ddbcpu 0
^^^^^^
finally hangs ddb
however, after reboot:
df -kh
Filesystem Size Used Avail Capacity Mounted on
/dev/sd0a 242M 94.7M 135M 41% /
/dev/sd0h 1.9G 1.6G 204M 89% /home
/dev/sd0g 96.7M 10.0K 91.8M 0% /tmp
/dev/sd0d 2.9G 745M 2.0G 26% /usr
/dev/sd0f 986M 195M 741M 21% /usr/X11R6
/dev/sd0e 2.9G 2.0G 791M 72% /usr/local
/dev/sd0j 17.0G 35.2M 16.1G 0% /var
root@daedalus:~# df -i
Filesystem 512-blocks Used Avail Capacity iused ifree %iused
Mounted on
/dev/sd0a 495580 193856 276948 41% 3137 28861 10% /
/dev/sd0h 4050876 3431596 416740 89% 274198 10920 96% /home
/dev/sd0g 197980 20 188064 0% 6 12792 0% /tmp
/dev/sd0d 6082908 1525800 4252964 26% 14918 399800 4% /usr
/dev/sd0f 2018844 399760 1518144 21% 9070 146448 6%
/usr/X11R6
/dev/sd0e 6082908 4158968 1619796 72% 163020 251698 39%
/usr/local
/dev/sd0j 35610492 72160 33757808 0% 2030 2304848 0% /var
doesn't really look like / or /var is full?
if I see it again, anything else to check?
Sebastian
