Hi Stefano,
On 9 February 2017 at 05:40, Stefano Stabellini <sstabell...@kernel.org> wrote:
> On Wed, 8 Feb 2017, Bhupinder Thakur wrote:
>> Hi Julien,
>>
>> On 3 February 2017 at 19:38, Julien Grall <julien.gr...@arm.com> wrote:
>> > So if I understand correctly, you don't receive anymore output. Correct?
>> > Have you tried to see whether the pl011 driver is receiving interrupt or
>> > even Linux calling it.
>>
>> The pl011 driver is not receiving the interrupts (I put a
>> HYPERVISOR_console_io() call inside the interrupt handler and I do not
>> see the print on the xen console) . When the console is in this state,
>> if I keep typing the characters on the console then Xen keeps raising
>> the interrupts (for each character typed) but all those interrupts are
>> marked as inflight. So it seems that interrupt is not getting cleared
>> in GIC. Which GIC function can I look at to further debug what is the
>> issue?
>>
>> Earlier, when I used to manually write characters to /dev/ttyAMA0 then
>> I used to receive the interrupts all fine.
>
> DomU doesn't get emulated PL011 interrupts from Xen? Make sure you don't
> mark the interrupt as a real hardware interrupt by mistake
> (GICH_V2_LR_HW). At the same time, check that the virtual PL011
> interrupt has been added correctly to the LRs
> (xen/arch/arm/gic-v2.c:gicv2_update_lr).
I verified that the PL011 interrupt is getting marked pending by Xen
but guest does not receive it. I am not sure if the local interrupt
delivery is disabled on the guest because of which it may not be
receiving the interrupts. Is there a way to verify that interrupts are
indeed disabled at that point. Since the interrupt delivery works fine
when I manually read/write characters to /dev/ttyAMA0, I think there
is no issue from the Xen side in terms of raising an interrupt.
One observation is that pl011_request_port () calls request_mem_region
() for the MMIO space allocated to pl011. The function returns -EBUSY
indicating that the memory region is already in use. But when I do cat
/proc/iomem, I do not see any overlaps. I am using 0x22000000 as the
start of the pl011 MMIO space. I see the same error while booting the
dom0 kernel also.
I have attached the guest boot logs when console=ttyAMA0. I have added
debug prints marked with [DEBUG]. I added debug prints in amba-pl011.c
and serial_core.c. The output from the console seems to suddenly stop
once the pl011 port is started. I have not added getty for ttyAMA0 in
the /etc/inittab file. I believe auto-serial-console should be able to
attach a getty session to ttyAMA0 automatically (i did try with adding
a getty command in the inittab file but that did not solve the issue).
Towards the end of the log, we can see that every time a character is
typed, Xen raises an interrupt and the characters keep getting
collected in the IN RING buffer (in_prod keeps incrementing). Later
interrupts remain in inflight as the earlier interrupt is not cleared.
Regards,
Bhupinder
(none):/root # cat domU.cfg
kernel = "/root/Image"
name = "guest"
memory = 512
vcpus = 1
extra = "console=ttyAMA0 root=/dev/xvda ro"
disk = [ 'phy:/dev/loop0,xvda,w' ]
(none):/root # xl create domU.cfg
Parsing config from domU.cfg
(XEN) allocated xen_consumer = 1 for port = 1, remote dom=0
(XEN) searching virq in < 32, 33 >
(XEN) HVM_PARAM_VPL011_VIRQ param get val=32
(XEN) HVM_PARAM_VPL011_CONSOLE_PFN = 0x39003
(XEN) HVM_PARAM_VPL011_CONSOLE_PFN param set val=0x39003
seq 1204 queued, 'add' 'xen-backend'
Validate module index
Check if link configuration needs reloading.
seq 1204 forked new worker [1556]
seq 1204 running
IMPORT builtin 'hwdb' /usr/lib/udev/rules.d/50-udev-default.rules:15
IMPORT builtin 'hwdb' returned non-zero
RUN 'kmod load $env{MODALIAS}' /usr/lib/udev/rules.d/80-drivers.rules:5
Execute 'load' 'xen-backend:vbd'
passed device to netlink monitor 0xaaaab38dad90
seq 1204 processed
cleanup idle workers
Unload module index
Unloaded link configuration context.
worker [1556] exited
seq 1205 queued, 'add' 'xen-backend'
seq 1205 forked new worker [1614]
seq 1205 running
IMPORT builtin 'hwdb' /usr/lib/udev/rules.d/50-udev-default.rules:15
IMPORT builtin 'hwdb' returned non-zero
RUN 'kmod load $env{MODALIAS}' /usr/lib/udev/rules.d/80-drivers.rules:5
Execute 'load' 'xen-backend:console'
passed device to netlink monitor 0xaaaab38dad90
seq 1205 processed
cleanup idle workers
Unload module index
Unloaded link configuration context.
worker [1614] exited
(none):/root # xl console guest
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.9.0-rc3-00243-g0c183d9-dirty (btuser@blr-ubuntu-linaro) (gcc version 4.9.3 20141031 (prerelease) (Linaro GCC 2014.11) ) #94 SMP PREEMPT Mon Feb 13 12:12:48 IST 2017
[ 0.000000] Boot CPU: AArch64 Processor [410fd000]
[ 0.000000] Xen 4.9 support found
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 16 MiB at 0x000000005f000000
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv0.2 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] percpu: Embedded 21 pages/cpu @ffff80001efdc000 s48000 r8192 d29824 u86016
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129024
[ 0.000000] Kernel command line: console=ttyAMA0 root=/dev/xvda ro
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.000000] Memory: 476280K/524288K available (8380K kernel code, 876K rwdata, 3676K rodata, 1024K init, 283K bss, 31624K reserved, 16384K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB)
[ 0.000000] .text : 0xffff000008080000 - 0xffff0000088b0000 ( 8384 KB)
[ 0.000000] .rodata : 0xffff0000088b0000 - 0xffff000008c50000 ( 3712 KB)
[ 0.000000] .init : 0xffff000008c50000 - 0xffff000008d50000 ( 1024 KB)
[ 0.000000] .data : 0xffff000008d50000 - 0xffff000008e2b200 ( 877 KB)
[ 0.000000] .bss : 0xffff000008e2b200 - 0xffff000008e71ec0 ( 284 KB)
[ 0.000000] fixed : 0xffff7dfffe7fd000 - 0xffff7dfffec00000 ( 4108 KB)
[ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB)
[ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum)
[ 0.000000] 0xffff7e0000000000 - 0xffff7e0000800000 ( 8 MB actual)
[ 0.000000] memory : 0xffff800000000000 - 0xffff800020000000 ( 512 MB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] Build-time adjustment of leaf fanout to 64.
[ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
[ 0.000000] NR_IRQS:64 nr_irqs:64 0
[ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 100.00MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns
[ 0.000024] sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns
[ 0.000660] Console: colour dummy device 80x25
[ 0.000720] Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=400000)
[ 0.000771] pid_max: default: 32768 minimum: 301
[ 0.001138] Security Framework initialized
[ 0.001338] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.001389] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.004333] ASID allocator initialised with 65536 entries
[ 0.005545] xen:grant_table: Grant tables using version 1 layout
[ 0.005653] Grant table initialized
[ 0.005731] xen:events: Using FIFO-based ABI
[ 0.005813] Xen: initializing cpu0
[ 0.038093] EFI services will not be available.
[ 0.052923] Brought up 1 CPUs
[ 0.052955] SMP: Total of 1 processors activated.
[ 0.052994] CPU features: detected feature: Privileged Access Never
[ 0.053040] CPU features: detected feature: User Access Override
[ 0.053094] CPU features: detected feature: 32-bit EL0 Support
[ 0.053252] CPU: All CPU(s) started at EL1
[ 0.053298] alternatives: patching kernel code
[ 0.057685] devtmpfs: initialized
[ 0.065647] DMI not present or invalid.
[ 0.066549] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.070126] pinctrl core: initialized pinctrl subsystem
[ 0.075662] NET: Registered protocol family 16
[ 0.090773] vdso: 2 pages (1 code @ ffff0000088b7000, 1 data @ ffff000008d54000)
[ 0.090827] hw-breakpoint: found 16 breakpoint and 16 watchpoint registers.
[ 0.096374] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.101159] Serial: AMBA PL011 UART driver
[ 0.107941] [DEBUG] DT baudrate=115200
[ 0.108053] [DEBUG] pl011_setup_port: mmiobase = 0x22000000
[ 0.108082] [DEBUG] registering pl011 port...
[ 0.108288] [DEBUG] pl011_request_port called with return -16
[ 0.108321] [DEBUG] pl011_config_port return -16
[ 0.108376] 22000000.sbsa-pl011: ttyAMA0 at MMIO 0x22000000 (irq = 5, base_baud = 0) is a SBSA
[ 0.951889] console [ttyAMA0] enabled
[ 1.079520] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 1.126153] ACPI: Interpreter disabled.
[ 1.157902] xen:balloon: Initialising balloon driver
[ 1.186169] xen_balloon: Initialising balloon driver
[ 1.221471] vgaarb: loaded
[ 1.253097] SCSI subsystem initialized
[ 1.261315] usbcore: registered new interface driver usbfs
[ 1.292045] usbcore: registered new interface driver hub
[ 1.294535] usbcore: registered new device driver usb
[ 1.299146] pps_core: LinuxPPS API ver. 1 registered
[ 1.301146] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giome...@linux.it>
[ 1.305592] PTP clock support registered
[ 1.309078] dmi: Firmware registration failed.
[ 1.312554] Advanced Linux Sound Architecture Driver Initialized.
[ 1.321390] clocksource: Switched to clocksource arch_sys_counter
[ 1.351568] VFS: Disk quotas dquot_6.6.0
[ 1.354063] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 1.357800] pnp: PnP ACPI: disabled
[ 1.414296] NET: Registered protocol family 2
[ 1.419180] TCP established hash table entries: 4096 (order: 3, 32768 bytes)
[ 1.423418] TCP bind hash table entries: 4096 (order: 4, 65536 bytes)
[ 1.427725] TCP: Hash tables configured (established 4096 bind 4096)
[ 1.431440] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 1.434356] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 1.437637] NET: Registered protocol family 1
[ 1.448956] RPC: Registered named UNIX socket transport module.
[ 1.479221] RPC: Registered udp transport module.
[ 1.482021] RPC: Registered tcp transport module.
[ 1.483433] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 1.487911] kvm [1]: HYP mode not available
[ 1.492819] futex hash table entries: 256 (order: 3, 32768 bytes)
[ 1.554422] audit: initializing netlink subsys (disabled)
[ 1.556524] audit: type=2000 audit(0.708:1): initialized
[ 1.565774] workingset: timestamp_bits=46 max_order=17 bucket_order=0
[ 1.671737] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 1.680715] NFS: Registering the id_resolver key type
[ 1.684352] Key type id_resolver registered
[ 1.687312] Key type id_legacy registered
[ 1.690059] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 1.693517] 9p: Installing v9fs 9p2000 file system support
[ 1.711701] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[ 1.715750] io scheduler noop registered
[ 1.719041] io scheduler cfq registered (default)
[ 1.725609] libphy: mdio_driver_register: phy-bcm-ns2-pci
[ 1.760436] xen:xen_evtchn: Event-channel device installed
[ 1.791998] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 1.804472] SuperH (H)SCI(F) driver initialized
[ 1.808884] msm_serial: driver initialized
[ 1.812963] Unable to detect cache hierarchy from DT for CPU 0
[ 1.870954] loop: module loaded
[ 1.873610] Invalid max_queues (4), will use default max: 1.
[ 1.885531] hisi_sas: driver version v1.6
[ 1.947514] libphy: Fixed MDIO Bus: probed
[42080.693216] xen-blkback: backend/vbd/3/51712: using 1 queues, protocol 1 (arm-abi) persistent grants
[ 2.004909] tun: Universal TUN/TAP device driver, 1.6
[ 2.062463] tun: (C) 1999-2004 Max Krasnyansky <m...@qualcomm.com>
[ 2.122473] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[ 2.153762] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 2.157244] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[ 2.161758] igb: Copyright (c) 2007-2014 Intel Corporation.
[ 2.165164] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.4.0-k
[ 2.169878] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[ 2.173102] sky2: driver version 1.30
[ 2.176530] QLogic/NetXen Network Driver v4.0.82
[ 2.182603] xen_netfront: Initialising Xen virtual ethernet driver
[ 2.193794] VFIO - User Level meta-driver version: 0.3
[ 2.206433] blkfront: xvda: flush diskcache: enabled; persistent grants: enabled; indirect descriptors: enabled;
[ 2.234777] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 2.234789] ehci-pci: EHCI PCI platform driver
[ 2.235235] ehci-platform: EHCI generic platform driver
[ 2.235603] ehci-exynos: EHCI EXYNOS driver
[ 2.235953] ehci-msm: Qualcomm On-Chip EHCI Host Controller
[ 2.237978] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 2.238005] ohci-pci: OHCI PCI platform driver
[ 2.238323] ohci-platform: OHCI generic platform driver
[ 2.238667] ohci-exynos: OHCI EXYNOS driver
[ 2.240473] usbcore: registered new interface driver usb-storage
[ 2.247037] mousedev: PS/2 mouse device common for all mice
[ 2.254747] i2c /dev entries driver
[ 2.264399] xen_wdt: Xen WatchDog Timer Driver v0.01
[ 2.267832] xen_wdt: initialized (timeout=60s, nowayout=0)
[ 2.269948] sdhci: Secure Digital Host Controller Interface driver
[ 2.269957] sdhci: Copyright(c) Pierre Ossman
[ 2.271099] Synopsys Designware Multimedia Card Interface Driver
[ 2.273296] sdhci-pltfm: SDHCI platform and OF driver helper
[ 2.276918] ledtrig-cpu: registered to indicate activity on CPUs
[ 2.284412] usbcore: registered new interface driver usbhid
[ 2.284420] usbhid: USB HID core driver
[ 2.296057] NET: Registered protocol family 17
[ 2.296431] 9pnet: Installing 9P2000 support
[ 2.297058] Key type dns_resolver registered
[ 2.305379] registered taskstats version 1
[ 2.409868] hctosys: unable to open rtc device (rtc0)
[ 2.412447] ALSA device list:
[ 2.414132] No soundcards found.
[ 2.415352] [DEBUG] uart_port_startup entry with port type = 32
[ 2.418303] [DEBUG] sbsa_uart_startup called...
[ 2.419736] [DEBUG] pl011_allocate_irq irq=5...
[ 2.421320] [DEBUG] sbsa_uart_startup done...
[ 2.568210] [DEBUG] uart_port_startup exit
[ 2.570649] EXT4-fs (xvda): mounting ext3 file system using the ext4 subsystem
[ 2.615662] EXT4-fs (xvda): mounted filesystem with ordered data mode. Opts: (null)
[ 2.644906] VFS: Mounted root (ext3 filesystem) readonly on device 202:0.
[ 2.653947] devtmpfs: mounted
[ 2.685517] Freeing unused kernel memory: 1024K (ffff800000c50000 - ffff800000d50000)
[ 2.750920] [DEBUG] uart_wait_until_sent entry ...
[ 2.780537] [DEBUG] pl011_tx_empty called with return 1
[ 2.781983] [DEBUG] uart_wait_until_sent exit ...
[ 2.783592] [DEBUG] uart_wait_until_sent entry ...
[ 2.786313] [DEBUG] pl011_tx_empty called with return 1
[ 2.787676] [DEBUG] uart_wait_until_sent exit ...
(XEN) [DEBUG]SENding guest pl011 intr 32 with intr=0x30, flag=0x80, out_cons=12494, out_prod=12494, in_cons=0, in_prod=1, status=0...
(XEN) [DEBUG]vgic_vcpu_inject_irq: raised
(XEN) [DEBUG]gic_raise_guest_irq: pending
(XEN) [DEBUG]irq 32 marked lr pending
(XEN) [DEBUG]setting lr 1 for irq 32
(XEN) [DEBUG]gicv2_update_lr: 1, 1
(XEN) [DEBUG]SENding guest pl011 intr 32 with intr=0x30, flag=0x80, out_cons=12494, out_prod=12494, in_cons=0, in_prod=2, status=0...
(XEN) [DEBUG]vgic_vcpu_inject_irq: inflight
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
