Re: [Xen-devel] [PATCH 08/17] x86emul: generate and make use of canonical opcode representation

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 16:03, wrote: > On 15/09/16 07:43, Jan Beulich wrote: > On 14.09.16 at 19:30, wrote: +#define X86EMUL_OPC_PFX_MASK 0x0300 +# define X86EMUL_OPC_66(ext, byte) (X86EMUL_OPC(ext, byte) | 0x0100) +# define X86EMUL_OPC_F3(ext, byte) (X86EMUL_OP

Re: [Xen-devel] [Patch] x86emul: simplify prefix handling for VMFUNC

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 19:43, wrote: > Finally found the vmfunc opcode page in Vol 3 30.3, VMX Instruction Reference. > Agreed, there's no mention of prefixes, "pfx", on this page. It appears > that the other VMX instructions in this section don't mention prefixes either. > Looking at Table A-6 "Opco

[Xen-devel] [PATCH v2 00/16] x86: split insn emulator decode and execution

2016-09-28 Thread Jan Beulich
..., complete the decoder, leverage decoding for SVM instruction sizing and PV 32-bit call gate emulation, and use the emulator for PV priv-op handling. 01: x86emul: split instruction decoding from execution 02: x86emul: fetch all insn bytes during the decode phase 03: x86emul: track only rIP in e

[Xen-devel] [PATCH v2 01/16] x86emul: split instruction decoding from execution

2016-09-28 Thread Jan Beulich
This is only the mechanical part, a subsequent patch will make non- mechanical adjustments to actually do all decoding in this new function. Signed-off-by: Jan Beulich --- v2: Fix a coding style issue. --- a/xen/arch/x86/x86_emulate/x86_emulate.c +++ b/xen/arch/x86/x86_emulate/x86_emulate.c @@ -

[Xen-devel] [PATCH v2 02/16] x86emul: fetch all insn bytes during the decode phase

2016-09-28 Thread Jan Beulich
This way we can offer to callers the service of just sizing instructions, and we also can better guarantee not to raise the wrong fault due to not having read all relevant bytes. Signed-off-by: Jan Beulich --- v2: Rename x86_decode_base() -> x86_decode_onebyte(). --- a/xen/arch/x86/x86_emulate/x

[Xen-devel] [PATCH v2 03/16] x86emul: track only rIP in emulator state

2016-09-28 Thread Jan Beulich
Now that all decoding happens in x86_decode() there's no need to keep the local registers copy in struct x86_emulate_state. Only rIP gets updated in the decode phase, so only that register needs tracking there. All other (read-only) registers can be read from the original structure (but sadly, due

[Xen-devel] [PATCH v2 05/16] x86emul: add XOP decoding

2016-09-28 Thread Jan Beulich
This way we can at least size (and e.g. skip) them if needed, and we also won't raise the wrong fault due to not having read all relevant bytes. Signed-off-by: Jan Beulich --- v2: Add a comment. --- a/xen/arch/x86/x86_emulate/x86_emulate.c +++ b/xen/arch/x86/x86_emulate/x86_emulate.c @@ -279,6 +

[Xen-devel] [PATCH v2 04/16] x86emul: complete decoding of two-byte instructions

2016-09-28 Thread Jan Beulich
This way we can at least size (and e.g. skip) them if needed, and we also won't raise the wrong fault due to not having read all relevant bytes. This at once adds correct raising of #UD for the three "ud" flavors (Intel names only "ud2", but AMD names all three of them in their opcode maps), as th

[Xen-devel] [PATCH v2 06/16] x86emul: add EVEX decoding

2016-09-28 Thread Jan Beulich
This way we can at least size (and e.g. skip) them if needed, and we also won't raise the wrong fault due to not having read all relevant bytes. Signed-off-by: Jan Beulich --- TBD: I'm kind of undecided whether to right away propagate evex.R into modrm_reg (and then also deal with the new me

[Xen-devel] [PATCH v2 07/16] x86emul: generate and make use of a canonical opcode representation

2016-09-28 Thread Jan Beulich
This representation is then being made available to interested callers, to facilitate replacing their custom decoding. This entails combining the three main switch statements into one. Signed-off-by: Jan Beulich --- v2: Extend comments. Use uint8_t cast in X86EMUL_OPC(). Rename X86EMUL_OPC_K

[Xen-devel] [PATCH v2 08/16] SVM: use generic instruction decoding

2016-09-28 Thread Jan Beulich
... instead of custom handling. To facilitate this break out init code from _hvm_emulate_one() into the new hvm_emulate_init(), and make hvmemul_insn_fetch( globally available. Signed-off-by: Jan Beulich --- v2: Add comment to caller field. Rename REG_POISON to PTR_POISON. Align opc_tab[] ini

[Xen-devel] [PATCH v2 09/16] x86/32on64: use generic instruction decoding for call gate emulation

2016-09-28 Thread Jan Beulich
... instead of custom handling. Note that we can't use generic emulation, as the emulator's far branch support is rather rudimentary at this point in time. Signed-off-by: Jan Beulich --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -28,6 +28,7 @@ #include #include #include +#includ

[Xen-devel] [PATCH v2 11/16] x86/PV: split out dealing with DRn from privileged instruction handling

2016-09-28 Thread Jan Beulich
This is in preparation for using the generic emulator here. Some care is needed temporarily to not unduly alter guest register state: The local variable "res" can only go away once this code got fully switched over to using x86_emulate(). Also switch to IS_ERR_VALUE() instead of (incorrectly) ope

[Xen-devel] [PATCH v2 12/16] x86/PV: split out dealing with MSRs from privileged instruction handling

2016-09-28 Thread Jan Beulich
This is in preparation for using the generic emulator here. Signed-off-by: Jan Beulich --- v2: Re-base. --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -2386,6 +2386,345 @@ static inline uint64_t guest_misc_enable return val; } +static inline bool is_cpufreq_controller(const str

[Xen-devel] [PATCH v2 10/16] x86/PV: split out dealing with CRn from privileged instruction handling

2016-09-28 Thread Jan Beulich
This is in preparation for using the generic emulator here. Signed-off-by: Jan Beulich --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -2255,6 +2255,107 @@ unsigned long guest_to_host_gpr_switch(u void (*pv_post_outb_hook)(unsigned int port, u8 value); +static int priv_op_read_cr(u

[Xen-devel] [PATCH v2 13/16] x86emul: support XSETBV

2016-09-28 Thread Jan Beulich
This is a prereq for switching PV privileged op emulation to the generic instruction emulator. Since handle_xsetbv() is already capable of dealing with all guest kinds, avoid introducing another hook here. Signed-off-by: Jan Beulich --- v2: Explicitly generate #UD when vex.pfx is non-zero. --- a

[Xen-devel] [PATCH v2 15/16] x86/PV: use generic emulator for privileged instruction handling

2016-09-28 Thread Jan Beulich
There's a new emulator return code being added to allow bypassing certain operations (see the code comment). Its handling in the epilogue code involves moving the raising of the single step trap until after registers were updated. This should probably have been that way from the beginning, to allow

[Xen-devel] [PATCH v2 14/16] x86emul: sort opcode 0f01 special case switch() statement

2016-09-28 Thread Jan Beulich
Sort the special case opcode 0f01 entries numerically, insert blank lines between each of the cases, and properly place opening braces. No functional change. Signed-off-by: Jan Beulich --- a/xen/arch/x86/x86_emulate/x86_emulate.c +++ b/xen/arch/x86/x86_emulate/x86_emulate.c @@ -4195,6 +4195,14

[Xen-devel] [PATCH v2 16/16] x86emul: don't assume a memory operand

2016-09-28 Thread Jan Beulich
Especially for x86_insn_operand_ea() to return dependable segment information even when the caller didn't consider applicability, we shouldn't have ea.type start out as OP_MEM. Make it OP_NONE instead, and set it to OP_MEM when we actually encounter memory like operands. This requires to eliminate

Re: [Xen-devel] [PATCH v2 13/30] xen: introduce a new format specifier to print sizes in human-readable form

2016-09-28 Thread Juergen Gross
On 27/09/16 17:57, Roger Pau Monne wrote: > Introduce a new %pB format specifier to print sizes (in bytes) in a Code suggests it is pZ. > human-readable form. > > Signed-off-by: Roger Pau Monné > --- > Cc: Andrew Cooper > Cc: George Dunlap > Cc: Ian Jackson > Cc: Jan Beulich > Cc: Konrad Rz

[Xen-devel] [PATCH] pvgrub: use printk() instead of grub_printf()

2016-09-28 Thread Juergen Gross
grub_printf() is supporting only a very limited number of formats. Especially some error messages suffer from that, e.g. %lx won't work. Switch to use printk() for error messages instead. Signed-off-by: Juergen Gross --- stubdom/grub/kexec.c | 22 +++--- 1 file changed, 11 insert

[Xen-devel] [PATCH v2 00/16] x86: split insn emulator decode and execution

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 09:59, wrote: > 01: x86emul: split instruction decoding from execution > 02: x86emul: fetch all insn bytes during the decode phase > 03: x86emul: track only rIP in emulator state > 04: x86emul: complete decoding of two-byte instructions > 05: x86emul: add XOP decoding > 06: x86e

Re: [Xen-devel] [PATCH RFC v7 07/14] efi: create new early memory allocator

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 19:49, wrote: > On Mon, Sep 26, 2016 at 07:37:45AM -0600, Jan Beulich wrote: >> >>> On 23.09.16 at 23:47, wrote: >> > --- a/xen/common/efi/boot.c >> > +++ b/xen/common/efi/boot.c >> > @@ -79,6 +79,10 @@ static size_t wstrlen(const CHAR16 * s); >> > static int set_color(u32 mas

Re: [Xen-devel] [PATCH v7 08/14] x86: add multiboot2 protocol support for EFI platforms

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 20:11, wrote: > On Mon, Sep 26, 2016 at 07:47:42AM -0600, Jan Beulich wrote: >> >>> On 23.09.16 at 23:47, wrote: >> > This way Xen can be loaded on EFI platforms using GRUB2 and >> > other boot loaders which support multiboot2 protocol. >> > >> > Signed-off-by: Daniel Kiper >>

Re: [Xen-devel] [PATCH v7 08/14] x86: add multiboot2 protocol support for EFI platforms

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 20:21, wrote: > On Mon, Sep 26, 2016 at 09:12:40AM -0600, Jan Beulich wrote: >> >>> On 26.09.16 at 16:40, wrote: >> > On 26/09/16 15:33, Jan Beulich wrote: >> > On 26.09.16 at 16:19, wrote: >> >>> On 23/09/16 22:47, Daniel Kiper wrote: >> +/* >> +

Re: [Xen-devel] [PATCH v7 12/14] x86: make Xen early boot code relocatable

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 21:55, wrote: > On Mon, Sep 26, 2016 at 09:03:30AM -0600, Jan Beulich wrote: >> >>> On 23.09.16 at 23:47, wrote: >> > @@ -426,32 +453,65 @@ trampoline_bios_setup: >> > xor %cl, %cl >> > >> > trampoline_setup: >> > +/* >> > + * Called on legacy BIOS

[Xen-devel] [xen-unstable test] 101175: tolerable FAIL

2016-09-28 Thread osstest service owner
flight 101175 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/101175/ Failures :-/ but no regressions. Tests which are failing intermittently (not blocking): test-armhf-armhf-xl-rtds 15 guest-start/debian.repeat fail pass in 101169 test-armhf-armhf-xl-arndale 15

Re: [Xen-devel] PCIe devices that are hotplugged after MMIO has been setup fail due to _CRS not covering 64-bit area

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 16:43, wrote: > If the guest is booted with 'pci' we nicely expand the MMIO region below > 4GB and try to fit in the BARs in there. If that fails (not enough > space) we move it above the memory (64-bit). And throughout all of this > we also update the _CRS field to cover these

[Xen-devel] [libvirt test] 101176: tolerable FAIL - PUSHED

2016-09-28 Thread osstest service owner
flight 101176 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/101176/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail never pass test-armhf-armhf-libvirt-xsm 14 guest-sav

Re: [Xen-devel] [PATCH v2 03/30] xen/x86: fix parameters and return value of *_set_allocation functions

2016-09-28 Thread Tim Deegan
At 17:56 +0200 on 27 Sep (1474999018), Roger Pau Monne wrote: > Return should be an int, and the number of pages should be an unsigned long. > > Signed-off-by: Roger Pau Monné Both those changes seem fine to me. Since you're changing the return types to int, can you please also change the two c

Re: [Xen-devel] [PATCH v4 1/6] VMX: Statically assign two PI hooks

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 08:48, wrote: > >> -Original Message- >> From: Jan Beulich [mailto:jbeul...@suse.com] >> Sent: Monday, September 26, 2016 8:10 PM >> To: Wu, Feng >> Cc: andrew.coop...@citrix.com; dario.faggi...@citrix.com; >> george.dun...@eu.citrix.com; Tian, Kevin ; xen- >> de...@l

Re: [Xen-devel] [PATCH v7 08/14] x86: add multiboot2 protocol support for EFI platforms

2016-09-28 Thread Daniel Kiper
On Wed, Sep 28, 2016 at 02:57:10AM -0600, Jan Beulich wrote: > >>> On 27.09.16 at 20:11, wrote: > > On Mon, Sep 26, 2016 at 07:47:42AM -0600, Jan Beulich wrote: > >> >>> On 23.09.16 at 23:47, wrote: > >> > This way Xen can be loaded on EFI platforms using GRUB2 and > >> > other boot loaders which

[Xen-devel] [PATCH v1 08/12] tmem: Handle 'struct tmem_info' as a seperate field in the

2016-09-28 Thread Konrad Rzeszutek Wilk
the struct tmem_op. No functional change. But it makes the code so much easier to read. Note: We still have to do this awkward 'guest_handle_cast' otherwise it will not compile on ARM - which defines _two_ of these macros (__guest_handle_64_xen_sysctl_tmem_client_t and __guest_handle_xen_sysctl_t

[Xen-devel] [PATCH v1 02/12] tmem: Retire XEN_SYSCTL_TMEM_OP_[SET_CAP|SAVE_GET_CLIENT_CAP]

2016-09-28 Thread Konrad Rzeszutek Wilk
It is not used by anything. Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Wei Liu v1: First submission --- docs/man/xl.pod.1.in | 4 tools/libxc/xc_tmem.c | 13 +++-- tools/libxl/libxl.c | 4 +--- tools/libxl/xl_cmdtable.c

[Xen-devel] [PATCH v1 03/12] tmem: Wrap tmem dedup code with CONFIG_TMEM_DEDUP

2016-09-28 Thread Konrad Rzeszutek Wilk
This config is not defined anywhere but it makes it way easier to figure out what code to deal with. Signed-off-by: Konrad Rzeszutek Wilk --- v1: First submission. --- xen/common/tmem.c | 57 ++ xen/common/tmem_control.c | 2 ++ xen/common/t

[Xen-devel] [PATCH v1 06/12] tmem: Move client weight, frozen, live_migrating, and compress

2016-09-28 Thread Konrad Rzeszutek Wilk
in its own structure. This paves the way to make only one hypercall to retrieve/set this information instead of multiple ones. Signed-off-by: Konrad Rzeszutek Wilk --- v1: First submission. --- xen/common/tmem.c | 40 +--- xen/common/tmem_control.c

[Xen-devel] [PATCH v1 04/12] tmem: Wrap tmem tze code with CONFIG_TMEM_TZE

2016-09-28 Thread Konrad Rzeszutek Wilk
. which is actually dependent on CONFIG_TMEM_DEDUP Signed-off-by: Konrad Rzeszutek Wilk --- v1: First submission. --- xen/common/tmem.c | 12 ++-- xen/common/tmem_xen.c | 4 xen/include/xen/tmem_xen.h | 4 3 files changed, 18 insertions(+), 2 deletions(-) diff

[Xen-devel] [PATCH v1 11/12] tmem/xc_tmem_control: Rename 'arg1' to 'len' and 'arg2' to arg.

2016-09-28 Thread Konrad Rzeszutek Wilk
That is what they are used for. Lets make it more clear. Of all the various sub-commands, the only one that needed semantic change is XEN_SYSCTL_TMEM_OP_SAVE_BEGIN. That in the past used 'arg1', and now we are moving it to use 'arg'. Since that code is only used during migration which is tied to t

[Xen-devel] [PATCH v1] Tmem cleanups/improvements for v4.8.

2016-09-28 Thread Konrad Rzeszutek Wilk
Hey! This batch of fixes slowly marches toward ripping out pieces of code in tmem that are hard to maintain and improve on code that was orginacally developed. I had hoped that I would have had the migration support all working, but it took longer than I thought to get to this point (and migratio

[Xen-devel] [PATCH v1 05/12] tmem: Delete deduplication (and tze) code.

2016-09-28 Thread Konrad Rzeszutek Wilk
Couple of reasons: - It can lead to security issues (see row-hammer, KSM and such attacks). - Code is quite complex. - Deduplication is good if the pages themselves are the same but that is hardly guaranteed. - We got some gains (if pages are deduped) but at the cost of making code les

[Xen-devel] [PATCH v1 09/12] tmem: Check version and maxpools when XEN_SYSCTL_TMEM_SET_CLIENT_INFO

2016-09-28 Thread Konrad Rzeszutek Wilk
is called. If they are different from what the hypervisor can support we will get the appropiate errors. Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Wei Liu v1: First submission. --- tools/libxc/xc_tmem.c | 1 - xen/common/tmem_control.c | 6 ++ xen/include/publi

[Xen-devel] [PATCH v1 12/12] tmem: Batch and squash XEN_SYSCTL_TMEM_OP_SAVE_GET_POOL_[FLAGS, NPAGES, UUID] in one sub-call: XEN_SYSCTL_TMEM_OP_GET_POOLS.

2016-09-28 Thread Konrad Rzeszutek Wilk
These operations are used during the save process of migration. Instead of doing 64 hypercalls lets do just one. We modify the 'struct tmem_client' structure (used in XEN_SYSCTL_TMEM_OP_[GET|SET]_CLIENT_INFO) to have an extra field 'nr_pools'. Armed with that the code slurping up pages from the hyp

[Xen-devel] [PATCH v1 10/12] tmem: Unify XEN_SYSCTL_TMEM_OP_[[SAVE_[BEGIN|END]|RESTORE_BEGIN]

2016-09-28 Thread Konrad Rzeszutek Wilk
return values. For success they used to be 1 ([SAVE,RESTORE]_BEGIN), 0 if guest did not have any tmem (but only for SAVE_BEGIN), and -1 for any type of failure. And SAVE_END (which you would think would mirror SAVE_BEGIN) had 0 for success and -1 if guest did not any tmem enabled for it. This is

[Xen-devel] [PATCH v1 01/12] libxc/tmem/restore: Remove call to XEN_SYSCTL_TMEM_OP_SAVE_GET_VERSION

2016-09-28 Thread Konrad Rzeszutek Wilk
The only thing this hypercall returns is TMEM_SPEC_VERSION. The comment around is also misleading - this call does not do any domain operation. Signed-off-by: Konrad Rzeszutek Wilk --- Cc: Ian Jackson Cc: Wei Liu v1: Initial submission. --- tools/libxc/xc_tmem.c | 4 1 file changed, 4

[Xen-devel] [PATCH v1 07/12] tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE]..

2016-09-28 Thread Konrad Rzeszutek Wilk
Specifically: XEN_SYSCTL_TMEM_OP_SET_[WEIGHT,COMPRESS] are now done via: XEN_SYSCTL_TMEM_SET_CLIENT_INFO and XEN_SYSCTL_TMEM_OP_SAVE_GET_[VERSION,MAXPOOLS, CLIENT_WEIGHT, CLIENT_FLAGS] can now be retrieved via: XEN_SYSCTL_TMEM_GET_CLIENT_INFO All this information is now in 'struct tmem_clien

[Xen-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Juergen Gross
Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds with uncommon config settings") introduced a bug which will break the build, as xc_minios.c will have defined those macros via xc_private.h. Fix this by defining the macros only if compiling a minios source. Signed-off-by: Juer

Re: [Xen-devel] [PATCH v4 2/6] VMX: Properly handle pi when all the assigned devices are removed

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 08:50, wrote: > >> -Original Message- >> From: Jan Beulich [mailto:jbeul...@suse.com] >> Sent: Monday, September 26, 2016 7:47 PM >> To: Wu, Feng >> Cc: andrew.coop...@citrix.com; dario.faggi...@citrix.com; >> george.dun...@eu.citrix.com; Tian, Kevin ; xen- >> de...@l

Re: [Xen-devel] PCIe devices that are hotplugged after MMIO has been setup fail due to _CRS not covering 64-bit area

2016-09-28 Thread Konrad Rzeszutek Wilk
On Wed, Sep 28, 2016 at 03:21:08AM -0600, Jan Beulich wrote: > >>> On 27.09.16 at 16:43, wrote: > > If the guest is booted with 'pci' we nicely expand the MMIO region below > > 4GB and try to fit in the BARs in there. If that fails (not enough > > space) we move it above the memory (64-bit). And t

Re: [Xen-devel] [PATCH v7 12/14] x86: make Xen early boot code relocatable

2016-09-28 Thread Daniel Kiper
On Wed, Sep 28, 2016 at 03:06:31AM -0600, Jan Beulich wrote: > >>> On 27.09.16 at 21:55, wrote: > > On Mon, Sep 26, 2016 at 09:03:30AM -0600, Jan Beulich wrote: > >> >>> On 23.09.16 at 23:47, wrote: > >> > @@ -426,32 +453,65 @@ trampoline_bios_setup: > >> > xor %cl, %cl > >> > > >> >

Re: [Xen-devel] [PATCH v4 5/6] VT-d: No need to set irq affinity for posted format IRTE

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 08:51, wrote: > >> -Original Message- >> From: Jan Beulich [mailto:jbeul...@suse.com] >> Sent: Monday, September 26, 2016 8:58 PM >> To: Wu, Feng >> Cc: andrew.coop...@citrix.com; dario.faggi...@citrix.com; >> george.dun...@eu.citrix.com; Tian, Kevin ; xen- >> de...@l

Re: [Xen-devel] [PATCH v6 16/16] libxl/arm: Add the size of ACPI tables to maxmem

2016-09-28 Thread Wei Liu
On Tue, Sep 27, 2016 at 11:43:38AM -0700, Shannon Zhao wrote: > > > On 2016/9/27 9:35, Wei Liu wrote: > >On Tue, Sep 27, 2016 at 09:01:00AM -0700, Shannon Zhao wrote: > >> > >> > >>On 2016/9/27 2:41, Wei Liu wrote: > >>>On Mon, Sep 26, 2016 at 02:54:55PM -0700, Shannon Zhao wrote: > > >

Re: [Xen-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 11:50:04AM +0200, Juergen Gross wrote: > Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds > with uncommon config settings") introduced a bug which will break > the build, as xc_minios.c will have defined those macros via > xc_private.h. > > Fix this by

Re: [Xen-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Juergen Gross
On 28/09/16 12:01, Wei Liu wrote: > On Wed, Sep 28, 2016 at 11:50:04AM +0200, Juergen Gross wrote: >> Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds >> with uncommon config settings") introduced a bug which will break >> the build, as xc_minios.c will have defined those macro

Re: [Xen-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 12:06:02PM +0200, Juergen Gross wrote: > On 28/09/16 12:01, Wei Liu wrote: > > On Wed, Sep 28, 2016 at 11:50:04AM +0200, Juergen Gross wrote: > >> Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds > >> with uncommon config settings") introduced a bug whic

Re: [Xen-devel] [Minios-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Juergen Gross
On 28/09/16 12:10, Wei Liu wrote: > On Wed, Sep 28, 2016 at 12:06:02PM +0200, Juergen Gross wrote: >> On 28/09/16 12:01, Wei Liu wrote: >>> On Wed, Sep 28, 2016 at 11:50:04AM +0200, Juergen Gross wrote: Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds with uncommon co

Re: [Xen-devel] [PATCH v7 08/14] x86: add multiboot2 protocol support for EFI platforms

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:39, wrote: > On Wed, Sep 28, 2016 at 02:57:10AM -0600, Jan Beulich wrote: >> >>> On 27.09.16 at 20:11, wrote: >> > On Mon, Sep 26, 2016 at 07:47:42AM -0600, Jan Beulich wrote: >> >> >>> On 23.09.16 at 23:47, wrote: >> >> > +/* >> >> > + * Initialize BSS (no

Re: [Xen-devel] [PATCH v7 12/14] x86: make Xen early boot code relocatable

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:56, wrote: > On Wed, Sep 28, 2016 at 03:06:31AM -0600, Jan Beulich wrote: >> >>> On 27.09.16 at 21:55, wrote: >> > On Mon, Sep 26, 2016 at 09:03:30AM -0600, Jan Beulich wrote: >> >> >>> On 23.09.16 at 23:47, wrote: >> >> > +1: mov %eax,sym_fs(l2_bootmap)-8(%ebx,%ec

Re: [Xen-devel] [PATCH v5 06/16] livepatch: ARM/x86: Check displacement of old_addr and new_addr

2016-09-28 Thread Ross Lagerwall
On 09/23/2016 04:59 PM, Konrad Rzeszutek Wilk wrote: On Fri, Sep 23, 2016 at 11:37:27AM -0400, Konrad Rzeszutek Wilk wrote: On Fri, Sep 23, 2016 at 03:36:27PM +0100, Ross Lagerwall wrote: On 09/21/2016 06:32 PM, Konrad Rzeszutek Wilk wrote: If the distance is too big we are in trouble - as our

Re: [Xen-devel] [PATCH v7] acpi: Prevent GPL-only code from seeping into non-GPL binaries

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 16:46, wrote: > --- a/tools/firmware/hvmloader/acpi/Makefile > +++ b/tools/firmware/hvmloader/acpi/Makefile > @@ -17,7 +17,8 @@ > XEN_ROOT = $(CURDIR)/../../../.. > include $(XEN_ROOT)/tools/firmware/Rules.mk > > -C_SRC = build.c dsdt_anycpu.c dsdt_15cpu.c static_tables.c >

[Xen-devel] [ovmf test] 101180: all pass - PUSHED

2016-09-28 Thread osstest service owner
flight 101180 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/101180/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf ab970515d2c6ec657fceab0ce571054bb43a22f2 baseline version: ovmf eab26788156436a549610

[Xen-devel] [xen-unstable-coverity test] 101181: all pass - PUSHED

2016-09-28 Thread osstest service owner
flight 101181 xen-unstable-coverity real [real] http://logs.test-lab.xenproject.org/osstest/logs/101181/ Perfect :-) All tests in this flight passed as required version targeted for testing: xen 1e75ed8b64bc1a9b47e540e6f100f17ec6d97f1b baseline version: xen a75f

Re: [Xen-devel] [PATCH v7 14/16] public/hvm/params.h: Add macros for HVM_PARAM_CALLBACK_TYPE_PPI

2016-09-28 Thread Jan Beulich
>>> On 27.09.16 at 17:27, wrote: > --- a/xen/include/public/hvm/params.h > +++ b/xen/include/public/hvm/params.h > @@ -30,6 +30,7 @@ > */ > > #define HVM_PARAM_CALLBACK_IRQ 0 > +#define HVM_PARAM_CALLBACK_IRQ_TYPE_MASK 0xFF00ULL Well, I have to admit that I'm somewhat disappointe

Re: [Xen-devel] [PATCH] pvgrub: use printk() instead of grub_printf()

2016-09-28 Thread Samuel Thibault
Juergen Gross, on Wed 28 Sep 2016 10:31:58 +0200, wrote: > grub_printf() is supporting only a very limited number of formats. > Especially some error messages suffer from that, e.g. %lx won't work. > Switch to use printk() for error messages instead. > > Signed-off-by: Juergen Gross Acked-by: Sa

Re: [Xen-devel] [PATCH] pvgrub: use printk() instead of grub_printf()

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 10:31:58AM +0200, Juergen Gross wrote: > grub_printf() is supporting only a very limited number of formats. > Especially some error messages suffer from that, e.g. %lx won't work. > Switch to use printk() for error messages instead. > > Signed-off-by: Juergen Gross Acked

Re: [Xen-devel] [PATCH v1 08/12] tmem: Handle 'struct tmem_info' as a seperate field in the

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:22AM -0400, Konrad Rzeszutek Wilk wrote: > the struct tmem_op. > > No functional change. But it makes the code so much easier > to read. > > Note: We still have to do this awkward 'guest_handle_cast' > otherwise it will not compile on ARM - which defines _two_ > of t

Re: [Xen-devel] [PATCH v1 02/12] tmem: Retire XEN_SYSCTL_TMEM_OP_[SET_CAP|SAVE_GET_CLIENT_CAP]

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:16AM -0400, Konrad Rzeszutek Wilk wrote: > It is not used by anything. > > Signed-off-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu > diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h > index 8197c14..847ec35 100644 > --- a/xen/include/public/sy

Re: [Xen-devel] [PATCH v1 01/12] libxc/tmem/restore: Remove call to XEN_SYSCTL_TMEM_OP_SAVE_GET_VERSION

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:15AM -0400, Konrad Rzeszutek Wilk wrote: > The only thing this hypercall returns is TMEM_SPEC_VERSION. > > The comment around is also misleading - this call does not > do any domain operation. > > Signed-off-by: Konrad Rzeszutek Wilk Acked-by: Wei Liu ___

Re: [Xen-devel] [PATCH v1 09/12] tmem: Check version and maxpools when XEN_SYSCTL_TMEM_SET_CLIENT_INFO

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:23AM -0400, Konrad Rzeszutek Wilk wrote: > is called. If they are different from what the hypervisor > can support we will get the appropiate errors. > > Signed-off-by: Konrad Rzeszutek Wilk > > --- > Cc: Ian Jackson > Cc: Wei Liu > > v1: First submission. > ---

Re: [Xen-devel] [PATCH] pvgrub: fix crash when booting kernel with p2m list outside kernel mapping

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 06:02:44AM +0200, Juergen Gross wrote: > When trying to boot a kernel with the p2m list not mapped by the > initial kernel mapping it can happen that pvgrub is failing as it is > keeping some page tables mapped. > > Unmap the additional page tables created for the special p

Re: [Xen-devel] [PATCH v1 07/12] tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE]..

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:21AM -0400, Konrad Rzeszutek Wilk wrote: > Specifically: > > XEN_SYSCTL_TMEM_OP_SET_[WEIGHT,COMPRESS] are now done via: > > XEN_SYSCTL_TMEM_SET_CLIENT_INFO > > and XEN_SYSCTL_TMEM_OP_SAVE_GET_[VERSION,MAXPOOLS, > CLIENT_WEIGHT, CLIENT_FLAGS] can now be retrieved vi

Re: [Xen-devel] [PATCH v1 10/12] tmem: Unify XEN_SYSCTL_TMEM_OP_[[SAVE_[BEGIN|END]|RESTORE_BEGIN]

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:24AM -0400, Konrad Rzeszutek Wilk wrote: > return values. For success they used to be 1 ([SAVE,RESTORE]_BEGIN), > 0 if guest did not have any tmem (but only for SAVE_BEGIN), and > -1 for any type of failure. > > And SAVE_END (which you would think would mirror SAVE_BE

Re: [Xen-devel] [PATCH v1 12/12] tmem: Batch and squash XEN_SYSCTL_TMEM_OP_SAVE_GET_POOL_[FLAGS, NPAGES, UUID] in one sub-call: XEN_SYSCTL_TMEM_OP_GET_POOLS.

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:26AM -0400, Konrad Rzeszutek Wilk wrote: > These operations are used during the save process of migration. > Instead of doing 64 hypercalls lets do just one. We modify > the 'struct tmem_client' structure (used in > XEN_SYSCTL_TMEM_OP_[GET|SET]_CLIENT_INFO) to have an

Re: [Xen-devel] [PATCH v1 11/12] tmem/xc_tmem_control: Rename 'arg1' to 'len' and 'arg2' to arg.

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 05:42:25AM -0400, Konrad Rzeszutek Wilk wrote: > That is what they are used for. Lets make it more clear. > > Of all the various sub-commands, the only one that needed > semantic change is XEN_SYSCTL_TMEM_OP_SAVE_BEGIN. That in the > past used 'arg1', and now we are moving

Re: [Xen-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Samuel Thibault
Juergen Gross, on Wed 28 Sep 2016 11:50:04 +0200, wrote: > Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds > with uncommon config settings") introduced a bug which will break > the build, as xc_minios.c will have defined those macros via > xc_private.h. > > Fix this by defini

Re: [Xen-devel] [Minios-devel] [PATCH] minios: fix build issue with xen_*mb defines

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 01:07:36PM +0200, Samuel Thibault wrote: > Juergen Gross, on Wed 28 Sep 2016 11:50:04 +0200, wrote: > > Commit e35295d724d64e6e025ec31c2e510e448b8641d1 ("mini-os: fix builds > > with uncommon config settings") introduced a bug which will break > > the build, as xc_minios.c w

[Xen-devel] [qemu-mainline test] 101177: tolerable FAIL - PUSHED

2016-09-28 Thread osstest service owner
flight 101177 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/101177/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop fail like 101172 test-amd64-amd64-xl-rtds

Re: [Xen-devel] [PATCH v2 13/30] xen: introduce a new format specifier to print sizes in human-readable form

2016-09-28 Thread Roger Pau Monne
On Wed, Sep 28, 2016 at 10:24:07AM +0200, Juergen Gross wrote: > On 27/09/16 17:57, Roger Pau Monne wrote: > > Introduce a new %pB format specifier to print sizes (in bytes) in a > > Code suggests it is pZ. Right, first implementation used pB, but then I realized this was already used by Linux t

[Xen-devel] [xen-unstable-smoke test] 101183: tolerable all pass - PUSHED

2016-09-28 Thread osstest service owner
flight 101183 xen-unstable-smoke real [real] http://logs.test-lab.xenproject.org/osstest/logs/101183/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-libvirt 12 migrate-support-checkfail never pass test-armhf-armhf-xl 1

[Xen-devel] [PATCH] pub-headers: reduce C99 dependencies

2016-09-28 Thread Jan Beulich
For consumers not using (fully) C99-aware compilers, limit the number of places where tweaking of the headers would be necessary: Introduce and use xen_mk_ullong(), allowing its helper macro to be overridden at once. For now don't touch public/io/, which also has a few offenders. The need to incl

Re: [Xen-devel] [PATCH v2 13/30] xen: introduce a new format specifier to print sizes in human-readable form

2016-09-28 Thread Andrew Cooper
On 28/09/16 12:56, Roger Pau Monne wrote: > On Wed, Sep 28, 2016 at 10:24:07AM +0200, Juergen Gross wrote: >> On 27/09/16 17:57, Roger Pau Monne wrote: >>> Introduce a new %pB format specifier to print sizes (in bytes) in a >> Code suggests it is pZ. > Right, first implementation used pB, but then

Re: [Xen-devel] [PATCH v1 02/12] tmem: Retire XEN_SYSCTL_TMEM_OP_[SET_CAP|SAVE_GET_CLIENT_CAP]

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > It is not used by anything. But that shouldn't be the only aspect. Are they also not useful for anything? > --- a/xen/common/tmem_control.c > +++ b/xen/common/tmem_control.c > @@ -103,9 +103,9 @@ static int tmemc_list_client(struct client *c, > tmem_cli_va_par

Re: [Xen-devel] [PATCH v1 03/12] tmem: Wrap tmem dedup code with CONFIG_TMEM_DEDUP

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > This config is not defined anywhere but it makes it way > easier to figure out what code to deal with. And the functionality thus removed is - dead code, - broken, - anything else? > --- a/xen/common/tmem.c > +++ b/xen/common/tmem.c > @@ -71,10 +71,14 @@ struct

Re: [Xen-devel] [PATCH v1 04/12] tmem: Wrap tmem tze code with CONFIG_TMEM_TZE

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > . which is actually dependent on CONFIG_TMEM_DEDUP Same comments as for patch 3. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 00/30] PVHv2 Dom0

2016-09-28 Thread Roger Pau Monne
On Tue, Sep 27, 2016 at 05:56:55PM +0200, Roger Pau Monne wrote: > Hello, > > This is the first "complete" PVHv2 implementation in the sense that > it has feature parity with classic PVH Dom0. It is still very experimental, > but I've managed to boot it on all Intel boxes I've tried (OK, only 3 so

Re: [Xen-devel] [PATCH v1 05/12] tmem: Delete deduplication (and tze) code.

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > Couple of reasons: > - It can lead to security issues (see row-hammer, KSM and such >attacks). > - Code is quite complex. > - Deduplication is good if the pages themselves are the same >but that is hardly guaranteed. > - We got some gains (if pages ar

Re: [Xen-devel] [PATCH v1 06/12] tmem: Move client weight, frozen, live_migrating, and compress

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > --- a/xen/include/public/sysctl.h > +++ b/xen/include/public/sysctl.h > @@ -785,6 +785,20 @@ struct tmem_handle { > xen_tmem_oid_t oid; > }; > > +struct tmem_client { I guess there's a reason for this to get added to the public header, but if it really i

Re: [Xen-devel] [PATCH v1 07/12] tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE]..

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > Hypervisor wise we had to do a bit of casting. The > 'struct xen_sysctl_tmem_op'->buf variable is a pointer to > char. Casting that using the guest_handle_cast to a structure > (struct tmem_client) does not work. Hence we first have to > cast it a void and then t

Re: [Xen-devel] [PATCH v7] acpi: Prevent GPL-only code from seeping into non-GPL binaries

2016-09-28 Thread Boris Ostrovsky
On 09/28/2016 06:38 AM, Jan Beulich wrote: On 27.09.16 at 16:46, wrote: >> --- a/tools/firmware/hvmloader/acpi/Makefile >> +++ b/tools/firmware/hvmloader/acpi/Makefile >> @@ -17,7 +17,8 @@ >> XEN_ROOT = $(CURDIR)/../../../.. >> include $(XEN_ROOT)/tools/firmware/Rules.mk >> >> -C_SRC = bu

Re: [Xen-devel] [PATCH v7] acpi: Prevent GPL-only code from seeping into non-GPL binaries

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 08:53:41AM -0400, Boris Ostrovsky wrote: > On 09/28/2016 06:38 AM, Jan Beulich wrote: > On 27.09.16 at 16:46, wrote: > >> --- a/tools/firmware/hvmloader/acpi/Makefile > >> +++ b/tools/firmware/hvmloader/acpi/Makefile > >> @@ -17,7 +17,8 @@ > >> XEN_ROOT = $(CURDIR)/..

Re: [Xen-devel] [PATCH v1 08/12] tmem: Handle 'struct tmem_info' as a seperate field in the

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > Note: We still have to do this awkward 'guest_handle_cast' > otherwise it will not compile on ARM - which defines _two_ > of these macros (__guest_handle_64_xen_sysctl_tmem_client_t > and __guest_handle_xen_sysctl_tmem_client_t). And if cast is > not used then a

Re: [Xen-devel] [PATCH v1 09/12] tmem: Check version and maxpools when XEN_SYSCTL_TMEM_SET_CLIENT_INFO

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > --- a/xen/common/tmem_control.c > +++ b/xen/common/tmem_control.c > @@ -270,6 +270,12 @@ static int __tmemc_set_var(struct client *client, > if ( copy_from_guest(&info, buf, 1) ) > return -EFAULT; > > +if ( info.version != TMEM_SPEC_VERSION )

Re: [Xen-devel] [PATCH v1 10/12] tmem: Unify XEN_SYSCTL_TMEM_OP_[[SAVE_[BEGIN|END]|RESTORE_BEGIN]

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > case XEN_SYSCTL_TMEM_OP_RESTORE_BEGIN: > if ( client == NULL && (client = client_create(cli_id)) != NULL ) > -return 1; > +rc = 0; > +else > +rc = -EEXIST; > break; I don't think the same error c

Re: [Xen-devel] [PATCH v7] acpi: Prevent GPL-only code from seeping into non-GPL binaries

2016-09-28 Thread Boris Ostrovsky
On 09/28/2016 08:53 AM, Wei Liu wrote: > On Wed, Sep 28, 2016 at 08:53:41AM -0400, Boris Ostrovsky wrote: >> On 09/28/2016 06:38 AM, Jan Beulich wrote: >> On 27.09.16 at 16:46, wrote: --- a/tools/firmware/hvmloader/acpi/Makefile +++ b/tools/firmware/hvmloader/acpi/Makefile @@ -1

Re: [Xen-devel] [PATCH v6 16/16] libxl/arm: Add the size of ACPI tables to maxmem

2016-09-28 Thread Shannon Zhao
On 2016/9/28 3:00, Wei Liu wrote: On Tue, Sep 27, 2016 at 11:43:38AM -0700, Shannon Zhao wrote: On 2016/9/27 9:35, Wei Liu wrote: On Tue, Sep 27, 2016 at 09:01:00AM -0700, Shannon Zhao wrote: On 2016/9/27 2:41, Wei Liu wrote: On Mon, Sep 26, 2016 at 02:54:55PM -0700, Shannon Zhao wrote:

Re: [Xen-devel] [PATCH v1 12/12] tmem: Batch and squash XEN_SYSCTL_TMEM_OP_SAVE_GET_POOL_[FLAGS, NPAGES, UUID] in one sub-call: XEN_SYSCTL_TMEM_OP_GET_POOLS.

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 11:42, wrote: > @@ -1592,6 +1594,7 @@ static int do_tmem_new_pool(domid_t this_cli_id, > > out: > tmem_client_info("pool_id=%d\n", d_poolid); > +client->info.nr_pools ++; Stray blank? > --- a/xen/common/tmem_control.c > +++ b/xen/common/tmem_control.c > @@ -276,6 +

Re: [Xen-devel] [PATCH v6 16/16] libxl/arm: Add the size of ACPI tables to maxmem

2016-09-28 Thread Wei Liu
On Wed, Sep 28, 2016 at 06:11:53AM -0700, Shannon Zhao wrote: > > > On 2016/9/28 3:00, Wei Liu wrote: > >On Tue, Sep 27, 2016 at 11:43:38AM -0700, Shannon Zhao wrote: > >> > >> > >>On 2016/9/27 9:35, Wei Liu wrote: > >>>On Tue, Sep 27, 2016 at 09:01:00AM -0700, Shannon Zhao wrote: > > >

Re: [Xen-devel] [PATCH] pub-headers: reduce C99 dependencies

2016-09-28 Thread George Dunlap
On 28/09/16 13:00, Jan Beulich wrote: > For consumers not using (fully) C99-aware compilers, limit the number > of places where tweaking of the headers would be necessary: Introduce > and use xen_mk_ullong(), allowing its helper macro to be overridden at > once. I'm just curious: Did you actually

[Xen-devel] [xtf test] 101185: all pass - PUSHED

2016-09-28 Thread osstest service owner
flight 101185 xtf real [real] http://logs.test-lab.xenproject.org/osstest/logs/101185/ Perfect :-) All tests in this flight passed as required version targeted for testing: xtf 962dd4b225c17ae48d36a978602b245dbf77bec5 baseline version: xtf efe6103963c6756f5c7b77

[Xen-devel] [ovmf baseline-only test] 67778: all pass

2016-09-28 Thread Platform Team regression test user
This run is configured for baseline tests only. flight 67778 ovmf real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/67778/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf ab970515d2c6ec657fceab0ce571054bb43a22f2 baseline v

Re: [Xen-devel] [PATCH] pub-headers: reduce C99 dependencies

2016-09-28 Thread Jan Beulich
>>> On 28.09.16 at 15:31, wrote: > On 28/09/16 13:00, Jan Beulich wrote: >> For consumers not using (fully) C99-aware compilers, limit the number >> of places where tweaking of the headers would be necessary: Introduce >> and use xen_mk_ullong(), allowing its helper macro to be overridden at >> on

  1   2   3   >