Re: [PATCH v2] xen/misra: add rules 1.4 and 2.1

2023-06-16 Thread Luca Fancellu


> On 15 Jun 2023, at 22:27, Stefano Stabellini  wrote:
> 
> From: Stefano Stabellini 
> 
> Also add a comment at the top of the file to say rules.rst could be
> changed.
> 
> Signed-off-by: Stefano Stabellini 

Hi Stefano,

Reviewed-by: Luca Fancellu 


While I was testing the patch with our script that translates the docs to 
cppcheck
Inputs, I noticed we might have a small issue there, seems that Directives and 
Rules
clashes, and from a quick look to cppcheck addon, seems that only the rules are 
needed.

I’ll have a look on that soon.

> 
> ---
> Changes in v2:
> - add link for 1.4
> - expand 1.4 comment to say it could be revisited
> - add comment at the top
> ---
> docs/misra/rules.rst | 15 +++
> 1 file changed, 15 insertions(+)
> 
> diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> index a88c284e7d..11b9c42b70 100644
> --- a/docs/misra/rules.rst
> +++ b/docs/misra/rules.rst
> @@ -32,6 +32,9 @@ violations are meant to be documented as deviations, while 
> some others
> should be fixed. Both compliance and documenting deviations on the
> existing codebase are work-in-progress.
> 
> +The list below might need to be updated over time. Reach out to THE REST
> +maintainers if you want to suggest a change.
> +
> .. list-table::
>:header-rows: 1
> 
> @@ -90,6 +93,18 @@ existing codebase are work-in-progress.
>behaviour
>  -
> 
> +   * - `Rule 1.4 
> `_
> + - Required
> + - Emergent language features shall not be used
> + - Emergent language features, such as C11 features, should not be
> +   confused with similar compiler extensions, which we use. When the
> +   time comes to adopt C11, this rule will be revisited.
> +
> +   * - `Rule 2.1 
> `_
> + - Required
> + - A project shall not contain unreachable code
> + -
> +
>* - `Rule 2.6 
> `_
>  - Advisory
>  - A function should not contain unused label declarations
> -- 
> 2.25.1
> 
> 



Re: Refactoring of a possibly unsafe pattern for variable initialization via function calls

2023-06-16 Thread Jan Beulich
On 15.06.2023 18:39, nicola wrote:
> while investigating possible patches regarding Mandatory Rule 9.1, I
> found the following pattern, that is likely to results in a lot possible
> positives from many (all) static analysis tools for this rule.
> 
> This is the current status (taken from `xen/common/device_tree.c:135')
> 
> 
> const struct dt_property *dt_find_property(const struct dt_device_node *np,
> const char *name, u32 *lenp)
> {
>  const struct dt_property *pp;
> 
>  if ( !np )
>  return NULL;
> 
>  for ( pp = np->properties; pp; pp = pp->next )
>  {
>  if ( dt_prop_cmp(pp->name, name) == 0 )
>  {
>  if ( lenp )
>  *lenp = pp->length;
>  break;
>  }
>  }
> 
>  return pp;
> }
> 
> 
> 
> 
> It's very hard to detect that the pointee is always written whenever a 
> non-NULL pointer for `lenp' is supplied, and it can safely be read in 
> the callee, so a sound analysis will err on the cautious side.

I'm having trouble seeing why this is hard to recognize: The loop can
only be exited two ways: pp == NULL or with *lenp written.

For rule 9.1 I'd rather expect the scanning tool (and often the compiler)
to get into trouble with the NULL return value case, and *lenp not being
written yet apparently consumed in the caller. Then, however, ...

> My proposal, in a future patch, is to refactor these kinds of functions 
> as follows:
> 
> 
> const struct dt_property *dt_find_property(const struct dt_device_node *np,
> const char *name, u32 *lenp)
> {
>  u32 len = 0;
>  const struct dt_property *pp;
> 
>  if ( !np )
>  return NULL;

... this path would be a problem as well.

>  for ( pp = np->properties; pp; pp = pp->next )
>  {
>  if ( dt_prop_cmp(pp->name, name) == 0 )
>  {
>  len = pp->length;
>  break;
>  }
>  }
> 
>  if ( lenp )
>  *lenp = len;
>  return pp;
> }
> 
> 
> The advantage here is that we can easily argue that `*lenp' is always
> initialized by the function (if not NULL) and inform the tool about
> this, which is a safer API and also resolves almost all subsequent
> "don't know"s about further uses of the variables involved (e.g. `lenp').

The disadvantage is that in a more complex case and with the function
e.g. being static, the initializer of "len" may prevent compiler /
tools from spotting cases where the variable would (otherwise) truly
(and wrongly) remain uninitialized (and that fact propagating up the
call chain, through - in this example - whatever variable's address
the caller passed for "lenp"). IOW - I don't think a common pattern
can be agreed upon up front for cases like this one.

Jan



Re: [XEN PATCH] docs/misra: document the C dialect and translation toolchain assumptions.

2023-06-16 Thread Roberto Bagnara

On 16/06/23 08:53, Jan Beulich wrote:

On 16.06.2023 01:26, Stefano Stabellini wrote:

On Thu, 15 Jun 2023, Roberto Bagnara wrote:
I have a few comments below, mostly to clarify the description of some
of the less documented GCC extensions, for the purpose of having all
community members be able to understand what they can and cannot use.


What do you mean by "can and cannot use"? Is this document intended to
forbid the use of any extensions we may not currently use, or we use
but which aren't enumerated here?

One of the reasons that kept me from replying to this submission is
that the full purpose of this new doc isn't stated in the description.


My full purpose was to give the community a starting point for the
discussion on the assumptions the project makes on the programming
language and the translation toolchains that are intended to be used
now or in the future.  As far as I know, no documentation is currently
provided on these topics, so I believe the document fills a gap and
I hope it is good enough as a starting point.


Which in turn leaves open whether certain items actually need to be
here (see e.g. the libc related remark below).


Because the analyzed build used to included some of the tools, which in turn
relied on libc for program termination.  Once confirmation is given
that the analyzed build is now what is intended, all references to
libc can be removed.


Another is that it's
hard to tell how to convince oneself of this being an exhaustive
enumeration. One extension we use extensively yet iirc is missing here
is omission of the middle operand of the ternary operator.


Not sure I understand: do you mean something different from the following
entry in the document?

   * - Binary conditional expression
 - ARM64, X86_64
 - See Section "6.8 Conditionals with Omitted Operands" of GCC_MANUAL.



+Reference Documentation
+___
+
+The following documents are referred to in the sequel:
+
+GCC_MANUAL:
+  https://gcc.gnu.org/onlinedocs/gcc-12.1.0/gcc.pdf
+CPP_MANUAL:
+  https://gcc.gnu.org/onlinedocs/gcc-12.1.0/cpp.pdf


Why 12.1 when meanwhile there's 12.3 and 13.1?


For no special reason: as I said, my purpose is only to provide
a starting point for discussion and customization of the
assumptions.


+ARM64_ABI_MANUAL:
+  
https://github.com/ARM-software/abi-aa/blob/60a8eb8c55e999d74dac5e368fc9d7e36e38dda4/aapcs64/aapcs64.rst
+X86_64_ABI_MANUAL:
+  
https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build
+ARM64_LIBC_MANUAL:
+  https://www.gnu.org/software/libc/manual/pdf/libc.pdf
+X86_64_LIBC_MANUAL:
+  https://www.gnu.org/software/libc/manual/pdf/libc.pdf


How is libc relevant to the hypervisor?


See above.


+   * - Empty declaration
+ - ARM64, X86_64
+ - Non-documented GCC extension.


For the non-documented GCC extensions, would it be possible to add a
very brief example or a couple of words in the "References" sections?
Otherwise I think people might not understand what we are talking about.

For instance in this case I would say:

An empty declaration is a semicolon with nothing before it.
Non-documented GCC extension.


Which then could be confused with empty statements. I think in a document
like this language needs to be very precise, to avoid ambiguities and
confusion as much as possible. (Iirc from going over this doc yesterday
this applies elsewhere as well.)


OK.


+   * - Ill-formed source detected by the parser


As we are documenting compiler extensions that we are using, I am a bit
confused by the name of this category of compiler extensions, and the
reason why they are bundled together. After all, they are all separate
compiler extensions? Should each of them have their own row?


+1


OK.


+
+   * - Unspecified escape sequence is encountered in a character constant or a 
string literal token
+ - X86_64
+ - \\m:
+  non-documented GCC extension.


Are you saying that we are using \m and \m is not allowed by the C
standard?


This exists in the __ASSEMBLY__ part of a header, and I had previously
commented on Roberto's diagnosis (possibly derived from Eclair's) here.
As per that I don't think the item should be here, but I'm of course
open to be shown that my understanding of translation phases is wrong.


I was not convinced by your explanation but, as I think I have said already,
I am not the one to be convinced.  In the specific case, independently
from __ASSEMBLY__ or any other considerations, that thing reaches the C
preprocessor and, to the best of my knowledge, the C preprocessor documentation
does not say how that would be handled.  I have spent a lot of time in the
past 10 years on the study of functional-safety standards, and what I
am providing is a honest opinion on what I believe is compliant
and what is not.  But I may be wrong of course: if you or anyone else feels
like they would not have any problems in arguing a different position
from mine in front of an assesso

Re: [XEN v8 1/5] xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64

2023-06-16 Thread Michal Orzel
Hi Julien,

On 15/06/2023 22:32, Julien Grall wrote:
> 
> 
> Hi Michal,
> 
> I notice you posted some comments but didn't add a Acked-by/Reviewed-by.
> Can you indicate if you are happy with the patch so long your comments
> are addressed?
> 
> If so, are you OK if I deal with them on commit?
I thought I added my tag but clearly not. With the remarks addressed on commit:
Reviewed-by: Michal Orzel 

~Michal



Re: Refactoring of a possibly unsafe pattern for variable initialization via function calls

2023-06-16 Thread Nicola Vetrini




On 16/06/23 09:19, Jan Beulich wrote:

On 15.06.2023 18:39, nicola wrote:

while investigating possible patches regarding Mandatory Rule 9.1, I
found the following pattern, that is likely to results in a lot possible
positives from many (all) static analysis tools for this rule.

This is the current status (taken from `xen/common/device_tree.c:135')


const struct dt_property *dt_find_property(const struct dt_device_node *np,
 const char *name, u32 *lenp)
{
  const struct dt_property *pp;

  if ( !np )
  return NULL;

  for ( pp = np->properties; pp; pp = pp->next )
  {
  if ( dt_prop_cmp(pp->name, name) == 0 )
  {
  if ( lenp )
  *lenp = pp->length;
  break;
  }
  }

  return pp;
}




It's very hard to detect that the pointee is always written whenever a
non-NULL pointer for `lenp' is supplied, and it can safely be read in
the callee, so a sound analysis will err on the cautious side.


I'm having trouble seeing why this is hard to recognize: The loop can
only be exited two ways: pp == NULL or with *lenp written.

For rule 9.1 I'd rather expect the scanning tool (and often the compiler)
to get into trouble with the NULL return value case, and *lenp not being
written yet apparently consumed in the caller. Then, however, ...



You're right, I made a mistake, thank you for finding it.
I meant to write on `*lenp' in all execution paths.
Please, take a look at this revised version:


const struct dt_property *dt_find_property(const struct dt_device_node *np,
   const char *name, u32 *lenp)
{
u32 len = 0;
const struct dt_property *pp = NULL;

if ( np )
{
for ( pp = np->properties; pp; pp = pp->next )
{
if ( dt_prop_cmp(pp->name, name) == 0 )
{
len = pp->length;
break;
}
}
}

if ( lenp )
*lenp = len;
return pp;
}


My proposal, in a future patch, is to refactor these kinds of functions 
as follows:



const struct dt_property *dt_find_property(const struct dt_device_node *np,
const char *name, u32 *lenp)
{
 u32 len = 0;
 const struct dt_property *pp;

 if ( !np )
 return NULL;


... this path would be a problem as well.


 for ( pp = np->properties; pp; pp = pp->next )
 {
 if ( dt_prop_cmp(pp->name, name) == 0 )
 {
 len = pp->length;
 break;
 }
 }

 if ( lenp )
 *lenp = len;
 return pp;
}


The advantage here is that we can easily argue that `*lenp' is always
initialized by the function (if not NULL) and inform the tool about
this, which is a safer API and also resolves almost all subsequent
"don't know"s about further uses of the variables involved (e.g. `lenp')

The disadvantage is that in a more complex case and with the function
e.g. being static, the initializer of "len" may prevent compiler /
tools from spotting cases where the variable would (otherwise) truly
(and wrongly) remain uninitialized (and that fact propagating up the
call chain, through - in this example - whatever variable's address
the caller passed for "lenp"). IOW - I don't think a common pattern
can be agreed upon up front for cases like this one.



That's ok, but perhaps we can agree that in a subset of functions as 
simple as this one the refactoring can help both developers and tools.


--
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)



Re: compat code lacks support for __attribute__

2023-06-16 Thread Olaf Hering
Wed, 14 Jun 2023 11:49:35 +0200 Jan Beulich :

> However, if you're after adding packed attributes, and if you're
> meaning to only communicate between Xen and the tool stack, then
> the requirement above doesn't exist. Yet then I would also wonder
> whether you need any compat translation in the first place. Those
> packed structures would better be arch-/bitness-agnostic, wouldn't
> they? So perhaps we could arrange for your additions to be excluded
> from the compat translation machinery?

The change below works for me. I wonder if any special compat handling
for t_buf and t_rec is required. To me it looks like only uint32_t is
used, which will most likely cause no unexpected alignment issues.


Olaf

--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -34,15 +34,6 @@
 #include 
 #include 
 
-#ifdef CONFIG_COMPAT
-#include 
-#define xen_t_buf t_buf
-CHECK_t_buf;
-#undef xen_t_buf
-#else
-#define compat_t_rec t_rec
-#endif
-
 /* opt_tbuf_size: trace buffer size (in pages) for each cpu */
 static unsigned int opt_tbuf_size;
 static unsigned int opt_tevt_mask;
--- a/xen/include/Makefile
+++ b/xen/include/Makefile
@@ -30,7 +30,7 @@ headers-$(CONFIG_HVM) += compat/hvm/hvm_op.h
 headers-$(CONFIG_HVM) += compat/hvm/hvm_vcpu.h
 headers-$(CONFIG_HYPFS)   += compat/hypfs.h
 headers-$(CONFIG_KEXEC)   += compat/kexec.h
-headers-$(CONFIG_TRACEBUFFER) += compat/trace.h
+headers-n += compat/trace.h
 headers-$(CONFIG_XENOPROF) += compat/xenoprof.h
 headers-$(CONFIG_XSM_FLASK) += compat/xsm/flask_op.h
 
--- a/xen/include/public/trace.h
+++ b/xen/include/public/trace.h
@@ -81,6 +81,15 @@
 #define TRC_TRACE_WRAP_BUFFER  (TRC_GEN + 2)
 #define TRC_TRACE_CPU_CHANGE(TRC_GEN + 3)
 
+#define TRC_a (TRC_GEN + 123)
+struct __attribute__((__packed__)) trc_a {
+unsigned a;
+};
+#define TRC_b (TRC_GEN + 321)
+typedef struct __attribute__((__packed__)) trc_b {
+unsigned b;
+} trc_b_t;
+
 #define TRC_SCHED_RUNSTATE_CHANGE   (TRC_SCHED_MIN + 1)
 #define TRC_SCHED_CONTINUE_RUNNING  (TRC_SCHED_MIN + 2)
 #define TRC_SCHED_DOM_ADD(TRC_SCHED_VERBOSE +  1)


pgpkpVFqhWdPr.pgp
Description: Digitale Signatur von OpenPGP


Re: [XEN PATCH] docs/misra: document the C dialect and translation toolchain assumptions.

2023-06-16 Thread Jan Beulich
On 16.06.2023 09:45, Roberto Bagnara wrote:
> On 16/06/23 08:53, Jan Beulich wrote:
>> On 16.06.2023 01:26, Stefano Stabellini wrote:
>> Another is that it's
>> hard to tell how to convince oneself of this being an exhaustive
>> enumeration. One extension we use extensively yet iirc is missing here
>> is omission of the middle operand of the ternary operator.
> 
> Not sure I understand: do you mean something different from the following
> entry in the document?
> 
> * - Binary conditional expression
>   - ARM64, X86_64
>   - See Section "6.8 Conditionals with Omitted Operands" of GCC_MANUAL.

Ah, yes, that is it. I find gcc's title misleading (because there are
far more "conditionals" than just the ternary operator), and hence
when going through you doc I didn't spot this (I'm sorry), and when
then searching for "ternary" and "?:" there were no hits.

 +   * - Unspecified escape sequence is encountered in a character constant 
 or a string literal token
 + - X86_64
 + - \\m:
 +  non-documented GCC extension.
>>>
>>> Are you saying that we are using \m and \m is not allowed by the C
>>> standard?
>>
>> This exists in the __ASSEMBLY__ part of a header, and I had previously
>> commented on Roberto's diagnosis (possibly derived from Eclair's) here.
>> As per that I don't think the item should be here, but I'm of course
>> open to be shown that my understanding of translation phases is wrong.
> 
> I was not convinced by your explanation but, as I think I have said already,
> I am not the one to be convinced.  In the specific case, independently
> from __ASSEMBLY__ or any other considerations, that thing reaches the C
> preprocessor and, to the best of my knowledge, the C preprocessor 
> documentation
> does not say how that would be handled.  I have spent a lot of time in the
> past 10 years on the study of functional-safety standards, and what I
> am providing is a honest opinion on what I believe is compliant
> and what is not.  But I may be wrong of course: if you or anyone else feels
> like they would not have any problems in arguing a different position
> from mine in front of an assessor, then please go for it, but please
> do not ask me to go beyond my judgment.

Well, disagreement on purely a technical matter can usually be resolved,
unless something is truly unspecified. Since you referred to translation
phases, and since I pointed out that preprocessing directives are carried
out before escape sequences are converted to the execution character set
(which is the point where unknown escape sequences would matter afaict),
there must be something you view differently in this process. It would be
helpful if you could point out what this is, possibly leading to me
recognizing a mistake of mine.

Actually, maybe I figured what you're concerned about: Already at the
stage of decomposing into preprocessing-token-s there is an issue, as
e.g. "\mode" doesn't form a valid string-literal. For other, unquoted
\m I would assume though that the final "each non-white-space character
that cannot be one of the above" (in the enumeration of what a
preprocessing-token is) would catch it.

Furthermore it is entirely unclear to me what it is that you suggest we
do instead. It can't reasonably be "name all you assembler macro
parameters such that they start with a, b, f, n, r, t, or v". Splitting
headers also wouldn't be very nice - we try to keep related things
together, after all. It also doesn't look like __stringify(\mode) would
be okay, as macro expansion shares a translation phase with execution
of preprocessing directives (so in principle the body of "#if 0" could
be macro-expanded before being discarded). (Plus I think this would
result in "\\mode", i.e. also wouldn't work in the first place. But it
would rule out other possible C macro trickery as well.)

Jan



Re: compat code lacks support for __attribute__

2023-06-16 Thread Jan Beulich
On 16.06.2023 11:51, Olaf Hering wrote:
> Wed, 14 Jun 2023 11:49:35 +0200 Jan Beulich :
> 
>> However, if you're after adding packed attributes, and if you're
>> meaning to only communicate between Xen and the tool stack, then
>> the requirement above doesn't exist. Yet then I would also wonder
>> whether you need any compat translation in the first place. Those
>> packed structures would better be arch-/bitness-agnostic, wouldn't
>> they? So perhaps we could arrange for your additions to be excluded
>> from the compat translation machinery?
> 
> The change below works for me. I wonder if any special compat handling
> for t_buf and t_rec is required. To me it looks like only uint32_t is
> used, which will most likely cause no unexpected alignment issues.

No special handling is needed, indeed, but ...

> --- a/xen/common/trace.c
> +++ b/xen/common/trace.c
> @@ -34,15 +34,6 @@
>  #include 
>  #include 
>  
> -#ifdef CONFIG_COMPAT
> -#include 
> -#define xen_t_buf t_buf
> -CHECK_t_buf;

... you're removing the line that's actually verifying this is the case.

Jan



Re: compat code lacks support for __attribute__

2023-06-16 Thread Olaf Hering
Fri, 16 Jun 2023 12:07:20 +0200 Jan Beulich :

> ... you're removing the line that's actually verifying this is the case.

Yeah, because it disappeared. I think the other approach is to teach
the python tool about __attribute__(()).

Let me know which way is preferred.


Olaf


pgpghOyGFRHEO.pgp
Description: Digitale Signatur von OpenPGP


[XEN PATCH] Config.mk: update OVMF to edk2-stable202305

2023-06-16 Thread Anthony PERARD
Update to OVMF's latest stable tag.

This is been prompt by trying to build Xen on Debian Bookworm,
where edk2-stable202108 doesn't build. Also, it's been too long since
the last update.

Signed-off-by: Anthony PERARD 
---

I've tested it in osstest, so the update should be fine:
http://logs.test-lab.xenproject.org/osstest/logs/181458/
---
 Config.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Config.mk b/Config.mk
index d12d4c2b8f..c529b1ba19 100644
--- a/Config.mk
+++ b/Config.mk
@@ -216,7 +216,7 @@ QEMU_TRADITIONAL_REVISION ?= $(QEMU_TAG)
 endif
 
 OVMF_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/ovmf.git
-OVMF_UPSTREAM_REVISION ?= 7b4a99be8a39c12d3a7fc4b8db9f0eab4ac688d5
+OVMF_UPSTREAM_REVISION ?= ba91d0292e593df8528b66f99c1b0b14fadc8e16
 
 QEMU_UPSTREAM_URL ?= https://xenbits.xen.org/git-http/qemu-xen.git
 QEMU_UPSTREAM_REVISION ?= master
-- 
Anthony PERARD




[linux-linus test] 181456: regressions - FAIL

2023-06-16 Thread osstest service owner
flight 181456 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181456/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-pvops 6 kernel-build fail REGR. vs. 180278
 build-armhf-pvops 6 kernel-build fail REGR. vs. 180278

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-examine  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-examine  1 build-check(1)   blocked  n/a
 test-amd64-amd64-examine-bios  1 build-check(1)   blocked  n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-freebsd11-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-freebsd12-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-

Re: compat code lacks support for __attribute__

2023-06-16 Thread Jan Beulich
On 16.06.2023 12:22, Olaf Hering wrote:
> Fri, 16 Jun 2023 12:07:20 +0200 Jan Beulich :
> 
>> ... you're removing the line that's actually verifying this is the case.
> 
> Yeah, because it disappeared. I think the other approach is to teach
> the python tool about __attribute__(()).
> 
> Let me know which way is preferred.

For the moment neither, I'm afraid. I don't know about other maintainers,
but I at least would first like to see the gain. And if there is agreement
that that's worth it, we could then try to determine what price we're
willing to pay.

Jan




Re: [XEN PATCH] Config.mk: update OVMF to edk2-stable202305

2023-06-16 Thread Jan Beulich
On 16.06.2023 12:29, Anthony PERARD wrote:
> Update to OVMF's latest stable tag.
> 
> This is been prompt by trying to build Xen on Debian Bookworm,
> where edk2-stable202108 doesn't build. Also, it's been too long since
> the last update.

Hmm, yes, almost 2 years.

> Signed-off-by: Anthony PERARD 

Acked-by: Jan Beulich 





Re: xentrace buffer size, maxcpus and online cpus

2023-06-16 Thread Olaf Hering
Wed, 31 May 2023 11:05:52 +0200 Jan Beulich :

> As said before, num_online_cpus() will under-report for the purpose
> here, as CPUs may have been brought offline, and may be brought online
> again later (independent of the use of "maxcpus=").

It turned out, commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39 broke
the expected behavior. But to me it is unclear what bug was fixed by
this commit.

If I read alloc_trace_bufs correctly, it already operates on online
cpus. And __trace_var will do nothing if called on a cpu which was
not online, t_bufs will likely be NULL.

To me it looks like commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39
could be reverted.


Olaf


pgpi3uO7iNlia.pgp
Description: Digitale Signatur von OpenPGP


Re: xentrace buffer size, maxcpus and online cpus

2023-06-16 Thread Jan Beulich
On 16.06.2023 13:47, Olaf Hering wrote:
> Wed, 31 May 2023 11:05:52 +0200 Jan Beulich :
> 
>> As said before, num_online_cpus() will under-report for the purpose
>> here, as CPUs may have been brought offline, and may be brought online
>> again later (independent of the use of "maxcpus=").
> 
> It turned out, commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39 broke
> the expected behavior. But to me it is unclear what bug was fixed by
> this commit.

Hmm, I find title and description quite clear there.

> If I read alloc_trace_bufs correctly, it already operates on online
> cpus. And __trace_var will do nothing if called on a cpu which was
> not online, t_bufs will likely be NULL.

Yielding an incomplete overall trace, at best.

> To me it looks like commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39
> could be reverted.

I don't think so. I'll add George to Cc as well, as he's the maintainer
of this stuff.

Jan



Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies

2023-06-16 Thread Jan Beulich
On 15.06.2023 20:17, Andrew Cooper wrote:
> On 15/06/2023 1:13 pm, Jan Beulich wrote:
>> On 15.06.2023 12:41, Andrew Cooper wrote:
>>> On 15/06/2023 9:30 am, Jan Beulich wrote:
 On 14.06.2023 20:12, Andrew Cooper wrote:
> On 13/06/2023 10:59 am, Jan Beulich wrote:
>> On 12.06.2023 18:13, Andrew Cooper wrote:
>>> The RSBA bit, "RSB Alternative", means that the RSB may use alternative
>>> predictors when empty.  From a practical point of view, this mean 
>>> "Retpoline
>>> not safe".
>>>
>>> Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously 
>>> IBRS_ATT) is a
>>> statement that IBRS is implemented in hardware (as opposed to the form
>>> retrofitted to existing CPUs in microcode).
>>>
>>> The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the 
>>> eIBRS
>>> property that predictions are tagged with the mode in which they were 
>>> learnt.
>>> Therefore, it means "when eIBRS is active, the RSB may fall back to
>>> alternative predictors but restricted to the current prediction mode".  
>>> As
>>> such, it's stronger statement than RSBA, but still means "Retpoline not 
>>> safe".
>>>
>>> CPUs are not expected to enumerate both RSBA and RRSBA.
>>>
>>> Add feature dependencies for EIBRS and RRSBA.  While technically 
>>> they're not
>>> linked, absolutely nothing good can come of letting the guest see RRSBA
>>> without EIBRS.  Nor a guest seeing EIBRS without IBRSB.  Furthermore, 
>>> we use
>>> this dependency to simplify the max derivation logic.
>>>
>>> The max policies gets RSBA and RRSBA unconditionally set (with the EIBRS
>>> dependency maybe hiding RRSBA).  We can run any VM, even if it has been 
>>> told
>>> "somewhere you might run, Retpoline isn't safe".
>>>
>>> The default policies are more complicated.  A guest shouldn't see both 
>>> bits,
>>> but it needs to see one if the current host suffers from any form of 
>>> RSBA, and
>>> which bit it needs to see depends on whether eIBRS is visible or not.
>>> Therefore, the calculation must be performed after 
>>> sanitise_featureset().
>>>
>>> Signed-off-by: Andrew Cooper 
>>> ---
>>> CC: Jan Beulich 
>>> CC: Roger Pau Monné 
>>> CC: Wei Liu 
>>>
>>> v3:
>>>  * Minor commit message adjustment.
>>>  * Drop changes to recalculate_cpuid_policy().  Deferred to a later 
>>> series.
>> With this dropped, with the title not saying "max/default", and with
>> the description also not mentioning "live" policies at all, I don't
>> think this patch is self-consistent (meaning in particular: leaving
>> aside the fact that there's no way right now to requests e.g. both
>> RSBA and RRSBA for a guest; aiui it is possible for Dom0).
>>
>> As you may imagine I'm also curious why you decided to drop this.
> Because when I tried doing levelling in Xapi, I remembered why I did it
> the way I did in v1, and why the v2 way was wrong.
>
> Xen cannot safely edit what the toolstack provides, so must not. 
 And this is the part I don't understand: Why can't we correct the
 (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least
 as long as ...

> Instead, failing the set_policy() call is an option, and is what we want
> to do longterm,
 ... we aren't there.

> but also happens to be wrong too in this case. An admin
> may know that a VM isn't using retpoline, and may need to migrate it
> anyway for a number of reasons, so any safety checks need to be in the
> toolstack, and need to be overrideable with something like --force.
 Possibly leading to an inconsistent policy exposed to a guest? I
 guess this may be the only option when we can't really resolve an
 ambiguity, but that isn't the case here, is it?
>>> Wrong.  Xen does not have any knowledge of other hosts the VM might
>>> migrate to.
>>>
>>> So while Xen can spot problem combinations *on this host*, which way to
>>> correct the problem combination depends on where the VM might migrate to.
>> I actually view this as two different levels: With a flawed policy, the
>> guest is liable to not work correctly at all. No point thinking about
>> it being able to migrate. With a fixed up policy it may fail to migrate,
>> but it'll at least work otherwise.
> 
> If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in
> the guest, even if you migrate it.
> 
> The consequence of getting RSBA and/or RRSBA wrong is the guest *might*
> think retpoline is safe to use, and *might* end up vulnerable to
> speculative attacks on this or other hardware.

Isn't that some sort of "malfunction"?

> And the admin might know that they overrode the default settings and
> forced the use of some other protection mechanism, so the guest is in
> fact safe despite having wrong RSBA/RRSBA s

[PATCH] x86: fix expansion of %XV

2023-06-16 Thread Jan Beulich
Only %LV should continue on to S handling; avoid emitting a stray 'l'
(typically) in suffix-always mode.

--- a/opcodes/i386-dis.c
+++ b/opcodes/i386-dis.c
@@ -11067,19 +11067,20 @@ putop (instr_info *ins, const char *in_t
  *ins->obufp++ = ' ';
  break;
case 'L':
- if (!(ins->rex & REX_W))
-   break;
- *ins->obufp++ = 'a';
- *ins->obufp++ = 'b';
- *ins->obufp++ = 's';
- break;
+ if (ins->rex & REX_W)
+   {
+ *ins->obufp++ = 'a';
+ *ins->obufp++ = 'b';
+ *ins->obufp++ = 's';
+   }
+ goto case_S;
default:
  abort ();
}
}
  else
abort ();
- goto case_S;
+ break;
case 'W':
  if (l == 0)
{



Re: [PATCH] x86: fix expansion of %XV

2023-06-16 Thread Jan Beulich
On 16.06.2023 14:33, Jan Beulich wrote:
> Only %LV should continue on to S handling; avoid emitting a stray 'l'
> (typically) in suffix-always mode.

Oops, I'm sorry, confusion of mailing lists.

Jan




Re: [PATCH v4 04/34] pgtable: Create struct ptdesc

2023-06-16 Thread Jason Gunthorpe
On Mon, Jun 12, 2023 at 02:03:53PM -0700, Vishal Moola (Oracle) wrote:
> Currently, page table information is stored within struct page. As part
> of simplifying struct page, create struct ptdesc for page table
> information.
> 
> Signed-off-by: Vishal Moola (Oracle) 
> ---
>  include/linux/pgtable.h | 51 +
>  1 file changed, 51 insertions(+)
> 
> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> index c5a51481bbb9..330de96ebfd6 100644
> --- a/include/linux/pgtable.h
> +++ b/include/linux/pgtable.h
> @@ -975,6 +975,57 @@ static inline void ptep_modify_prot_commit(struct 
> vm_area_struct *vma,
>  #endif /* __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION */
>  #endif /* CONFIG_MMU */
>  
> +
> +/**
> + * struct ptdesc - Memory descriptor for page tables.
> + * @__page_flags: Same as page flags. Unused for page tables.
> + * @pt_list: List of used page tables. Used for s390 and x86.
> + * @_pt_pad_1: Padding that aliases with page's compound head.
> + * @pmd_huge_pte: Protected by ptdesc->ptl, used for THPs.
> + * @_pt_s390_gaddr: Aliases with page's mapping. Used for s390 gmap only.
> + * @pt_mm: Used for x86 pgds.
> + * @pt_frag_refcount: For fragmented page table tracking. Powerpc and s390 
> only.
> + * @ptl: Lock for the page table.
> + *
> + * This struct overlays struct page for now. Do not modify without a good
> + * understanding of the issues.
> + */
> +struct ptdesc {
> + unsigned long __page_flags;
> +
> + union {
> + struct list_head pt_list;
> + struct {
> + unsigned long _pt_pad_1;
> + pgtable_t pmd_huge_pte;
> + };
> + };
> + unsigned long _pt_s390_gaddr;
> +
> + union {
> + struct mm_struct *pt_mm;
> + atomic_t pt_frag_refcount;
> + };
> +
> +#if ALLOC_SPLIT_PTLOCKS
> + spinlock_t *ptl;
> +#else
> + spinlock_t ptl;
> +#endif
> +};

I think you should include the memcg here too? It needs to be valid
for a ptdesc, even if we don't currently deref it through the ptdesc
type.

Also, do you see a way to someday put a 'struct rcu_head' into here?

Thanks,
Jason



[PATCH v1] xen/trace: remove duplicate check for tb_init_done

2023-06-16 Thread Olaf Hering
The single caller of next_record already checked for
tb_init_done. The functions are called with t_lock
held. The call stack look like this:

__trace_var
  tb_init_done?
  insert_wrap_record
  insert_lost_records
__insert_record
  next_record
tb_init_done?

Signed-off-by: Olaf Hering 
---
 xen/common/trace.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/common/trace.c b/xen/common/trace.c
index 582c58c884..8d56124ec9 100644
--- a/xen/common/trace.c
+++ b/xen/common/trace.c
@@ -514,7 +514,7 @@ static unsigned char *next_record(const struct t_buf *buf, 
uint32_t *next,
 
 barrier(); /* must read buf->prod and buf->cons only once */
 *next = x;
-if ( !tb_init_done || bogus(x, cons) )
+if ( bogus(x, cons) )
 return NULL;
 
 if ( x >= data_size )



[PATCH 00/13] lib{xc,xl}: support for guest MSR features

2023-06-16 Thread Roger Pau Monne
Hello,

The following series adds support for handling guest MSR features as
defined in arch-x86/cpufeatureset.h.

The end result is the user being able to use such features with the
xl.cfg(5) cpuid option.  This also involves adding support to all the
underlying layers, so both libxl and libxc also get new functionality in
order to properly parse those.

In order for such support to be as transparent as possible for existing
users of libxl, both libxl_cpuid_policy_list and libxl_cpuid_policy are
modified, so that the libxl_cpuid_policy_list type is no longer an array
anymore, and libxl_cpuid_policy is converted into a structure with
two fields to hold both a CPUID and MSR arrays.  It's my thinking that
current users of libxl had no business in poking at
libxl_cpuid_policy_list, since the underlying type (struct
xc_xend_cpuid) is opaque in that context.  Also the format of the array
(with a terminating empty element) is not documented in the public
headers.

Some of the patches had been salvaged from a previous series of mine to
introduce better cpu_policy management support in libxc and libxl, and
hence contain some version notes about changes, or are already reviewed.

Thanks, Roger.

Roger Pau Monne (13):
  libs/guest: simplify xc_cpuid_apply_policy()
  libx86: introduce helper to fetch cpuid leaf
  libs/guest: allow fetching a specific CPUID leaf from a cpu policy
  libx86: introduce helper to fetch msr entry
  libs/guest: allow fetching a specific MSR entry from a cpu policy
  libs/guest: replace usage of host featureset in
xc_cpuid_apply_policy()
  libs/guest: rework xc_cpuid_xend_policy
  libs/guest: introduce support for setting guest MSRs
  libxl: change the type of libxl_cpuid_policy_list
  libxl: introduce MSR data in libxl_cpuid_policy
  libxl: split logic to parse user provided CPUID features
  libxl: use the cpuid feature names from cpufeatureset.h
  libxl: add support for parsing MSR features

 docs/man/xl.cfg.5.pod.in |  24 +-
 tools/include/libxl.h|   7 +-
 tools/include/xenctrl.h  |  21 +-
 tools/include/xenguest.h |  13 +
 tools/libs/guest/xg_cpuid_x86.c  | 383 +
 tools/libs/light/gentest.py  |   2 +-
 tools/libs/light/libxl_cpuid.c   | 504 +++
 tools/tests/cpu-policy/test-cpu-policy.c | 220 +-
 tools/xl/xl_parse.c  |   3 +
 xen/arch/x86/cpuid.c |  55 +--
 xen/include/xen/lib/x86/cpu-policy.h |  37 +-
 xen/lib/x86/cpuid.c  |  52 +++
 xen/lib/x86/msr.c|  41 +-
 13 files changed, 821 insertions(+), 541 deletions(-)

-- 
2.40.0




[PATCH 02/13] libx86: introduce helper to fetch cpuid leaf

2023-06-16 Thread Roger Pau Monne
Introduce a helper based on the current Xen guest_cpuid code in order
to fetch a cpuid leaf from a policy. The newly introduced function in
cpuid.c should not be directly called and instead the provided
x86_cpuid_get_leaf macro should be used that will properly deal with
const and non-const inputs.

Also add a test to check that the introduced helper doesn't go over
the bounds of the policy.

Note the code in x86_cpuid_copy_from_buffer is not switched to use the
new function because of the boundary checks against the max fields of
the policy, which might not be properly set at the point where
x86_cpuid_copy_from_buffer get called, for example when filling an
empty policy from scratch.

Signed-off-by: Roger Pau Monné 
---
Changes since v6:
 - Add more tests.
 - Drop Jan R-b.

Changes since v4:
 - Rename _x86_cpuid_get_leaf to x86_cpuid_get_leaf_const.

Changes since v3:
 - New in this version.
---
 tools/tests/cpu-policy/test-cpu-policy.c | 177 +++
 xen/arch/x86/cpuid.c |  55 +--
 xen/include/xen/lib/x86/cpu-policy.h |  19 +++
 xen/lib/x86/cpuid.c  |  52 +++
 4 files changed, 255 insertions(+), 48 deletions(-)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c 
b/tools/tests/cpu-policy/test-cpu-policy.c
index 301df2c00285..a11c8f067aad 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -1,6 +1,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -565,6 +566,180 @@ static void test_cpuid_out_of_range_clearing(void)
 }
 }
 
+static void test_cpuid_get_leaf_failure(void)
+{
+static const struct test {
+struct cpu_policy p;
+const char *name;
+uint32_t leaf, subleaf;
+} tests[] = {
+/* Test for invalid configurations in the object itself. */
+{
+.name = "Basic max leaf >= array size",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC,
+},
+},
+{
+.name = "Feature max leaf >= array size",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+.feat.max_subleaf = CPUID_GUEST_NR_FEAT,
+},
+.leaf = 0x7,
+},
+{
+.name = "Extended max leaf >= array size",
+.p = {
+.extd.max_leaf = 0x8000 + CPUID_GUEST_NR_EXTD,
+},
+.leaf = 0x8000,
+},
+
+/* Test out-of-bounds checks in the accessor. */
+{
+.name = "Basic leaf >= max leaf",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+},
+.leaf = CPUID_GUEST_NR_BASIC,
+},
+{
+.name = "Cache leaf >= cache array size",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+},
+.leaf = 0x4,
+.subleaf = CPUID_GUEST_NR_CACHE,
+},
+{
+.name = "Feature leaf >= max leaf",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+.feat.max_subleaf = CPUID_GUEST_NR_FEAT - 1,
+},
+.leaf = 0x7,
+.subleaf = CPUID_GUEST_NR_FEAT,
+},
+{
+.name = "Extended Topo leaf >= cache array size",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+},
+.leaf = 0xb,
+.subleaf = CPUID_GUEST_NR_TOPO,
+},
+{
+.name = "Xstate leaf >= cache array size",
+.p = {
+.basic.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+},
+.leaf = 0xd,
+.subleaf = CPUID_GUEST_NR_XSTATE,
+},
+{
+.name = "Extended leaf >= max leaf",
+.p = {
+.extd.max_leaf = 0x8000 + CPUID_GUEST_NR_EXTD - 1,
+},
+.leaf = 0x8000 + CPUID_GUEST_NR_EXTD,
+},
+
+/* Test fetching Xsave without present. */
+{
+.name = "Fetch Xsave without present",
+.p = {
+.basic = {
+.max_leaf = CPUID_GUEST_NR_BASIC - 1,
+.xsave = false,
+},
+},
+.leaf = 0xd,
+},
+
+};
+const struct cpu_policy pc = {};
+const struct cpuid_leaf *lc;
+struct cpu_policy p = {};
+struct cpuid_leaf *l;
+
+/* Constness build test. */
+lc = x86_cpuid_get_leaf(&pc, 0, 0);
+l = x86_cpuid_get_leaf(&p, 0, 0);
+
+printf("Testing CPUID get leaf bound checking:\n");
+
+for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+{
+const struct test *t = &tests[i];
+const struct cpu_policy *p = memdup(&t->p);
+
+if ( x86_cpuid_get_leaf_const(p, t->leaf, t->subleaf) )
+fail("  Test %s get leaf fail\n", t->na

[PATCH 01/13] libs/guest: simplify xc_cpuid_apply_policy()

2023-06-16 Thread Roger Pau Monne
Introduce a local xc_cpu_policy_t and use it to simplify some of the
logic in the function:

 * Populate the introduced xc_cpu_policy_t with the current domain
   policy, which will already be the default for the given domain
   type.  This avoids fetching and processing any default policy.

 * Use xc_cpu_policy_set_domain() to apply the adjusted policy to the
   domain, thus avoiding open-coding the policy serialization.

Note that some error messages are removed, as the newly used
xc_cpu_policy_ functions already print an error message themselves.

No functional change intended.

Signed-off-by: Roger Pau Monné 
---
 tools/libs/guest/xg_cpuid_x86.c | 51 ++---
 1 file changed, 9 insertions(+), 42 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 5b035223f4f5..67e0dc9b4ad2 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -431,10 +431,9 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 int rc;
 bool hvm;
 xc_domaininfo_t di;
-unsigned int i, nr_leaves, nr_msrs;
-xen_cpuid_leaf_t *leaves = NULL;
-struct cpu_policy *p = NULL;
-uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
+unsigned int i;
+xc_cpu_policy_t *policy = NULL;
+struct cpu_policy *p;
 uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
 uint32_t len = ARRAY_SIZE(host_featureset);
 
@@ -446,17 +445,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 }
 hvm = di.flags & XEN_DOMINF_hvm_guest;
 
-rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-if ( rc )
-{
-PERROR("Failed to obtain policy info size");
-rc = -errno;
-goto out;
-}
-
 rc = -ENOMEM;
-if ( (leaves = calloc(nr_leaves, sizeof(*leaves))) == NULL ||
- (p = calloc(1, sizeof(*p))) == NULL )
+if ( (policy = xc_cpu_policy_init()) == NULL )
 goto out;
 
 /* Get the host policy. */
@@ -470,26 +460,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 goto out;
 }
 
-/* Get the domain's default policy. */
-nr_msrs = 0;
-rc = get_system_cpu_policy(xch, hvm ? XEN_SYSCTL_cpu_policy_hvm_default
-: XEN_SYSCTL_cpu_policy_pv_default,
-   &nr_leaves, leaves, &nr_msrs, NULL);
+/* Get the domain's policy. */
+rc = xc_cpu_policy_get_domain(xch, domid, policy);
 if ( rc )
 {
-PERROR("Failed to obtain %s default policy", hvm ? "hvm" : "pv");
 rc = -errno;
 goto out;
 }
-
-rc = x86_cpuid_copy_from_buffer(p, leaves, nr_leaves,
-&err_leaf, &err_subleaf);
-if ( rc )
-{
-ERROR("Failed to deserialise CPUID (err leaf %#x, subleaf %#x) (%d = 
%s)",
-  err_leaf, err_subleaf, -rc, strerror(-rc));
-goto out;
-}
+p = &policy->policy;
 
 if ( restore )
 {
@@ -643,19 +621,9 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 }
 }
 
-rc = x86_cpuid_copy_to_buffer(p, leaves, &nr_leaves);
-if ( rc )
-{
-ERROR("Failed to serialise CPUID (%d = %s)", -rc, strerror(-rc));
-goto out;
-}
-
-rc = xc_set_domain_cpu_policy(xch, domid, nr_leaves, leaves, 0, NULL,
-  &err_leaf, &err_subleaf, &err_msr);
+rc = xc_cpu_policy_set_domain(xch, domid, policy);
 if ( rc )
 {
-PERROR("Failed to set d%d's policy (err leaf %#x, subleaf %#x, msr 
%#x)",
-   domid, err_leaf, err_subleaf, err_msr);
 rc = -errno;
 goto out;
 }
@@ -666,8 +634,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 rc = 0;
 
 out:
-free(p);
-free(leaves);
+xc_cpu_policy_destroy(policy);
 
 return rc;
 }
-- 
2.40.0




[PATCH 03/13] libs/guest: allow fetching a specific CPUID leaf from a cpu policy

2023-06-16 Thread Roger Pau Monne
Introduce an interface that returns a specific leaf/subleaf from a cpu
policy in xen_cpuid_leaf_t format.

This is useful to callers can peek data from the opaque
xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

Signed-off-by: Roger Pau Monné 
Reviewed-by: Jan Beulich 
---
Changes since v6:
 - Add newline before return.

Changes since v5:
 - Zero out parameter.

Changes since v3:
 - Use x86_cpuid_get_leaf.

Changes since v1:
 - Use find leaf.
---
 tools/include/xenguest.h|  3 +++
 tools/libs/guest/xg_cpuid_x86.c | 26 ++
 2 files changed, 29 insertions(+)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index e01f494b772a..0a6fd9930627 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -807,6 +807,9 @@ int xc_cpu_policy_update_cpuid(xc_interface *xch, 
xc_cpu_policy_t *policy,
uint32_t nr);
 int xc_cpu_policy_update_msrs(xc_interface *xch, xc_cpu_policy_t *policy,
   const xen_msr_entry_t *msrs, uint32_t nr);
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
+uint32_t leaf, uint32_t subleaf,
+xen_cpuid_leaf_t *out);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 67e0dc9b4ad2..630d0018529f 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -828,6 +828,32 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, 
xc_cpu_policy_t *policy,
 return rc;
 }
 
+int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
+uint32_t leaf, uint32_t subleaf,
+xen_cpuid_leaf_t *out)
+{
+const struct cpuid_leaf *tmp;
+
+*out = (xen_cpuid_leaf_t){};
+
+tmp = x86_cpuid_get_leaf(&policy->policy, leaf, subleaf);
+if ( !tmp )
+{
+/* Unable to find a matching leaf. */
+errno = ENOENT;
+return -1;
+}
+
+out->leaf = leaf;
+out->subleaf = subleaf;
+out->a = tmp->a;
+out->b = tmp->b;
+out->c = tmp->c;
+out->d = tmp->d;
+
+return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
  xc_cpu_policy_t *guest)
 {
-- 
2.40.0




[PATCH 04/13] libx86: introduce helper to fetch msr entry

2023-06-16 Thread Roger Pau Monne
Use such helper in order to replace the code in
x86_msr_copy_from_buffer. Note the introduced helper should not be
directly called and instead x86_msr_get_entry should be used that will
properly deal with const and non-const inputs.

Note this requires making the raw fields uint64_t so that it can
accommodate the maximum size of MSRs values, and in turn removing the
truncation tests.

Signed-off-by: Roger Pau Monné 
---
Changes since v4:
 - Rename _x86_msr_get_entry to x86_msr_get_entry_const.
 - Add newline before endif.

Changes since v3:
 - New in this version.
---
 tools/tests/cpu-policy/test-cpu-policy.c | 43 +---
 xen/include/xen/lib/x86/cpu-policy.h | 18 +-
 xen/lib/x86/msr.c| 41 +++---
 3 files changed, 75 insertions(+), 27 deletions(-)

diff --git a/tools/tests/cpu-policy/test-cpu-policy.c 
b/tools/tests/cpu-policy/test-cpu-policy.c
index a11c8f067aad..2ce6ee96f91f 100644
--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -387,11 +387,6 @@ static void test_msr_deserialise_failure(void)
 .msr = { .idx = 0xce, .flags = 1 },
 .rc = -EINVAL,
 },
-{
-.name = "truncated val",
-.msr = { .idx = 0xce, .val = ~0ull },
-.rc = -EOVERFLOW,
-},
 };
 
 printf("Testing MSR deserialise failure:\n");
@@ -740,6 +735,43 @@ static void test_cpuid_get_leaf(void)
 }
 }
 
+static void test_msr_get_entry(void)
+{
+static const struct test {
+const char *name;
+unsigned int idx;
+bool success;
+} tests[] = {
+{
+.name = "bad msr index",
+.idx = -1,
+},
+{
+.name = "good msr index",
+.idx = 0xce,
+.success = true,
+},
+};
+const struct cpu_policy pc = { };
+const uint64_t *ec;
+struct cpu_policy p = { };
+uint64_t *e;
+
+/* Constness build test. */
+ec = x86_msr_get_entry(&pc, 0);
+e = x86_msr_get_entry(&p, 0);
+
+printf("Testing MSR get leaf:\n");
+
+for ( size_t i = 0; i < ARRAY_SIZE(tests); ++i )
+{
+const struct test *t = &tests[i];
+
+if ( !!x86_msr_get_entry(&pc, t->idx) != t->success )
+fail("  Test %s failed\n", t->name);
+}
+}
+
 static void test_is_compatible_success(void)
 {
 static struct test {
@@ -840,6 +872,7 @@ int main(int argc, char **argv)
 
 test_msr_serialise_success();
 test_msr_deserialise_failure();
+test_msr_get_entry();
 
 test_is_compatible_success();
 test_is_compatible_failure();
diff --git a/xen/include/xen/lib/x86/cpu-policy.h 
b/xen/include/xen/lib/x86/cpu-policy.h
index 3fcc02c729db..877879fc8c16 100644
--- a/xen/include/xen/lib/x86/cpu-policy.h
+++ b/xen/include/xen/lib/x86/cpu-policy.h
@@ -337,7 +337,7 @@ struct cpu_policy
  * is dependent on real hardware support.
  */
 union {
-uint32_t raw;
+uint64_t raw;
 struct {
 uint32_t :31;
 bool cpuid_faulting:1;
@@ -559,6 +559,22 @@ const struct cpuid_leaf *x86_cpuid_get_leaf_const(const 
struct cpu_policy *p,
 #define x86_cpuid_get_leaf(p, l, s) \
 ((__typeof__(&(p)->basic.raw[0]))x86_cpuid_get_leaf_const(p, l, s))
 
+/**
+ * Get a MSR entry from a policy object.
+ *
+ * @param policy  The msr_policy object.
+ * @param idx The index.
+ * @returns a pointer to the requested leaf or NULL in case of error.
+ *
+ * Do not call this function directly and instead use x86_msr_get_entry that
+ * will deal with both const and non-const policies returning a pointer with
+ * constness matching that of the input.
+ */
+const uint64_t *x86_msr_get_entry_const(const struct cpu_policy *p,
+uint32_t idx);
+#define x86_msr_get_entry(p, i) \
+((__typeof__(&(p)->platform_info.raw))x86_msr_get_entry_const(p, i))
+
 #endif /* !XEN_LIB_X86_POLICIES_H */
 
 /*
diff --git a/xen/lib/x86/msr.c b/xen/lib/x86/msr.c
index e04b9ca01302..25f6e8e0be92 100644
--- a/xen/lib/x86/msr.c
+++ b/xen/lib/x86/msr.c
@@ -74,6 +74,8 @@ int x86_msr_copy_from_buffer(struct cpu_policy *p,
 
 for ( i = 0; i < nr_entries; i++ )
 {
+uint64_t *val;
+
 if ( copy_from_buffer_offset(&data, msrs, i, 1) )
 return -EFAULT;
 
@@ -83,31 +85,13 @@ int x86_msr_copy_from_buffer(struct cpu_policy *p,
 goto err;
 }
 
-switch ( data.idx )
+val = x86_msr_get_entry(p, data.idx);
+if ( !val )
 {
-/*
- * Assign data.val to p->field, checking for truncation if the
- * backing storage for field is smaller than uint64_t
- */
-#define ASSIGN(field) \
-({\
-if ( (typeof(p->field))data.val != data.val ) \
-{ \
- 

[PATCH 05/13] libs/guest: allow fetching a specific MSR entry from a cpu policy

2023-06-16 Thread Roger Pau Monne
Introduce an interface that returns a specific MSR entry from a cpu
policy in xen_msr_entry_t format.

This is useful to callers can peek data from the opaque
xc_cpu_policy_t type.

No caller of the interface introduced on this patch.

Signed-off-by: Roger Pau Monné 
Reviewed-by: Jan Beulich 
---
Changes since v3:
 - Use x86_msr_get_entry.

Changes since v1:
 - Introduce a helper to perform a binary search of the MSR entries
   array.
---
 tools/include/xenguest.h|  2 ++
 tools/libs/guest/xg_cpuid_x86.c | 20 
 2 files changed, 22 insertions(+)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index 0a6fd9930627..2672fd043cf9 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -810,6 +810,8 @@ int xc_cpu_policy_update_msrs(xc_interface *xch, 
xc_cpu_policy_t *policy,
 int xc_cpu_policy_get_cpuid(xc_interface *xch, const xc_cpu_policy_t *policy,
 uint32_t leaf, uint32_t subleaf,
 xen_cpuid_leaf_t *out);
+int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
+  uint32_t msr, xen_msr_entry_t *out);
 
 /* Compatibility calculations. */
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 630d0018529f..c67e8c458f24 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -854,6 +854,26 @@ int xc_cpu_policy_get_cpuid(xc_interface *xch, const 
xc_cpu_policy_t *policy,
 return 0;
 }
 
+int xc_cpu_policy_get_msr(xc_interface *xch, const xc_cpu_policy_t *policy,
+  uint32_t msr, xen_msr_entry_t *out)
+{
+const uint64_t *val;
+
+*out = (xen_msr_entry_t){};
+
+val = x86_msr_get_entry(&policy->policy, msr);
+if ( !val )
+{
+errno = ENOENT;
+return -1;
+}
+
+out->idx = msr;
+out->val = *val;
+
+return 0;
+}
+
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
  xc_cpu_policy_t *guest)
 {
-- 
2.40.0




[PATCH 06/13] libs/guest: replace usage of host featureset in xc_cpuid_apply_policy()

2023-06-16 Thread Roger Pau Monne
Further changes to the function will require a host policy, hence
switch usages now in order to avoid having both a host featureset and
a host policy in the same context.

No functional change intended.

Signed-off-by: Roger Pau Monné 
---
 tools/libs/guest/xg_cpuid_x86.c | 27 ---
 1 file changed, 12 insertions(+), 15 deletions(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index c67e8c458f24..1e532b255c21 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -432,10 +432,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 bool hvm;
 xc_domaininfo_t di;
 unsigned int i;
-xc_cpu_policy_t *policy = NULL;
+xc_cpu_policy_t *policy = NULL, *host = NULL;
 struct cpu_policy *p;
-uint32_t host_featureset[FEATURESET_NR_ENTRIES] = {};
-uint32_t len = ARRAY_SIZE(host_featureset);
 
 if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
 {
@@ -446,16 +444,14 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 hvm = di.flags & XEN_DOMINF_hvm_guest;
 
 rc = -ENOMEM;
-if ( (policy = xc_cpu_policy_init()) == NULL )
+if ( (policy = xc_cpu_policy_init()) == NULL ||
+ (host = xc_cpu_policy_init()) == NULL )
 goto out;
 
 /* Get the host policy. */
-rc = xc_get_cpu_featureset(xch, XEN_SYSCTL_cpu_featureset_host,
-   &len, host_featureset);
-/* Tolerate "buffer too small", as we've got the bits we need. */
-if ( rc && errno != ENOBUFS )
+rc = xc_cpu_policy_get_system(xch, XEN_SYSCTL_cpu_policy_host, host);
+if ( rc )
 {
-PERROR("Failed to obtain host featureset");
 rc = -errno;
 goto out;
 }
@@ -485,13 +481,13 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
  * - Re-enable features which have become (possibly) off by default.
  */
 
-p->basic.rdrand = test_bit(X86_FEATURE_RDRAND, host_featureset);
-p->feat.hle = test_bit(X86_FEATURE_HLE, host_featureset);
-p->feat.rtm = test_bit(X86_FEATURE_RTM, host_featureset);
+p->basic.rdrand = host->policy.basic.rdrand;
+p->feat.hle = host->policy.feat.hle;
+p->feat.rtm = host->policy.feat.rtm;
 
 if ( hvm )
 {
-p->feat.mpx = test_bit(X86_FEATURE_MPX, host_featureset);
+p->feat.mpx = host->policy.feat.mpx;
 }
 
 p->basic.max_leaf = min(p->basic.max_leaf, 0xdu);
@@ -560,8 +556,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
  * On hardware without CPUID Faulting, PV guests see real topology.
  * As a consequence, they also need to see the host htt/cmp fields.
  */
-p->basic.htt   = test_bit(X86_FEATURE_HTT, host_featureset);
-p->extd.cmp_legacy = test_bit(X86_FEATURE_CMP_LEGACY, host_featureset);
+p->basic.htt   = host->policy.basic.htt;
+p->extd.cmp_legacy = host->policy.extd.cmp_legacy;
 }
 else
 {
@@ -635,6 +631,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 
 out:
 xc_cpu_policy_destroy(policy);
+xc_cpu_policy_destroy(host);
 
 return rc;
 }
-- 
2.40.0




[PATCH 07/13] libs/guest: rework xc_cpuid_xend_policy

2023-06-16 Thread Roger Pau Monne
Rename xc_cpuid_xend_policy to xc_cpu_policy_apply_cpuid and make it
public. Modify the function internally to use the new xc_cpu_policy_*
set of functions. Also don't apply the passed policy to a domain
directly, and instead modify the provided xc_cpu_policy_t. The caller
will be responsible of applying the modified cpu policy to the domain.

The find_leaf helper and related comparison function is also removed,
as it's no longer needed to search for cpuid leafs as finding the
matching leaves is now done using xc_cpu_policy_get_cpuid.

For the purpose of the 'x' modifier, assume that the passed
xc_cpu_policy_t 'policy' parameter contains the default policy, and
hence just skip making any modifications in that case.  This
simplifies some of the logic and avoids having to fetch the default
policy for the domain.

No functional change intended.

Signed-off-by: Roger Pau Monné 
---
Changes since v6:
 - Pass a host policy to xc_cpuid_apply_policy.

Changes since v3:
 - Drop find_leaf and comparison helper.
---
 tools/include/xenctrl.h |   2 +-
 tools/include/xenguest.h|   5 +
 tools/libs/guest/xg_cpuid_x86.c | 201 +---
 3 files changed, 64 insertions(+), 144 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index dba33d5d0f39..45f05a2d3d7e 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1839,7 +1839,7 @@ int xc_cpuid_apply_policy(xc_interface *xch,
   uint32_t domid, bool restore,
   const uint32_t *featureset,
   unsigned int nr_features, bool pae, bool itsc,
-  bool nested_virt, const struct xc_xend_cpuid *xend);
+  bool nested_virt, const struct xc_xend_cpuid *cpuid);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
 xc_cpumap_t cpumap, unsigned int nr_cpus);
diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index 2672fd043cf9..705a93a058fb 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -817,6 +817,11 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const 
xc_cpu_policy_t *policy,
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
  xc_cpu_policy_t *guest);
 
+/* Apply an array of xc_xend_cpuid leafs to the policy. */
+int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
+  const struct xc_xend_cpuid *cpuid,
+  const xc_cpu_policy_t *host);
+
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
   uint32_t *nr_features, uint32_t *featureset);
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 1e532b255c21..0d0970d4bd69 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -254,145 +254,67 @@ int xc_set_domain_cpu_policy(xc_interface *xch, uint32_t 
domid,
 return ret;
 }
 
-static int compare_leaves(const void *l, const void *r)
+int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
+  const struct xc_xend_cpuid *cpuid,
+  const xc_cpu_policy_t *host)
 {
-const xen_cpuid_leaf_t *lhs = l;
-const xen_cpuid_leaf_t *rhs = r;
-
-if ( lhs->leaf != rhs->leaf )
-return lhs->leaf < rhs->leaf ? -1 : 1;
-
-if ( lhs->subleaf != rhs->subleaf )
-return lhs->subleaf < rhs->subleaf ? -1 : 1;
-
-return 0;
-}
-
-static xen_cpuid_leaf_t *find_leaf(
-xen_cpuid_leaf_t *leaves, unsigned int nr_leaves,
-const struct xc_xend_cpuid *xend)
-{
-const xen_cpuid_leaf_t key = { xend->leaf, xend->subleaf };
-
-return bsearch(&key, leaves, nr_leaves, sizeof(*leaves), compare_leaves);
-}
-
-static int xc_cpuid_xend_policy(
-xc_interface *xch, uint32_t domid, const struct xc_xend_cpuid *xend)
-{
-int rc;
-bool hvm;
-xc_domaininfo_t di;
-unsigned int nr_leaves, nr_msrs;
-uint32_t err_leaf = -1, err_subleaf = -1, err_msr = -1;
-/*
- * Three full policies.  The host, default for the domain type,
- * and domain current.
- */
-xen_cpuid_leaf_t *host = NULL, *def = NULL, *cur = NULL;
-unsigned int nr_host, nr_def, nr_cur;
-
-if ( (rc = xc_domain_getinfo_single(xch, domid, &di)) < 0 )
-{
-PERROR("Failed to obtain d%d info", domid);
-rc = -errno;
-goto fail;
-}
-hvm = di.flags & XEN_DOMINF_hvm_guest;
-
-rc = xc_cpu_policy_get_size(xch, &nr_leaves, &nr_msrs);
-if ( rc )
-{
-PERROR("Failed to obtain policy info size");
-rc = -errno;
-goto fail;
-}
-
-rc = -ENOMEM;
-if ( (host = calloc(nr_leaves, sizeof(*host))) == NULL ||
- (def  = calloc(nr_leave

[PATCH 09/13] libxl: change the type of libxl_cpuid_policy_list

2023-06-16 Thread Roger Pau Monne
Currently libxl_cpuid_policy_list is an opaque type to the users of
libxl, and internally it's an array of xc_xend_cpuid objects.

Change the type to instead be a structure that contains one array for
CPUID policies, in preparation for it also holding another array for
MSR policies.

Signed-off-by: Roger Pau Monné 
---
 tools/include/libxl.h  |  6 +++--
 tools/libs/light/gentest.py|  2 +-
 tools/libs/light/libxl_cpuid.c | 49 +++---
 3 files changed, 32 insertions(+), 25 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index cac641a7eba2..41e19f2af7f5 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1459,8 +1459,10 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
  * libxl_cpuid_policy is opaque in the libxl ABI.  Users of both libxl and
  * libxc may not make assumptions about xc_xend_cpuid.
  */
-typedef struct xc_xend_cpuid libxl_cpuid_policy;
-typedef libxl_cpuid_policy * libxl_cpuid_policy_list;
+typedef struct libxl_cpu_policy {
+struct xc_xend_cpuid *cpuid;
+} libxl_cpuid_policy;
+typedef libxl_cpuid_policy libxl_cpuid_policy_list;
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
 int libxl_cpuid_policy_list_length(const libxl_cpuid_policy_list *l);
 void libxl_cpuid_policy_list_copy(libxl_ctx *ctx,
diff --git a/tools/libs/light/gentest.py b/tools/libs/light/gentest.py
index 1cc7eebc826d..207f988a741d 100644
--- a/tools/libs/light/gentest.py
+++ b/tools/libs/light/gentest.py
@@ -194,7 +194,7 @@ static void 
libxl_cpuid_policy_list_rand_init(libxl_cpuid_policy_list *pp)
 };
 const int nr_options = sizeof(options)/sizeof(options[0]);
 char buf[64];
-libxl_cpuid_policy_list p = NULL;
+libxl_cpuid_policy_list p = { };
 
 for (i = 0; i < nr_policies; i++) {
 int opt = test_rand(nr_options);
diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index c96aeb3bce46..ded0d0b8bc15 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -19,10 +19,10 @@ int libxl__cpuid_policy_is_empty(libxl_cpuid_policy_list 
*pl)
 return !libxl_cpuid_policy_list_length(pl);
 }
 
-void libxl_cpuid_dispose(libxl_cpuid_policy_list *p_cpuid_list)
+void libxl_cpuid_dispose(libxl_cpuid_policy_list *policy)
 {
 int i, j;
-libxl_cpuid_policy_list cpuid_list = *p_cpuid_list;
+struct xc_xend_cpuid *cpuid_list = policy->cpuid;
 
 if (cpuid_list == NULL)
 return;
@@ -33,8 +33,8 @@ void libxl_cpuid_dispose(libxl_cpuid_policy_list 
*p_cpuid_list)
 cpuid_list[i].policy[j] = NULL;
 }
 }
-free(cpuid_list);
-*p_cpuid_list = NULL;
+free(policy->cpuid);
+policy->cpuid = NULL;
 return;
 }
 
@@ -62,9 +62,10 @@ struct cpuid_flags {
 /* go through the dynamic array finding the entry for a specified leaf.
  * if no entry exists, allocate one and return that.
  */
-static libxl_cpuid_policy_list cpuid_find_match(libxl_cpuid_policy_list *list,
-  uint32_t leaf, uint32_t subleaf)
+static struct xc_xend_cpuid *cpuid_find_match(libxl_cpuid_policy *policy,
+  uint32_t leaf, uint32_t subleaf)
 {
+struct xc_xend_cpuid **list = &policy->cpuid;
 int i = 0;
 
 if (*list != NULL) {
@@ -86,7 +87,7 @@ static libxl_cpuid_policy_list 
cpuid_find_match(libxl_cpuid_policy_list *list,
  * Will overwrite earlier entries and thus can be called multiple
  * times.
  */
-int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str)
+int libxl_cpuid_parse_config(libxl_cpuid_policy_list *policy, const char* str)
 {
 #define NA XEN_CPUID_INPUT_UNUSED
 static const struct cpuid_flags cpuid_flags[] = {
@@ -345,7 +346,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*cpuid, const char* str)
 if (flag->name == NULL) {
 return 2;
 }
-entry = cpuid_find_match(cpuid, flag->leaf, flag->subleaf);
+entry = cpuid_find_match(policy, flag->leaf, flag->subleaf);
 resstr = entry->policy[flag->reg - 1];
 num = strtoull(val, &endptr, 0);
 flags[flag->length] = 0;
@@ -400,7 +401,7 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*cpuid, const char* str)
  * the strings for each register were directly exposed to the user.
  * Used for maintaining compatibility with older config files
  */
-int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
+int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *policy,
   const char* str)
 {
 char *endptr;
@@ -427,7 +428,7 @@ int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list 
*cpuid,
 return 3;
 }
 str = endptr + 1;
-entry = cpuid_find_match(cpuid, leaf, subleaf);
+entry = cpuid_find_match(policy, leaf, subleaf);
 for (str = endptr + 1; *str != 0;) {
 if (str[0] != 'e' || str[2] != 'x') {
 return 4;
@@ -502,7 +503,7 @@ int lib

Re: [PATCH v3 4/4] x86/cpu-policy: Derive RSBA/RRSBA for guest policies

2023-06-16 Thread Andrew Cooper
On 16/06/2023 1:12 pm, Jan Beulich wrote:
> On 15.06.2023 20:17, Andrew Cooper wrote:
>> On 15/06/2023 1:13 pm, Jan Beulich wrote:
>>> On 15.06.2023 12:41, Andrew Cooper wrote:
 On 15/06/2023 9:30 am, Jan Beulich wrote:
> On 14.06.2023 20:12, Andrew Cooper wrote:
>> On 13/06/2023 10:59 am, Jan Beulich wrote:
>>> On 12.06.2023 18:13, Andrew Cooper wrote:
 The RSBA bit, "RSB Alternative", means that the RSB may use alternative
 predictors when empty.  From a practical point of view, this mean 
 "Retpoline
 not safe".

 Enhanced IBRS (officially IBRS_ALL in Intel's docs, previously 
 IBRS_ATT) is a
 statement that IBRS is implemented in hardware (as opposed to the form
 retrofitted to existing CPUs in microcode).

 The RRSBA bit, "Restricted-RSBA", is a combination of RSBA, and the 
 eIBRS
 property that predictions are tagged with the mode in which they were 
 learnt.
 Therefore, it means "when eIBRS is active, the RSB may fall back to
 alternative predictors but restricted to the current prediction mode". 
  As
 such, it's stronger statement than RSBA, but still means "Retpoline 
 not safe".

 CPUs are not expected to enumerate both RSBA and RRSBA.

 Add feature dependencies for EIBRS and RRSBA.  While technically 
 they're not
 linked, absolutely nothing good can come of letting the guest see RRSBA
 without EIBRS.  Nor a guest seeing EIBRS without IBRSB.  Furthermore, 
 we use
 this dependency to simplify the max derivation logic.

 The max policies gets RSBA and RRSBA unconditionally set (with the 
 EIBRS
 dependency maybe hiding RRSBA).  We can run any VM, even if it has 
 been told
 "somewhere you might run, Retpoline isn't safe".

 The default policies are more complicated.  A guest shouldn't see both 
 bits,
 but it needs to see one if the current host suffers from any form of 
 RSBA, and
 which bit it needs to see depends on whether eIBRS is visible or not.
 Therefore, the calculation must be performed after 
 sanitise_featureset().

 Signed-off-by: Andrew Cooper 
 ---
 CC: Jan Beulich 
 CC: Roger Pau Monné 
 CC: Wei Liu 

 v3:
  * Minor commit message adjustment.
  * Drop changes to recalculate_cpuid_policy().  Deferred to a later 
 series.
>>> With this dropped, with the title not saying "max/default", and with
>>> the description also not mentioning "live" policies at all, I don't
>>> think this patch is self-consistent (meaning in particular: leaving
>>> aside the fact that there's no way right now to requests e.g. both
>>> RSBA and RRSBA for a guest; aiui it is possible for Dom0).
>>>
>>> As you may imagine I'm also curious why you decided to drop this.
>> Because when I tried doing levelling in Xapi, I remembered why I did it
>> the way I did in v1, and why the v2 way was wrong.
>>
>> Xen cannot safely edit what the toolstack provides, so must not. 
> And this is the part I don't understand: Why can't we correct the
> (EIBRS,RSBA,RRSBA) tuple to a combination that is "legal"? At least
> as long as ...
>
>> Instead, failing the set_policy() call is an option, and is what we want
>> to do longterm,
> ... we aren't there.
>
>> but also happens to be wrong too in this case. An admin
>> may know that a VM isn't using retpoline, and may need to migrate it
>> anyway for a number of reasons, so any safety checks need to be in the
>> toolstack, and need to be overrideable with something like --force.
> Possibly leading to an inconsistent policy exposed to a guest? I
> guess this may be the only option when we can't really resolve an
> ambiguity, but that isn't the case here, is it?
 Wrong.  Xen does not have any knowledge of other hosts the VM might
 migrate to.

 So while Xen can spot problem combinations *on this host*, which way to
 correct the problem combination depends on where the VM might migrate to.
>>> I actually view this as two different levels: With a flawed policy, the
>>> guest is liable to not work correctly at all. No point thinking about
>>> it being able to migrate. With a fixed up policy it may fail to migrate,
>>> but it'll at least work otherwise.
>> If you get RSBA and/or RRSBA wrong, nothing is going to malfunction in
>> the guest, even if you migrate it.
>>
>> The consequence of getting RSBA and/or RRSBA wrong is the guest *might*
>> think retpoline is safe to use, and *might* end up vulnerable to
>> speculative attacks on this or other hardware.
> Isn't that some sort of "malfunction"?

Perhaps, there's a difference between 

[PATCH 11/13] libxl: split logic to parse user provided CPUID features

2023-06-16 Thread Roger Pau Monne
Move the CPUID value parsers out of libxl_cpuid_parse_config() into a
newly created cpuid_add() local helper.  This is in preparation for
also adding MSR feature parsing support.

No functional change intended.

Signed-off-by: Roger Pau Monné 
---
 tools/libs/light/libxl_cpuid.c | 120 +
 1 file changed, 63 insertions(+), 57 deletions(-)

diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index 7261c1f1fd82..d0ac5b2bc102 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -88,6 +88,66 @@ static struct xc_xend_cpuid 
*cpuid_find_match(libxl_cpuid_policy *policy,
 return *list + i;
 }
 
+static int cpuid_add(libxl_cpuid_policy *policy, const struct cpuid_flags 
*flag,
+ const char *val)
+{
+struct xc_xend_cpuid *entry = cpuid_find_match(policy, flag->leaf,
+   flag->subleaf);
+unsigned long num;
+char flags[33], *resstr, *endptr;
+unsigned int i;
+
+resstr = entry->policy[flag->reg - 1];
+num = strtoull(val, &endptr, 0);
+flags[flag->length] = 0;
+if (endptr != val) {
+/* if this was a valid number, write the binary form into the string */
+for (i = 0; i < flag->length; i++) {
+flags[flag->length - 1 - i] = "01"[!!(num & (1 << i))];
+}
+} else {
+switch(val[0]) {
+case 'x': case 'k': case 's':
+memset(flags, val[0], flag->length);
+break;
+default:
+return 3;
+}
+}
+
+if (resstr == NULL) {
+resstr = strdup("");
+}
+
+/* the family and model entry is potentially split up across
+ * two fields in Fn_0001_EAX, so handle them here separately.
+ */
+if (!strcmp(flag->name, "family")) {
+if (num < 16) {
+memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
+memcpy(resstr + (32 - 8) - 20, "", 8);
+} else {
+num -= 15;
+memcpy(resstr + (32 - 4) - flag->bit, "", 4);
+for (i = 0; i < 7; i++) {
+flags[7 - i] = "01"[num & 1];
+num >>= 1;
+}
+memcpy(resstr + (32 - 8) - 20, flags, 8);
+}
+} else if (!strcmp(flag->name, "model")) {
+memcpy(resstr + (32 - 4) - 16, flags, 4);
+memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
+} else {
+memcpy(resstr + (32 - flag->length) - flag->bit, flags,
+   flag->length);
+}
+entry->policy[flag->reg - 1] = resstr;
+
+return 0;
+
+}
+
 /* parse a single key=value pair and translate it into the libxc
  * used interface using 32-characters strings for each register.
  * Will overwrite earlier entries and thus can be called multiple
@@ -332,12 +392,8 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*policy, const char* str)
 {NULL, 0, NA, CPUID_REG_INV, 0, 0}
 };
 #undef NA
-char *sep, *val, *endptr;
-int i;
+const char *sep, *val;
 const struct cpuid_flags *flag;
-struct xc_xend_cpuid *entry;
-unsigned long num;
-char flags[33], *resstr;
 
 sep = strchr(str, '=');
 if (sep == NULL) {
@@ -347,60 +403,10 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*policy, const char* str)
 }
 for (flag = cpuid_flags; flag->name != NULL; flag++) {
 if(!strncmp(str, flag->name, sep - str) && flag->name[sep - str] == 0)
-break;
-}
-if (flag->name == NULL) {
-return 2;
-}
-entry = cpuid_find_match(policy, flag->leaf, flag->subleaf);
-resstr = entry->policy[flag->reg - 1];
-num = strtoull(val, &endptr, 0);
-flags[flag->length] = 0;
-if (endptr != val) {
-/* if this was a valid number, write the binary form into the string */
-for (i = 0; i < flag->length; i++) {
-flags[flag->length - 1 - i] = "01"[!!(num & (1 << i))];
-}
-} else {
-switch(val[0]) {
-case 'x': case 'k': case 's':
-memset(flags, val[0], flag->length);
-break;
-default:
-return 3;
-}
-}
-
-if (resstr == NULL) {
-resstr = strdup("");
+return cpuid_add(policy, flag, val);
 }
 
-/* the family and model entry is potentially split up across
- * two fields in Fn_0001_EAX, so handle them here separately.
- */
-if (!strncmp(str, "family", sep - str)) {
-if (num < 16) {
-memcpy(resstr + (32 - 4) - flag->bit, flags + 4, 4);
-memcpy(resstr + (32 - 8) - 20, "", 8);
-} else {
-num -= 15;
-memcpy(resstr + (32 - 4) - flag->bit, "", 4);
-for (i = 0; i < 7; i++) {
-flags[7 - i] = "01"[num & 1];
-num >>= 1;
-}
-memcpy(resstr + (3

[PATCH 10/13] libxl: introduce MSR data in libxl_cpuid_policy

2023-06-16 Thread Roger Pau Monne
Add a new array field to libxl_cpuid_policy in order to store the MSR
policies.

Note that libxl_cpuid_policy_list_{copy,length,parse_json,gen_json}
are not adjusted to deal with the new MSR array now part of
libxl_cpuid_policy_list.

Adding the MSR data in the libxl_cpuid_policy_list type is done so
that existing users can seamlessly pass MSR features as part of the
CPUID data, without requiring the introduction of a separate
domain_build_info field, and a new set of handlers functions.

Signed-off-by: Roger Pau Monné 
---
 tools/include/libxl.h  |  1 +
 tools/libs/light/libxl_cpuid.c | 31 +++
 2 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 41e19f2af7f5..4e7b08ab5027 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1461,6 +1461,7 @@ void libxl_bitmap_dispose(libxl_bitmap *map);
  */
 typedef struct libxl_cpu_policy {
 struct xc_xend_cpuid *cpuid;
+struct xc_msr *msr;
 } libxl_cpuid_policy;
 typedef libxl_cpuid_policy libxl_cpuid_policy_list;
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *cpuid_list);
diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index ded0d0b8bc15..7261c1f1fd82 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -21,20 +21,26 @@ int libxl__cpuid_policy_is_empty(libxl_cpuid_policy_list 
*pl)
 
 void libxl_cpuid_dispose(libxl_cpuid_policy_list *policy)
 {
-int i, j;
 struct xc_xend_cpuid *cpuid_list = policy->cpuid;
 
-if (cpuid_list == NULL)
-return;
-for (i = 0; cpuid_list[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) {
-for (j = 0; j < 4; j++)
-if (cpuid_list[i].policy[j] != NULL) {
-free(cpuid_list[i].policy[j]);
-cpuid_list[i].policy[j] = NULL;
-}
+if (cpuid_list) {
+unsigned int i, j;
+
+for (i = 0; cpuid_list[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) {
+for (j = 0; j < 4; j++)
+if (cpuid_list[i].policy[j] != NULL) {
+free(cpuid_list[i].policy[j]);
+cpuid_list[i].policy[j] = NULL;
+}
+}
+free(policy->cpuid);
+policy->cpuid = NULL;
+}
+
+if (policy->msr) {
+free(policy->msr);
+policy->msr = NULL;
 }
-free(policy->cpuid);
-policy->cpuid = NULL;
 return;
 }
 
@@ -503,7 +509,8 @@ int libxl__cpuid_legacy(libxl_ctx *ctx, uint32_t domid, 
bool restore,
 info->tsc_mode == LIBXL_TSC_MODE_ALWAYS_EMULATE);
 
 r = xc_cpuid_apply_policy(ctx->xch, domid, restore, NULL, 0,
-  pae, itsc, nested_virt, info->cpuid.cpuid, NULL);
+  pae, itsc, nested_virt, info->cpuid.cpuid,
+  info->cpuid.msr);
 if (r)
 LOGEVD(ERROR, -r, domid, "Failed to apply CPUID policy");
 
-- 
2.40.0




[PATCH 08/13] libs/guest: introduce support for setting guest MSRs

2023-06-16 Thread Roger Pau Monne
Like it's done with CPUID, introduce support for passing MSR values to
xc_cpuid_apply_policy().  The chosen format for expressing MSR policy
data matches the current one used for CPUID.  Note that existing
callers of xc_cpuid_apply_policy() can pass NULL as the value for the
newly introduced 'msr' parameter in order to preserve the same
functionality, and in fact that's done in libxl on this patch.

Signed-off-by: Roger Pau Monné 
---
 tools/include/xenctrl.h | 21 -
 tools/include/xenguest.h|  5 ++-
 tools/libs/guest/xg_cpuid_x86.c | 76 -
 tools/libs/light/libxl_cpuid.c  |  2 +-
 4 files changed, 99 insertions(+), 5 deletions(-)

diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h
index 45f05a2d3d7e..786061282c91 100644
--- a/tools/include/xenctrl.h
+++ b/tools/include/xenctrl.h
@@ -1822,6 +1822,21 @@ struct xc_xend_cpuid {
 char *policy[4];
 };
 
+/*
+ * MSR policy data.
+ *
+ * The format of the policy string is the following:
+ *   '1' -> force to 1
+ *   '0' -> force to 0
+ *   'x' -> we don't care (use default)
+ *   'k' -> pass through host value
+ */
+struct xc_msr {
+uint32_t index;
+char policy[65];
+};
+#define XC_MSR_INPUT_UNUSED 0xu
+
 /*
  * Make adjustments to the CPUID settings for a domain.
  *
@@ -1833,13 +1848,15 @@ struct xc_xend_cpuid {
  * Either pass a full new @featureset (and @nr_features), or adjust individual
  * features (@pae, @itsc, @nested_virt).
  *
- * Then (optionally) apply legacy XEND overrides (@xend) to the result.
+ * Then (optionally) apply legacy XEND CPUID overrides (@cpuid) or MSR (@msr)
+ * to the result.
  */
 int xc_cpuid_apply_policy(xc_interface *xch,
   uint32_t domid, bool restore,
   const uint32_t *featureset,
   unsigned int nr_features, bool pae, bool itsc,
-  bool nested_virt, const struct xc_xend_cpuid *cpuid);
+  bool nested_virt, const struct xc_xend_cpuid *cpuid,
+  const struct xc_msr *msr);
 int xc_mca_op(xc_interface *xch, struct xen_mc *mc);
 int xc_mca_op_inject_v2(xc_interface *xch, unsigned int flags,
 xc_cpumap_t cpumap, unsigned int nr_cpus);
diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index 705a93a058fb..be61ff0af7fe 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -817,10 +817,13 @@ int xc_cpu_policy_get_msr(xc_interface *xch, const 
xc_cpu_policy_t *policy,
 bool xc_cpu_policy_is_compatible(xc_interface *xch, xc_cpu_policy_t *host,
  xc_cpu_policy_t *guest);
 
-/* Apply an array of xc_xend_cpuid leafs to the policy. */
+/* Apply an array of xc_xend_cpuid leafs or xc_msrs to the policy. */
 int xc_cpu_policy_apply_cpuid(xc_interface *xch, xc_cpu_policy_t *policy,
   const struct xc_xend_cpuid *cpuid,
   const xc_cpu_policy_t *host);
+int xc_cpu_policy_apply_msr(xc_interface *xch, xc_cpu_policy_t *policy,
+const struct xc_msr *msr,
+const xc_cpu_policy_t *host);
 
 int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
 int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index 0d0970d4bd69..09a012960a43 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -331,10 +331,74 @@ int xc_cpu_policy_apply_cpuid(xc_interface *xch, 
xc_cpu_policy_t *policy,
 return 0;
 }
 
+int xc_cpu_policy_apply_msr(xc_interface *xch, xc_cpu_policy_t *policy,
+const struct xc_msr *msr,
+const xc_cpu_policy_t *host)
+{
+for ( ; msr->index != XC_MSR_INPUT_UNUSED; ++msr )
+{
+xen_msr_entry_t cur_msr, host_msr;
+int rc;
+
+rc = xc_cpu_policy_get_msr(xch, policy, msr->index, &cur_msr);
+if ( rc )
+{
+ERROR("Failed to get current MSR %#x", msr->index);
+return rc;
+}
+rc = xc_cpu_policy_get_msr(xch, host, msr->index, &host_msr);
+if ( rc )
+{
+ERROR("Failed to get host policy MSR %#x", msr->index);
+return rc;
+}
+
+for ( unsigned int i = 0; i < ARRAY_SIZE(msr->policy) - 1; i++ )
+{
+bool val;
+
+switch ( msr->policy[i] )
+{
+case '1':
+val = true;
+break;
+
+case '0':
+val = false;
+break;
+
+case 'x':
+/* Leave alone: the current policy is the default one. */
+continue;
+
+case 'k':
+val = test_bit(63 - i, &host_msr.val);
+break;
+
+default:
+ERROR("Bad charact

[PATCH 13/13] libxl: add support for parsing MSR features

2023-06-16 Thread Roger Pau Monne
Introduce support for handling MSR features in
libxl_cpuid_parse_config().  The MSR policies are added to the
libxl_cpuid_policy like the CPUID one, which gets passed to
xc_cpuid_apply_policy().

This allows existing users of libxl to provide MSR related features as
key=value pairs to libxl_cpuid_parse_config() without requiring the
usage of a different API.

Signed-off-by: Roger Pau Monné 
---
 tools/libs/light/libxl_cpuid.c | 57 +-
 1 file changed, 56 insertions(+), 1 deletion(-)

diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index cbbd5d31d63b..804dddb446c3 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -149,6 +149,53 @@ static int cpuid_add(libxl_cpuid_policy *policy, const 
struct cpuid_flags *flag,
 return 0;
 }
 
+static struct xc_msr *msr_find_match(libxl_cpuid_policy *policy, uint32_t 
index)
+{
+unsigned int i = 0;
+
+if (policy->msr != NULL)
+for (i = 0; policy->msr[i].index != XC_MSR_INPUT_UNUSED; i++)
+if (policy->msr[i].index == index)
+return &policy->msr[i];
+
+policy->msr = realloc(policy->msr, sizeof(struct xc_msr) * (i + 2));
+policy->msr[i].index = index;
+memset(policy->msr[i].policy, 'x', ARRAY_SIZE(policy->msr[0].policy) - 1);
+policy->msr[i].policy[ARRAY_SIZE(policy->msr[0].policy) - 1] = '\0';
+policy->msr[i + 1].index = XC_MSR_INPUT_UNUSED;
+
+return &policy->msr[i];
+}
+
+static int msr_add(libxl_cpuid_policy *policy, uint32_t index, unsigned int 
bit,
+   const char *val)
+{
+struct xc_msr *entry = msr_find_match(policy, index);
+
+/* Only allow options taking a character for MSRs, no values allowed. */
+if (strlen(val) != 1)
+return 3;
+
+switch (val[0]) {
+case '0':
+case '1':
+case 'x':
+case 'k':
+entry->policy[63 - bit] = val[0];
+break;
+
+case 's':
+/* Translate s -> k as xc_msr doesn't support the deprecated 's'. */
+entry->policy[63 - bit] = 'k';
+break;
+
+default:
+return 3;
+}
+
+return 0;
+}
+
 struct feature_name {
 const char *name;
 unsigned int bit;
@@ -328,7 +375,15 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*policy, const char* str)
 }
 
 case FEAT_MSR:
-return 2;
+{
+unsigned int bit = feat->bit % 32;
+
+if (feature_to_policy[feat->bit / 32].msr.reg == CPUID_REG_EDX)
+bit += 32;
+
+return msr_add(policy, feature_to_policy[feat->bit / 32].msr.index,
+   bit, val);
+}
 }
 
 return 2;
-- 
2.40.0




[PATCH 12/13] libxl: use the cpuid feature names from cpufeatureset.h

2023-06-16 Thread Roger Pau Monne
The current implementation in libxl_cpuid_parse_config() requires
keeping a list of cpuid feature bits that should be mostly in sync
with the contents of cpufeatureset.h.

Avoid such duplication by using the automatically generated list of
cpuid features in INIT_FEATURE_NAMES in order to map feature names to
featureset bits, and then translate from featureset bits into cpuid
leaf, subleaf, register tuple.

Note that the full contents of the previous cpuid translation table
can't be removed.  That's because some feature names allowed by libxl
are not described in the featuresets, or because naming has diverged
and the previous nomenclature is preserved for compatibility reasons.

Should result in no functional change observed by callers, albeit some
new cpuid features will be available as a result of the change.

While there constify cpuid_flags name field.

Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - const unnamed structure cast.
 - Declare struct feature_name outside the function.
 - Use strcmp.
 - Fix indentation.
 - Add back missing feature name options.
 - Return ERROR_NOMEM if allocation fails.
 - Improve xl.cfg documentation about how to reference the features
   described in the public header.
---
 docs/man/xl.cfg.5.pod.in   |  24 +--
 tools/libs/light/libxl_cpuid.c | 267 -
 tools/xl/xl_parse.c|   3 +
 3 files changed, 107 insertions(+), 187 deletions(-)

diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 3979be2a590a..55161856f4c7 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -2010,24 +2010,16 @@ proccount procpkg stepping
 
 =back
 
-List of keys taking a character:
+List of keys taking a character can be found in the public header file
+Lhttps://xenbits.xen.org/docs/unstable/hypercall/x86_64/include,public,arch-x86,cpufeatureset.h.html>
 
-=over 4
-
-3dnow 3dnowext 3dnowprefetch abm acpi adx aes altmovcr8 apic arat avx avx2
-avx512-4fmaps avx512-4vnniw avx512bw avx512cd avx512dq avx512er avx512f
-avx512ifma avx512pf avx512vbmi avx512vl bmi1 bmi2 clflushopt clfsh clwb cmov
-cmplegacy cmpxchg16 cmpxchg8 cmt cntxid dca de ds dscpl dtes64 erms est extapic
-f16c ffxsr fma fma4 fpu fsgsbase fxsr hle htt hypervisor ia64 ibs invpcid
-invtsc lahfsahf lm lwp mca mce misalignsse mmx mmxext monitor movbe mpx msr
-mtrr nodeid nx ospke osvw osxsave pae page1gb pat pbe pcid pclmulqdq pdcm
-perfctr_core perfctr_nb pge pku popcnt pse pse36 psn rdrand rdseed rdtscp rtm
-sha skinit smap smep smx ss sse sse2 sse3 sse4.1 sse4.2 sse4_1 sse4_2 sse4a
-ssse3 svm svm_decode svm_lbrv svm_npt svm_nrips svm_pausefilt svm_tscrate
-svm_vmcbclean syscall sysenter tbm tm tm2 topoext tsc tsc-deadline tsc_adjust
-umip vme vmx wdt x2apic xop xsave xtpr
+The feature names described in C should be specified in all
+lowercase letters, and with underscores converted to hyphens.  For example in
+order to reference feature C the string C should be used.
 
-=back
+Note that C is described as an option that takes a value, and that
+takes precedence over the C flag in C.  The feature
+flag must be referenced as C.
 
 =back
 
diff --git a/tools/libs/light/libxl_cpuid.c b/tools/libs/light/libxl_cpuid.c
index d0ac5b2bc102..cbbd5d31d63b 100644
--- a/tools/libs/light/libxl_cpuid.c
+++ b/tools/libs/light/libxl_cpuid.c
@@ -14,6 +14,8 @@
 
 #include "libxl_internal.h"
 
+#include 
+
 int libxl__cpuid_policy_is_empty(libxl_cpuid_policy_list *pl)
 {
 return !libxl_cpuid_policy_list_length(pl);
@@ -57,7 +59,7 @@ void libxl_cpuid_dispose(libxl_cpuid_policy_list *policy)
  * Used for the static structure describing all features.
  */
 struct cpuid_flags {
-char* name;
+const char *name;
 uint32_t leaf;
 uint32_t subleaf;
 int reg;
@@ -145,7 +147,19 @@ static int cpuid_add(libxl_cpuid_policy *policy, const 
struct cpuid_flags *flag,
 entry->policy[flag->reg - 1] = resstr;
 
 return 0;
+}
+
+struct feature_name {
+const char *name;
+unsigned int bit;
+};
+
+static int search_feature(const void *a, const void *b)
+{
+const char *key = a;
+const char *feat = ((const struct feature_name *)b)->name;
 
+return strcmp(key, feat);
 }
 
 /* parse a single key=value pair and translate it into the libxc
@@ -168,208 +182,42 @@ int libxl_cpuid_parse_config(libxl_cpuid_policy_list 
*policy, const char* str)
 {"proccount",0x0001, NA, CPUID_REG_EBX, 16,  8},
 {"localapicid",  0x0001, NA, CPUID_REG_EBX, 24,  8},
 
-{"sse3", 0x0001, NA, CPUID_REG_ECX,  0,  1},
-{"pclmulqdq",0x0001, NA, CPUID_REG_ECX,  1,  1},
-{"dtes64",   0x0001, NA, CPUID_REG_ECX,  2,  1},
-{"monitor",  0x0001, NA, CPUID_REG_ECX,  3,  1},
-{"dscpl",0x0001, NA, CPUID_REG_ECX,  4,  1},
-{"vmx",  0x0001, NA, CPUID_REG_ECX,  5,  1},
-{"smx",  0x0001, NA, CPUID_REG_ECX,  6,  1},
 {"est",

Re: [PATCH v2 2/2] x86/vPIT: account for "counter stopped" time

2023-06-16 Thread Roger Pau Monné
On Thu, Jun 15, 2023 at 04:56:22PM +0200, Jan Beulich wrote:
> For an approach like that used in "x86: detect PIT aliasing on ports
> other than 0x4[0-3]" [1] to work, channel 2 may not (appear to) continue
> counting when "gate" is low. Record the time when "gate" goes low, and
> adjust pit_get_{count,out}() accordingly. Additionally for most of the
> modes a rising edge of "gate" doesn't mean just "resume counting", but
> "initiate counting", i.e. specifically the reloading of the counter with
> its init value.
> 
> No special handling for state save/load: See the comment near the end of
> pit_load().
> 
> [1] https://lists.xen.org/archives/html/xen-devel/2023-05/msg00898.html
> 
> Signed-off-by: Jan Beulich 

Acked-by: Roger Pau Monné 

Albeit I have one request below I would like you to considerate.

> ---
> TBD: "gate" can only ever be low for chan2 (with "x86/vPIT: check/bound
>  values loaded from state save record" [2] in place), so in
>  principle we could get away without a new pair of arrays, but just
>  two individual fields. At the expense of more special casing in
>  code.

One bit I'm missing is how is the gate for timers 0 and 1 is accessed.
Is such line simply not accessible?

My i8254 spec doesn't mention this, and the (kind of random) copy of
the ICH7 Spec I'm looking at also doesn't mention enable bits for
timers 0 and 1 being available.  I assume those are always enabled?

> 
> TBD: Should we deal with other aspects of "gate low" in pit_get_out()
>  here as well, right away? I was hoping to get away without ...
>  (Note how the two functions also disagree in their placement of the
>  "default" labels, even if that's largely benign when taking into
>  account that modes 6 and 7 are transformed to 2 and 3 respectively
>  by pit_load(). A difference would occur only before the guest first
>  sets the mode, as pit_reset() sets it to 7.)

I wouldn't, but as mentioned before I would also avoid touching the
PIT much unless it's fixing an issue that's reproducible.  I think the
chances of us messing up while modifying the code is high due to the
lack of testing.

> 
> Other observations:
> - Loading of new counts occurs too early in some of the modes (2/3: at
>   end of current sequence or when gate goes high; 1/5: only when gate
>   goes high).
> - BCD counting doesn't appear to be properly supported either (at least
>   that's mentioned in the public header).
> 
> [2] https://lists.xen.org/archives/html/xen-devel/2023-05/msg00887.html
> ---
> v2: In pit_load_count() also set count_stop_time from count_load_time
> (in case the counter is stopped). Correct spelling in comments.
> Correct calculations in pit_get_{count,out}().
> 
> --- a/xen/arch/x86/emul-i8254.c
> +++ b/xen/arch/x86/emul-i8254.c
> @@ -65,7 +65,10 @@ static int pit_get_count(PITState *pit,
>  
>  ASSERT(spin_is_locked(&pit->lock));
>  
> -d = muldiv64(get_guest_time(v) - pit->count_load_time[channel],
> +d = pit->hw.channels[channel].gate || (c->mode & 3) == 1
> +? get_guest_time(v)
> +: pit->count_stop_time[channel];
> +d = muldiv64(d - pit->count_load_time[channel] - 
> pit->stopped_time[channel],
>   PIT_FREQ, SYSTEM_TIME_HZ);
>  
>  switch ( c->mode )
> @@ -110,6 +113,10 @@ static void pit_load_count(PITState *pit
>  pit->count_load_time[channel] = 0;
>  else
>  pit->count_load_time[channel] = get_guest_time(v);
> +
> +pit->count_stop_time[channel] = pit->count_load_time[channel];
> +pit->stopped_time[channel] = 0;
> +
>  s->count = val;
>  period = DIV_ROUND(val * SYSTEM_TIME_HZ, PIT_FREQ);
>  
> @@ -148,7 +155,10 @@ static int pit_get_out(PITState *pit, in
>  
>  ASSERT(spin_is_locked(&pit->lock));
>  
> -d = muldiv64(get_guest_time(v) - pit->count_load_time[channel], 
> +d = pit->hw.channels[channel].gate || (s->mode & 3) == 1
> +? get_guest_time(v)
> +: pit->count_stop_time[channel];
> +d = muldiv64(d - pit->count_load_time[channel] - 
> pit->stopped_time[channel],
>   PIT_FREQ, SYSTEM_TIME_HZ);

The above logic is repeated here and in pit_get_count(), maybe could
be abstracted into a common helper to keep both in sync? (get_counter())

Thanks, Roger.



Re: xentrace buffer size, maxcpus and online cpus

2023-06-16 Thread George Dunlap
On Fri, Jun 16, 2023 at 12:52 PM Jan Beulich  wrote:

> On 16.06.2023 13:47, Olaf Hering wrote:
> > Wed, 31 May 2023 11:05:52 +0200 Jan Beulich :
> >
> >> As said before, num_online_cpus() will under-report for the purpose
> >> here, as CPUs may have been brought offline, and may be brought online
> >> again later (independent of the use of "maxcpus=").
> >
> > It turned out, commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39 broke
> > the expected behavior. But to me it is unclear what bug was fixed by
> > this commit.
>
> Hmm, I find title and description quite clear there.
>
[SNIP]

> > To me it looks like commit 74584a367051bc0d6f4b96fd360fa7bc6538fc39
> > could be reverted.
>
> I don't think so. I'll add George to Cc as well, as he's the maintainer
> of this stuff.
>

I agree; the clear implication is that with smt=0, you might have
num_online_cpus() return 4, but cpuids that look like {1, 3, 5, 7}; so you
need the trace buffer array to be of size 8.


> > If I read alloc_trace_bufs correctly, it already operates on online
> > cpus. And __trace_var will do nothing if called on a cpu which was
> > not online, t_bufs will likely be NULL.
>
> Yielding an incomplete overall trace, at best.
>

Historically we've avoided the thorny problems of synchronization wrt the
trace buffers by saying that you get one chance to set them up the way you
want, and that's what you get until you reboot the host.  If someone felt
like sorting all that out I wouldn't oppose it.  But in the meantime, the
current "policy" is that it's OK if *tracing* breaks when changing the
number of cpus, as long as it doesn't cause the *hypervisor* to break.

The proposed change to calculate_tbuf_size() wouldn't work as-is, because
it looks like at the moment alloc_trace_bufs() leaves a gap in the mfn list
for "offline" CPUs.  But it looks like maybe the "interface" would allow
those "holes" to be compacted?

 -George


Re: [PATCH 2/3] acpi/processor: sanitize _PDC buffer bits when running as Xen dom0

2023-06-16 Thread Roger Pau Monné
On Wed, Jun 14, 2023 at 03:57:11PM -0400, Jason Andryuk wrote:
> Hi, Roger,
> 
> On Mon, Nov 21, 2022 at 10:04 AM Roger Pau Monné  wrote:
> >
> > On Mon, Nov 21, 2022 at 03:10:36PM +0100, Jan Beulich wrote:
> > > On 21.11.2022 11:21, Roger Pau Monne wrote:
> > > > --- a/drivers/acpi/processor_pdc.c
> > > > +++ b/drivers/acpi/processor_pdc.c
> > > > @@ -137,6 +137,14 @@ acpi_processor_eval_pdc(acpi_handle handle, struct 
> > > > acpi_object_list *pdc_in)
> > > > buffer[2] &= ~(ACPI_PDC_C_C2C3_FFH | ACPI_PDC_C_C1_FFH);
> > > >
> > > > }
> > > > +   if (xen_initial_domain())
> > > > +   /*
> > > > +* When Linux is running as Xen dom0 it's the hypervisor the
> > > > +* entity in charge of the processor power management, and 
> > > > so
> > > > +* Xen needs to check the OS capabilities reported in the 
> > > > _PDC
> > > > +* buffer matches what the hypervisor driver supports.
> > > > +*/
> > > > +   xen_sanitize_pdc((uint32_t 
> > > > *)pdc_in->pointer->buffer.pointer);
> > > > status = acpi_evaluate_object(handle, "_PDC", pdc_in, NULL);
> > >
> > > Again looking at our old XenoLinux forward port we had this inside the
> > > earlier if(), as an _alternative_ to the &= (I don't think it's valid
> > > to apply both the kernel's and Xen's adjustments). That would also let
> > > you use "buffer" rather than re-calculating it via yet another (risky
> > > from an abstract pov) cast.
> >
> > Hm, I've wondered this and decided it wasn't worth to short-circuit
> > the boot_option_idle_override conditional because ACPI_PDC_C_C2C3_FFH
> > and ACPI_PDC_C_C1_FFH will be set anyway by Xen in
> > arch_acpi_set_pdc_bits() as part of ACPI_PDC_C_CAPABILITY_SMP.
> >
> > I could re-use some of the code in there, but didn't want to make it
> > more difficult to read just for the benefit of reusing buffer.
> >
> > > It was the very nature of requiring Xen-specific conditionals which I
> > > understand was the reason why so far no attempt was made to get this
> > > (incl the corresponding logic for patch 1) into any upstream kernel.
> >
> > Yes, well, it's all kind of ugly.  Hence my suggestion to simply avoid
> > doing any ACPI Processor object handling in Linux with the native code
> > and handle it all in a Xen specific driver.  That requires the Xen
> > driver being able to fetch more data itself form the ACPI Processor
> > methods, but also unties it from the dependency on the data being
> > filled by the generic code, and the 'tricks' is plays into fooling
> > generic code to think certain processors are online.
> 
> Are you working on this patch anymore?

Not really, I didn't get any feedback from maintainers (apart from
Jans comments, which I do value), and wasn't aware of this causing
issues, or being required by any other work, hence I kind of dropped
it (I have plenty of other stuff to work on).

> My Xen HWP patches need a
> Linux patch like this one to set bit 12 in the PDC.  I had an affected
> user test with this patch and it worked, serving as an equivalent of
> Linux commit a21211672c9a ("ACPI / processor: Request native thermal
> interrupt handling via _OSC").
> 
> Another idea is to use Linux's arch_acpi_set_pdc_bits() to make the
> hypercall to Xen.  It occurs earlier:
> acpi_processor_set_pdc()
> acpi_processor_alloc_pdc()
> acpi_set_pdc_bits()
> arch_acpi_set_pdc_bits()
> acpi_processor_eval_pdc()
> 
> So the IDLE_NOMWAIT masking in acpi_processor_eval_pdc() would still
> apply.  arch_acpi_set_pdc_bits() is provided the buffer, so it's a
> little cleaner in that respect.

I see.  My reasoning for placing the Xen filtering in
acpi_processor_eval_pdc() is so that there are no further
modifications to the buffer by Linux after the call to sanitize the
buffer (XENPF_set_processor_pminfo).

I think if the filtering done by Xen is moved to
arch_acpi_set_pdc_bits() we would then need to disable the evaluation
of boot_option_idle_override in acpi_processor_eval_pdc() as we don't
want dom0 choices affecting the selection of _PDC features done by
Xen?

In any case, feel free to pick this patch and re-submit upstream if
you want.

Thanks, Roger.



Re: [XEN PATCH] docs/misra: document the C dialect and translation toolchain assumptions.

2023-06-16 Thread Roberto Bagnara

On 16/06/23 12:03, Jan Beulich wrote:

On 16.06.2023 09:45, Roberto Bagnara wrote:

On 16/06/23 08:53, Jan Beulich wrote:

On 16.06.2023 01:26, Stefano Stabellini wrote:

+   * - Unspecified escape sequence is encountered in a character constant or a 
string literal token
+ - X86_64
+ - \\m:
+  non-documented GCC extension.


Are you saying that we are using \m and \m is not allowed by the C
standard?


This exists in the __ASSEMBLY__ part of a header, and I had previously
commented on Roberto's diagnosis (possibly derived from Eclair's) here.
As per that I don't think the item should be here, but I'm of course
open to be shown that my understanding of translation phases is wrong.


I was not convinced by your explanation but, as I think I have said already,
I am not the one to be convinced.  In the specific case, independently
from __ASSEMBLY__ or any other considerations, that thing reaches the C
preprocessor and, to the best of my knowledge, the C preprocessor documentation
does not say how that would be handled.  I have spent a lot of time in the
past 10 years on the study of functional-safety standards, and what I
am providing is a honest opinion on what I believe is compliant
and what is not.  But I may be wrong of course: if you or anyone else feels
like they would not have any problems in arguing a different position
from mine in front of an assessor, then please go for it, but please
do not ask me to go beyond my judgment.


Well, disagreement on purely a technical matter can usually be resolved,
unless something is truly unspecified. Since you referred to translation
phases, and since I pointed out that preprocessing directives are carried
out before escape sequences are converted to the execution character set
(which is the point where unknown escape sequences would matter afaict),
there must be something you view differently in this process. It would be
helpful if you could point out what this is, possibly leading to me
recognizing a mistake of mine.

Actually, maybe I figured what you're concerned about: Already at the
stage of decomposing into preprocessing-token-s there is an issue, as
e.g. "\mode" doesn't form a valid string-literal. For other, unquoted
\m I would assume though that the final "each non-white-space character
that cannot be one of the above" (in the enumeration of what a
preprocessing-token is) would catch it.


Yes but, more generally, my concern is that the behavior in presence
of unspecified escape sequences is not specified in the C99 standard
and it is not a documented extension according to the documentation
I have examined.  For this reason, I don't think that feature is
usable for safety-related development unless other (potentially
quite expensive) activities are performed (such as prescribing
extra validation activities for the preprocessor).


Furthermore it is entirely unclear to me what it is that you suggest we
do instead. It can't reasonably be "name all you assembler macro
parameters such that they start with a, b, f, n, r, t, or v". Splitting
headers also wouldn't be very nice - we try to keep related things
together, after all. It also doesn't look like __stringify(\mode) would
be okay, as macro expansion shares a translation phase with execution
of preprocessing directives (so in principle the body of "#if 0" could
be macro-expanded before being discarded). (Plus I think this would
result in "\\mode", i.e. also wouldn't work in the first place. But it
would rule out other possible C macro trickery as well.)


My suggestion is avoiding the use of the C preprocessor
outside its specification.  This includes, among other
possibilities:

a) using a different preprocessor or substitution mechanism;
b) amend the preprocessor specification by, e.g., submitting
   patches with suitable additions for "The C Preprocessor"
   manual of GCC.

In view of that, naming macro parameters so that you never
have an unspecified escape sequence is probably the cheapest
(yet bulletproof) solution.
Kind regards,

   Roberto



Re: [PATCH v3 1/4] automation: Add container for ppc64le builds

2023-06-16 Thread Andrew Cooper
On 13/06/2023 3:49 pm, Shawn Anastasio wrote:
> Add a container for cross-compiling xen for ppc64le.
>
> Signed-off-by: Shawn Anastasio 

Acked-by: Andrew Cooper 

I've built and deployed this container properly, replacing the prototype
from earlier.



Re: xentrace buffer size, maxcpus and online cpus

2023-06-16 Thread Olaf Hering
Fri, 16 Jun 2023 15:22:24 +0100 George Dunlap :

> I agree; the clear implication is that with smt=0, you might have
> num_online_cpus() return 4, but cpuids that look like {1, 3, 5, 7}; so you
> need the trace buffer array to be of size 8.

I see. Apparently some remapping is required to map a cpuid to an index
into the trace buffer metadata.


Olaf


pgp_RQUVqiH0Q.pgp
Description: Digitale Signatur von OpenPGP


Re: [XEN PATCH] docs/misra: document the C dialect and translation toolchain assumptions.

2023-06-16 Thread Roberto Bagnara

On 16/06/23 01:26, Stefano Stabellini wrote:

On Thu, 15 Jun 2023, Roberto Bagnara wrote:

This document specifies the C language dialect used by Xen and
the assumptions Xen makes on the translation toolchain.

Signed-off-by: Roberto Bagnara 


Thanks Roberto for the amazing work of research and archaeology.

I have a few comments below, mostly to clarify the description of some
of the less documented GCC extensions, for the purpose of having all
community members be able to understand what they can and cannot use.

+   * - Arithmetic operator on void type
+ - ARM64, X86_64
+ - See Section "6.24 Arithmetic on void- and Function-Pointers" of 
GCC_MANUAL."
+
+   * - GNU statement expression


"GNU statement expression" is not very clear, at least for me. I would
call it "Statements and Declarations in Expressions".


Agreed.


+   * - Empty declaration
+ - ARM64, X86_64
+ - Non-documented GCC extension.


For the non-documented GCC extensions, would it be possible to add a
very brief example or a couple of words in the "References" sections?
Otherwise I think people might not understand what we are talking about.


Ok.


+   * - Incomplete enum declaration
+ - ARM64
+ - Non-documented GCC extension.


Is this 6.49 of the GCC manual perhaps?


Indeed, on a second reading, I think that section covers also the case
of an enum declaration that is never completed in the course of the
translation unit.


+   * - Implicit conversion from a pointer to an incompatible pointer
+ - ARM64, X86_64
+ - Non-documented GCC extension.


Is this related to -Wincompatible-pointer-types?


In my opinion, this does not specify what the result of the
conversion is.


+   * - Pointer to a function is converted to a pointer to an object or a 
pointer to an object is converted to a pointer to a function
+ - X86_64
+ - Non-documented GCC extension.


Is this J.5.7 of n1570?
https://www.iso-9899.info/n1570.html


This says that function pointer casts are a common extension.
What we need here is documentation for GCC that assures us
that the extension is implemented and what its semantics is.


Or maybe we should link https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83584


My opinion is that this might not be accepted by an assessor:
if I was an assessor, I would not accept it.


+   * - Ill-formed source detected by the parser


As we are documenting compiler extensions that we are using, I am a bit
confused by the name of this category of compiler extensions, and the
reason why they are bundled together. After all, they are all separate
compiler extensions? Should each of them have their own row?


Agreed.


+ - ARM64, X86_64
+ - token pasting of ',' and __VA_ARGS__ is a GNU extension:
+  see Section "6.21 Macros with a Variable Number of Arguments" of 
GCC_MANUAL.
+   must specify at least one argument for '...' parameter of variadic 
macro:
+  see Section "6.21 Macros with a Variable Number of Arguments" of 
GCC_MANUAL.
+   void function should not return void expression:


I understand that GCC does a poor job at documenting several of these
extensions. In fact a few of them are not even documented at all.
However, if they are extensions, they should be described for what they
do, not for the rule they violate. What do you think?


The point is that we don't know what they do.  We might make observations,
and our observations might substantiate what we believe they do.
But this would not allow us to generalize them.


For example, in this case maybe we should say "void function can return
a void expression" ?


We can certainly say that, but this might not convince an assessor.
One possibility would be to submit patches to the GCC manual and see
whether they are accepted.


+  see the documentation for -Wreturn-type in Section "3.8 Options to 
Request or Suppress Warnings" of GCC_MANUAL.
+   use of GNU statement expression extension from macro expansion:
+  see Section "6.1 Statements and Declarations in Expressions" of 
GCC_MANUAL.
+   invalid application of sizeof to a void type:
+  see Section "6.24 Arithmetic on void- and Function-Pointers" of 
GCC_MANUAL.
+   redeclaration of already-defined enum is a GNU extension:
+  see Section "6.49 Incomplete enum Types" of GCC_MANUAL.
+   static function is used in an inline function with external linkage:
+  non-documented GCC extension.


I am not sure if I follow about this one. Did you mean "static is used
in an inline function with external linkage" ?


An inline function with external linkage can be inlined everywhere.
If that calls a static functions, which is not available everywhere,
the behavior is not defined.


+   struct may not be nested in a struct due to flexible array member:
+  see Section "6.18 Arrays of Length Zero" of GCC_MANUAL.
+   struct may not be used as an array element due to flexible array member:
+  see Sectio

Re: xentrace buffer size, maxcpus and online cpus

2023-06-16 Thread Andrew Cooper
On 16/06/2023 4:37 pm, Olaf Hering wrote:
> Fri, 16 Jun 2023 15:22:24 +0100 George Dunlap :
>
>> I agree; the clear implication is that with smt=0, you might have
>> num_online_cpus() return 4, but cpuids that look like {1, 3, 5, 7}; so you
>> need the trace buffer array to be of size 8.
> I see. Apparently some remapping is required to map a cpuid to an index
> into the trace buffer metadata.

The xentrace mapping interface is horrible, and makes a lot of
assumptions which date from the early PV-only days.

If you want to improve things, we've got all the building blocks now for
a much more sane interface.

XENMEM_acquire_resource is a new mapping interface with far more sane
semantics which, amongst other things, will work in PVH guests too.

If we specify a new mapping space of type xentrace, using cpu id's as
the sub-index space (see vmtrace as an example), then you'll remove that
entire opencoded mechanism of passing mfns around, as well as reducing
the number of unstable hypercalls that the xentrace infrastructure uses.

I can talk you through it further if you feel up to tackling this.

~Andrew



Re: [PATCH v7 01/12] vpci: introduce per-domain lock to protect vpci structure

2023-06-16 Thread Roger Pau Monné
On Tue, Jun 13, 2023 at 10:32:26AM +, Volodymyr Babchuk wrote:
> From: Oleksandr Andrushchenko 
> 
> Introduce a per-domain read/write lock to check whether vpci is present,
> so we are sure there are no accesses to the contents of the vpci struct
> if not. This lock can be used (and in a few cases is used right away)
> so that vpci removal can be performed while holding the lock in write
> mode. Previously such removal could race with vpci_read for example.
> 
> 1. Per-domain's vpci_rwlock is used to protect pdev->vpci structure
> from being removed.
> 
> 2. Writing the command register and ROM BAR register may trigger
> modify_bars to run, which in turn may access multiple pdevs while
> checking for the existing BAR's overlap. The overlapping check, if done
> under the read lock, requires vpci->lock to be acquired on both devices
> being compared, which may produce a deadlock. It is not possible to
> upgrade read lock to write lock in such a case. So, in order to prevent
> the deadlock, check which registers are going to be written and acquire
> the lock in the appropriate mode from the beginning.
> 
> All other code, which doesn't lead to pdev->vpci destruction and does not
> access multiple pdevs at the same time, can still use a combination of the
> read lock and pdev->vpci->lock.
> 
> 3. Optimize if ROM BAR write lock required detection by caching offset
> of the ROM BAR register in vpci->header->rom_reg which depends on
> header's type.
> 
> 4. Reduce locked region in vpci_remove_device as it is now possible
> to set pdev->vpci to NULL early right after the write lock is acquired.
> 
> 5. Reduce locked region in vpci_add_handlers as it is possible to
> initialize many more fields of the struct vpci before assigning it to
> pdev->vpci.
> 
> 6. vpci_{add|remove}_register are required to be called with the write lock
> held, but it is not feasible to add an assert there as it requires
> struct domain to be passed for that. So, add a comment about this requirement
> to these and other functions with the equivalent constraints.
> 
> 7. Drop const qualifier where the new rwlock is used and this is appropriate.
> 
> 8. Do not call process_pending_softirqs with any locks held. For that unlock
> prior the call and re-acquire the locks after. After re-acquiring the
> lock there is no need to check if pdev->vpci exists:
>  - in apply_map because of the context it is called (no race condition
>possible)
>  - for MSI/MSI-X debug code because it is called at the end of
>pdev->vpci access and no further access to pdev->vpci is made
> 
> 9. Check for !pdev->vpci in vpci_{read|write} after acquiring the lock
> and if so, allow reading or writing the hardware register directly. This is
> acceptable as we only deal with Dom0 as of now. Once DomU support is
> added the write will need to be ignored and read return all 0's for the
> guests, while Dom0 can still access the registers directly.
> 
> 10. Introduce pcidevs_trylock, so there is a possibility to try locking
> the pcidev's lock.
> 
> 11. Use pcidev's lock around for_each_pdev and pci_get_pdev_by_domain
> while accessing pdevs in vpci code.
> 
> 12. This is based on the discussion at [1].
> 
> [1] https://lore.kernel.org/all/20220204063459.680961-4-andr2...@gmail.com/
> 
> Suggested-by: Roger Pau Monné 
> Suggested-by: Jan Beulich 
> Signed-off-by: Oleksandr Andrushchenko 

Thanks.

I haven't looked in full detail, but I'm afraid there's an ABBA
deadlock with the per-domain vpci lock and the pcidevs lock in
modify_bars() vs  vpci_add_handlers() and vpci_remove_device().

I've made some comments below.

> ---
> This was checked on x86: with and without PVH Dom0.
> ---
>  xen/arch/x86/hvm/vmsi.c   |   7 +++
>  xen/common/domain.c   |   3 +
>  xen/drivers/passthrough/pci.c |   5 ++
>  xen/drivers/vpci/header.c |  52 +
>  xen/drivers/vpci/msi.c|  25 +++-
>  xen/drivers/vpci/msix.c   |  51 +---
>  xen/drivers/vpci/vpci.c   | 107 +-
>  xen/include/xen/pci.h |   1 +
>  xen/include/xen/sched.h   |   3 +
>  xen/include/xen/vpci.h|   6 ++
>  10 files changed, 223 insertions(+), 37 deletions(-)
> 
> diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
> index 3cd4923060..f188450b1b 100644
> --- a/xen/arch/x86/hvm/vmsi.c
> +++ b/xen/arch/x86/hvm/vmsi.c
> @@ -895,6 +895,9 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>  {
>  unsigned int i;
>  
> +ASSERT(rw_is_locked(&msix->pdev->domain->vpci_rwlock));
> +ASSERT(pcidevs_locked());
> +
>  for ( i = 0; i < msix->max_entries; i++ )
>  {
>  const struct vpci_msix_entry *entry = &msix->entries[i];
> @@ -913,7 +916,11 @@ int vpci_msix_arch_print(const struct vpci_msix *msix)
>  struct pci_dev *pdev = msix->pdev;
>  
>  spin_unlock(&msix->pdev->vpci->lock);
> +pcidevs_unlock();
> +read_unlock(&pdev->domain->vp

Re: [PATCH v3 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Shawn Anastasio
On Thu Jun 15, 2023 at 1:47 AM CDT, Jan Beulich wrote:
> On 14.06.2023 18:36, Shawn Anastasio wrote:
> > On Wed Jun 14, 2023 at 10:51 AM CDT, Jan Beulich wrote:
> >> On 13.06.2023 16:50, Shawn Anastasio wrote:
> >>> +DECL_SECTION(.bss) { /* BSS */
> >>> +__bss_start = .;
> >>> +*(.bss.stack_aligned)
> >>> +. = ALIGN(PAGE_SIZE);
> >>> +*(.bss.page_aligned)
> >>
> >> ... the one between the two .bss parts looks unmotivated. Within
> >> a section definition ALIGN() typically only makes sense when followed
> >> by a label definition, like ...
> > 
> > Correct me if I'm wrong, but wouldn't the ALIGN here serve to ensure
> > that the subsequent '.bss.page_aligned' section has the correct alignment
> > that its name implies?
>
> Yes and no. Thing is that every contribution to .bss.page_aligned already
> needs to specify page alignment itself, or else it may break if any earlier
> contribution was page-aligned, but not a full page in size (which I think
> the compiler wouldn't allow to happen, but assembly code can result in
> such). Note how this very ALIGN() was recently dropped from RISC-V code,
> and my respective Arm side patch is also about to go in.

That makes sense, thanks. I'll get rid of the ALIGN here in v4.

> Jan

Thanks,
Shawn




[xen-unstable-smoke test] 181466: tolerable all pass - PUSHED

2023-06-16 Thread osstest service owner
flight 181466 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181466/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e0586a4ff514590eec50185e2440b97f9a31cb7f
baseline version:
 xen  c4e492a1399b763400daf3ca35fcef3028c7a7da

Last test of basis   181457  2023-06-15 21:00:25 Z0 days
Testing same since   181466  2023-06-16 14:01:56 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   c4e492a139..e0586a4ff5  e0586a4ff514590eec50185e2440b97f9a31cb7f -> smoke



[PATCH v4 3/4] automation: Add ppc64le cross-build jobs

2023-06-16 Thread Shawn Anastasio
Add build jobs to cross-compile Xen for ppc64le.

Signed-off-by: Shawn Anastasio 
---
 automation/gitlab-ci/build.yaml | 60 +
 1 file changed, 60 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 420ffa5acb..bd8c7332db 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -183,6 +183,33 @@
   variables:
 <<: *gcc
 
+.ppc64le-cross-build-tmpl:
+  <<: *build
+  variables:
+XEN_TARGET_ARCH: ppc64
+  tags:
+- x86_64
+
+.ppc64le-cross-build:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+debug: n
+
+.ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-tmpl
+  variables:
+debug: y
+
+.gcc-ppc64le-cross-build:
+  extends: .ppc64le-cross-build
+  variables:
+<<: *gcc
+
+.gcc-ppc64le-cross-build-debug:
+  extends: .ppc64le-cross-build-debug
+  variables:
+<<: *gcc
+
 .yocto-test:
   stage: build
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
@@ -516,6 +543,39 @@ archlinux-current-gcc-riscv64-debug-randconfig:
 EXTRA_FIXED_RANDCONFIG:
   CONFIG_COVERAGE=n
 
+# Power cross-build
+debian-bullseye-gcc-ppc64le:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+CONTAINER: debian:bullseye-ppc64le
+KBUILD_DEFCONFIG: openpower_defconfig
+HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-debug:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+CONTAINER: debian:bullseye-ppc64le
+KBUILD_DEFCONFIG: openpower_defconfig
+HYPERVISOR_ONLY: y
+
+debian-bullseye-gcc-ppc64le-randconfig:
+  extends: .gcc-ppc64le-cross-build
+  variables:
+CONTAINER: debian:bullseye-ppc64le
+KBUILD_DEFCONFIG: openpower_defconfig
+RANDCONFIG: y
+EXTRA_FIXED_RANDCONFIG:
+  CONFIG_COVERAGE=n
+
+debian-bullseye-gcc-ppc64le-debug-randconfig:
+  extends: .gcc-ppc64le-cross-build-debug
+  variables:
+CONTAINER: debian:bullseye-ppc64le
+KBUILD_DEFCONFIG: openpower_defconfig
+RANDCONFIG: y
+EXTRA_FIXED_RANDCONFIG:
+  CONFIG_COVERAGE=n
+
 # Yocto test jobs
 yocto-qemuarm64:
   extends: .yocto-test-arm64
-- 
2.30.2




[PATCH v4 0/4] Initial support for Power

2023-06-16 Thread Shawn Anastasio
Hello all,

This patch series adds support for building a minimal image
for Power ISA 2.07B+ (POWER8+) systems. In addition to a patch adding
support to the build system and a simple infinite loop at the
entrypoint, patches to add ppc64le support to the CI as well as a
MAINTAINERS update are included.

Since Xen previously had support for a much older version of the ISA in
version 3.2.3, we were able to carry over some headers and support
routines from that version. Unlike that initial port though, this effort
focuses solely on POWER8+ CPUs that are capable of running in Little
Endian mode.

With an appropriate powerpc64le-linux-gnu cross-toolchain, the minimal
image can be built with:

$ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen build

The resulting binary can then be booted in a standard QEMU/pseries VM:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/xen \
-vga none -serial mon:stdio -nographic

Thanks,
Shawn

--
Changes from v4:
  - Change '$(@D)/$(@F)' to '$@' in ppc/Makefile
  - Add -m64, -mlittle-endian to CFLAGS to allow using ppc{32,64be}
toolchains
  - Change IBM-specific '$' to '.' in head.S
  - Drop unnecessary ALIGN() in linker script in .bss

Changes from v3:
  - Fix formatting of MAINTAINERS patch

Changes from v2:
  - Add ppc64le cross-build container patch
  - Add ppc64le cross build CI job patch
  - Drop serial output patch (will be in future patch series)
  - Drop setup.c and unneeded headers from minimal build patch
  - Fixed ordering of MAINTAINERS patch + add F: line
  - Fix config/ppc64.mk option names
  - Clarify Kconfig Baseline ISA option help strings

Shawn Anastasio (4):
  automation: Add container for ppc64le builds
  xen: Add files needed for minimal ppc64le build
  automation: Add ppc64le cross-build jobs
  maintainers: Add ppc64 maintainer

 MAINTAINERS   |   4 +
 .../build/debian/bullseye-ppc64le.dockerfile  |  28 +++
 automation/gitlab-ci/build.yaml   |  60 ++
 automation/scripts/containerize   |   1 +
 config/ppc64.mk   |   5 +
 xen/Makefile  |   5 +-
 xen/arch/ppc/Kconfig  |  42 +
 xen/arch/ppc/Kconfig.debug|   0
 xen/arch/ppc/Makefile |  16 ++
 xen/arch/ppc/Rules.mk |   0
 xen/arch/ppc/arch.mk  |  12 ++
 xen/arch/ppc/configs/openpower_defconfig  |  13 ++
 xen/arch/ppc/include/asm/config.h |  63 +++
 xen/arch/ppc/include/asm/page-bits.h  |   7 +
 xen/arch/ppc/ppc64/Makefile   |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c  |   0
 xen/arch/ppc/ppc64/head.S |  27 +++
 xen/arch/ppc/xen.lds.S| 172 ++
 18 files changed, 454 insertions(+), 2 deletions(-)
 create mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/openpower_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S

--
2.30.2




[PATCH v4 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Shawn Anastasio
Add the build system changes required to build for ppc64le (POWER8+).
As of now the resulting image simply boots to an infinite loop.

$ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
$ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen build

This port targets POWER8+ CPUs running in Little Endian mode specifically,
and does not boot on older machines. Additionally, this initial skeleton
only implements the PaPR/pseries boot protocol which allows it to be
booted in a standard QEMU virtual machine:

$ qemu-system-ppc64 -M pseries-5.2 -m 256M -kernel xen/xen

Signed-off-by: Shawn Anastasio 
---
 config/ppc64.mk  |   5 +
 xen/Makefile |   5 +-
 xen/arch/ppc/Kconfig |  42 ++
 xen/arch/ppc/Kconfig.debug   |   0
 xen/arch/ppc/Makefile|  16 +++
 xen/arch/ppc/Rules.mk|   0
 xen/arch/ppc/arch.mk |  12 ++
 xen/arch/ppc/configs/openpower_defconfig |  13 ++
 xen/arch/ppc/include/asm/config.h|  63 +
 xen/arch/ppc/include/asm/page-bits.h |   7 +
 xen/arch/ppc/ppc64/Makefile  |   1 +
 xen/arch/ppc/ppc64/asm-offsets.c |   0
 xen/arch/ppc/ppc64/head.S|  27 
 xen/arch/ppc/xen.lds.S   | 172 +++
 14 files changed, 361 insertions(+), 2 deletions(-)
 create mode 100644 config/ppc64.mk
 create mode 100644 xen/arch/ppc/Kconfig
 create mode 100644 xen/arch/ppc/Kconfig.debug
 create mode 100644 xen/arch/ppc/Makefile
 create mode 100644 xen/arch/ppc/Rules.mk
 create mode 100644 xen/arch/ppc/arch.mk
 create mode 100644 xen/arch/ppc/configs/openpower_defconfig
 create mode 100644 xen/arch/ppc/include/asm/config.h
 create mode 100644 xen/arch/ppc/include/asm/page-bits.h
 create mode 100644 xen/arch/ppc/ppc64/Makefile
 create mode 100644 xen/arch/ppc/ppc64/asm-offsets.c
 create mode 100644 xen/arch/ppc/ppc64/head.S
 create mode 100644 xen/arch/ppc/xen.lds.S

diff --git a/config/ppc64.mk b/config/ppc64.mk
new file mode 100644
index 00..597f0668c3
--- /dev/null
+++ b/config/ppc64.mk
@@ -0,0 +1,5 @@
+CONFIG_PPC := y
+CONFIG_PPC64 := y
+CONFIG_PPC_$(XEN_OS) := y
+
+CONFIG_XEN_INSTALL_SUFFIX :=
diff --git a/xen/Makefile b/xen/Makefile
index e89fc461fc..db5454fb58 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -38,7 +38,7 @@ EFI_MOUNTPOINT ?= $(BOOT_DIR)/efi
 ARCH=$(XEN_TARGET_ARCH)
 SRCARCH=$(shell echo $(ARCH) | \
   sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
-  -e s'/riscv.*/riscv/g')
+  -e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 export ARCH SRCARCH
 
 # Allow someone to change their config file
@@ -244,7 +244,7 @@ include $(XEN_ROOT)/Config.mk
 export TARGET_SUBARCH  := $(XEN_TARGET_ARCH)
 export TARGET_ARCH := $(shell echo $(XEN_TARGET_ARCH) | \
 sed -e 's/x86.*/x86/' -e s'/arm\(32\|64\)/arm/g' \
--e s'/riscv.*/riscv/g')
+-e s'/riscv.*/riscv/g' -e s'/ppc.*/ppc/g')
 
 export CONFIG_SHELL := $(SHELL)
 export CC CXX LD NM OBJCOPY OBJDUMP ADDR2LINE
@@ -563,6 +563,7 @@ _clean:
$(Q)$(MAKE) $(clean)=xsm
$(Q)$(MAKE) $(clean)=crypto
$(Q)$(MAKE) $(clean)=arch/arm
+   $(Q)$(MAKE) $(clean)=arch/ppc
$(Q)$(MAKE) $(clean)=arch/riscv
$(Q)$(MAKE) $(clean)=arch/x86
$(Q)$(MAKE) $(clean)=test
diff --git a/xen/arch/ppc/Kconfig b/xen/arch/ppc/Kconfig
new file mode 100644
index 00..a0a70adef4
--- /dev/null
+++ b/xen/arch/ppc/Kconfig
@@ -0,0 +1,42 @@
+config PPC
+   def_bool y
+
+config PPC64
+   def_bool y
+   select 64BIT
+
+config ARCH_DEFCONFIG
+   string
+   default "arch/ppc/configs/openpower_defconfig"
+
+menu "Architecture Features"
+
+source "arch/Kconfig"
+
+endmenu
+
+menu "ISA Selection"
+
+choice
+   prompt "Base ISA"
+   default POWER_ISA_2_07B if PPC64
+   help
+ This selects the base ISA version that Xen will target.
+
+config POWER_ISA_2_07B
+   bool "Power ISA 2.07B"
+   help
+ Target version 2.07B of the Power ISA (POWER8)
+
+config POWER_ISA_3_00
+   bool "Power ISA 3.00"
+   help
+ Target version 3.00 of the Power ISA (POWER9)
+
+endchoice
+
+endmenu
+
+source "common/Kconfig"
+
+source "drivers/Kconfig"
diff --git a/xen/arch/ppc/Kconfig.debug b/xen/arch/ppc/Kconfig.debug
new file mode 100644
index 00..e69de29bb2
diff --git a/xen/arch/ppc/Makefile b/xen/arch/ppc/Makefile
new file mode 100644
index 00..98220648af
--- /dev/null
+++ b/xen/arch/ppc/Makefile
@@ -0,0 +1,16 @@
+obj-$(CONFIG_PPC64) += ppc64/
+
+$(TARGET): $(TARGET)-syms
+   cp -f $< $@
+
+$(TARGET)-syms: $(objtree)/prelink.o $(obj)/xen.lds
+   $(LD) $(XEN_LDFLAGS) -T $(obj)/xen.lds -N $< $(build_id_linker) -o $@
+   $(NM) -pa --format=sysv $@ \
+   | $(objtree)/tools/symbols --all-symbols --xensyms --sysv 
--sort \
+

[PATCH v4 1/4] automation: Add container for ppc64le builds

2023-06-16 Thread Shawn Anastasio
Add a container for cross-compiling xen for ppc64le.

Signed-off-by: Shawn Anastasio 
Acked-by: Andrew Cooper 
---
 .../build/debian/bullseye-ppc64le.dockerfile  | 28 +++
 automation/scripts/containerize   |  1 +
 2 files changed, 29 insertions(+)
 create mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile

diff --git a/automation/build/debian/bullseye-ppc64le.dockerfile 
b/automation/build/debian/bullseye-ppc64le.dockerfile
new file mode 100644
index 00..8a87631b52
--- /dev/null
+++ b/automation/build/debian/bullseye-ppc64le.dockerfile
@@ -0,0 +1,28 @@
+FROM debian:bullseye-slim
+LABEL maintainer.name="The Xen Project" \
+  maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+# Add compiler path
+ENV CROSS_COMPILE powerpc64le-linux-gnu-
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+apt-get --quiet --yes --no-install-recommends install \
+bison \
+build-essential \
+checkpolicy \
+flex \
+gawk \
+gcc-powerpc64le-linux-gnu \
+make \
+python3-minimal \
+&& \
+apt-get autoremove -y && \
+apt-get clean && \
+rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index 5476ff0ea1..6d46f63665 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -33,6 +33,7 @@ case "_${CONTAINER}" in
 _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
 _jessie) CONTAINER="${BASE}/debian:jessie" ;;
 _jessie-i386) CONTAINER="${BASE}/debian:jessie-i386" ;;
+_bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;;
 _stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
 _stretch-i386) CONTAINER="${BASE}/debian:stretch-i386" ;;
 _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
-- 
2.30.2




[PATCH v4 4/4] maintainers: Add ppc64 maintainer

2023-06-16 Thread Shawn Anastasio
Signed-off-by: Shawn Anastasio 
---
 MAINTAINERS | 4 
 1 file changed, 4 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 1bb7a6a839..25139fe4a3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -460,6 +460,10 @@ X: xen/arch/x86/acpi/lib.c
 F: xen/drivers/cpufreq/
 F: xen/include/acpi/cpufreq/
 
+PPC64
+M: Shawn Anastasio 
+F: xen/arch/ppc/
+
 PUBLIC I/O INTERFACES AND PV DRIVERS DESIGNS
 M: Juergen Gross 
 S: Supported
-- 
2.30.2




[PATCH 0.5/2] x86/boot: Clean up early error asm

2023-06-16 Thread Andrew Cooper
The asm forming early error handling is a mix of local and non-local symbols,
and has some pointless comments.  Drop the "# Error message" comments,
tweaking the style on modified lines, and make the symbols local.

However, leave behind one real symbol so this logic disassembles nicely
without merging in to acpi_boot_init(), which is the thing that happens to be
immediately prior in my build.

No functional change.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
CC: Alejandro Vallejo 

Done in order to simplfy Alejandro's NX series a little.
---
 xen/arch/x86/boot/head.S | 33 +++--
 1 file changed, 19 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
index 09bebf8635d0..d52dbc752e29 100644
--- a/xen/arch/x86/boot/head.S
+++ b/xen/arch/x86/boot/head.S
@@ -142,25 +142,27 @@ efi_platform:
 
 .section .init.text, "ax", @progbits
 
-bad_cpu:
-add $sym_offs(.Lbad_cpu_msg),%esi   # Error message
+early_error:
+
+.Lbad_cpu:
+add $sym_offs(.Lbad_cpu_msg), %esi
 jmp .Lget_vtb
-not_multiboot:
-add $sym_offs(.Lbad_ldr_msg),%esi   # Error message
+.Lnot_multiboot:
+add $sym_offs(.Lbad_ldr_msg), %esi
 jmp .Lget_vtb
 .Lnot_aligned:
-add $sym_offs(.Lbag_alg_msg),%esi   # Error message
+add $sym_offs(.Lbag_alg_msg), %esi
 jmp .Lget_vtb
 .Lmb2_no_st:
 /*
  * Here we are on EFI platform. vga_text_buffer was zapped earlier
  * because there is pretty good chance that VGA is unavailable.
  */
-add $sym_offs(.Lbad_ldr_nst),%esi   # Error message
+add $sym_offs(.Lbad_ldr_nst), %esi
 jmp .Lget_vtb
 .Lmb2_no_ih:
 /* Ditto. */
-add $sym_offs(.Lbad_ldr_nih),%esi   # Error message
+add $sym_offs(.Lbad_ldr_nih), %esi
 jmp .Lget_vtb
 .Lmb2_no_bs:
 /*
@@ -168,7 +170,7 @@ not_multiboot:
  * via start label. Then reliable vga_text_buffer zap is impossible
  * in Multiboot2 scanning loop and we have to zero %edi below.
  */
-add $sym_offs(.Lbad_ldr_nbs),%esi   # Error message
+add $sym_offs(.Lbad_ldr_nbs), %esi
 xor %edi,%edi   # No VGA text buffer
 jmp .Lprint_err
 .Lmb2_efi_ia_32:
@@ -176,11 +178,11 @@ not_multiboot:
  * Here we are on EFI IA-32 platform. Then reliable vga_text_buffer 
zap is
  * impossible in Multiboot2 scanning loop and we have to zero %edi 
below.
  */
-add $sym_offs(.Lbad_efi_msg),%esi   # Error message
+add $sym_offs(.Lbad_efi_msg), %esi
 xor %edi,%edi   # No VGA text buffer
 jmp .Lprint_err
 .Lget_vtb:
-mov sym_esi(vga_text_buffer),%edi
+mov sym_esi(vga_text_buffer), %edi
 .Lprint_err:
 lodsb
 test%al,%al# Terminate on '\0' sentinel
@@ -202,6 +204,9 @@ not_multiboot:
 .Lhalt: hlt
 jmp .Lhalt
 
+.size early_error, . - early_error
+.type early_error, @function
+
 .code64
 
 __efi64_mb2_start:
@@ -221,8 +226,8 @@ __efi64_mb2_start:
 cmp $MULTIBOOT2_BOOTLOADER_MAGIC,%eax
 je  .Lefi_multiboot2_proto
 
-/* Jump to not_multiboot after switching CPU to x86_32 mode. */
-lea not_multiboot(%rip),%r15
+/* Jump to .Lnot_multiboot after switching CPU to x86_32 mode. */
+lea .Lnot_multiboot(%rip), %r15
 jmp x86_32_switch
 
 .Lefi_multiboot2_proto:
@@ -464,7 +469,7 @@ __start:
 
 /* Check for Multiboot bootloader. */
 cmp $MULTIBOOT_BOOTLOADER_MAGIC,%eax
-jne not_multiboot
+jne .Lnot_multiboot
 
 /* Get mem_lower from Multiboot information. */
 testb   $MBI_MEMLIMITS,MB_flags(%ebx)
@@ -655,7 +660,7 @@ trampoline_setup:
 
 /* Check for availability of long mode. */
 bt  $cpufeat_bit(X86_FEATURE_LM),%edx
-jnc bad_cpu
+jnc .Lbad_cpu
 
 /* Stash TSC to calculate a good approximation of time-since-boot */
 rdtsc
-- 
2.30.2




Re: [PATCH v4 1/4] automation: Add container for ppc64le builds

2023-06-16 Thread Stefano Stabellini
On Fri, 16 Jun 2023, Shawn Anastasio wrote:
> Add a container for cross-compiling xen for ppc64le.
> 
> Signed-off-by: Shawn Anastasio 
> Acked-by: Andrew Cooper 

Acked-by: Stefano Stabellini 

> ---
>  .../build/debian/bullseye-ppc64le.dockerfile  | 28 +++
>  automation/scripts/containerize   |  1 +
>  2 files changed, 29 insertions(+)
>  create mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
> 
> diff --git a/automation/build/debian/bullseye-ppc64le.dockerfile 
> b/automation/build/debian/bullseye-ppc64le.dockerfile
> new file mode 100644
> index 00..8a87631b52
> --- /dev/null
> +++ b/automation/build/debian/bullseye-ppc64le.dockerfile
> @@ -0,0 +1,28 @@
> +FROM debian:bullseye-slim
> +LABEL maintainer.name="The Xen Project" \
> +  maintainer.email="xen-devel@lists.xenproject.org"
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV USER root
> +
> +# Add compiler path
> +ENV CROSS_COMPILE powerpc64le-linux-gnu-
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +# build depends
> +RUN apt-get update && \
> +apt-get --quiet --yes --no-install-recommends install \
> +bison \
> +build-essential \
> +checkpolicy \
> +flex \
> +gawk \
> +gcc-powerpc64le-linux-gnu \
> +make \
> +python3-minimal \
> +&& \
> +apt-get autoremove -y && \
> +apt-get clean && \
> +rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> diff --git a/automation/scripts/containerize b/automation/scripts/containerize
> index 5476ff0ea1..6d46f63665 100755
> --- a/automation/scripts/containerize
> +++ b/automation/scripts/containerize
> @@ -33,6 +33,7 @@ case "_${CONTAINER}" in
>  _focal) CONTAINER="${BASE}/ubuntu:focal" ;;
>  _jessie) CONTAINER="${BASE}/debian:jessie" ;;
>  _jessie-i386) CONTAINER="${BASE}/debian:jessie-i386" ;;
> +_bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;;
>  _stretch|_) CONTAINER="${BASE}/debian:stretch" ;;
>  _stretch-i386) CONTAINER="${BASE}/debian:stretch-i386" ;;
>  _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;;
> -- 
> 2.30.2
> 



Re: [PATCH v4 3/4] automation: Add ppc64le cross-build jobs

2023-06-16 Thread Stefano Stabellini
On Fri, 16 Jun 2023, Shawn Anastasio wrote:
> Add build jobs to cross-compile Xen for ppc64le.
> 
> Signed-off-by: Shawn Anastasio 

Acked-by: Stefano Stabellini 


> ---
>  automation/gitlab-ci/build.yaml | 60 +
>  1 file changed, 60 insertions(+)
> 
> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
> index 420ffa5acb..bd8c7332db 100644
> --- a/automation/gitlab-ci/build.yaml
> +++ b/automation/gitlab-ci/build.yaml
> @@ -183,6 +183,33 @@
>variables:
>  <<: *gcc
>  
> +.ppc64le-cross-build-tmpl:
> +  <<: *build
> +  variables:
> +XEN_TARGET_ARCH: ppc64
> +  tags:
> +- x86_64
> +
> +.ppc64le-cross-build:
> +  extends: .ppc64le-cross-build-tmpl
> +  variables:
> +debug: n
> +
> +.ppc64le-cross-build-debug:
> +  extends: .ppc64le-cross-build-tmpl
> +  variables:
> +debug: y
> +
> +.gcc-ppc64le-cross-build:
> +  extends: .ppc64le-cross-build
> +  variables:
> +<<: *gcc
> +
> +.gcc-ppc64le-cross-build-debug:
> +  extends: .ppc64le-cross-build-debug
> +  variables:
> +<<: *gcc
> +
>  .yocto-test:
>stage: build
>image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> @@ -516,6 +543,39 @@ archlinux-current-gcc-riscv64-debug-randconfig:
>  EXTRA_FIXED_RANDCONFIG:
>CONFIG_COVERAGE=n
>  
> +# Power cross-build
> +debian-bullseye-gcc-ppc64le:
> +  extends: .gcc-ppc64le-cross-build
> +  variables:
> +CONTAINER: debian:bullseye-ppc64le
> +KBUILD_DEFCONFIG: openpower_defconfig
> +HYPERVISOR_ONLY: y
> +
> +debian-bullseye-gcc-ppc64le-debug:
> +  extends: .gcc-ppc64le-cross-build-debug
> +  variables:
> +CONTAINER: debian:bullseye-ppc64le
> +KBUILD_DEFCONFIG: openpower_defconfig
> +HYPERVISOR_ONLY: y
> +
> +debian-bullseye-gcc-ppc64le-randconfig:
> +  extends: .gcc-ppc64le-cross-build
> +  variables:
> +CONTAINER: debian:bullseye-ppc64le
> +KBUILD_DEFCONFIG: openpower_defconfig
> +RANDCONFIG: y
> +EXTRA_FIXED_RANDCONFIG:
> +  CONFIG_COVERAGE=n
> +
> +debian-bullseye-gcc-ppc64le-debug-randconfig:
> +  extends: .gcc-ppc64le-cross-build-debug
> +  variables:
> +CONTAINER: debian:bullseye-ppc64le
> +KBUILD_DEFCONFIG: openpower_defconfig
> +RANDCONFIG: y
> +EXTRA_FIXED_RANDCONFIG:
> +  CONFIG_COVERAGE=n
> +
>  # Yocto test jobs
>  yocto-qemuarm64:
>extends: .yocto-test-arm64
> -- 
> 2.30.2
> 



Re: [PATCH v2 1/2] x86/boot: Clear XD_DISABLE from the early boot path

2023-06-16 Thread Andrew Cooper
On 15/06/2023 4:31 pm, Alejandro Vallejo wrote:
> diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
> index 09bebf8635..ce62eae6f3 100644
> --- a/xen/arch/x86/boot/head.S
> +++ b/xen/arch/x86/boot/head.S
> @@ -142,8 +142,8 @@ efi_platform:
>  
>  .section .init.text, "ax", @progbits
>  
> -bad_cpu:
> -add $sym_offs(.Lbad_cpu_msg),%esi   # Error message
> +.Lbad_cpu:
> +add $sym_offs(.Lbad_cpu_msg),%esi
>  jmp .Lget_vtb
>  not_multiboot:
>  add $sym_offs(.Lbad_ldr_msg),%esi   # Error message
> @@ -647,15 +647,59 @@ trampoline_setup:
>  cpuid
>  1:  mov %edx, CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM) + 
> sym_esi(boot_cpu_data)
>  
> -/* Check for NX. Adjust EFER setting if available. */
> +/* Check for availability of long mode. */
> +bt  $cpufeat_bit(X86_FEATURE_LM),%edx
> +jnc .Lbad_cpu
> +
> +/* Check for NX */
>  bt  $cpufeat_bit(X86_FEATURE_NX), %edx
> +jc .Lhas_nx_bit

This part of the diff is far more legible when rebased over the 0.5/2
patch I've just emailed out.  I've got the rebase locally if you want it.

Although I'd suggest that this label name be .Lgot_nx.

> +
> +/*
> + * NX appears to be unsupported, but it might be hidden.
> + *
> + * Intel CPUs (may) implement MSR_IA32_MISC_ENABLE. Among other
> + * things this MSR has a bit that artificially hides NX support in
> + * CPUID. Xen _really_ wants that feature enabled if present, so we
> + * have to determine if (a) the MSR exists and if so (b) clear the
> + * bit.
> + *
> + * For native boots this is perfectly fine because the MSR was
> + * introduced before Netburst, which was the first family to
> + * provide 64bit support. So we're safe simply accessing it as long
> + * as long mode support has already been checked.
> + *
> + * For the virtualized case the MSR might not be emulated though,
> + * so we make sure to do an initial check for NX in order to bypass
> + * this MSR read

I'd suggest reordering this a bit, and swapping some what for why.  How
about:

---
NX appears to be unsupported, but it might be hidden.

The feature is part of the AMD64 spec, but the very first Intel 64bit
CPUs lacked the feature, and thereafter there was a firmware knob to
disable the feature.  Undo the disable if possible.

All 64bit Intel CPUs support this MSR.  If virtualised, expect the
hypervisor to either emulate the MSR or give us NX.
---

> + */
> +xor %eax,%eax
> +cpuid
> +cmpl$X86_VENDOR_INTEL_EBX,%ebx

Where possible, spaces after commas and without size suffixes (the l on
cmpl here).  While not relevant here

> +jnz .Lno_nx_bit

The "_bit" is still redundant.  Simply .Lno_nx will do.

> +cmpl$X86_VENDOR_INTEL_EDX,%edx
> +jnz .Lno_nx_bit
> +cmpl$X86_VENDOR_INTEL_ECX,%ecx
> +jnz .Lno_nx_bit
> +
> +/* Clear the XD_DISABLE bit */
> +movl$MSR_IA32_MISC_ENABLE, %ecx
> +rdmsr
> +btrl$2, %edx

It's unfortunate that we don't have an asm-safe ilog2().  Oh well.

>  jnc 1f
> -orb $EFER_NXE >> 8, 1 + sym_esi(trampoline_efer)
> -1:
> +wrmsr
> +orb $MSR_IA32_MISC_ENABLE_XD_DISABLE >> 32, 4 + 
> sym_esi(trampoline_misc_enable_off)
>  
> -/* Check for availability of long mode. */
> -bt  $cpufeat_bit(X86_FEATURE_LM),%edx
> -jnc bad_cpu
> +1:  /* Check again for NX */

Failing to clear XD_DISABLE wants to go to .Lno_nx, not to re-scan CPUID
having done nothing to the MSR.

> +mov $0x8001,%eax
> +cpuid
> +bt  $cpufeat_bit(X86_FEATURE_NX), %edx
> +jnc .Lno_nx_bit
> +
> +.Lhas_nx_bit:
> +/* Adjust EFER is NX is present */

"Adjust EFER, given that NX is present".

> diff --git a/xen/arch/x86/cpu/intel.c b/xen/arch/x86/cpu/intel.c
> index 168cd58f36..46b0cd8dbb 100644
> --- a/xen/arch/x86/cpu/intel.c
> +++ b/xen/arch/x86/cpu/intel.c
> @@ -305,23 +305,23 @@ static void cf_check early_init_intel(struct 
> cpuinfo_x86 *c)
>   c->x86_cache_alignment = 128;
>  
>   /* Unmask CPUID levels and NX if masked: */
> - rdmsrl(MSR_IA32_MISC_ENABLE, misc_enable);
> -
> - disable = misc_enable & (MSR_IA32_MISC_ENABLE_LIMIT_CPUID |
> -  MSR_IA32_MISC_ENABLE_XD_DISABLE);
> - if (disable) {
> - wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable & ~disable);
> - bootsym(trampoline_misc_enable_off) |= disable;
> - bootsym(trampoline_efer) |= EFER_NXE;
> - }
> + if (rdmsr_safe(MSR_IA32_MISC_ENABLE, misc_enable) == 0) {

There's no need to change rdmsrl() to rdmsr_safe(), and not doing so
will shrink this diff a lot.

The only thing you need to

Re: [PATCH v2 2/2] x86: Add Kconfig option to require NX bit support

2023-06-16 Thread Andrew Cooper
On 15/06/2023 4:31 pm, Alejandro Vallejo wrote:
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 406445a358..fa97d4 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -307,6 +307,22 @@ config MEM_SHARING
>   bool "Xen memory sharing support (UNSUPPORTED)" if UNSUPPORTED
>   depends on HVM
>  
> +config REQUIRE_NX
> + bool "Require NX bit support"

"Require NX (No eXecute) support".

> + help
> +   No-eXecute (also called XD "eXecute Disable" and DEP "Data
> +   Execution Prevention") is a security feature designed originally
> +   to combat buffer overflow attacks by marking regions of memory
> +   which the CPU must not interpret as instructions.
> +
> +   The NX feature exists in every 64bit CPU except for some very
> +   early Pentium 4 Prescott machines.
> +
> +   Enabling this option will improve Xen's security by removing
> +   cases where Xen could be tricked into thinking that the feature
> +   was unavailable. However, if enabled, Xen will no longer boot on
> +   any CPU which is lacking NX support.
> +
>  endmenu
>  
>  source "common/Kconfig"
> diff --git a/xen/arch/x86/boot/head.S b/xen/arch/x86/boot/head.S
> index ce62eae6f3..ec1e80ef68 100644
> --- a/xen/arch/x86/boot/head.S
> +++ b/xen/arch/x86/boot/head.S
> @@ -123,6 +123,7 @@ multiboot2_header:
>  .Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
>  .Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
>  .Lbag_alg_msg: .asciz "ERR: Xen must be loaded at a 2Mb boundary!"
> +.Lno_nx_bit_msg: .asciz "ERR: Not an NX-bit capable CPU!"

Still two too many "bit"'s in this line.

With these two adjusted, Reviewed-by: Andrew Cooper




Re: [PATCH v4 1/4] automation: Add container for ppc64le builds

2023-06-16 Thread Andrew Cooper
On 16/06/2023 6:48 pm, Shawn Anastasio wrote:
> Add a container for cross-compiling xen for ppc64le.
>
> Signed-off-by: Shawn Anastasio 
> Acked-by: Andrew Cooper 

Just to say that I already committed this patch, when rebuilding the
container.  We likely raced given the time you posted the series, but
you'll want to drop it as part of rebasing onto the staging branch (if
another series is needed).

~Andrew



Re: [PATCH v4 1/4] automation: Add container for ppc64le builds

2023-06-16 Thread Shawn Anastasio
On 6/16/23 2:49 PM, Andrew Cooper wrote:
> On 16/06/2023 6:48 pm, Shawn Anastasio wrote:
>> Add a container for cross-compiling xen for ppc64le.
>>
>> Signed-off-by: Shawn Anastasio 
>> Acked-by: Andrew Cooper 
> Just to say that I already committed this patch, when rebuilding the
> container.  We likely raced given the time you posted the series, but
> you'll want to drop it as part of rebasing onto the staging branch (if
> another series is needed).

Apologies. If a v5 is needed I'll definitely exclude this patch then.

> ~Andrew
Thanks,
Shawn



Re: [PATCH v2] xen/misra: add rules 1.4 and 2.1

2023-06-16 Thread Stefano Stabellini
On Fri, 16 Jun 2023, Luca Fancellu wrote:
> > On 15 Jun 2023, at 22:27, Stefano Stabellini  wrote:
> > 
> > From: Stefano Stabellini 
> > 
> > Also add a comment at the top of the file to say rules.rst could be
> > changed.
> > 
> > Signed-off-by: Stefano Stabellini 
> 
> Hi Stefano,
> 
> Reviewed-by: Luca Fancellu 
> 
> 
> While I was testing the patch with our script that translates the docs to 
> cppcheck
> Inputs, I noticed we might have a small issue there, seems that Directives 
> and Rules
> clashes, and from a quick look to cppcheck addon, seems that only the rules 
> are needed.
> 
> I’ll have a look on that soon.

Noted, thanks!


> > 
> > ---
> > Changes in v2:
> > - add link for 1.4
> > - expand 1.4 comment to say it could be revisited
> > - add comment at the top
> > ---
> > docs/misra/rules.rst | 15 +++
> > 1 file changed, 15 insertions(+)
> > 
> > diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
> > index a88c284e7d..11b9c42b70 100644
> > --- a/docs/misra/rules.rst
> > +++ b/docs/misra/rules.rst
> > @@ -32,6 +32,9 @@ violations are meant to be documented as deviations, 
> > while some others
> > should be fixed. Both compliance and documenting deviations on the
> > existing codebase are work-in-progress.
> > 
> > +The list below might need to be updated over time. Reach out to THE REST
> > +maintainers if you want to suggest a change.
> > +
> > .. list-table::
> >:header-rows: 1
> > 
> > @@ -90,6 +93,18 @@ existing codebase are work-in-progress.
> >behaviour
> >  -
> > 
> > +   * - `Rule 1.4 
> > `_
> > + - Required
> > + - Emergent language features shall not be used
> > + - Emergent language features, such as C11 features, should not be
> > +   confused with similar compiler extensions, which we use. When the
> > +   time comes to adopt C11, this rule will be revisited.
> > +
> > +   * - `Rule 2.1 
> > `_
> > + - Required
> > + - A project shall not contain unreachable code
> > + -
> > +
> >* - `Rule 2.6 
> > `_
> >  - Advisory
> >  - A function should not contain unused label declarations
> > -- 
> > 2.25.1
> > 
> > 
> 
> 

Re: [PATCH v4 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Andrew Cooper
On 16/06/2023 6:48 pm, Shawn Anastasio wrote:
> Add the build system changes required to build for ppc64le (POWER8+).
> As of now the resulting image simply boots to an infinite loop.
>
> $ make XEN_TARGET_ARCH=ppc64 -C xen openpower_defconfig
> $ make XEN_TARGET_ARCH=ppc64 SUBSYSTEMS=xen -C xen build

I think the first of these isn't needed, given the config ARCH_DEFCONFIG
default.  I'd suggest dropping it.

On the second, what is the SUBSYSTEMS=xen?  It's not needed given the
stripped down build system, but I don't see why we'd ever be compiling
Xen with some kind of subsystem configuration for something else.

> diff --git a/config/ppc64.mk b/config/ppc64.mk
> new file mode 100644
> index 00..597f0668c3
> --- /dev/null
> +++ b/config/ppc64.mk
> @@ -0,0 +1,5 @@
> +CONFIG_PPC := y
> +CONFIG_PPC64 := y
> +CONFIG_PPC_$(XEN_OS) := y

I know you're copying the existing architectures, but I'm pretty certain
these $(XEN_OS) expressions are pretty bogus.  The userspace stuff in
tools/ may need to know the host OS it's being built for, but Xen really
doesn't.

I'm pretty sure it will compile with this dropped, so please do.  I'll
see about patching it out of the other architectures.

> diff --git a/xen/arch/ppc/Kconfig b/xen/arch/ppc/Kconfig
> new file mode 100644
> index 00..a0a70adef4
> --- /dev/null
> +++ b/xen/arch/ppc/Kconfig
> @@ -0,0 +1,42 @@
> +config PPC
> + def_bool y
> +
> +config PPC64
> + def_bool y
> + select 64BIT
> +
> +config ARCH_DEFCONFIG
> + string
> + default "arch/ppc/configs/openpower_defconfig"
> +
> +menu "Architecture Features"
> +
> +source "arch/Kconfig"
> +
> +endmenu
> +
> +menu "ISA Selection"
> +
> +choice
> + prompt "Base ISA"
> + default POWER_ISA_2_07B if PPC64
> + help
> +   This selects the base ISA version that Xen will target.
> +
> +config POWER_ISA_2_07B
> + bool "Power ISA 2.07B"
> + help
> +   Target version 2.07B of the Power ISA (POWER8)
> +
> +config POWER_ISA_3_00
> + bool "Power ISA 3.00"
> + help
> +   Target version 3.00 of the Power ISA (POWER9)

For both of these, it will be helpful for anyone who isn't as
PPC-knowledgeable if the POWER8/9 was in the title too, seeing as
they're the most common name.

But as I'm a noob here too, how different are Power8 and 9?  Given they
share a head.S, they're presumably not too disjoint in terms of ISA.

While being able to target a specific CPU is something we're trying to
retrofit to Xen, by default we do expect it to run on as broad a set of
systems as possible.

If that's not feasible, then fine, but if it is, it ought to be the
default.  Which might be as simple as saying "or later" somewhere in
this text, or might be a giant can of worms that I shouldn't open...
 
> diff --git a/xen/arch/ppc/include/asm/page-bits.h 
> b/xen/arch/ppc/include/asm/page-bits.h
> new file mode 100644
> index 00..4c01bf9716
> --- /dev/null
> +++ b/xen/arch/ppc/include/asm/page-bits.h
> @@ -0,0 +1,7 @@
> +#ifndef __PPC_PAGE_BITS_H__
> +#define __PPC_PAGE_BITS_H__
> +
> +#define PAGE_SHIFT  16 /* 64 KiB Pages */
> +#define PADDR_BITS  48

Is 64k the minimum granularity?  Or is 4k an option?

I ask because Xen has some very short sighted ABIs which we're still
working on removing.  There are still quite a few expectations of
PAGE_SHIFT being 12.

To be clear, we're looking to fix all of these ABIs, but I suspect it
will be an easier lift to begin with at 4k.  (Or perhaps the right thing
is to double down and just get them fixed.)

> +
> +#endif /* __PPC_PAGE_BITS_H__ */
> diff --git a/xen/arch/ppc/ppc64/Makefile b/xen/arch/ppc/ppc64/Makefile
> new file mode 100644
> index 00..3340058c08
> --- /dev/null
> +++ b/xen/arch/ppc/ppc64/Makefile
> @@ -0,0 +1 @@
> +obj-y += head.o
> diff --git a/xen/arch/ppc/ppc64/asm-offsets.c 
> b/xen/arch/ppc/ppc64/asm-offsets.c
> new file mode 100644
> index 00..e69de29bb2
> diff --git a/xen/arch/ppc/ppc64/head.S b/xen/arch/ppc/ppc64/head.S
> new file mode 100644
> index 00..0b289c713a
> --- /dev/null
> +++ b/xen/arch/ppc/ppc64/head.S
> @@ -0,0 +1,27 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +.section .text.header, "ax", %progbits
> +
> +ENTRY(start)
> +/*
> + * Depending on how we were booted, the CPU could be running in either
> + * Little Endian or Big Endian mode. The following trampoline from Linux
> + * cleverly uses an instruction that encodes to a NOP if the CPU's
> + * endianness matches the assumption of the assembler (LE, in our case)
> + * or a branch to code that performs the endian switch in the other case.
> + */
> +tdi 0, 0, 0x48/* Reverse endian of b . + 8  */
> +b . + 44  /* Skip trampoline if endian is good  */
> +.long 0xa600607d  /* mfmsr r11  */
> +.long 0x01006b69  /* xori r11,r11,1 */
> +.long 0x4039  /* li r10,0

Re: [PATCH v4 3/4] automation: Add ppc64le cross-build jobs

2023-06-16 Thread Andrew Cooper
On 16/06/2023 6:48 pm, Shawn Anastasio wrote:
> Add build jobs to cross-compile Xen for ppc64le.
>
> Signed-off-by: Shawn Anastasio 

Acked-by: Andrew Cooper 



Re: [PATCH v4 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Andrew Cooper
On 16/06/2023 6:48 pm, Shawn Anastasio wrote:
> diff --git a/xen/arch/ppc/ppc64/head.S b/xen/arch/ppc/ppc64/head.S
> new file mode 100644
> index 00..0b289c713a
> --- /dev/null
> +++ b/xen/arch/ppc/ppc64/head.S
> @@ -0,0 +1,27 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +.section .text.header, "ax", %progbits
> +
> +ENTRY(start)
> +/*
> + * Depending on how we were booted, the CPU could be running in either
> + * Little Endian or Big Endian mode. The following trampoline from Linux
> + * cleverly uses an instruction that encodes to a NOP if the CPU's
> + * endianness matches the assumption of the assembler (LE, in our case)
> + * or a branch to code that performs the endian switch in the other case.
> + */

Sorry, I also meant to ask.  How prevalent is Big Endian in practice in
the Power world?

It's another area (like 4k pages) where I expect there to be plenty of
fun to be had with the codebase.

~Andrew



Re: [XEN v8 1/5] xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64

2023-06-16 Thread Julien Grall

Hi,

On 15/06/2023 09:05, Michal Orzel wrote:

  /*
   * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
   * with IPA bits == PA bits, compare against "pabits".
@@ -2291,6 +2299,7 @@ void __init setup_virt_paging(void)
   */
  if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
  max_vmid = MAX_VMID_16_BIT;
+#endif

  /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
  for ( i = 0; i < ARRAY_SIZE(pa_range_info); i++ )
@@ -2304,26 +2313,41 @@ void __init setup_virt_paging(void)

  /* pa_range is 4 bits but we don't support all modes */

this comment makes sense really only on arm64 as it refers to PARange field of 
ID_AA64MMFR0_EL1.


  if ( pa_range >= ARRAY_SIZE(pa_range_info) || 
!pa_range_info[pa_range].pabits )
-panic("Unknown encoding of ID_AA64MMFR0_EL1.PARange %x\n", pa_range);
+{
+/*
+ * In case of ARM_64, we do not support this encoding of
+ * ID_AA64MMFR0_EL1.PARange
+ */
+panic("Unsupported value for p2m_ipa_bits = 0x%x\n", p2m_ipa_bits);

NIT: Putting variable names in messages visible by users is not a great idea 
IMO.
"Unsupported IPA size" would read better. Furthermore, I do not think printing 
IPA size in hex
is beneficial. I would use "%u bits" (i.e. 32 bits reads better than 0x20 bits).


I went with the following:

-/* pa_range is 4 bits but we don't support all modes */
+/* Check if we found the associated entry in the array */
 if ( pa_range >= ARRAY_SIZE(pa_range_info) || 
!pa_range_info[pa_range].pabits )

-{
-/*
- * In case of ARM_64, we do not support this encoding of
- * ID_AA64MMFR0_EL1.PARange
- */
-panic("Unsupported value for p2m_ipa_bits = 0x%x\n", p2m_ipa_bits);
-}
+panic("%-bit P2M is not supported\n", p2m_ipa_bits);

This should be generic enough.

--
Julien Grall



Re: [XEN v8 1/5] xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64

2023-06-16 Thread Julien Grall




On 16/06/2023 08:59, Michal Orzel wrote:

Hi Julien,

On 15/06/2023 22:32, Julien Grall wrote:



Hi Michal,

I notice you posted some comments but didn't add a Acked-by/Reviewed-by.
Can you indicate if you are happy with the patch so long your comments
are addressed?

If so, are you OK if I deal with them on commit?

I thought I added my tag but clearly not. With the remarks addressed on commit:
Reviewed-by: Michal Orzel 


Thanks. The series is now committed.

Cheers,

--
Julien Grall



Re: [PATCH v4 04/34] pgtable: Create struct ptdesc

2023-06-16 Thread Matthew Wilcox
On Thu, Jun 15, 2023 at 12:57:19AM -0700, Hugh Dickins wrote:
> Probably just trivial collisions in most architectures, which either
> of us can easily adjust to the other; powerpc likely to be more awkward,
> but fairly easily resolved; s390 quite a problem.
> 
> I've so far been unable to post a v2 of my series (and powerpc and s390
> were stupidly wrong in the v1), because a good s390 patch is not yet
> decided - Gerald Schaefer and I are currently working on that, on the
> s390 list (I took off most Ccs until we are settled and I can post v2).
> 
> As you have no doubt found yourself, s390 has sophisticated handling of
> free half-pages already, and I need to add rcu_head usage in there too:
> it's tricky to squeeze it all in, and ptdesc does not appear to help us
> in any way (though mostly it's just changing some field names, okay).
> 
> If ptdesc were actually allowing a flexible structure which architectures
> could add into, that would (in some future) be nice; but of course at
> present it's still fitting it all into one struct page, and mandating
> new restrictions which just make an architecture's job harder.

The intent is to get ptdescs to be dynamically allocated at some point
in the ~2-3 years out future when we have finished the folio project ...
which is not a terribly helpful thing for me to say.

I have three suggestions, probably all dreadful:

1. s390 could change its behaviour to always allocate page tables in
pairs.  That is, it fills in two pmd_t entries any time it takes a fault
in either of them.

2. We could allocate two or four pages at a time for s390 to allocate
2kB pages from.  That gives us a lot more space to store RCU heads.

3. We could use s390 as a guinea-pig for dynamic ptdesc allocation.
Every time we allocate a struct page, we have a slab cache for an
s390-special definition of struct ptdesc, we allocate a ptdesc and store
a pointer to that in compound_head.

We could sweeten #3 by doing that not just for s390 but also for every
configuration which has ALLOC_SPLIT_PTLOCKS today.  That would get rid
of the ambiguity between "is ptl a pointer or a lock".

> But I've no desire to undo powerpc's use of pt_frag_refcount:
> just warning that we may want to undo any use of it in s390.

I would dearly love ppc & s390 to use the _same_ scheme to solve the
same problem.



Re: [PATCH v4 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Julien Grall

Hi,

On 16/06/2023 21:24, Andrew Cooper wrote:

--- /dev/null
+++ b/xen/arch/ppc/ppc64/head.S
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+.section .text.header, "ax", %progbits
+
+ENTRY(start)
+/*
+ * Depending on how we were booted, the CPU could be running in either
+ * Little Endian or Big Endian mode. The following trampoline from Linux
+ * cleverly uses an instruction that encodes to a NOP if the CPU's
+ * endianness matches the assumption of the assembler (LE, in our case)
+ * or a branch to code that performs the endian switch in the other case.
+ */
+tdi 0, 0, 0x48/* Reverse endian of b . + 8  */
+b . + 44  /* Skip trampoline if endian is good  */
+.long 0xa600607d  /* mfmsr r11  */
+.long 0x01006b69  /* xori r11,r11,1 */
+.long 0x4039  /* li r10,0   */
+.long 0x6401417d  /* mtmsrd r10,1   */
+.long 0x05009f42  /* bcl 20,31,$+4  */
+.long 0xa602487d  /* mflr r10   */
+.long 0x14004a39  /* addi r10,r10,20*/
+.long 0xa6035a7d  /* mtsrr0 r10 */
+.long 0xa6037b7d  /* mtsrr1 r11 */
+.long 0x244c  /* rfid   */
+
+/* Now that the endianness is confirmed, continue */
+1:  b 1b


.size start, . - start
.type start, %function


Shouldn't we introduce ENDPROC()/END() to avoid open-coding these two 
lines everywhere?


Cheers,

--
Julien Grall



[xen-unstable-smoke test] 181467: tolerable all pass - PUSHED

2023-06-16 Thread osstest service owner
flight 181467 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181467/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e533438e3d28158602dce051b032811bdd26377d
baseline version:
 xen  e0586a4ff514590eec50185e2440b97f9a31cb7f

Last test of basis   181466  2023-06-16 14:01:56 Z0 days
Testing same since   181467  2023-06-16 18:00:24 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Shawn Anastasio 
  Shawn Anastasio 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   e0586a4ff5..e533438e3d  e533438e3d28158602dce051b032811bdd26377d -> smoke



Re: [XEN PATCH] docs/misra: document the C dialect and translation toolchain assumptions.

2023-06-16 Thread Stefano Stabellini
On Fri, 16 Jun 2023, Roberto Bagnara wrote:
> > > +   * - Implicit conversion from a pointer to an incompatible pointer
> > > + - ARM64, X86_64
> > > + - Non-documented GCC extension.
> > 
> > Is this related to -Wincompatible-pointer-types?
> 
> In my opinion, this does not specify what the result of the
> conversion is.

Fair enough. However, if -Wincompatible-pointer-types and "Implicit
conversion from a pointer to an incompatible pointer" are related, it
would add -Wincompatible-pointer-types as extra info about it. See also
below.


> > > +   * - Pointer to a function is converted to a pointer to an object or a
> > > pointer to an object is converted to a pointer to a function
> > > + - X86_64
> > > + - Non-documented GCC extension.
> > 
> > Is this J.5.7 of n1570?
> > https://www.iso-9899.info/n1570.html
> 
> This says that function pointer casts are a common extension.
> What we need here is documentation for GCC that assures us
> that the extension is implemented and what its semantics is.
> 
> > Or maybe we should link https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83584
> 
> My opinion is that this might not be accepted by an assessor:
> if I was an assessor, I would not accept it.

I understand your point and I think it is valid. My observation was
that it is better to provide as much information for these undocumented
extensions as we can. Not necessarily to help with an assessors, but for
a new engineer working on this project, reading this document and
understanding what can be done. 

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83584 might not be an
official documentation of the extension but it is better than no
documentation at all. Even better might be a code example.

I am not saying we should document ourselves what GCC failed to
document. I am only saying we should add enough description to
understand what we are talking about.

For instance, I read "Pointer to a function is converted to a pointer to
an object or a pointer to an object is converted to a pointer to a
function" and I have an idea about what this is but I am not really
sure. I googled the sentence and found information on Stackoverflow. I
think it is better to link
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83584 or a couple of
sentences from it, although it might not be official.


> > > +   * - Ill-formed source detected by the parser
> > 
> > As we are documenting compiler extensions that we are using, I am a bit
> > confused by the name of this category of compiler extensions, and the
> > reason why they are bundled together. After all, they are all separate
> > compiler extensions? Should each of them have their own row?
> 
> Agreed.
> 
> > > + - ARM64, X86_64
> > > + - token pasting of ',' and __VA_ARGS__ is a GNU extension:
> > > +  see Section "6.21 Macros with a Variable Number of Arguments"
> > > of GCC_MANUAL.
> > > +   must specify at least one argument for '...' parameter of variadic
> > > macro:
> > > +  see Section "6.21 Macros with a Variable Number of Arguments"
> > > of GCC_MANUAL.
> > > +   void function should not return void expression:
> > 
> > I understand that GCC does a poor job at documenting several of these
> > extensions. In fact a few of them are not even documented at all.
> > However, if they are extensions, they should be described for what they
> > do, not for the rule they violate. What do you think?
> 
> The point is that we don't know what they do.  We might make observations,
> and our observations might substantiate what we believe they do.
> But this would not allow us to generalize them.
>
> > For example, in this case maybe we should say "void function can return
> > a void expression" ?
> 
> We can certainly say that, but this might not convince an assessor.
> One possibility would be to submit patches to the GCC manual and see
> whether they are accepted.

I think we have two different target audiences for this document. One
target is an assessors, and I understand that extra unofficial
information might not help there.

However another target is the community. This document should help the
Xen community write better code, not just the assessors raise red flags.
Right? It should help us have better compiler compatibility, and making
sure that we are clear about the C dialect we use. Actually, I think
this document could be of great help. Do you agree?

>From that point of view "void function should not return void
expression" is not understandable. At least I don't understand it.

A different approach would be to say:

- this is a MISRA C violation or compiler warning/error
- it is not C99 compliant
- it is not documented behavior by GCC

Not try to describe what the extension is at all, and instead focus on
what the MISRA C violation or compiler warning is.

I think it is OK to go down that route, but in that case we need to
reorganize the document so that:
- all documented extensions are referred to as extensions
- all undocumen

Re: [PATCH v3 8/8] SUPPORT.md: write down restriction of 32-bit tool stacks

2023-06-16 Thread Julien Grall

Hi Jan,

Sorry for the late reply.

On 26/04/2022 11:27, Jan Beulich wrote:

Let's try to avoid giving the impression that 32-bit tool stacks are as
capable as 64-bit ones.

Signed-off-by: Jan Beulich 


Acked-by: Julien Grall 

Cheers,

--
Julien Grall



Re: Asking for help to debug xen efi on Kunpeng machine

2023-06-16 Thread Julien Grall

Hi,

On 15/06/2023 22:52, Jiatong Shen wrote:

 Thank you for your answer! Adding console=hvc0 indeed provides kernel
output serial console. Looking at the log message,
I found dom0 kernel failed to initialize a lot of device drivers (network
cards, raid cards, etc).

Is it related to iommu? 


I am afraid I can't tell without seen some logs.

>I remember iommu is not enabled during the xen

kernel booting stage.


At the moment, the IOMMU is not hidden from dom0. So Linux may try to 
use it. However, IIRC, it is only becoming a problem when booting a guest.


Best regards,

---
Julien Grall



Re: Refactoring of a possibly unsafe pattern for variable initialization via function calls

2023-06-16 Thread Stefano Stabellini
On Fri, 16 Jun 2023, Nicola Vetrini wrote:
> On 16/06/23 09:19, Jan Beulich wrote:
> > On 15.06.2023 18:39, nicola wrote:
> > > while investigating possible patches regarding Mandatory Rule 9.1, I
> > > found the following pattern, that is likely to results in a lot possible
> > > positives from many (all) static analysis tools for this rule.
> > > 
> > > This is the current status (taken from `xen/common/device_tree.c:135')
> > > 
> > > 
> > > const struct dt_property *dt_find_property(const struct dt_device_node
> > > *np,
> > >  const char *name, u32 *lenp)
> > > {
> > >   const struct dt_property *pp;
> > > 
> > >   if ( !np )
> > >   return NULL;
> > > 
> > >   for ( pp = np->properties; pp; pp = pp->next )
> > >   {
> > >   if ( dt_prop_cmp(pp->name, name) == 0 )
> > >   {
> > >   if ( lenp )
> > >   *lenp = pp->length;
> > >   break;
> > >   }
> > >   }
> > > 
> > >   return pp;
> > > }
> > > 
> > > 
> > > 
> > > 
> > > It's very hard to detect that the pointee is always written whenever a
> > > non-NULL pointer for `lenp' is supplied, and it can safely be read in
> > > the callee, so a sound analysis will err on the cautious side.
> > 
> > I'm having trouble seeing why this is hard to recognize: The loop can
> > only be exited two ways: pp == NULL or with *lenp written.
> > 
> > For rule 9.1 I'd rather expect the scanning tool (and often the compiler)
> > to get into trouble with the NULL return value case, and *lenp not being
> > written yet apparently consumed in the caller. Then, however, ...
> 
> 
> You're right, I made a mistake, thank you for finding it.
> I meant to write on `*lenp' in all execution paths.
> Please, take a look at this revised version:
> 
> 
> const struct dt_property *dt_find_property(const struct dt_device_node *np,
>const char *name, u32 *lenp)
> {
> u32 len = 0;
> const struct dt_property *pp = NULL;
> 
> if ( np )
> {
> for ( pp = np->properties; pp; pp = pp->next )
> {
> if ( dt_prop_cmp(pp->name, name) == 0 )
> {
> len = pp->length;
> break;
> }
> }
> }
> 
> if ( lenp )
> *lenp = len;
> return pp;
> }

Nesting more will make the code less readable and also cause other code
quality metrics to deteriorate (cyclomatic complexity).

Would the below work?


const struct dt_property *dt_find_property(const struct dt_device_node *np,
   const char *name, u32 *lenp)
{
u32 len = 0;
const struct dt_property *pp = NULL;

if ( !np )
return NULL

for ( pp = np->properties; pp; pp = pp->next )
{
if ( dt_prop_cmp(pp->name, name) == 0 )
{
len = pp->length;
break;
}
}

if ( lenp )
*lenp = len;
return pp;
}




Re: [PATCH v4 2/4] xen: Add files needed for minimal ppc64le build

2023-06-16 Thread Shawn Anastasio
On 6/16/23 3:27 PM, Andrew Cooper wrote:
> Sorry, I also meant to ask.  How prevalent is Big Endian in practice in
> the Power world?

Modern systems support operating in either endianness, but historically
most operating systems targeted Big Endian-only, and some older systems
didn't support Little Endian at all.

These days Little Endian is more prevalent (at least in the Open Source
world), and many Linux distributions only target LE. Despite this though,
most firmware on Power systems still operates in Big Endian mode
exclusively,
so it's the responsibility of the kernel to handle the switch to LE at
its entrypoint.

The FW being BE also needs to be considered whenever the kernel calls
into firmware, since it's the responsibility of the kernel to switch to BE
before making the call, and also to switch itself back to LE after the FW
routine returns. Typically it's just handled by sprinkling this trampoline
(via a macro) at all of the entry/return points.

> It's another area (like 4k pages) where I expect there to be plenty of
> fun to be had with the codebase.

Hopefully the choice to run in Little Endian mode will reduce the number
of pain points encountered.

Also it's worth mentioning that both 4k and 64k pages are supported,
though 64k is probably the more common choice.

> ~Andrew

Thanks,
Shawn







[xen-unstable test] 181460: trouble: broken/fail/pass

2023-06-16 Thread osstest service owner
flight 181460 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181460/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-vhd  broken

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-vhd   5 host-install(5) broken like 181440
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail like 
181440
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 181440
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 181440
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 181440
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 181440
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 181440
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 181440
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 181440
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 181440
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 181440
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 181440
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 181440
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 181440
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-arm64-arm64-xl-vhd  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-checkfail never pass

version targeted for testing:
 xen  1512a68721620338daf477bf717d23befbb4faf5
baseline version:
 xen  87c621d0ef75e5f95987d66811ed1fd7129208d1

Last test of basis   181440  2023-06-15 06:54:36 Z1 days
Testing same since   181460  2023-06-16 00:10:47 Z0 days1 attempts

---

[qemu-mainline test] 181463: regressions - FAIL

2023-06-16 Thread osstest service owner
flight 181463 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181463/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 180691
 build-arm64   6 xen-buildfail REGR. vs. 180691
 build-arm64-xsm   6 xen-buildfail REGR. vs. 180691
 build-amd64   6 xen-buildfail REGR. vs. 180691
 build-i3866 xen-buildfail REGR. vs. 180691
 build-i386-xsm6 xen-buildfail REGR. vs. 180691
 build-armhf   6 xen-buildfail REGR. vs. 180691

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-vhd1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-vhd   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-de

Re: [PATCH v4 04/34] pgtable: Create struct ptdesc

2023-06-16 Thread Vishal Moola
On Thu, Jun 15, 2023 at 12:57 AM Hugh Dickins  wrote:
>
> On Mon, 12 Jun 2023, Vishal Moola (Oracle) wrote:
>
> > Currently, page table information is stored within struct page. As part
> > of simplifying struct page, create struct ptdesc for page table
> > information.
> >
> > Signed-off-by: Vishal Moola (Oracle) 
>
> Vishal, as I think you have already guessed, your ptdesc series and
> my pte_free_defer() "mm: free retracted page table by RCU" series are
> on a collision course.
>
> Probably just trivial collisions in most architectures, which either
> of us can easily adjust to the other; powerpc likely to be more awkward,
> but fairly easily resolved; s390 quite a problem.
>
> I've so far been unable to post a v2 of my series (and powerpc and s390
> were stupidly wrong in the v1), because a good s390 patch is not yet
> decided - Gerald Schaefer and I are currently working on that, on the
> s390 list (I took off most Ccs until we are settled and I can post v2).
>
> As you have no doubt found yourself, s390 has sophisticated handling of
> free half-pages already, and I need to add rcu_head usage in there too:
> it's tricky to squeeze it all in, and ptdesc does not appear to help us
> in any way (though mostly it's just changing some field names, okay).
>
> If ptdesc were actually allowing a flexible structure which architectures
> could add into, that would (in some future) be nice; but of course at
> present it's still fitting it all into one struct page, and mandating
> new restrictions which just make an architecture's job harder.

A goal of ptdescs is to make architecture's jobs simpler and standardized.
Unfortunately, ptdescs are nowhere near isolated from struct page yet.
This version of struct ptdesc contains the exact number of fields architectures
need right now, just reorganized to be located next to each other. It *probably*
shouldn't make an architectures job harder, aside from discouraging their use
of yet even more members of struct page.

> Some notes on problematic fields below FYI.
>
> > ---
> >  include/linux/pgtable.h | 51 +
> >  1 file changed, 51 insertions(+)
> >
> > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> > index c5a51481bbb9..330de96ebfd6 100644
> > --- a/include/linux/pgtable.h
> > +++ b/include/linux/pgtable.h
> > @@ -975,6 +975,57 @@ static inline void ptep_modify_prot_commit(struct 
> > vm_area_struct *vma,
> >  #endif /* __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION */
> >  #endif /* CONFIG_MMU */
> >
> > +
> > +/**
> > + * struct ptdesc - Memory descriptor for page tables.
> > + * @__page_flags: Same as page flags. Unused for page tables.
> > + * @pt_list: List of used page tables. Used for s390 and x86.
> > + * @_pt_pad_1: Padding that aliases with page's compound head.
> > + * @pmd_huge_pte: Protected by ptdesc->ptl, used for THPs.
> > + * @_pt_s390_gaddr: Aliases with page's mapping. Used for s390 gmap only.
> > + * @pt_mm: Used for x86 pgds.
> > + * @pt_frag_refcount: For fragmented page table tracking. Powerpc and s390 
> > only.
> > + * @ptl: Lock for the page table.
> > + *
> > + * This struct overlays struct page for now. Do not modify without a good
> > + * understanding of the issues.
> > + */
> > +struct ptdesc {
> > + unsigned long __page_flags;
> > +
> > + union {
> > + struct list_head pt_list;
>
> I shall be needing struct rcu_head rcu_head (or pt_rcu_head or whatever,
> if you prefer) in this union too.  Sharing the lru or pt_list with rcu_head
> is what's difficult to get right and efficient on s390 - and if ptdesc gave
> us an independent rcu_head for each page table, that would be a blessing!
> but sadly not, it still has to squeeze into a struct page.

I can add a pt_rcu_head along with a comment to deter aliasing issues :)
Independent rcu_heads aren't coming any time soon though :(

> > + struct {
> > + unsigned long _pt_pad_1;
> > + pgtable_t pmd_huge_pte;
> > + };
> > + };
> > + unsigned long _pt_s390_gaddr;
> > +
> > + union {
> > + struct mm_struct *pt_mm;
> > + atomic_t pt_frag_refcount;
>
> Whether s390 will want pt_mm is not yet decided: I want to use it,
> Gerald prefers to go without it; but if we do end up using it,
> then pt_frag_refcount is a luxury we would have to give up.

I don't like the use of pt_mm for s390 either. s390 uses space equivalent
to all five words allocated in the page table struct (albeit in various places
of struct page). Using extra space (especially allocated for unrelated
reasons) just because it exists makes things more complicated and
confusing, and s390 is already confusing enough as a result of that.

If having access to pt_mm is necessary I can drop the
pt_frag_refcount patch, but I'd rather avoid it.

> s390 does very well already with its _refcount tricks, and I'd expect
> powerpc's simpler but more wasteful implementation to work as w

[xen-unstable-smoke test] 181470: regressions - trouble: blocked/broken/pass

2023-06-16 Thread osstest service owner
flight 181470 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181470/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-armhf  broken
 build-armhf   5 host-build-prep  fail REGR. vs. 181467

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  7a25a1501ca941c3e01b0c4e624ace05417f1587
baseline version:
 xen  e533438e3d28158602dce051b032811bdd26377d

Last test of basis   181467  2023-06-16 18:00:24 Z0 days
Testing same since   181470  2023-06-16 21:08:31 Z0 days1 attempts


People who touched revisions under test:
  Ayan Kumar Halder 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  broken  
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job build-armhf broken

Not pushing.


commit 7a25a1501ca941c3e01b0c4e624ace05417f1587
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:54 2023 +0100

xen/arm: p2m: Enable support for 32bit IPA for ARM_32

Refer ARM DDI 0406C.d ID040418, B3-1345,

"A stage 2 translation with an input address range of 31-34 bits can
start the translation either:

- With a first-level lookup, accessing a first-level translation
  table with 2-16 entries.

- With a second-level lookup, accessing a set of concatenated
  second-level translation tables"

Thus, for 32 bit IPA, there will be no concatenated root level tables.
So, the root-order is 0.

Also, Refer ARM DDI 0406C.d ID040418, B3-1348
"Determining the required first lookup level for stage 2 translations

For a stage 2 translation, the output address range from the stage 1
translations determines the required input address range for the stage 2
translation. The permitted values of VTCR.SL0 are:
0b00 Stage 2 translation lookup must start at the second level.
0b01 Stage 2 translation lookup must start at the first level.

VTCR.T0SZ must indicate the required input address range. The size of
the input address region is 2^(32-T0SZ) bytes."

Thus VTCR.SL0 = 1 (maximum value) and VTCR.T0SZ = 0 when the size of
input address region is 2^32 bytes.

Signed-off-by: Ayan Kumar Halder 
Reviewed-by: Michal Orzel 
Acked-by: Julien Grall 

commit 7c72147baa221cb49da80498bb0360c4d24a759f
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:53 2023 +0100

xen/arm: Restrict zeroeth_table_offset for ARM_64

When 32 bit physical addresses are used (ie PHYS_ADDR_T_32=y),
"va >> ZEROETH_SHIFT" causes an overflow.
Also, there is no zeroeth level page table on Arm32.

Also took the opportunity to clean up dump_pt_walk(). One could use
DECLARE_OFFSETS() macro instead of declaring an array of page table
offsets.

Signed-off-by: Ayan Kumar Halder 
Reviewed-by: Michal Orzel 
Acked-by: Julien Grall 

commit c3aabf7bd20eefa2c0fa297e53e087126ed9a06a
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:52 2023 +0100

xen/arm: guest_walk: LPAE specific bits should be enclosed within "ifndef 
CONFIG_PHYS_ADDR_T_32"

As the previous patch introduces CONFIG_PHYS_ADDR_T_32 to support 32 bit
physical addresses, the code specific to "Large Physical Address Extension"
(ie LPAE) should be enclosed within "ifndef CONFIG_PHYS_ADDR_T_32".

Refer xen/

[linux-5.4 test] 181462: regressions - trouble: broken/fail/pass/starved

2023-06-16 Thread osstest service owner
flight 181462 linux-5.4 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181462/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-rtds broken
 test-armhf-armhf-xl  broken  in 181448
 build-amd64-pvops 6 kernel-build   fail in 181448 REGR. vs. 181363
 build-i386-pvops  6 kernel-build   fail in 181448 REGR. vs. 181363
 build-arm64-pvops 6 kernel-build   fail in 181448 REGR. vs. 181363

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl  5 host-install(5) broken in 181448 pass in 181462
 test-armhf-armhf-xl-rtds  5 host-install(5)  broken pass in 181448
 test-armhf-armhf-libvirt-raw 13 guest-start  fail in 181448 pass in 181462
 test-armhf-armhf-libvirt-qcow2 12 debian-di-installfail pass in 181448

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 
181448 n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
in 181448 n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)blocked in 181448 n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 
181448 n/a
 test-arm64-arm64-examine  1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-examine   1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-examine-bios  1 build-check(1)  blocked in 181448 n/a
 test-amd64-i386-pair  1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)blocked in 181448 n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)  blocked in 181448 n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)blocked in 181448 n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-examine-uefi  1 build-check(1)  blocked in 181448 n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1) blocked in 181448 n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)blocked in 181448 n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 1 build-check(1) blocked in 181448 
n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-vhd1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)blocked in 181448 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow 1 build-check(1) blocked in 
181448 n/a
 test-amd64-amd64-examine  1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked in 181448 n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked in 
181448 n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64 1 build-check(1) blocked in 181448 
n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
in 181448 n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 181448 
n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-examine-uefi  1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked in 181448 n/a
 test-arm64-arm64-xl-vhd   1 build-check(1)   blocked in 181448 n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked in 181448 n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)  blocked in 181448 n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64 1 build-check(1) blocked in 181448 n/a
 test-amd64-i386-qemut-rhel6hvm-amd  1 build-check(1) blocked in 181448 n/a
 test-

[xen-unstable-smoke test] 181473: trouble: broken/pass

2023-06-16 Thread osstest service owner
flight 181473 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181473/

Failures and problems with tests :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl  broken
 test-armhf-armhf-xl   5 host-install(5)broken REGR. vs. 181467

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  7a25a1501ca941c3e01b0c4e624ace05417f1587
baseline version:
 xen  e533438e3d28158602dce051b032811bdd26377d

Last test of basis   181467  2023-06-16 18:00:24 Z0 days
Testing same since   181470  2023-06-16 21:08:31 Z0 days2 attempts


People who touched revisions under test:
  Ayan Kumar Halder 
  Julien Grall 
  Stefano Stabellini 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  broken  
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-job test-armhf-armhf-xl broken
broken-step test-armhf-armhf-xl host-install(5)

Not pushing.


commit 7a25a1501ca941c3e01b0c4e624ace05417f1587
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:54 2023 +0100

xen/arm: p2m: Enable support for 32bit IPA for ARM_32

Refer ARM DDI 0406C.d ID040418, B3-1345,

"A stage 2 translation with an input address range of 31-34 bits can
start the translation either:

- With a first-level lookup, accessing a first-level translation
  table with 2-16 entries.

- With a second-level lookup, accessing a set of concatenated
  second-level translation tables"

Thus, for 32 bit IPA, there will be no concatenated root level tables.
So, the root-order is 0.

Also, Refer ARM DDI 0406C.d ID040418, B3-1348
"Determining the required first lookup level for stage 2 translations

For a stage 2 translation, the output address range from the stage 1
translations determines the required input address range for the stage 2
translation. The permitted values of VTCR.SL0 are:
0b00 Stage 2 translation lookup must start at the second level.
0b01 Stage 2 translation lookup must start at the first level.

VTCR.T0SZ must indicate the required input address range. The size of
the input address region is 2^(32-T0SZ) bytes."

Thus VTCR.SL0 = 1 (maximum value) and VTCR.T0SZ = 0 when the size of
input address region is 2^32 bytes.

Signed-off-by: Ayan Kumar Halder 
Reviewed-by: Michal Orzel 
Acked-by: Julien Grall 

commit 7c72147baa221cb49da80498bb0360c4d24a759f
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:53 2023 +0100

xen/arm: Restrict zeroeth_table_offset for ARM_64

When 32 bit physical addresses are used (ie PHYS_ADDR_T_32=y),
"va >> ZEROETH_SHIFT" causes an overflow.
Also, there is no zeroeth level page table on Arm32.

Also took the opportunity to clean up dump_pt_walk(). One could use
DECLARE_OFFSETS() macro instead of declaring an array of page table
offsets.

Signed-off-by: Ayan Kumar Halder 
Reviewed-by: Michal Orzel 
Acked-by: Julien Grall 

commit c3aabf7bd20eefa2c0fa297e53e087126ed9a06a
Author: Ayan Kumar Halder 
Date:   Fri Jun 2 13:07:52 2023 +0100

xen/arm: guest_walk: LPAE specific bits should be enclosed within "ifndef 
CONFIG_PHYS_ADDR_T_32"

As the previous patch introduces CONFIG_PHYS_ADDR_T_32 to support 32 bit
physical addresses, the code specific to "Large Physical Address Extension"
(ie LPAE) should be enclosed within "ifndef CONFIG_PHYS_ADDR_T_32".

Refer xe

[qemu-mainline test] 181472: regressions - FAIL

2023-06-16 Thread osstest service owner
flight 181472 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/181472/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 180691
 build-arm64   6 xen-buildfail REGR. vs. 180691
 build-arm64-xsm   6 xen-buildfail REGR. vs. 180691
 build-amd64   6 xen-buildfail REGR. vs. 180691
 build-i3866 xen-buildfail REGR. vs. 180691
 build-i386-xsm6 xen-buildfail REGR. vs. 180691
 build-armhf   6 xen-buildfail REGR. vs. 180691

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-shadow 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-vhd1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-raw  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit1   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-credit2   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-vhd   1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-raw   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-de

[linux-linus test] 181464: regressions - FAIL

2023-06-16 Thread osstest service owner
flight 181464 linux-linus real [real]
flight 181475 linux-linus real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/181464/
http://logs.test-lab.xenproject.org/osstest/logs/181475/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 180278

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-pvhv2-intel 22 guest-start/debian.repeat fail pass in 
181475-retest
 test-arm64-arm64-libvirt-raw 17 guest-start/debian.repeat fail pass in 
181475-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-multivcpu  8 xen-boot fail like 180278
 test-armhf-armhf-xl   8 xen-boot fail  like 180278
 test-armhf-armhf-xl-credit2   8 xen-boot fail  like 180278
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 180278
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 180278
 test-armhf-armhf-libvirt-raw  8 xen-boot fail  like 180278
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 180278
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 180278
 test-armhf-armhf-libvirt  8 xen-boot fail  like 180278
 test-armhf-armhf-xl-arndale   8 xen-boot fail  like 180278
 test-armhf-armhf-examine  8 reboot   fail  like 180278
 test-armhf-armhf-libvirt-qcow2  8 xen-bootfail like 180278
 test-armhf-armhf-xl-rtds  8 xen-boot fail  like 180278
 test-armhf-armhf-xl-vhd   8 xen-boot fail  like 180278
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 180278
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qcow2 14 migrate-support-checkfail never pass
 test-arm64-arm64-xl-vhd  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  15 saverestore-support-checkfail   never pass

version targeted for testing:
 linux40f71e7cd3c6ac04293556ab0504a372393838ff
baseline version:
 linux6c538e1adbfc696ac4747fb10d63e704344f763d

Last test of basis   180278  2023-04-16 19:41:46 Z   61 days
Failing since180281  2023-04-17 06:24:36 Z   61 days  115 attempts
Testing same since   181464  2023-06-16 10:38:42 Z0 days1 attempts


2717 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-arm64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-arm64-pvopspass