Re: [Qemu-devel] [PATCH] tests/migration: Enable the migration test on s390x, too

2018-08-23 Thread Cornelia Huck
On Thu, 23 Aug 2018 17:00:55 +0200 Thomas Huth wrote: > On 2018-08-23 16:54, Cornelia Huck wrote: > > On Thu, 23 Aug 2018 14:25:20 +0200 > > Thomas Huth wrote: > > > >> We can re-use the s390-ccw bios code to implement a small firmware > >> for a s390x guest which prints out the "A" and "B" c

[Qemu-devel] [PULL 3/4] configure: require libseccomp 2.2.0

2018-08-23 Thread Eduardo Otubo
From: Marc-André Lureau The following patch is going to require TSYNC, which is only available since libseccomp 2.2.0. libseccomp 2.2.0 was released February 12, 2015. According to repology, libseccomp version in different distros: RHEL-7: 2.3.1 Debian (Stretch): 2.3.1 OpenSUSE Leap 15:

Re: [Qemu-devel] [PATCH 11/20] target/arm: Clear unused predicate bits for LD1RQ

2018-08-23 Thread Richard Henderson
On 08/23/2018 08:21 AM, Peter Maydell wrote: > On 9 August 2018 at 05:21, Richard Henderson > wrote: >> The 16-byte load only uses 16 predicate bits. But while >> reusing the other load infrastructure, we find other bits >> that are set and trigger an assert. To avoid this and >> retain the asse

[Qemu-devel] [PATCH v3 0/3] block nodes graph visualization

2018-08-23 Thread Vladimir Sementsov-Ogievskiy
Hi all! On the way of backup schemes development (and in general any complicated developments in Qemu block layer) it would be good to have an ability to print out graph of block nodes with their permissions. Just look at attached picture. v3: again, major rework, after long discussion with Max:

[Qemu-devel] [PATCH v3 1/3] qapi: add x-debug-query-block-graph

2018-08-23 Thread Vladimir Sementsov-Ogievskiy
Add a new command, returning block nodes (and their users) graph. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 91 +++ include/block/block.h | 1 + include/sysemu/block-backend.h | 2 + block.c| 129 +++

[Qemu-devel] [PATCH v3 3/3] not-for-commit: example of new command usage for debugging

2018-08-23 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/222 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 0ead56d574..91d88aa5c0 100644 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -137,6 +137,8 @@ with iotes

[Qemu-devel] [PATCH v3 2/3] scripts: add render_block_graph function for QEMUMachine

2018-08-23 Thread Vladimir Sementsov-Ogievskiy
Render block nodes graph with help of graphviz. This new function is for debugging, so there is no sense to put it into qemu.py as a method of QEMUMachine. Let's instead put it separately. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/render_block_graph.py | 120 +++

Re: [Qemu-devel] [PATCH v4 10/16] linux-user: Split out some simple file syscalls

2018-08-23 Thread Laurent Vivier
Le 23/08/2018 à 00:58, Richard Henderson a écrit : > On 08/21/2018 05:50 PM, Laurent Vivier wrote: >> I don't understand why you need/want to duplicate the list of syscalls here. >> >> If I modify your patch as following, it works without duplicating the list: >> >> diff --git a/linux-user/syscall-

Re: [Qemu-devel] [PATCH 12/20] target/arm: Rewrite helper_sve_ld1*_r using pages

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:21, Richard Henderson wrote: > Uses tlb_vaddr_to_host for correct operation with softmmu. > Optimize for accesses within a single page or pair of pages. > > Perf report comparison for cortex-strings test-strlen > with aarch64-linux-user: > > before: >1.59% qemu-aarch64

Re: [Qemu-devel] [PATCH 13/20] target/arm: Rewrite helper_sve_ld[234]*_r

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:21, Richard Henderson wrote: > Use the same *_tlb primitives as we use for ld1. This is not > a significant change, but does (for linux-user) hoist the set > of helper_retaddr, and (for softmmu) hoist the computation of > the current mmu_idx outside the loop. > > This does

Re: [Qemu-devel] [PATCH 14/20] target/arm: Rewrite helper_sve_st[1234]*_r

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:22, Richard Henderson wrote: > This fixes the endianness problem for softmmu, and does > move the main loop out of a macro and into an inlined function. > > Signed-off-by: Richard Henderson Reviewed-by: Peter Maydell thanks -- PMM

Re: [Qemu-devel] [PATCH 17/20] target/arm: Rewrite vector gather loads

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:22, Richard Henderson wrote: > This fixes the endianness problem for softmmu, and does > move the main loop out of a macro and into an inlined function. > > Signed-off-by: Richard Henderson > --- > target/arm/helper-sve.h| 84 + > target/arm/sve_helper.c

Re: [Qemu-devel] [libvirt] clean/simple Q35 support in libvirt+QEMU for guest OSes that don't support virtio-1.0

2018-08-23 Thread Markus Armbruster
Eduardo Habkost writes: > On Wed, Aug 22, 2018 at 01:26:01PM +0100, Daniel P. Berrangé wrote: >> On Wed, Aug 22, 2018 at 09:01:35AM -0300, Eduardo Habkost wrote: >> > On Wed, Aug 22, 2018 at 12:36:27PM +0200, Andrea Bolognani wrote: >> > > On Tue, 2018-08-21 at 14:21 -0400, Laine Stump wrote: >>

Re: [Qemu-devel] [PATCH 19/20] target/arm: Rewrite vector gather first-fault loads

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:22, Richard Henderson wrote: > This implements the feature for softmmu, and moves the > main loop out of a macro and into a function. > > Signed-off-by: Richard Henderson > --- > target/arm/helper-sve.h| 84 --- > target/arm/sve_helper.c| 290 +

Re: [Qemu-devel] [PATCH 18/20] target/arm: Rewrite vector gather stores

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:22, Richard Henderson wrote: > This fixes the endianness problem for softmmu, and does > move the main loop out of a macro and into an inlined function. > > Signed-off-by: Richard Henderson > --- > target/arm/helper-sve.h| 52 ++ > target/arm/sve_helper.c

Re: [Qemu-devel] [PULL v3 00/69] Misc patches for 2018-08-21

2018-08-23 Thread Peter Maydell
On 23 August 2018 at 14:35, Paolo Bonzini wrote: > The following changes since commit 659b11e7a7239529cfdb4968418268ff9aa22d88: > > Merge remote-tracking branch > 'remotes/vivier2/tags/linux-user-for-3.1-pull-request' into staging > (2018-08-21 11:36:15 +0100) > > are available in the Git repo

Re: [Qemu-devel] [PATCH] scripts: report on author emails that are mangled by the mailing list

2018-08-23 Thread Markus Armbruster
Peter Maydell writes: > On 23 August 2018 at 11:25, Daniel P. Berrangé wrote: >> In some cases the Author: email address in patches submitted to the >> list gets mangled such that it says >> >> John Doe via Qemu-devel >> >> This change is a result of workarounds for DMARC policies. > > I do

Re: [Qemu-devel] [PATCH 20/20] target/arm: Pass TCGMemOpIdx to sve memory helpers

2018-08-23 Thread Peter Maydell
On 9 August 2018 at 05:22, Richard Henderson wrote: > There is quite a lot of code required to compute cpu_mem_index, > or even put together the full TCGMemOpIdx. This can easily be > done at translation time. > > Signed-off-by: Richard Henderson > --- > target/arm/internals.h | 5 ++ > t

Re: [Qemu-devel] [PULL v3 00/69] Misc patches for 2018-08-21

2018-08-23 Thread Paolo Bonzini
On 23/08/2018 18:16, Peter Maydell wrote: > > tests/libqtest.c: In function 'qtest_is_old_versioned_machine': > tests/libqtest.c:1019:13: warning: array subscript has type 'char' > [-Wchar-subscripts] > if (!isdigit(*chr) && *chr != '.') { > ^ > > Code should be using qe

[Qemu-devel] [PATCH v3 29/58] json: Fix \uXXXX for surrogate pairs

2018-08-23 Thread Markus Armbruster
The JSON parser treats each half of a surrogate pair as unpaired surrogate. Fix it to recognize surrogate pairs. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-parser.c | 60 --- tests/check-qjson.c | 3 +-- 2 files changed,

[Qemu-devel] [PATCH v3 34/58] json: Redesign the callback to consume JSON values

2018-08-23 Thread Markus Armbruster
The classical way to structure parser and lexer is to have the client call the parser to get an abstract syntax tree, the parser call the lexer to get the next token, and the lexer call some function to get input characters. Another way to structure them would be to have the client feed characters

[Qemu-devel] [PATCH v3 26/58] json: Leave rejecting invalid escape sequences to parser

2018-08-23 Thread Markus Armbruster
Both lexer and parser reject invalid escape sequences in strings. The parser's check is useless. The lexer ends the token right after the first non-well-formed byte. This tends to lead to suboptimal error reporting. For instance, input {"abc\@ijk": 1} produces the tokens JSON_LCURLY

[Qemu-devel] [PATCH v3 12/58] check-qjson: Cover UTF-8 in single quoted strings

2018-08-23 Thread Markus Armbruster
utf8_string() tests only double quoted strings. Cover single quoted strings, too: store the strings to test without quotes, then wrap them in either kind of quote. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 427 ++-

[Qemu-devel] [PATCH v3 35/58] json: Don't pass null @tokens to json_parser_parse()

2018-08-23 Thread Markus Armbruster
json_parser_parse() normally returns the QObject on success. Except it returns null when its @tokens argument is null. Its only caller json_message_process_token() passes null @tokens when emitting a lexical error. The call is a rather opaque way to say json = NULL then. Simplify matters by lif

[Qemu-devel] [PATCH v3 21/58] check-qjson: Document we expect invalid UTF-8 to be rejected

2018-08-23 Thread Markus Armbruster
The JSON parser rejects some invalid sequences, but accepts others without correcting the problem. We should either reject all invalid sequences, or minimize overlong sequences and replace all other invalid sequences by a suitable replacement character. A common choice for replacement is U+FFFD.

[Qemu-devel] [PATCH v3 22/58] json: Reject invalid UTF-8 sequences

2018-08-23 Thread Markus Armbruster
We reject bytes that can't occur in valid UTF-8 (\xC0..\xC1, \xF5..\xFF in the lexer. That's insufficient; there's plenty of invalid UTF-8 not containing these bytes, as demonstrated by check-qjson: * Malformed sequences - Unexpected continuation bytes - Missing continuation bytes after sta

[Qemu-devel] [PATCH v3 39/58] json: Pass lexical errors and limit violations to callback

2018-08-23 Thread Markus Armbruster
The callback to consume JSON values takes QObject *json, Error *err. If both are null, the callback is supposed to make up an error by itself. This sucks. qjson.c's consume_json() neglects to do so, which makes qobject_from_json() null instead of failing. I consider that a bug. The culprit is j

[Qemu-devel] [PATCH v3 33/58] json: Have lexer call streamer directly

2018-08-23 Thread Markus Armbruster
json_lexer_init() takes the function to process a token as an argument. It's always json_message_process_token(). Makes the code harder to understand for no actual gain. Drop the indirection. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- include/qapi/qmp/json-lexer.h| 13 +

[Qemu-devel] [PATCH v3 32/58] json-parser: simplify and avoid JSONParserContext allocation

2018-08-23 Thread Markus Armbruster
From: Marc-André Lureau parser_context_new/free() are only used from json_parser_parse(). We can fold the code there and avoid an allocation altogether. Signed-off-by: Marc-André Lureau Message-Id: <20180719184111.5129-9-marcandre.lur...@redhat.com> Reviewed-by: Markus Armbruster --- qobject/

[Qemu-devel] [PATCH v3 13/58] check-qjson: Simplify utf8_string()

2018-08-23 Thread Markus Armbruster
The previous commit made utf8_string()'s test_cases[].utf8_in superfluous: we can use .json_in instead. Except for the case testing U+. \x00 doesn't work in C strings, so it tests \\u instead. But testing \\u is escaped_string()'s job. It's covered there. Test U+0001 here, and drop .

[Qemu-devel] [PATCH v3 31/58] json: remove useless return value from lexer/parser

2018-08-23 Thread Markus Armbruster
From: Marc-André Lureau The lexer always returns 0 when char feeding. Furthermore, none of the caller care about the return value. Signed-off-by: Marc-André Lureau Message-Id: <20180326150916.9602-10-marcandre.lur...@redhat.com> Reviewed-by: Markus Armbruster Reviewed-by: Thomas Huth Signed-o

[Qemu-devel] [PATCH v3 44/58] json: Fix latent parser aborts at end of input

2018-08-23 Thread Markus Armbruster
json-parser.c carefully reports end of input like this: token = parser_context_pop_token(ctxt); if (token == NULL) { parse_error(ctxt, NULL, "premature EOI"); goto out; } Except parser_context_pop_token() can't return null, it fails its assertion instead. Same for par

[Qemu-devel] [PATCH v3 17/58] json: Fix lexer to include the bad character in JSON_ERROR token

2018-08-23 Thread Markus Armbruster
json_lexer[] maps (lexer state, input character) to the new lexer state. The input character is consumed unless the new state is terminal and the input character doesn't belong to this token, i.e. the state transition uses look-ahead. When this is the case, input character '\0' would result in th

[Qemu-devel] [PATCH v3 36/58] json: Don't create JSON_ERROR tokens that won't be used

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-streamer.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 0c33186e8e..fa595a8761 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-st

[Qemu-devel] [PATCH v3 37/58] json: Rename token JSON_ESCAPE & friends to JSON_INTERP

2018-08-23 Thread Markus Armbruster
The JSON parser optionally supports interpolation. The code calls it "escape". Awkward, because it uses the same term for escape sequences within strings. The latter usage is consistent with RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange Format" and ISO C. Call the former "inte

[Qemu-devel] [PATCH v3 28/58] json: Reject invalid \uXXXX, fix \u0000

2018-08-23 Thread Markus Armbruster
The JSON parser translates invalid \u to garbage instead of rejecting it, and swallows \u. Fix by using mod_utf8_encode() instead of flawed wchar_to_utf8(). Valid surrogate pairs are now differently broken: they're rejected instead of translated to garbage. The next commit will fix them.

[Qemu-devel] [PATCH v3 27/58] json: Simplify parse_string()

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-parser.c | 42 +++--- 1 file changed, 19 insertions(+), 23 deletions(-) diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 7437827c24..9cb363f7e1 100644 --- a/qobject/json

[Qemu-devel] [PATCH v3 18/58] json: Reject unescaped control characters

2018-08-23 Thread Markus Armbruster
Fix the lexer to reject unescaped control characters in JSON strings, in accordance with RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange Format". Bonus: we now recover more nicely from unclosed strings. E.g. {"one: 1}\n{"two": 2} now recovers cleanly after the newline, wher

[Qemu-devel] [PATCH v3 19/58] json: Revamp lexer documentation

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-lexer.c | 80 +++- 1 file changed, 71 insertions(+), 9 deletions(-) diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index e85e9a78ff..902fe60846 100644 --- a/qobject/json-l

[Qemu-devel] [PATCH v3 10/58] check-qjson: Cover escaped characters more thoroughly, part 2

2018-08-23 Thread Markus Armbruster
Cover escaped single quote, surrogates, invalid escapes, and noncharacters. This demonstrates that valid surrogate pairs are misinterpreted, and invalid surrogates and noncharacters aren't rejected. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 62 +

[Qemu-devel] [PATCH v3 15/58] check-qjson qmp-test: Cover control characters more thoroughly

2018-08-23 Thread Markus Armbruster
RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange Format" requires control characters in strings to be escaped. Demonstrate the JSON parser accepts U+0001 .. U+001F unescaped. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 36 ++

[Qemu-devel] [PATCH v3 24/58] json: Leave rejecting invalid UTF-8 to parser

2018-08-23 Thread Markus Armbruster
Both the lexer and the parser (attempt to) validate UTF-8 in JSON strings. The lexer rejects bytes that can't occur in valid UTF-8: \xC0..\xC1, \xF5..\xFF. This rejects some, but not all invalid UTF-8. It also rejects ASCII control characters \x00..\x1F, in accordance with RFC 8259 (see recent c

[Qemu-devel] [PATCH v3 04/58] check-qjson: Cover whitespace more thoroughly

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/check-qjson.c b/tests/check-qjson.c index 4daadc272b..188f683317 100644 --- a/tests/check-qjson.c +++ b/tests/check-qjson.c @@ -1236,7 +1236,

[Qemu-devel] [PATCH v3 16/58] check-qjson: Cover interpolation more thoroughly

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 166 +++- 1 file changed, 102 insertions(+), 64 deletions(-) diff --git a/tests/check-qjson.c b/tests/check-qjson.c index a586189d87..1688b2f5c1 100644 --- a/tests/check-qj

[Qemu-devel] [PATCH v3 20/58] json: Tighten and simplify qstring_from_escaped_str()'s loop

2018-08-23 Thread Markus Armbruster
Simplify loop control, and assert that the string ends with the appropriate quote (the lexer ensures it does). Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-parser.c | 30 +++--- 1 file changed, 7 insertions(+), 23 deletions(-) diff --git a/q

[Qemu-devel] [PATCH v3 23/58] json: Report first rather than last parse error

2018-08-23 Thread Markus Armbruster
Quiz time! When a parser reports multiple errors, but the user gets to see just one, which one is (on average) the least useful one? Yes, you're right, it's the last one! You're clearly familiar with compilers. Which one does QEMU report? Right again, the last one! You're clearly familiar wit

[Qemu-devel] [PATCH v3 02/58] check-qjson: Cover multiple JSON objects in same string

2018-08-23 Thread Markus Armbruster
qobject_from_json() & friends misbehave when the JSON text has more than one JSON value. Add test coverage to demonstrate the bugs. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 20 1 file changed, 20 insertions(+) diff --git a/tests/c

[Qemu-devel] [PATCH v3 25/58] json: Accept overlong \xC0\x80 as U+0000 ("modified UTF-8")

2018-08-23 Thread Markus Armbruster
Since the JSON grammer doesn't accept U+ anywhere, this merely exchanges one kind of parse error for another. It's purely for consistency with qobject_to_json(), which accepts \xC0\x80 (see commit e2ec3f97680). Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-lexer

[Qemu-devel] [PATCH v3 30/58] check-qjson: Fix and enable utf8_string()'s disabled part

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/tests/check-qjson.c b/tests/check-qjson.c index 343f8af36a..defc21fa04 100644 --- a/tests/check-qjson.c +++ b/tests/check-qjson.c @@ -7

[Qemu-devel] [PATCH v3 42/58] json: Improve names of lexer states related to numbers

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-lexer.c | 34 +- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c index f1a4b5a430..01417dca9d 100644 --- a/qobject/json-lexer.c ++

[Qemu-devel] [PATCH v3 14/58] check-qjson: Fix utf8_string() to test all invalid sequences

2018-08-23 Thread Markus Armbruster
Some of utf8_string()'s test_cases[] contain multiple invalid sequences. Testing that qobject_from_json() fails only tests we reject at least one invalid sequence. That's incomplete. Additionally test each non-space sequence in isolation. This demonstrates that the JSON parser accepts invalid s

[Qemu-devel] [PATCH v3 03/58] check-qjson: Cover blank and lexically erroneous input

2018-08-23 Thread Markus Armbruster
qobject_from_json() can return null without setting an error on lexical errors. I call that a bug. Add test coverage to demonstrate it. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 40 +--- 1 file changed, 37 insertions

[Qemu-devel] [PATCH v3 07/58] test-qga: Clean up how we test QGA synchronization

2018-08-23 Thread Markus Armbruster
To permit recovering from arbitrary JSON parse errors, the JSON parser resets itself on lexical errors. We recommend sending a 0xff byte for that purpose, and test-qga covers this usage since commit 5229564b832. That commit had to add an ugly hack to qmp_fd_vsend() to make capable of sending this

[Qemu-devel] [PATCH v3 48/58] json: Enforce token count and size limits more tightly

2018-08-23 Thread Markus Armbruster
Token count and size limits exist to guard against excessive heap usage. We check them only after we created the token on the heap. That's assigning a cowboy to the barn to lasso the horse after it has bolted. Close the barn door instead: check before we create the token. Signed-off-by: Markus A

[Qemu-devel] [PATCH v3 05/58] qmp-cmd-test: Split off qmp-test

2018-08-23 Thread Markus Armbruster
qmp-test is for QMP protocol tests. Commit e4a426e75ef added generic, basic tests of query commands to it. Move them to their own test program qmp-cmd-test, to keep qmp-test focused on the protocol. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- MAINTAINERS| 1 + t

[Qemu-devel] [PATCH v3 09/58] check-qjson: Streamline escaped_string()'s test strings

2018-08-23 Thread Markus Armbruster
Merge a few closely related test strings, and drop a few redundant ones. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 12 +--- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/tests/check-qjson.c b/tests/check-qjson.c index 008b6e95e4..

[Qemu-devel] [PATCH v3 43/58] qjson: Fix qobject_from_json() & friends for multiple values

2018-08-23 Thread Markus Armbruster
qobject_from_json() & friends use the consume_json() callback to receive either a value or an error from the parser. When they are fed a string that contains more than either one JSON value or one JSON syntax error, consume_json() gets called multiple times. When the last call receives a value, q

[Qemu-devel] [PATCH v3 46/58] json: Assert json_parser_parse() consumes all tokens on success

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-parser.c | 1 + 1 file changed, 1 insertion(+) diff --git a/qobject/json-parser.c b/qobject/json-parser.c index e3ee2a273a..685e9dac24 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -544,6 +544,7 @@ QOb

[Qemu-devel] [PATCH v3 11/58] check-qjson: Consolidate partly redundant string tests

2018-08-23 Thread Markus Armbruster
simple_string() and single_quote_string() have become redundant with escaped_string(), except for embedded single and double quotes. Replace them by a test that covers just that. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 64 ++

[Qemu-devel] [PATCH v3 08/58] check-qjson: Cover escaped characters more thoroughly, part 1

2018-08-23 Thread Markus Armbruster
escaped_string() first tests double quoted strings, then repeats a few tests with single quotes. Repeat all of them: store the strings to test without quotes, and wrap them in either kind of quote for testing. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/check-qjson.c | 9

[Qemu-devel] [PATCH v3 51/58] json: Make JSONToken opaque outside json-parser.c

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- include/qapi/qmp/json-parser.h | 4 include/qapi/qmp/json-streamer.h | 7 --- qobject/json-parser.c| 19 +++ qobject/json-streamer.c | 8 +--- 4 files changed, 24 insertions(+), 14

[Qemu-devel] [PATCH v3 38/58] json: Treat unwanted interpolation as lexical error

2018-08-23 Thread Markus Armbruster
The JSON parser optionally supports interpolation. The lexer recognizes interpolation tokens unconditionally. The parser rejects them when interpolation is disabled, in parse_interpolation(). However, it neglects to set an error then, which can make json_parser_parse() fail without setting an err

[Qemu-devel] [PATCH v3 01/58] docs/interop/qmp-spec: How to force known good parser state

2018-08-23 Thread Markus Armbruster
Section "QGA Synchronization" specifies that sending "a raw 0xFF sentinel byte" makes the server "reset its state and discard all pending data prior to the sentinel." What actually happens there is a lexical error, which will produce one or more error responses. Moreover, it's not specific to QGA.

[Qemu-devel] [PATCH v3 06/58] qmp-test: Cover syntax and lexical errors

2018-08-23 Thread Markus Armbruster
qmp-test neglects to cover QMP input that isn't valid JSON. libqtest doesn't let us such input. Add qtest_qmp_send_raw() for this purpose, and put it to use in qmp-test. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- tests/libqtest.c | 17 + tests/libqtest.h | 11

[Qemu-devel] [PATCH v3 57/58] json: Support %% in JSON strings when interpolating

2018-08-23 Thread Markus Armbruster
The previous commit makes JSON strings containing '%' awkward to express in templates: you'd have to mask the '%' with an Unicode escape \u0025. No template currently contains such JSON strings. Support the printf conversion specification %% in JSON strings as a convenience anyway, because it's tr

[Qemu-devel] [PATCH v3 41/58] json: Replace %I64d, %I64u by %PRId64, %PRIu64

2018-08-23 Thread Markus Armbruster
Support for %I64d got added in commit 2c0d4b36e7f "json: fix PRId64 on Win32". We had to hard-code I64d because we used the lexer's finite state machine to check interpolations. No more, so clean this up. Additional conversion specifications would be easy enough to implement when needed. Signed

[Qemu-devel] [PATCH v3 00/58] json: Fixes, error reporting improvements, cleanups

2018-08-23 Thread Markus Armbruster
JSON is such a simple language, so writing a parser should be easy, shouldn't it? Well, the evidence is in, and it's a lot of patches. Summary of fixes: * Reject ASCII control characters in strings as RFC 7159 specifies * Reject all invalid UTF-8 sequences, not just some * Reject invalid \u

[Qemu-devel] [PATCH v3 52/58] qobject: Drop superfluous includes of qemu-common.h

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- include/qapi/qmp/json-parser.h | 1 - qobject/json-lexer.c | 1 - qobject/json-streamer.c| 1 - qobject/qbool.c| 1 - qobject/qlist.c| 1 - qobject/qnull.c| 1 - qobject

[Qemu-devel] [PULL 14/69] tests: Skip old versioned machine types in quick testing mode

2018-08-23 Thread Paolo Bonzini
From: Thomas Huth The tests that check something for all machine types currently spend a lot of time checking old machine types (like "pc-i440fx-2.0" for example). The chances that we find something new there in addition to checking the latest version of a machine type are pretty low, so we shoul

Re: [Qemu-devel] [PATCH] vl.c: make sure maxcpus matches topology to prevent migration failure

2018-08-23 Thread Paolo Bonzini
On 23/08/2018 16:51, Igor Mammedov wrote: > Topology (threads*cores*sockets) must match maxcpus to be valid, > otherwise we could start QEMU with invalid topology that throws > a error on migration destination side, that should not be reachable: > Source: > -smp 8,maxcpus=64,cores=1,threads=8,soc

Re: [Qemu-devel] [PULL v3 00/69] Misc patches for 2018-08-21

2018-08-23 Thread Peter Maydell
On 23 August 2018 at 17:25, Paolo Bonzini wrote: > On 23/08/2018 18:16, Peter Maydell wrote: >> >> tests/libqtest.c: In function 'qtest_is_old_versioned_machine': >> tests/libqtest.c:1019:13: warning: array subscript has type 'char' >> [-Wchar-subscripts] >> if (!isdigit(*chr) && *chr

[Qemu-devel] [PATCH v3 45/58] json: Fix streamer not to ignore trailing unterminated structures

2018-08-23 Thread Markus Armbruster
json_message_process_token() accumulates tokens until it got the sequence of tokens that comprise a single JSON value (it counts curly braces and square brackets to decide). It feeds those token sequences to json_parser_parse(). If a non-empty sequence of tokens remains at the end of the parse, i

Re: [Qemu-devel] [libvirt] clean/simple Q35 support in libvirt+QEMU for guest OSes that don't support virtio-1.0

2018-08-23 Thread Daniel P . Berrangé
On Thu, Aug 23, 2018 at 06:08:55PM +0200, Markus Armbruster wrote: > Eduardo Habkost writes: > > > On Wed, Aug 22, 2018 at 01:26:01PM +0100, Daniel P. Berrangé wrote: > >> On Wed, Aug 22, 2018 at 09:01:35AM -0300, Eduardo Habkost wrote: > >> > On Wed, Aug 22, 2018 at 12:36:27PM +0200, Andrea Bolo

Re: [Qemu-devel] [libvirt] clean/simple Q35 support in libvirt+QEMU for guest OSes that don't support virtio-1.0

2018-08-23 Thread Eduardo Habkost
On Thu, Aug 23, 2018 at 05:26:47PM +0100, Daniel P. Berrangé wrote: [...] > There are countless mistakes in both QEMU & libvirt, but only some of > them are worth the cost of changing. I'm not seeing a compelling reason > why this change is worthwhile. The impact of the design mistake is narrow > a

[Qemu-devel] [PATCH v3 50/58] json: Unbox tokens queue in JSONMessageParser

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- include/qapi/qmp/json-streamer.h | 2 +- qobject/json-parser.c| 1 - qobject/json-streamer.c | 30 +++--- 3 files changed, 12 insertions(+), 21 deletions(-) diff --git a/include/qapi/qmp/

[Qemu-devel] [PATCH v3 49/58] json: Streamline json_message_process_token()

2018-08-23 Thread Markus Armbruster
Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- qobject/json-streamer.c | 13 + 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index 810aae521f..954bf9d468 100644 --- a/qobject/json-streamer.c +++ b/qobject

[Qemu-devel] [PATCH v3 56/58] json: Improve safety of qobject_from_jsonf_nofail() & friends

2018-08-23 Thread Markus Armbruster
The JSON parser optionally supports interpolation. This is used to build QObjects by parsing string templates. The templates are C literals, so parse errors (such as invalid interpolation specifications) are actually programming errors. Consequently, the functions providing parsing with interpol

Re: [Qemu-devel] [PATCH v3 2/3] kvm: Add support to KVM_GET_MSR_FEATURE_INDEX_LIST and KVM_GET_MSRS system ioctl

2018-08-23 Thread Eduardo Habkost
On Thu, Aug 23, 2018 at 02:28:28PM +0800, Robert Hoo wrote: > On Sat, 2018-08-18 at 12:05 -0300, Eduardo Habkost wrote: [...] > > We don't want QEMU to refuse to run if the kernel doesn't have > > KVM_CAP_GET_MSR_FEATURES. We can treat missing capability as > > equivalent to returning an empty lis

[Qemu-devel] [PATCH v3 47/58] qjson: Have qobject_from_json() & friends reject empty and blank

2018-08-23 Thread Markus Armbruster
The last case where qobject_from_json() & friends return null without setting an error is empty or blank input. Callers: * block.c's parse_json_protocol() reports "Could not parse the JSON options". It's marked as a work-around, because it also covered actual bugs, but they got fixed in the

[Qemu-devel] [PATCH v3 58/58] json: Update references to RFC 7159 to RFC 8259

2018-08-23 Thread Markus Armbruster
RFC 8259 (December 2017) obsoletes RFC 7159 (March 2014). Signed-off-by: Markus Armbruster --- include/qapi/qmp/qnum.h | 2 +- qapi/introspect.json| 2 +- qobject/json-parser.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/qapi/qmp/qnum.h b/include/qapi/qmp

[Qemu-devel] [PATCH v3 53/58] json: Clean up headers

2018-08-23 Thread Markus Armbruster
The JSON parser has three public headers, json-lexer.h, json-parser.h, json-streamer.h. They all contain stuff that is of no interest outside qobject/json-*.c. Collect the public interface in include/qapi/qmp/json-parser.h, and everything else in qobject/json-parser-int.h. Signed-off-by: Markus

Re: [Qemu-devel] [PATCH v3 2/3] kvm: Add support to KVM_GET_MSR_FEATURE_INDEX_LIST and KVM_GET_MSRS system ioctl

2018-08-23 Thread Paolo Bonzini
On 23/08/2018 19:11, Eduardo Habkost wrote: >>> We don't want QEMU to refuse to run if the kernel doesn't have >>> KVM_CAP_GET_MSR_FEATURES. We can treat missing capability as >>> equivalent to returning an empty list of MSRs. >> Yes. I'll let caller (kvm_arch_init) ignore the return value but a >

[Qemu-devel] [PATCH v3 54/58] tests/drive_del-test: Fix harmless JSON interpolation bug

2018-08-23 Thread Markus Armbruster
test_after_failed_device_add() does this: response = qmp("{'execute': 'device_add'," " 'arguments': {" " 'driver': 'virtio-blk-%s'," " 'drive': 'drive0'" "}}", qvirtio_get_dev_type()); Wrong. An interpolation spe

[Qemu-devel] [PATCH v3 55/58] json: Keep interpolation state in JSONParserContext

2018-08-23 Thread Markus Armbruster
The recursive descent parser passes along a pointer to JSONParserContext. It additionally passes a pointer to interpolation state (a va_alist *) as needed to reach its consumer parse_interpolation(). Stuffing the latter pointer into JSONParserContext saves us the trouble of passing it along, so d

[Qemu-devel] [PATCH v3 40/58] json: Leave rejecting invalid interpolation to parser

2018-08-23 Thread Markus Armbruster
Both lexer and parser reject invalid interpolation specifications. The parser's check is useless. The lexer ends the token right after the first bad character. This tends to lead to suboptimal error reporting. For instance, input [ %04d ] produces the tokens JSON_LSQUARE [ JSON_E

[Qemu-devel] [PULL v4 00/69] Misc patches for 2018-08-21

2018-08-23 Thread Paolo Bonzini
The following changes since commit 659b11e7a7239529cfdb4968418268ff9aa22d88: Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-3.1-pull-request' into staging (2018-08-21 11:36:15 +0100) are available in the Git repository at: git://github.com/bonzini/qemu.git tags/for-upstr

Re: [Qemu-devel] [PATCH v6 2/2] target: arm: Add support for VCPU event states

2018-08-23 Thread Peter Maydell
On 23 August 2018 at 16:45, Dongjiu Geng wrote: > This patch extends the qemu-kvm state sync logic with support for > KVM_GET/SET_VCPU_EVENTS, giving access to yet missing SError exception. > And also it can support the exception state migration. > > Signed-off-by: Dongjiu Geng Did you forget to

[Qemu-devel] [Bug 1788665] [NEW] Low 2D graphics performance with Windows 10 (1803) VGA passthrough VM using "Spectre" protection

2018-08-23 Thread Heiko Sieger
Public bug reported: Windows 10 (1803) VM using VGA passthrough via qemu script. After upgrading Windows 10 Pro VM to version 1803, or possibly after applying the March/April security updates from Microsoft, the VM would show low 2D graphics performance (sluggishness in 2D applications and low Pa

Re: [Qemu-devel] [PATCH v3 06/58] qmp-test: Cover syntax and lexical errors

2018-08-23 Thread Eric Blake
On 08/23/2018 11:39 AM, Markus Armbruster wrote: qmp-test neglects to cover QMP input that isn't valid JSON. libqtest doesn't let us such input. Add qtest_qmp_send_raw() for this purpose, s/us/us send/ and put it to use in qmp-test. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake

Re: [Qemu-devel] [PATCH 11/22] hw/misc/tz-msc: Model TrustZone Master Security Controller

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > Implement a model of the TrustZone Master Securtiy Controller, > as documented in the Arm CoreLink SIE-200 System IP for > Embedded TRM (DDI0571G): > https://developer.arm.com/products/architecture/m-profile/docs/ddi0571/g > > The MSC is intended t

Re: [Qemu-devel] [PATCH 20/22] hw/ssi/pl022: Correct wrong DMACR and ICR handling

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > In the PL022, register offset 0x20 is the ICR, a write-only > interrupt-clear register. Register offset 0x24 is DMACR, the DMA > control register. We were incorrectly implementing (a stub version > of) DMACR at 0x20, and not implementing anything at

Re: [Qemu-devel] [PATCH 12/22] hw/misc/iotkit-secctl: Wire up registers for controlling MSCs

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > The IoTKit does not have any Master Security Contollers itself, > but it does provide registers in the secure privilege control > block which allow control of MSCs in the external system. > Add support for these registers. > > Signed-off-by: Peter May

Re: [Qemu-devel] [PATCH 15/22] hw/ssi/pl022: Allow use as embedded-struct device

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > Create a new include file for the pl022's device struct, > type macros, etc, so that it can be instantiated using > the "embedded struct" coding style. > > While we're adding the new file to MAINTAINERS, add > also the .c file, which was missing an en

Re: [Qemu-devel] [PATCH] pc: acpi: revert back to 1 SRAT entry for hotpluggable area

2018-08-23 Thread Eduardo Habkost
On Thu, Aug 23, 2018 at 10:14:06AM +0200, Igor Mammedov wrote: > On Wed, 22 Aug 2018 15:01:12 -0300 > Eduardo Habkost wrote: [...] > > However, have you considered keeping adding separate entries for > > NVDIMM devices only (so we follow the spec), but add a single > > (numa_nodes-1, MEM_AFFINITY_

Re: [Qemu-devel] [PATCH 22/22] hw/arm/mps2-tz: Fix MPS2 SCC config register values

2018-08-23 Thread Peter Maydell
On 23 August 2018 at 18:45, Peter Maydell wrote: > On 23 August 2018 at 18:42, Richard Henderson > wrote: >> On 08/20/2018 07:11 AM, Peter Maydell wrote: >>> +mmc->scc_id = 0x4104 | (0x505 << 4); >> >> Is it any less clear to say 0x41045050? > > It makes it a bit clearer that that field i

Re: [Qemu-devel] [PATCH 13/22] hw/arm/iotkit: Wire up the lines for MSCs

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > The IoTKit doesn't have any MSCs itself but it does need > some wiring to connect the external signals from MSCs > in the outer board model up to the registers and the > NVIC IRQ line. > > We also need to expose a MemoryRegion corresponding to > the A

Re: [Qemu-devel] [PATCH v3 58/58] json: Update references to RFC 7159 to RFC 8259

2018-08-23 Thread Eric Blake
On 08/23/2018 11:40 AM, Markus Armbruster wrote: RFC 8259 (December 2017) obsoletes RFC 7159 (March 2014). Signed-off-by: Markus Armbruster --- include/qapi/qmp/qnum.h | 2 +- qapi/introspect.json| 2 +- qobject/json-parser.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)

Re: [Qemu-devel] [PATCH 12/22] hw/misc/iotkit-secctl: Wire up registers for controlling MSCs

2018-08-23 Thread Peter Maydell
On 23 August 2018 at 18:21, Richard Henderson wrote: > On 08/20/2018 07:11 AM, Peter Maydell wrote: >> The IoTKit does not have any Master Security Contollers itself, >> but it does provide registers in the secure privilege control >> block which allow control of MSCs in the external system. >> Ad

Re: [Qemu-devel] [PATCH 17/22] hw/ssi/pl022: Don't directly call vmstate_register()

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > Use the DeviceState vmsd pointer rather than calling vmstate_register() > directly. > > Signed-off-by: Peter Maydell > --- > hw/ssi/pl022.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Richard Henderson r~

Re: [Qemu-devel] [PATCH 14/22] hw/arm/mps2-tz: Create PL081s and MSCs

2018-08-23 Thread Richard Henderson
On 08/20/2018 07:11 AM, Peter Maydell wrote: > The AN505 FPGA image includes four PL081 DMA controllers, each > of which is gated by a Master Security Controller that allows > the guest to prevent a non-secure DMA controller from accessing > memory that is used by secure guest code. Create and wire

  1   2   3   4   >