Re: [PATCH 09/17] Move unicode to ASCII conversion to shared function.

[Adam Borowski]

On Mon, Sep 16, 2013 at 09:11:25PM -0700, Roy Franz wrote:
> +/* Convert the unicode UEFI command line to ASCII to pass to kernel.
> + * Size of memory allocated return in *cmd_line_len.
> + * Returns NULL on error.
> + */
> +static char *efi_convert_cmdline_to_ascii(efi_system_table_t *sys_table_arg,

> +   int load_options_size = image->load_options_size / 2; /* ASCII */

> + for (i = 0; i < options_size - 1; i++)
> + *s1++ = *s2++;

I'm afraid both this commit and comments inside are misnamed.  What you're
changing here is the encoding rather than character set.

In fact, these days it's 8-bit encodings that are more likely to be Unicode
than 16-bit ones: UTF-8 is ubiquitous, while you usually get UCS2 at most.
In either case, though, we have here is a 7-bit charset encoded as either
8-bit or 16-bit units.  What this function does is blindly truncating upper
byte.  The supported payload is in both cases ASCII.

I'd thus rename the function to what it already does: truncating u16 to u8,
and adjust comments accordingly.

Replacing values above 126 with a token character like '?' would be good
too: that'd avoid producing corrupted characters and/or random ASCII chars.

Your commit only moves things around, so it might be out of scope for now,
but I wonder: what if the kernel actually supported Unicode here?  Few
cmdline arguments take values where non-ASCII makes sense, but at least some
do: for example, a Russian guy is not unlikely to name subvolumes using
cyrillic.  Supporting that would be easy (estimating the length then
utf16s_to_utf8s()).  There's just one problem: which encoding to use, but
these days, most distributions have either dropped non-UTF8 or hardly pay
lip service, so we could get away with hard-coding UTF-8: those few who
use ancient charsets can stick to ASCII.  Would this be ok?  If so, shout,
I can code this if you don't care enough.

-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 09/17] Move unicode to ASCII conversion to shared function.

[Adam Borowski]

On Wed, Sep 18, 2013 at 09:48:44PM -0700, Roy Franz wrote:
> On Wed, Sep 18, 2013 at 8:44 PM, Adam Borowski  wrote:
> > [UCS2 truncation]
> 
> I stuck to re-arranging the code that was there, as I don't know enough
> about character encodings to propose changes.

I on the other hand don't know the kernel (lurking because of my first
patch), but I'm on a crusade against mangled Unicode (so far in the
userland).  Can't let such a blatant error slip through on my watch :)

> Also, this code is running as part of the kernel decompressor, rather than
> the kernel itself, so it doesn't have access to any kernel facilities, and
> it also needs to be position independent.

Ok, so it can't reuse common libraries.  No problem, a simplified, sanitized
and optimized copy of utf16s_to_utf8s() can be done in quite less code than
the original.

> It's running in a quite limited environment - the decompressor has
> its own copy of strstr(), and other string functions.

I'd need nothing but a way to alloc the new string.  And I see this is
already done (efi_{low,high_alloc()).

> I checked the UEFI specification, and it states that all 16 bit strings
> are UCS-2, unless otherwise noted.

... which means it will either get upgraded to UTF-16 in a subsequent
version, or some Unicode strings get mangled.  I'd ignore this bit and
implement full UTF-16 from the start: every legal UCS-2 string can be
decoded as UTF-16 so it's a strict superset.

> The load options that the command line is provided through a void pointer
> specified as: [snip]

Either a null pointer or a 16-bit string, that sounds clear enough.

I see not a word about endianness (does anything do EFI on big endian?),
but "same as host" seems to be a reasonable assumption.

> Would it be acceptable to fix the naming/comments, and convert values
> above 126 to '?' in the current patchset, and address a more thorough fix
> in another patch set?  The ARM and ARM64 EFI stub patchsets that are
> mostly complete depend on this one, so getting this merged soon would be
> helpful.

I don't want to hinder your work, so what about putting in your version
as-is and fixing it later?

> > There's just one problem: which encoding to use, but
> > these days, most distributions have either dropped non-UTF8 or hardly pay
> > lip service, so we could get away with hard-coding UTF-8: those few who
> > use ancient charsets can stick to ASCII.

Not being able to use regular kernel facilities makes supporting ancient
charsets a lost cause.  I'm so weeping about them... not.

> I would certainly appreciate your help improving this

Are we on the same page so far?  If so, I can make a patch atop yours.

-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH] vt: properly ignore xterm-256 colour codes

[Adam Borowski]

This is not a bug on our side, but a misdesign in ITU T.416, yet with
all popular terminals supporting these codes, people consider this to
be a bug in Linux.  By breaking the design principles behind SGR codes
(gracefully ignoring unsupported ones should not require knowing about
them), 256 colour ones tend to turn blinking on before invoking an
arbitrary unrelated command.

This commit doesn't add such support, merely skips such codes without
ill effects.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 26 --
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c677829..f7aaa28 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1300,13 +1300,27 @@ static void csi_m(struct vc_data *vc)
case 27:
vc->vc_reverse = 0;
break;
-   case 38: /* ANSI X3.64-1979 (SCO-ish?)
- * Enables underscore, white foreground
- * with white underscore (Linux - use
- * default foreground).
+   case 38:
+   case 48: /* ITU T.416
+ * Higher colour modes.
+ * They break the usual properties of SGR codes
+ * and thus need to be detected and ignored by
+ * hand.  Strictly speaking, that standard also
+ * wants : rather than ; as separators, 
contrary
+ * to ECMA-48, but no one produces such codes
+ * and almost no one accepts them.
  */
-   vc->vc_color = (vc->vc_def_color & 0x0f) | 
(vc->vc_color & 0xf0);
-   vc->vc_underline = 1;
+   i++;
+   if (i > vc->vc_npar)
+   break;
+   if (vc->vc_par[i] == 5)  /* 256 colours */
+   i++; /* ubiquitous */
+   else if (vc->vc_par[i] == 2) /* 24 bit colours 
*/
+   i += 3;  /* extremely rare 
*/
+   /* Subcommands 3 (CMY) and 4 (CMYK) are so 
insane
+* that detecting them is not worth the few 
extra
+* bytes of kernel's size.
+*/
break;
case 39: /* ANSI X3.64-1979 (SCO-ish?)
  * Disable underline option.
-- 
1.8.4.rc3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] vt: properly ignore xterm-256 colour codes

[Adam Borowski]

On Mon, Sep 09, 2013 at 05:53:19PM +0200, David Herrmann wrote:
> On Fri, Jul 12, 2013 at 10:23 PM, Adam Borowski  wrote:
> >  drivers/tty/vt/vt.c | 26 --
> >  1 file changed, 20 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> > index c677829..f7aaa28 100644
> > --- a/drivers/tty/vt/vt.c
> > +++ b/drivers/tty/vt/vt.c
> > @@ -1300,13 +1300,27 @@ static void csi_m(struct vc_data *vc)
[...]
> > -   case 38: /* ANSI X3.64-1979 (SCO-ish?)
> > - * Enables underscore, white foreground
> > - * with white underscore (Linux - use
> > - * default foreground).
> > -   vc->vc_color = (vc->vc_def_color & 0x0f) | 
> > (vc->vc_color & 0xf0);
> > -   vc->vc_underline = 1;
> 
> You break the old behavior here. _Iff_ this is what you want, then
> please do that in another commit. Explicitly state that "38" is used
> for 256color and shouldn't turn on underline+default-col. The SCO-ish
> behavior is weird, indeed, but breaking it silently is not ok.

This is implied by the description; none among modern terminal emulators
support this.  Would an additional comment in the commit message be
enough, or do I need to change the replacement into a pair of commits?
 
> > +   i++;
> > +   if (i > vc->vc_npar)
> 
> This should be ">=", but the for()-loop does allow your ">". So unless
> someone fixes the for-loop to use "<" (do a ++vc->vc_npar before it,
> if it's correct. But blindly doing "<=" is really irritating) I think
> this is ok.

The loop this switch is in does:
for (i = 0; i <= vc->vc_npar; i++)
which is obviously contrary to what we're used to, but I did not want to
rewrite nearby code to match my preferences.

The change you suggest would deoptimize the code by a single unnecessary
dereference and increment, which is negligible, but since the whole cost
of speedier version is having <= instead of < in the loop, I'm not so
certain this is a good idea.

[...]
> Btw., you should put Greg Kroah-Hartman and Andrew Morton on CC. Both
> are the most likely to pick this up.

Thanks for the suggestion.  I've sent the patch two days ago to Jiri Slaby
(listed as a maintainer besides Greg) together with a newbie question, but
he's apparently busy.

I've got more changes for the vt, but there's no hurry, I wanted to test
the waters with a single minor one in 3.12 first.

-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 1/2] vt: break a couple of obsolete SCOish codes.

[Adam Borowski]

No modern terminal supports them, and SGR 38 conflicts with detecting
xterm-256 colours.  This also makes SGR 39 consistent with other popular
terminals.  Neither are used by ncurses' terminfo.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c |   15 +--
 1 file changed, 1 insertion(+), 14 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c677829..ba528bb 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1300,21 +1300,8 @@ static void csi_m(struct vc_data *vc)
case 27:
vc->vc_reverse = 0;
break;
-   case 38: /* ANSI X3.64-1979 (SCO-ish?)
- * Enables underscore, white foreground
- * with white underscore (Linux - use
- * default foreground).
- */
-   vc->vc_color = (vc->vc_def_color & 0x0f) | 
(vc->vc_color & 0xf0);
-   vc->vc_underline = 1;
-   break;
-   case 39: /* ANSI X3.64-1979 (SCO-ish?)
- * Disable underline option.
- * Reset colour to default? It did this
- * before...
- */
+   case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) | 
(vc->vc_color & 0xf0);
-   vc->vc_underline = 0;
break;
case 49:
vc->vc_color = (vc->vc_def_color & 0xf0) | 
(vc->vc_color & 0x0f);
-- 
1.7.10.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 2/2] vt: properly ignore xterm-256 colour codes

[Adam Borowski]

This is not a bug on our side, but a misdesign in ITU T.416, yet with
all popular terminals supporting these codes, people consider this to
be a bug in Linux.  By breaking the design principles behind SGR codes
(gracefully ignoring unsupported ones should not require knowing about
them), 256 colour ones tend to turn blinking on before invoking an
arbitrary unrelated command.

This commit doesn't add such support, merely skips such codes without
ill effects.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c |   22 ++
 1 file changed, 22 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index ba528bb..bc441cf 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1300,6 +1300,28 @@ static void csi_m(struct vc_data *vc)
case 27:
vc->vc_reverse = 0;
break;
+   case 38:
+   case 48: /* ITU T.416
+ * Higher colour modes.
+ * They break the usual properties of SGR codes
+ * and thus need to be detected and ignored by
+ * hand.  Strictly speaking, that standard also
+ * wants : rather than ; as separators, 
contrary
+ * to ECMA-48, but no one produces such codes
+ * and almost no one accepts them.
+ */
+   i++;
+   if (i > vc->vc_npar)
+   break;
+   if (vc->vc_par[i] == 5)  /* 256 colours */
+   i++; /* ubiquitous */
+   else if (vc->vc_par[i] == 2) /* 24 bit colours 
*/
+   i += 3;  /* extremely rare 
*/
+   /* Subcommands 3 (CMY) and 4 (CMYK) are so 
insane
+* that detecting them is not worth the few 
extra
+* bytes of kernel's size.
+*/
+   break;
case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) | 
(vc->vc_color & 0xf0);
break;
-- 
1.7.10.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] vt: properly ignore xterm-256 colour codes

[Adam Borowski]

On Thu, Sep 12, 2013 at 02:37:26PM +0200, David Herrmann wrote: 
> On Mon, Sep 9, 2013 at 6:46 PM, Adam Borowski  wrote:
> > On Mon, Sep 09, 2013 at 05:53:19PM +0200, David Herrmann wrote:
> > [...]
> >> Btw., you should put Greg Kroah-Hartman and Andrew Morton on CC. Both
> >> are the most likely to pick this up.
> >
> > Thanks for the suggestion.  I've sent the patch two days ago to Jiri Slaby
> > (listed as a maintainer besides Greg) together with a newbie question, but
> > he's apparently busy.
> 
> Jiri Slaby maintains the TTY subsystem (together with Greg). This does
> not include the VT layer, though. drivers/tty/vt/ and
> drivers/video/console/ are unmaintained. You need to get the attention
> of any maintainer who is willing to take it through their tree (hint:
> most maintainers don't dare touching the VT layer. Greg and Andrew
> were brave enough in the past.).
> 
> I'm willing to review your patches

Thanks.  So I shouldn't bother anyone else for now to get My First Kernel
Patch(tm) into 3.12 (or, if it's too late, into something included in 3.13),
right?

> > I've got more changes for the vt, but there's no hurry, I wanted to test
> > the waters with a single minor one in 3.12 first.

> drivers/tty/vt/ and drivers/video/console/ are unmaintained.

> [...] but history taught me touching the VT layer is a waste of time.

Could you tell me why?  The console is an important tool when something
fails.  Of course, fancy schmancy stuff like combining characters, etc,
could be better done in that legendary userspace alternate console layer,
but the built-in VT must remain at least functional.  And getting corrupted
text is not nice; the VT has fallen woefully behind what works on any other
modern terminal.

Back by ~2000, I'd say it worked better than rxvt, xterm, or, Cthulhu help
us, Solaris' terminal.  It just needs some maintenance.

I had a bunch of other improvements planned; if you say that's a waste of
time perhaps I should scale that back.  But I'd still want to at least
make sure programs that don't use terminfo (terminfo is a bad joke) won't
spew ANSI codes to the screen; at least more popular ones like "set window
title", etc.

You might know of other clean-up and fixes that need to be done here,
though.


-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Copy on write hard links?

[Adam Borowski]

On Tue, Sep 24, 2013 at 08:36:56PM +0200, Thomas Meyer wrote:
> Is there such a thing?

In mainline, AFAIK no.  The vserver patchset, on the other hand, adds a new
xattr, iunlink, that copies the whole file when needed.  That works on most
filesystems.

That's quite a hack, though, and I think you'd be better off using btrfs
which does cow transparently at a lower level than hardlinks.

-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 0/2] blackfin: Remove dead DSA code

[Adam Borowski]

On Sun, May 28, 2017 at 04:59:47PM -0700, Florian Fainelli wrote:
> Hello? anyone still maintaining blackfin here?

Looks like people edit arch/blackfin/ a lot whenever it interferes with some
other work, but the only blackfin-specific fixes seem to be a couple of
drive-by ones by Al Viro, then nothing until first half of 2015.

Last maintainer pull request: 668b54a1 on 2015-04-24.

There was some mailing list traffic in Jan 2016:
https://sourceforge.net/p/adi-buildroot/mailman/adi-buildroot-devel/

So it looks pretty dead...

-- 
Don't be racist.  White, amber or black, all beers should be judged based
solely on their merits.  Heck, even if occasionally a cider applies for a
beer's job, why not?
On the other hand, corpo lager is not a race.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Tue, Jun 19, 2018 at 11:34:34AM -0400, Nicolas Pitre wrote:
> On Tue, 19 Jun 2018, Adam Borowski wrote:
> > Thus, it'd be nice to use the structure you add to implement full Unicode
> > range for the vast majority of people.  This includes even U+2800..FF.  :)
> 
> Be my guest if you want to use this structure. As for U+2800..FF, like I 
> said earlier, this is not what most people use when communicating, so it 
> is of little interest even to blind users except for displaying native 
> braille documents, or showing off. ;-)

It's meant for displaying braille to _sighted_ people.  And in real world,
the main [ab]use is a way to show images that won't get corrupted by
proportional fonts. :-þ

> If the core console code makes the switch to full unicode then yes, that 
> would be the way to go to maintain backward compatibility. However 
> vgacon users would see a performance drop when switching between VT's 
> and we used to brag about how fast the Linux console used to be 20 years 
> ago. Does it still matter today?

I've seen this slowness.  A long time ago, on a server that someone gave an
_ISA_ graphics card (it was an old machine, and it was 1.5 decades ago). 
Indeed, switching VTs took around a second.  But this was drawing speed, not
Unicode conversion.

There are three cases when a character can enter the screen:
* being printed by the tty.  This is the only case not sharply rate-limited.
  It already has to do the conversion.  If we eliminate the old struct, it
  might even be a speed-up when lots of text gets blasted to a non-active
  VT.
* VT switch
* scrollback

The last two cases are initiated by the user, and within human reaction time
you need to convert usually 2000 -- up to 20k-ish -- characters.  The
conversion is done by a 3-level array.  I think a ZX Spectrum can handle
this fine without a visible slowdown.

> > > I'm a prime user of this feature, as well as the BRLTTY maintainer Dave 
> > > Mielke
> > > who implemented support for this in BRLTTY. There is therefore a vested
> > > interest in maintaining this feature as necessary. And this received
> > > extensive testing as well at this point.
> > 
> > So, you care only about people with faulty wetware.  Thus, it sounds like
> > work that benefits sighted people would need to be done by people other than
> > you. 
> 
> Hard for me to contribute more if I can't enjoy the result.

Obviously.

The primary users would be:
* people who want symbols uncorrupted (especially if their language uses a
  non-latin script)
* CJK people (as discussed below)

It could also simplify the life for distros -- less required configuration:
a single font needed for currently supported charsets together has mere
~1000 glyphs, at 8x16 that's 16000 bytes (+ mapping).  Obviously for CJK
that's more.
 
> > So I'm only mentioning possible changes; they could possibly go after
> > your patchset goes in:
> > 
> > A) if memory is considered to be at premium, what about storing only one
> >32-bit value, masked 21 bits char 11 bits attr?  On non-vgacon, there's
> >no reason to keep the old structures.
> 
> Absolutely. As soon as vgacon is officially relegated to second class 
> citizen i.e. perform the glyph translation each time it requires 
> a refresh instead of dictating how the core console code works then the 
> central glyph buffer can go.

Per the analysis above, on-the-fly translation is so unobtrusive that it
shouldn't be a problem.

> > B) if being this frugal wrt memory is ridiculous today, what about instead
> >going for 32 bits char (wasteful) 32 bits attr?  This would be much nicer
> >15 bit fg color + 15 bit bg color + underline + CJK or something.
> > You already triple memory use; variant A) above would reduce that to 2x,
> > variant B) to 4x.
> 
> You certainly won't find any objections from me.

Right, let's see if your patchset gets okayed before building atop it.
 
> In the mean time, both systems may work in parallel for a smooth 
> transition.

Sounds like a good idea.


WRT support for fonts >512 glyphs: I talked to a Chinese hacker (log
starting at 15:32 on https://irclog.whitequark.org/linux-sunxi/2018-06-19),
she said there are multiple popular non-mainline patchsets implementing CJK
on console.  None of them got accepted because of pretty bad code like
https://github.com/Gentoo-zh/linux-cjktty/commit/b6160f85ef5bc5c2cae460f6c0a1aba3e417464f
but getting this done cleanly would require just:
* your patchset here
* console driver using the Unicode structure
* loading such larger fonts (the one in cjktty is built-in)
* double-width characters in vt.c


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Wed, Jun 20, 2018 at 10:21:37PM -0400, Dave Mielke wrote:
> [quoted lines by Adam Borowski on 2018/06/21 at 03:43 +0200]
> 
> >It's meant for displaying braille to _sighted_ people.  And in real world,
> >the main [ab]use is a way to show images that won't get corrupted by
> >proportional fonts. :-þ
> 
> It's not abuse at all. I often use U+28xx to show sighted people what the
> braille for something looks like. I often need to do this when, for example, I
> need them to comapre what I'm showing them to what's on an actual braille
> display. U+28xx is the only way for me to do this without a lengthy 
> description
> containing sequences of dot number combinations.

What you describe is the intended use.  Abuse is when people use these
glyphs to write text like this:

⡎⠉⠂⠠⠤⡀⣄⠤⡀⠀⠀⠀⡄⠀⡄⡠⠤⡀⡄⠀⡄⠀⠀⠀⣄⠤⡀⡠⠤⡀⠠⠤⡀⡠⠤⡇⠀⠀⠀⠤⡧⠄⣇⠤⡀⠠⡅⠀⡠⠤⠄⠎⢉⠆
⠣⠤⠂⠪⠭⠇⠇⠀⠇⠀⠀⠀⠨⠭⠃⠣⠤⠃⠣⠤⠃⠀⠀⠀⠇⠀⠀⠫⠭⠁⠪⠭⠇⠣⠤⠇⠣⠄⠇⠀⠇⠀⠣⠀⠬⠭⠂⠀⠅⠀

(Not sure if you're completely blind or merely very weakly sighted; if the
former, this is my way to show you how actual Latin letters look like,
without a lengthy description of letter shapes. :) )

or for graphs.  Here's commits per UTC hour of day:

⡀⠀⠀⢀⣴⣤⣴⣶⣶⣶⣾⣦
⣿⣷⣾⣿

git log --pretty=format:'%at'|
perl -pe 'use integer;/^(\d+)$/ or die;$_=$1/3600%24 ."\n"'|
sort -n|uniq -c|cut -c3-7|braillegraph -y 8

or arbitrary images, like my .sig in all my mails in this thread.

But your patch set doesn't special-case braille in any way, thus allowing
such abuse to work on the console is merely an unintended side effect.

> >The primary users would be:
> >* people who want symbols uncorrupted (especially if their language uses a
> >  non-latin script)
> >* CJK people (as discussed below)
> 
> Again, that's not true. Why aren't braille users included in this list? After
> all, it's we who motivated this enhancement. I guess actual blind people
> mustn't count just because there are relatively fewer of us. :-(

Well, I meant users of Unicode display fonts, ie, _additional_ functionality
that's not yet coded but would rely on this patchset.  What you guys want is
already included.

The reason I'm raising this issue now is because if the Unicode struct would
be the primary one, there's no point in keeping vc_data in addition to
uni_screen.  And that would require designing the struct well from the
start, to avoid unnecessary changes in the future.

But then, taking a bitmask from that 32-bit value wouldn't be a big change
-- you already take variously 8 or 9 bits out of a 16-bit field, depending
on 256 vs 512 glyph mode.

The other point is a quite pointless assumption that existing scrollback is
"optimized".  Even vgacon mostly uses software scrollback these days, as the
amount of VGA display memory is really small.

I don't know much about console display drivers in general, though, and it
looks like most of them are unmaintained (just noticed that sisusb for
example hasn't seen a maintainer action for 13 years, and that person's
domain expired in 2006).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Wed, Jun 20, 2018 at 10:59:08PM -0400, Nicolas Pitre wrote:
> On Thu, 21 Jun 2018, Adam Borowski wrote:
> 
> > On Tue, Jun 19, 2018 at 11:34:34AM -0400, Nicolas Pitre wrote:
> > > On Tue, 19 Jun 2018, Adam Borowski wrote:
> > > > Thus, it'd be nice to use the structure you add to implement full 
> > > > Unicode
> > > > range for the vast majority of people.  This includes even U+2800..FF.  
> > > > :)

> > > If the core console code makes the switch to full unicode then yes, that 
> > > would be the way to go to maintain backward compatibility. However 
> > > vgacon users would see a performance drop when switching between VT's 
> > > and we used to brag about how fast the Linux console used to be 20 years 
> > > ago. Does it still matter today?
> 
> > * VT switch
> > * scrollback
> > 
> > The last two cases are initiated by the user, and within human reaction time
> > you need to convert usually 2000 -- up to 20k-ish -- characters.  The
> > conversion is done by a 3-level array.  I think a ZX Spectrum can handle
> > this fine without a visible slowdown.
> 
> In the scrollback case, currently each driver is doing its own thing. 
> The vgacon driver is probably the most efficient as it only moves the 
> base memory register around without copying anything at all. And that 
> part doesn't have to change.

As long as the data is still in video memory, yeah.  Soft scrollback is not
yet the default, because some userspace tools assume vt switch clears
scrollback and do so for security reasons.  All known tools that do so have
been fixed (at least in Debian), but as you can run new kernels with
arbitrarily old userspace, it's better to wait a bit longer before switching
to something effectively identical to soft scrollback.  Failure mode: after
logout, you can scroll back to the supposedly cleared content of the old
session.

Your code avoids this, at the cost of losing data about anything
representable by the currently loaded charset for anything inside
scrollback.

But in the near future, it'd be good to have soft scrollback work the same
for all drivers.

> > Right, let's see if your patchset gets okayed before building atop it.
> 
> May I add your ACK to it?

I don't believe I'm knowledgeful nor active enough in this part for my ACKs
to be meaningful.  On the other hand, I've analyzed your patchset long
enough to see no problems with it, thus if you have an use for my tags, then
sure, you have my ACK.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: Kernel-only deployments?

[Adam Borowski]

On Thu, Aug 23, 2018 at 10:43:59AM -0700, Paul E. McKenney wrote:
> The mkinitramfs approach results in about 40MB of initrd, and dracut
> about 10MB.  Most of this is completely useless for rcutorture, which
> isn't interested in mounting filesystems, opening devices, and almost
> all of the other interesting things that mkinitramfs and dracut enable.
> 
> Those who know me will not be at all surprised to learn that I went
> overboard making the resulting initrd as small as possible.  I started
> by throwing out everything not absolutely needed by the dash and sleep
> binaries, which got me down to about 2.5MB, 1.8MB of which was libc.
> This situation of course prompted me to create an initrd containing
> a statically linked binary named "init" and absolutely nothing else
> (not even /dev or /tmp directories), which weighs in at not quite 800KB.
> This is a great improvement over 10MB, to say nothing of 40MB, but 800KB
> for a C-language "for" loop containing nothing more than a single call to
> sleep()?

.globl _start
.data
req:.8byte 9, 9
.text
_start:
mov $35, %rax   # syscall: nanosleep
mov $req, %rdi
xor %rsi, %rsi
syscall
jmp _start


as sl.s -o sl.o
ld sl.o -o init

'Ere you go, no libc needed.  If your arch is not amd64, just say so.

If you want to do anything more complex, though -- you really want musl
or another lightweight libc instead.  Glibc is utterly unfit for static
linking.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .globl _start↵.data↵rc: .ascii "/etc/init.d/rcS\0"↵.text↵_start
⣾⠁⢰⠒⠀⣿⡁ mov $57,%rax↵syscall↵cmp $0,%rax↵jne child↵parent:↵mov $61,%rax
⢿⡄⠘⠷⠚⠋⠀ mov $-1,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall↵jmp parent↵child:
⠈⠳⣄ mov $59,%rax↵mov $rc,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Thu, Jul 19, 2018 at 11:47:49AM +0100, Alan Cox wrote:
> On Wed, 18 Jul 2018 05:01:52 +0200
> Adam Borowski  wrote:
> 
> > Hi!
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> 
> That's a matter of taste so needs to configurable. Changing the default
> ought I think to be its own patch as it's a separate discussion to having
> the choice there and easy to make.

For vgacon yeah, you have a good point, as it can support either (and to be
exact, exactly one of the two as they share a hardware bit).  Only reason to
not have this configurable would be avoiding bloating the kernel with knobs
hardly anyone flips -- but you can already set minutiae like replacement
color for underline/italic/dim.  Thus you're right that if/when killing
blink on vgacon is implemented, it probably should be settable.

This is not an option on fbcon, though -- it can't blink (doable but I'm not
seeing anyone wanting to implement that) and already interprets that
attribute bit the way VGA would.  Thus, this patch merely makes vt behave a
way to match what the driver does.  This fixes some visual corruption in
certain user programs.

> For the palette why does it needs changing and exactly what standards
> document defines 'right', especially given we don't do ICC in console
> mode ? Have you tested the values used against multiple monitor types and
> cards with a light meter ?

You don't need a light meter for a difference of 53-out-of-256.  Most
desktop environments don't do ICC out of the box either, but there's an
assumption that whatever your monitor does, it gives the same result for the
same input (in the same lighting conditions).

That patch's purpose is:
* behave consistently between two APIs to set the same thing (\e[38;5;m vs
  \e[38;2;m) in a way that matches other terminals
* in case a future driver has better color handling we'd be different from
  other terminals

By the way, you can have ICC on console: "apt install vtgamma"
(https://github.com/kilobyte/vtgamma).

> BTW visibly breaking the Nvidia crud is also fine. They'll then actually
> bother to fix it and uually quite soon.

Yeah, but they haven't fixed 512-glyph yet (at least the last time I
looked), broken since day one.  In any case, this patchset doesn't support
vgacon yet, thus this is moot for now.  I picked the easy case (fbcon which
is always unblinking) over writing stuff to ports, we can update other
drivers later.

And vgacon has code that looks like it can do CGA and MDA (redundant with
mdacon?), either of which I haven't used in... quite a while.  Might be
tricky getting access to such hardware to test...


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Sat, Jul 21, 2018 at 09:43:19AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jul 18, 2018 at 05:01:52AM +0200, Adam Borowski wrote:
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> > * due to my error, 256-color mode uses a much darker palette for conversion,
> >   resulting in behaving inconsistently with 24-bit mode.
> > 
> > The new code uses bright backgrounds when possible, enabled with \e[100m or
> > \e[48;m.
> > 
> > Despite the whole idea following a VGA capability, this patchset doesn't
> > change vgacon yet, just fbcon.  [Not breaking nvidia-proprietary.]
> > 
> > Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
> > register 0x10) later.
> > 
> > This fixes the display of catimg and similar tools.
> 
> I've applied the first patch, as it was obvious :)
> 
> For the rest, can you make it a config option as Alan said?  And I
> agree, we don't care about breaking nvidia systems, go ahead :)

The only thing such an option would be able to set is disabling blinking on
vgacon, which is not yet implemented.  fbcon never blinks, and the only case
it can't display bright background is 1-bit black&white.

I probably should change the line in fbcon:
+   vc->vc_unblinking = vc->vc_can_do_color;
to always use 1, but it doesn't matter either way.  Only difference would be
matching the variable name ("unblinking" vs "shows_bright_bg").

Console drivers can interpret that bit of attribute as:
* blinking (vc_unblinking = 0)
* bright background (vc_unblinking = 1)
* neither (result is ignored down the pipeline)

The matrix is:
color b&w
---+---+--
fbbright bg neither
mda  N/A blink
newport  ?bright bg? ?N/A?
sti   ?neither??
vga  CGA/VGA: hardware switch (write to port)  MDA: blink
sisusb??

Technically, every console can be made to blink by drawing/clearing affected
characters a few times per second, but that'd be quite a waste of coding
time and kernel size.  There's a reason browsers dropped support for 
and text-decoration:blink.

This patchset doesn't drop blink in any case that was supported before,
though.  I did the easy part used by most people first (fbcon), so it can be
reviewed/merged.  Doing the hard part is quite pointless if this is NAKed
(but obviously doesn't require actual merging).

But perhaps by "option" you mean letting the user drop blink with no
replacement, which might also be a good idea if configurable?


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: get_maintainer.pl and change of email

[Adam Borowski]

On Wed, Aug 01, 2018 at 11:36:23AM -0700, Nick Desaulniers wrote:
> It seems that get_maintainer.pl will make recommendations based on
> commit history to a file, but over time, people change emails that
> they commit from, then get_maintainer.pl recommends the possibly now
> invalid email address.
> 
> I feel like we should have a look up table / file like MAINTAINERS
> that tracks previous and new email addresses, which get_maintainer.pl
> would parse, that way we have a record of movement between committer
> email addresses allowing get_maintainer.pl to make more accurate
> recommendations.  Thoughts?

.mailmap

-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Mon, Jul 23, 2018 at 10:53:29AM +0200, Geert Uytterhoeven wrote:
> On Sat, Jul 21, 2018 at 11:39 PM Adam Borowski  wrote:
> > Technically, every console can be made to blink by drawing/clearing affected
> > characters a few times per second, but that'd be quite a waste of coding
> > time and kernel size.  There's a reason browsers dropped support for 
> > and text-decoration:blink.
> 
> It's very simple and fast to implement in fbcon for FB_TYPE_PLANES or
> FB_TYPE_INTERLEAVED_PLANES and FB_VISUAL_PSEUDOCOLOR ;-)

Interesting...  I'm still not going to do the effort to implement that
(which would require learning fbdev internals first), though.  But while I
dislike this feature, someone else might want it.

The main problem here is that there are only 8 or 7 bits available for
attributes, thus it's better to use them for something more useful.  And
here, fbcon already interprets this bit as bright background, thus this
patchset makes vt use it instead of non-existant blink.

There'll be more bits available once attributes get migrated into uniscr --
either 11 or 32 bits depending on chosen implementation.  But I still
wouldn't go too wild with them: the console is meant for recovery tasks as
on any properly working system you can have an X terminal configured for
pixel-to-pixel identical behaviour as anything console can do.  Thus, only
cheap improvements to attributes make sense.  This patchset is currently at
+3 net lines, this certainly counts as cheap.

On the other hand, displaying the _glyph_ better would be worth some effort.
Nicolas' uniscr that's already in Greg's tree did most of the work, what
remains is a way to load and store a font with more glyphs (needs a sparse
representation).


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH v3 1/3] vt: preserve unicode values corresponding to screen characters

[Adam Borowski]

On Wed, Jul 11, 2018 at 01:39:56PM -0700, Kees Cook wrote:
> On Wed, Jul 11, 2018 at 2:18 AM, Greg Kroah-Hartman
>  wrote:
> > On Tue, Jul 10, 2018 at 05:52:01PM -0700, Kees Cook wrote:
> >> On Tue, Jun 26, 2018 at 8:56 PM, Nicolas Pitre  
> >> wrote:
> >> > +++ b/drivers/tty/vt/vt.c
> >> > [...]
> >> > +static void vc_uniscr_scroll(struct vc_data *vc, unsigned int t, 
> >> > unsigned int b,
> >> > +enum con_scroll dir, unsigned int nr)
> >> > +{
> >> > +   struct uni_screen *uniscr = get_vc_uniscr(vc);
> >> > +
> >> > +   if (uniscr) {
> >> > +   unsigned int s, d, rescue, clear;
> >> > +   char32_t *save[nr];
> >>
> >> Can you adjust this to avoid the VLA here? I've almost gotten all VLAs
> >> removed from the kernel[1], and this is introducing a new one. :)
> >
> Yup, that's fine. (It's how I noticed it: linux-next VLA build tests.)
> I'm just hoping it can get solved before the merge window opens. :)

This one is actually nasty: max console height is 32767, if you resize it
that big then issue a large scroll request, boom it goes.

> There are still a bunch of VLAs I'm chipping away at, but this was a
> newly added one, so I was hoping Nicolas (when he's back from
> vacation) will have ideas on how to best avoid it.

Nicolas: what about just moving line pointers one at a time?  Rotating an
array slice in-place isn't that slow -- and optimizing that much when
reasonable sizes don't exceed 100ish (depending on how good your eyes are)
is quite ridiculous.  Thanks to your change, we don't need to move actual
contents, just line pointers -- that's fast enough.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] .gitignore: add ZSTD-compressed files

[Adam Borowski]

For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more
will come, thus let's be consistent with all other compressors.

Signed-off-by: Adam Borowski 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 97ba6b79834c..0d09cf1c053c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@
 *.tab.[ch]
 *.tar
 *.xz
+*.zst
 Module.symvers
 modules.builtin
 
-- 
2.18.0

Re: [PATCH v3] drm/nouveau: Move irq setup/teardown to pci ctor/dtor

[Adam Borowski]

On Thu, Jan 25, 2018 at 06:29:53PM -0500, Lyude Paul wrote:
> This was made apparent by what appeared to be a regression in the
> mainline kernel that started introducing suspend/resume issues for
> nouveau:
> 
> a0c9259dc4e1 (irq/matrix: Spread interrupts on allocation)

I'm just a dumb user here, but I confirm:
CPU: AMD Phenom II X6 1055T, GPU: GTX 560 Ti
100% fail to resume GPU on 4.15-rc*, 100% ok with your patch.

And that spinning rust hates being repeatedly powered off and on... :/

> ---
> Changes since v2:
>  - Remove teardown, just reuse pci->irq to indicate when we're tearing
>down the driver
> 
>  drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c | 44 
> +-
>  1 file changed, 29 insertions(+), 15 deletions(-)

Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven big-ass trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...

Re: [git pull] uaccess-related bits of vfs.git

[Adam Borowski]

On Sat, May 13, 2017 at 07:57:45AM +0100, Al Viro wrote:
> On Fri, May 12, 2017 at 06:00:44PM -0700, Linus Torvalds wrote:
> > But the *first* thing I'd like to do would be to just get rid of
> > __get_user/__put_user as a thing. It really does generate nasty code,
> > and we might as well just make it do that function call.
> 
> But IMO the first step is not to convert __get_user()/__put_user() to
> aliases of get_user()/put_user() - it's to get rid of their infestations
> outside of arch/*.  They are concentrated in relatively few places, so
> we can deal with those individually and then just fucking ban their use
> outside of arch/*.  That's easy enough to watch for - simple git grep will do,
> so anything creeping back will be immediately spotted.

As someone from the peanuts gallery, I took a look for __put_user() in my
usual haunt, drivers/tty/vt/

* use 1: con_[gs]et_trans_*():
  Copies a linear array of 256 bytes/shorts, one by one.
  The obvious patch has 9 insertions(+), 22 deletions(-).

* use 2: con_[gs]et_unimap():
  Ditto, up to 65535*2 shorts, also in a nice linear array.

* use 3: tioclinux():
  Does a __put into a place that was checked only for read.  This got me
  frightened as it initially looked like something that can allow an user to
  write where they shouldn't.  Fortunately, it turns out the first argument
  to access_ok() is ignored on every single architecture -- why does it even
  exist then?  I imagined it's there for some odd arch that allows writes
  when in privileged mode, but unlike what the docs say, VERIFY_WRITE is
  exactly same as VERIFY_READ.

Ie, every use in this sample is wrong.  I suspect the rest of the kernel
should be similar.


Meow!
-- 
Don't be racist.  White, amber or black, all beers should be judged based
solely on their merits.  Heck, even if occasionally a cider applies for a
beer's job, why not?
On the other hand, corpo lager is not a race.

[PATCH 2/2] vsprintf: don't dereference pointers to the first or last page

[Adam Borowski]

As old code to avoid so is inconsistent, let's unify it within a single
macro.

Signed-off-by: Adam Borowski 
---
 lib/vsprintf.c | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 1c2c3cc5a321..4914dac03f0a 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -47,6 +47,8 @@
 #include 
 #include "kstrtox.h"
 
+#define IS_BAD_PTR(x) ((unsigned long)(x) >= (unsigned long)-PAGE_SIZE \
+   || (unsigned long)(x) < PAGE_SIZE)
 #define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : "(invalid)")
 
 /**
@@ -589,7 +591,7 @@ char *string(char *buf, char *end, const char *s, struct 
printf_spec spec)
int len = 0;
size_t lim = spec.precision;
 
-   if ((unsigned long)s < PAGE_SIZE)
+   if (IS_BAD_PTR(s))
s = BAD_PTR_STRING(s);
 
while (lim--) {
@@ -1583,7 +1585,7 @@ char *device_node_string(char *buf, char *end, struct 
device_node *dn,
if (!IS_ENABLED(CONFIG_OF))
return string(buf, end, "(!OF)", spec);
 
-   if ((unsigned long)dn < PAGE_SIZE)
+   if (IS_BAD_PTR(dn))
return string(buf, end, BAD_PTR_STRING(dn), spec);
 
/* simple case without anything any more format specifiers */
@@ -1851,7 +1853,7 @@ char *pointer(const char *fmt, char *buf, char *end, void 
*ptr,
 {
const int default_width = 2 * sizeof(void *);
 
-   if (!ptr && *fmt != 'K' && *fmt != 'x') {
+   if (IS_BAD_PTR(ptr) && *fmt != 'K' && *fmt != 'x') {
/*
 * Print (null)/etc with the same width as a pointer so it
 * makes tabular output look nice.
@@ -2575,8 +2577,7 @@ int vbin_printf(u32 *bin_buf, size_t size, const char 
*fmt, va_list args)
const char *save_str = va_arg(args, char *);
size_t len;
 
-   if ((unsigned long)save_str > (unsigned long)-PAGE_SIZE
-   || (unsigned long)save_str < PAGE_SIZE)
+   if (IS_BAD_PTR(save_ptr))
save_str = BAD_PTR_STRING(save_str);
len = strlen(save_str) + 1;
if (str + len < end)
-- 
2.16.2

[PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

Attempting to print an object pointed to by a bad (usually ERR_PTR) pointer
is a not so surprising error.  Our code handles them inconsistently:
 * two places print (null) if ptr
---
 lib/vsprintf.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index d7a708f82559..1c2c3cc5a321 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -47,6 +47,8 @@
 #include 
 #include "kstrtox.h"
 
+#define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : "(invalid)")
+
 /**
  * simple_strtoull - convert a string to an unsigned long long
  * @cp: The start of the string
@@ -588,7 +590,7 @@ char *string(char *buf, char *end, const char *s, struct 
printf_spec spec)
size_t lim = spec.precision;
 
if ((unsigned long)s < PAGE_SIZE)
-   s = "(null)";
+   s = BAD_PTR_STRING(s);
 
while (lim--) {
char c = *s++;
@@ -1582,7 +1584,7 @@ char *device_node_string(char *buf, char *end, struct 
device_node *dn,
return string(buf, end, "(!OF)", spec);
 
if ((unsigned long)dn < PAGE_SIZE)
-   return string(buf, end, "(null)", spec);
+   return string(buf, end, BAD_PTR_STRING(dn), spec);
 
/* simple case without anything any more format specifiers */
fmt++;
@@ -1851,12 +1853,12 @@ char *pointer(const char *fmt, char *buf, char *end, 
void *ptr,
 
if (!ptr && *fmt != 'K' && *fmt != 'x') {
/*
-* Print (null) with the same width as a pointer so it makes
-* tabular output look nice.
+* Print (null)/etc with the same width as a pointer so it
+* makes tabular output look nice.
 */
if (spec.field_width == -1)
spec.field_width = default_width;
-   return string(buf, end, "(null)", spec);
+   return string(buf, end, BAD_PTR_STRING(ptr), spec);
}
 
switch (*fmt) {
@@ -2575,7 +2577,7 @@ int vbin_printf(u32 *bin_buf, size_t size, const char 
*fmt, va_list args)
 
if ((unsigned long)save_str > (unsigned long)-PAGE_SIZE
|| (unsigned long)save_str < PAGE_SIZE)
-   save_str = "(null)";
+   save_str = BAD_PTR_STRING(save_str);
len = strlen(save_str) + 1;
if (str + len < end)
memcpy(str, save_str, len);
-- 
2.16.2

Re: [PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

On Tue, Mar 06, 2018 at 09:22:17PM +0300, Alexey Dobriyan wrote:
> > +#define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : 
> > "(invalid)")
> 
> This is getting ridiculous.
> 
> Instead of simply printing a pointer as %08lx or %016llx, not only glibc
> (null) stupidity is propagated but expanded and "improved".

This is not about printing a pointer, this is about attempting to print an
object referenced by such a bad pointer.  Which leads to a crash: in
userspace, you get a segfault; in the kernel, at least in the case I tested,
the system is dead without even a squeal on either console or serial.

> I assure you reading  is just as obvious as (null) and
> reading fffa is just as good as -ENOMEM.
> 
> In fact printing with hex is more information. Maybe it is important
> that buggy pointer is small value but it's value is lost.
>
> Sure don't dereference a pointer for very small or very erry values
> but print it without all the bell and whistles.

That's a reasonable suggestion, but it still needs to be special cased.
Note the difference between printk("%px", 42) and printk("%s", 42).

See this part:
-   if (!ptr && *fmt != 'K' && *fmt != 'x') {
+   if (IS_BAD_PTR(ptr) && *fmt != 'K' && *fmt != 'x') {
Printing the pointer is already excluded; what I'm fixing is:
1. lying that the bad pointer was (null) when it was -ENOMEM (commit 1)
2. crash when some bad code tries to printk("%s", -ENOMEM) (commit 2)

So, if what you propose is applying commit 2, and changing 1 to print the
raw value instead of (null)/(err)/(invalid), that sounds good.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

On Wed, Mar 07, 2018 at 03:17:19PM +0200, Andy Shevchenko wrote:
> On Tue, 2018-03-06 at 19:11 +0100, Adam Borowski wrote:
> 
> Thanks for the patch, my comments below.

(Review snipped.)

It looks pretty obvious that it'd take a lot less of your time to roll new
patch[es] from scratch than to try to educate me wrt how you'd want to see
it done; thus I'll sit out this one.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH] [NOT FOR MERGING] lib: Be noisy about used decompression method.

[Adam Borowski]

It's too easy to build the initrd with wrong options during testing, after
which it may silently work anyway.

Signed-off-by: Adam Borowski 
---
 lib/decompress.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/decompress.c b/lib/decompress.c
index ab3fc90ffc64..556b9d916e10 100644
--- a/lib/decompress.c
+++ b/lib/decompress.c
@@ -78,7 +78,9 @@ decompress_fn __init decompress_method(const unsigned char 
*inbuf, long len,
break;
 
}
-   if (name)
+   if (name) {
*name = cf->name;
+   printk("Decompressing using %s.\n", *name);
+   }
return cf->decompressor;
 }
-- 
2.16.2

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Wed, Mar 21, 2018 at 06:29:41PM -0700, Nick Terrell wrote:
> This patch set adds support for a ZSTD-compressed kernel and ramdisk
> images in the kernel boot process. It only integrates the support with
> x86, though the first patch is generic to all architectures.

I'm running this patch set since October on amd64 armhf arm64, no
explosions -- besides the obvious when it's not applied yet I forget to
reconfigure initramfs-tools.  Which is getting tedious, so let's merge this
already. :)

I've tested initrd on all of these archs; here's a debug patch to check if
you're actually using it.

As for compressing the kernel itself: the second patch works as submitted
(ie, x86 only).  Porting it to other architectures is straightforward, but
eg. on arm, most boards use u-boot which insists on decompressing the kernel
by itself instead of passing control.  EFI/grub should work, but I haven't
tested that yet.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Thu, Mar 22, 2018 at 12:09:45PM +0100, René Rebe wrote:
> Should this currently just work without any arch change on e.g.
> ppc64, sparc64 et al.? I could do a test build and boot if that is
> of any value, ...

Initrd: no reason it wouldn't work, although for anything related to the
boot process testing is a very good idea.  If you test, you'd really want to
apply that printk patch I posted, it's too easy to accidentally get a silent
fallback to gzip or uncompressed.  I wonder, perhaps such a printk could be
permanently added, at a low message priority?

Kernel itself: needs per-arch porting, but it's not advertised in kconfig
outside x86 so that's not a problem.  

If you are knowledgeful about bootloaders on ppc64, sparc64, etc, such
information would be great.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: Contributors can not just rescind the license (Legal Opinion)

[Adam Borowski]

On Fri, Sep 28, 2018 at 11:31:32AM +0200, www.Advocati.org wrote:
> CONTRIBUTORS CAN NOT JUST RESCIND THE LICENSE
> THEY GRANTED UNDER GPLv2. IT IS *COPYLEFTED*

> A letter from 17.09.2018 by @observerofaffairs
> titled 'GPL version 2 is a bare license. Recind. (Regarding (future)
> Code of Conduct Bannings)'
> https://lkml.org/lkml/2018/9/17/1174,

> repeated by another letter from 20.09.2018 by @unconditionedwitness
> titled 'Re: A Plea to Unfuck our Codes of Conduct'
> https://lulz.com/linux-devs-threaten-killswitch-coc-controversy-1252/

You're responding to trolling by our dear MikeeUSA, whom most of us long
since learned to recognize and ignore.  There are better uses for your time
than responding to a notorious cartooney.

He's as nasty a troll as CoralineAda, and about as harmful to the community.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.

Re: Official Linux system wrapper library?

[Adam Borowski]

On Sun, Nov 11, 2018 at 12:46:35PM +0100, Florian Weimer wrote:
> A lot of multi-threaded applications assume that most high-level
> functionality remains usable even after fork in a multi-threaded
> process.

How would this be even possible?  Currently fork kills all threads
(save for the caller).

Glibc's manpage also warns:

# After a fork() in a multithreaded program, the child can safely call only
# async-signal-safe functions (see signal-safety(7)) until such time as it
# calls execve(2).

Which makes sense as its malloc uses a mutex, and you can't take a breath
without a library call using malloc somewhere (or in C++, the language
itself).

So any functionality remaining usable after fork is pretty strictly
limited...


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢰⠒⠀⣿⡁ productivity. You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so. I recommend Skepticism
⠈⠳⣄ (funeral doom metal).

[PATCH 01/17] lib: Add support for ZSTD-compressed kernel

[Adam Borowski]

From: Nick Terrell 

Add support for extracting ZSTD-compressed kernel images, as well as
ZSTD-compressed ramdisk images in the kernel boot process.

When neither `fill' nor `flush' are used, the decompression function
requires a constant amount of memory (192 KB is sufficient). When either
is used the decompression function requires memory proportional to the
window size used during compression, which is limited to 8 MB. The maximum
memory usage is just over 8 MB.

Fix up lib/zstd and lib/xxhash.c for the preboot environment. They avoid
declaring themselves as modules. A memcpy() call needs to be a
__builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was
fixed in gcc-7.2, so it can be gated, since it hurts performance.

Signed-off-by: Nick Terrell 
---
 include/linux/decompress/unzstd.h |  26 +++
 init/Kconfig  |  14 +-
 lib/Kconfig   |   4 +
 lib/Makefile  |   1 +
 lib/decompress.c  |   5 +
 lib/decompress_unzstd.c   | 341 ++
 lib/xxhash.c  |  21 +-
 lib/zstd/decompress.c |   2 +
 lib/zstd/fse_decompress.c |   9 +-
 lib/zstd/zstd_internal.h  |  10 +-
 scripts/Makefile.lib  |  15 ++
 usr/Kconfig   |  22 ++
 12 files changed, 451 insertions(+), 19 deletions(-)
 create mode 100644 include/linux/decompress/unzstd.h
 create mode 100644 lib/decompress_unzstd.c

diff --git a/include/linux/decompress/unzstd.h 
b/include/linux/decompress/unzstd.h
new file mode 100644
index ..6f3022cd0955
--- /dev/null
+++ b/include/linux/decompress/unzstd.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2017 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#ifndef LINUX_DECOMPRESS_UNZSTD_H
+#define LINUX_DECOMPRESS_UNZSTD_H
+
+int unzstd(unsigned char *inbuf, long len,
+  long (*fill)(void*, unsigned long),
+  long (*flush)(void*, unsigned long),
+  unsigned char *output,
+  long *pos,
+  void (*error_fn)(char *x));
+#endif
diff --git a/init/Kconfig b/init/Kconfig
index a4112e95724a..ffa5ae4abc88 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -134,13 +134,16 @@ config HAVE_KERNEL_LZO
 config HAVE_KERNEL_LZ4
bool
 
+config HAVE_KERNEL_ZSTD
+   bool
+
 config HAVE_KERNEL_UNCOMPRESSED
bool
 
 choice
prompt "Kernel compression mode"
default KERNEL_GZIP
-   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED
+   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_ZSTD || 
HAVE_KERNEL_UNCOMPRESSED
help
  The linux kernel is a kind of self-extracting executable.
  Several compression algorithms are available, which differ
@@ -219,6 +222,15 @@ config KERNEL_LZ4
  is about 8% bigger than LZO. But the decompression speed is
  faster than LZO.
 
+config KERNEL_ZSTD
+   bool "ZSTD"
+   depends on HAVE_KERNEL_ZSTD
+   help
+ ZSTD is a compression algorithm targeting intermediate compression
+ with fast decompression speed. It will compress better than GZIP and
+ decompress around the same speed as LZO, but slower than LZ4. You
+ will need at least 192 KB RAM or more for booting.
+
 config KERNEL_UNCOMPRESSED
bool "None"
depends on HAVE_KERNEL_UNCOMPRESSED
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..e7ab43fd5461 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -304,6 +304,10 @@ config DECOMPRESS_LZ4
select LZ4_DECOMPRESS
tristate
 
+config DECOMPRESS_ZSTD
+   select ZSTD_DECOMPRESS
+   tristate
+
 #
 # Generic allocator support is selected if needed
 #
diff --git a/lib/Makefile b/lib/Makefile
index db06d1237898..58b48993f48a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -139,6 +139,7 @@ lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o
 lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o
 lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o
 lib-$(CONFIG_DECOMPRESS_LZ4) += decompress_unlz4.o
+lib-$(CONFIG_DECOMPRESS_ZSTD) += decompress_unzstd.o
 
 obj-$(CONFIG_TEXTSEARCH) += textsearch.o
 obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o
diff --git a/lib/decompress.c b/lib/decompress.c
index

[PATCH 16/17] Kconfig: Update the prose for selection of compression algorithm

[Adam Borowski]

It was really obsolete, and some entries contradicted each other.

Let's not recommend ZSTD for kernel compression yet as it's available
only on x86, and some distros might not have the tool installed.
Proposing ZSTD for initrd is safer but let's test it first.

Signed-off-by: Adam Borowski 
---
 init/Kconfig | 25 +
 usr/Kconfig  | 24 +++-
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 412ba93673fa..7c0180c41a3c 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -153,26 +153,27 @@ choice
  version of this functionality (bzip2 only), for 2.4, was
  supplied by Christian Ludwig)
 
- High compression options are mostly useful for users, who
- are low on disk space (embedded systems), but for whom ram
- size matters less.
+ High compression options tend to be more useful in most cases,
+ as bootloaders are often egregiously slow to read the kernel
+ from the disk/SD card/network/etc, overcoming any boot time
+ savings you would get from faster decompression.
 
- If in doubt, select 'gzip'
+ If in doubt, select 'xz'
 
 config KERNEL_GZIP
bool "Gzip"
depends on HAVE_KERNEL_GZIP
help
- The old and tried gzip compression. It provides a good balance
- between compression ratio and decompression speed.
+ The old and tried gzip compression. You generally want it if
+ some tool you use doesn't support more modern compressors.
 
 config KERNEL_LZMA
bool "LZMA"
depends on HAVE_KERNEL_LZMA
help
- This compression algorithm's ratio is best.  Decompression speed
- is between gzip and bzip2.  Compression is slowest.
- The kernel size is about 33% smaller with LZMA in comparison to gzip.
+ An old version of xz, like it providing strong compression at slow
+ speed. It lacks a header and support for filters or uncompressed
+ blocks, thus it's usually better to pick xz.
 
 config KERNEL_XZ
bool "XZ"
@@ -193,9 +194,9 @@ config KERNEL_LZO
bool "LZO"
depends on HAVE_KERNEL_LZO
help
- Its compression ratio is the poorest among the choices. The kernel
- size is about 10% bigger than gzip; however its speed
- (both compression and decompression) is the fastest.
+ Its compression ratio is pretty poor (but still better than
+ LZ4). You want to pick ZSTD (similar speed but much better
+ compression) or LZ4 (much better speed) instead.
 
 config KERNEL_LZ4
bool "LZ4"
diff --git a/usr/Kconfig b/usr/Kconfig
index 8d99edacabc9..f6e871585f05 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -131,17 +131,15 @@ config INITRAMFS_COMPRESSION_NONE
  on those architectures that support this. However, not compressing the
  initramfs may lead to slightly higher memory consumption during a
  short time at boot, while both the cpio image and the unpacked
- filesystem image will be present in memory simultaneously
+ filesystem image will be present in memory simultaneously.
 
 config INITRAMFS_COMPRESSION_GZIP
bool "Gzip"
depends on RD_GZIP
help
- Use the old and well tested gzip compression algorithm. Gzip provides
- a good balance between compression ratio and decompression speed and
- has a reasonable compression speed. It is also more likely to be
- supported by your build system as the gzip tool is present by default
- on most distros.
+ Use the old and well tested gzip compression algorithm. Gzip doesn't
+ provide very good compression nor speed, but it's the safest choice,
+ with wide support.
 
 config INITRAMFS_COMPRESSION_XZ
bool "XZ"
@@ -150,20 +148,20 @@ config INITRAMFS_COMPRESSION_XZ
  XZ uses the LZMA2 algorithm and has a large dictionary which may cause
  problems on memory constrained systems. The initramfs size is about
  30% smaller with XZ in comparison to gzip. Decompression speed is
- better than that of bzip2 but worse than gzip and LZO. Compression is
- slow.
+ okayish but still slowest of all currently available algorithms.
+ Compression is slow.
 
- If you choose this, keep in mind that you may need to install the xz
- tool to be able to compress the initram.
+ Any modern distro provides xz in its default install, but on
+ minimal build systems you might need to install xz-utils to be
+ able to compress the initram.
 
 config INITRAMFS_COMPRESSION_LZO
bool "LZO"
depends on RD_LZO
help
  It's compression ratio is the second poorest amongst

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Sun, Jun 17, 2018 at 03:07:02PM -0400, Nicolas Pitre wrote:
> The vt code translates UTF-8 strings into glyph index values and stores
> those glyph values directly in the screen buffer. Because there can only
> be at most 512 glyphs, it is impossible to represent most unicode
> characters, in which case a default glyph (often '?') is displayed
> instead. The original unicode value is then lost.
> 
> The 512-glyph limitation is inherent to VGA displays, but users of
> /dev/vcs* shouldn't have to be restricted to a narrow unicode space from
> lossy screen content because of that. This is especially true for
> accessibility applications such as BRLTTY that rely on /dev/vcs to rander
> screen content onto braille terminals.

You're thinking small.  That 256 possible values for Braille are easily
encodable within the 512-glyph space (256 char + stolen fg brightness bit,
another CGA peculiarity).  Your patchset, though, can be used for proper
Unicode support for the rest of us.

The 256/512 value limitation applies only to CGA-compatible hardware; these
days this means vgacon.  But most people use other drivers.  Nouveau forces
graphical console, on arm* there's no such thing as VGA[1], etc.

Thus, it'd be nice to use the structure you add to implement full Unicode
range for the vast majority of people.  This includes even U+2800..FF.  :)

> This patch series introduces unicode support to /dev/vcs* devices,
> allowing full unicode access from userspace to the vt console which
> can, amongst other purposes, appropriately translate actual unicode
> screen content into braille. Memory is allocated, and possible CPU
> overhead introduced, only if /dev/vcsu is read at least once.

What about doing so if any updated console driver is loaded?  Possibly, once
the vt in question has been switched to (>99% people never see anything but
tty1 during boot-up, all others showing nothing but getty).  Or perhaps the
moment any non-ASCII character is output to the given vt.

If memory usage is a concern, it's possible to drop the old structure and
convert back only in the rare case the driver is unloaded; reads of old-
style /dev/vc{s,sa}\d* are not speed-critical thus can use conversion on the
fuly.  Unicode takes only 21 bits out of 32 you allocate, that's plenty of
space for attributes: they currently take 8 bits; naive way gives us free 3
bits that could be used for additional attributes.

Especially underline is in common use these days; efficient support for CJK
would also use one bit to mark left/right half.  And it's decades overdue to
drop blink, which is not even supported by anything but vgacon anyway!
(Graphical drivers tend to show this bit as bright background, but don't
accept SGR codes other thank blink[2].)

> I'm a prime user of this feature, as well as the BRLTTY maintainer Dave Mielke
> who implemented support for this in BRLTTY. There is therefore a vested
> interest in maintaining this feature as necessary. And this received
> extensive testing as well at this point.

So, you care only about people with faulty wetware.  Thus, it sounds like
work that benefits sighted people would need to be done by people other than
you.  So I'm only mentioning possible changes; they could possibly go after
your patchset goes in:

A) if memory is considered to be at premium, what about storing only one
   32-bit value, masked 21 bits char 11 bits attr?  On non-vgacon, there's
   no reason to keep the old structures.
B) if being this frugal wrt memory is ridiculous today, what about instead
   going for 32 bits char (wasteful) 32 bits attr?  This would be much nicer
   15 bit fg color + 15 bit bg color + underline + CJK or something.
You already triple memory use; variant A) above would reduce that to 2x,
variant B) to 4x.

Considering that modern machines can draw complex scenes of several
megapixels 60 times a second, it could be reasonable to drop the complexity
of two structures even on vgacon: converting characters on the fly during vt
switch is beyond notice on any hardware Linux can run.

> This is also available on top of v4.18-rc1 here:
> 
>   git://git.linaro.org/people/nicolas.pitre/linux vt-unicode



Meow!

[1]. config VGA_CONSOLE
  depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC &&  !SUPERH && \
  (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \
  !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !NDS32 && !S390

[2]. Sounds like an easy improvement; not so long ago I added "\e[48;5;m",
"\e[48;2;m" and "\e[100m" which could be improved when on unblinking drivers.
Heck, even VGA can be switched to unblinking by flipping bit 3 of the
Attribute Mode Control Register -- like we already flip foreground
brightness when 512 glyphs are needed.
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go 

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Tue, Jun 19, 2018 at 09:52:13AM -0400, Dave Mielke wrote:
> [quoted lines by Adam Borowski on 2018/06/19 at 15:09 +0200]
> 
> >You're thinking small.  That 256 possible values for Braille are easily
> >encodable within the 512-glyph space (256 char + stolen fg brightness bit,
> >another CGA peculiarity).  
> 
> Not at all. We braille users, especially when working with languages other 
> than
> English, need more than 256 non-braille characters. Even for those who can 
> live
> with just 256 non-braille characters, it's still a major pain having to come 
> up
> with a usable braille-capable font for every needed 256 non-braille characters
> set. I can assure you, as an actual braille user, that the limitation has been
> a very long-standing problem and it's a great relief that it's finally been
> resolved.

Ok, I thought Braille is limited to 2x3 dots, recently extended to 2x4;
thanks for the explanation!

But those of us who are sighted, are greatly annoyed by characters that are
usually taken for granted being randomly missing.  For example, no console
font+mapping shipped with Debian supports ░▒▓▄▀ (despite them being a
commonly used part of the BIOS charset), so unless you go out of your way to
beat them back they'll be corrupted (usually into ♦).  Then Perl6 wants ｢｣⚛,
and so on.  All these problems would instantly disappear the moment console
sheds the limit of 256/512 glyphs.

So I'm pretty happy seeing this patch set.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

[PATCH resend] scripts: teach extract-vmlinux about LZ4 and ZSTD

[Adam Borowski]

Note that the LZ4 signature is different than that of modern LZ4 as we
use the "legacy" format which suffers from some downsides like inability
to disable compression.

Signed-off-by: Adam Borowski 
---
The first time this was sent I managed to screw up both the subject and
scissors line, so it's likely whatever filter you're using didn't notice
the patch inside a thread.

ZSTD compression for vmlinux is not yet in, but being able to unpack it
can't hurt.  LZ4 is in for quite a while.


 scripts/extract-vmlinux | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
index 5061abcc2540..e6239f39abad 100755
--- a/scripts/extract-vmlinux
+++ b/scripts/extract-vmlinux
@@ -57,6 +57,8 @@ try_decompress '\3757zXZ\000' abcde unxz
 try_decompress 'BZh'  xybunzip2
 try_decompress '\135\0\0\0'   xxx   unlzma
 try_decompress '\211\114\132' xy'lzop -d'
+try_decompress '\002!L\030'   xxx   'lz4 -d'
+try_decompress '(\265/\375'   xxx   unzstd
 
 # Bail out:
 echo "$me: Cannot find vmlinux." >&2
-- 
2.18.0

Re: Linux 4.18.1

[Adam Borowski]

Hi!

On Thu, Aug 16, 2018 at 12:14:29PM +0200, Greg KH wrote:
> I'm announcing the release of the 4.18.1 kernel.

I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:

ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
(.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'

gcc 8.2.0-4, as shipped in Debian unstable.
.config attached.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


config.xz
Description: application/xz

Re: Linux 4.18.1

[Adam Borowski]

On Thu, Aug 16, 2018 at 05:11:21PM +0200, Greg KH wrote:
> On Thu, Aug 16, 2018 at 03:59:58PM +0200, Sven Joachim wrote:
> > On 2018-08-16 15:05 +0200, Adam Borowski wrote:
> > > I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:
> > >
> > > ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
> > > (.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'
> > 
> > Same here and also in 4.17.15, but not in Linus' tree.
> > 
> > > CONFIG_KVM=y
> > > # CONFIG_KVM_INTEL is not set
> > > CONFIG_KVM_AMD=y
> > 
> > I have CONFIG_KVM{,AMD}=m and CONFIG_KVM_INTEL is not set either.
> 
> This is fixed in my queue, if it really bothers you, apply commit
> 1eb46908b35d ("x86/l1tf: Fix build error seen if CONFIG_KVM_INTEL is
> disabled") to your tree, it will be in the next round of stable kernel
> releases.

Probably obvious, but: cherry-picking that commit fixes the problem, it
builds, boots and works ok.

Thanks!


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Aug 17, 2018 at 09:54:03AM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 06:15:45PM +0200, René Rebe wrote:
> > Hey,
> > 
> > is there any mainline future for this zstd support?
> > Currently my most favourite compressor for this, and for what it’s worth
> > zstd/initrd now even tested on #t2sde / hp/pa-risc
> 
> Can you remove some other compressor for the kernel image if you merge this?

Funny you say this.  I want to post this after the merge window is over:
https://github.com/kilobyte/linux/commits/nobz2-next-20180731

Sorry for a tree atop next, but there are conflicts that needed to be
adressed.  The patchset is also undertested (works for me, but not even
compile-tested on archs I don't have a toolchain for at hand).

> The "favourite compressor" seems to roughly change every year, so if
> we keep adding new ones things will get more and more convoluted.

The above patchset drops just bzip2.  It is the only one that's strictly
beaten in every way (ratio, time, memory usage), there are also no other
uses of bzip2 anywhere in the kernel so we'd get to drop its code
completely: 900 lines of Linus' happiness.

Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
but those are used elsewhere thus there's hardly any gain.  If you want them
gone, please say so -- I'll include their droppage.

> Surely zstd is universally better than some existing compressor
> the kernel already uses for itself for a particular sweet spot?
> If it isn't it's likely not needed.

zstd is better in the balanced niche: unless you want extreme speed (lz4) or
best compression (xz), zstd handily beats algorithms in the middle.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Aug 17, 2018 at 12:22:44PM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 07:57:46PM +0200, Adam Borowski wrote:
> > > The "favourite compressor" seems to roughly change every year, so if
> > > we keep adding new ones things will get more and more convoluted.
> > 
> > The above patchset drops just bzip2.  It is the only one that's strictly
> > beaten in every way (ratio, time, memory usage), there are also no other
> 
> Does time include build time? I've been reverting back to gzip recently
> because I care very much about that.

Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while
ago), here's copypasta of a random 16824672 byte executable, in userspace,
with default level setting:

compdecomp  size
xz  8.038s  0.356s  4320292
bz2 2.265s  0.730s  5234516
zst 0.274s  0.102s  5657626
gz  0.880s  0.152s  6515505
Z   0.499s  0.133s  8932459
lzo 0.100s  0.095s  9198874

As you can see, zstd's compression time is drastically better than gzip,
while ratio is better.  The default level is very low (-3 on -1..-22 scale)
but you can crank it up for stronger compression.

The defaults fit your use case.

> > uses of bzip2 anywhere in the kernel so we'd get to drop its code
> > completely: 900 lines of Linus' happiness.
> 
> Great!
> 
> > Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
> > but those are used elsewhere thus there's hardly any gain.  If you want them
> > gone, please say so -- I'll include their droppage.
> 
> Yes would be good to remove the kernel image support for those too
> just to simplify the config process, even if it doesn't save much code.

There's one caveat: fast choices are quite new:
* lz4 userspace tools are not even in current Debian stable (just unstable)
* uncompressed kernel got in only this merge window
* zstd has userspace tools in Debian stable but is not merged into the
  kernel yet
(other dists are probably similar)

Thus, it might be a good idea to keep lzo for a while longer.

Bare lzma can probably go -- xz filters are nice for binaries of archs it
knows (disabled otherwise), and lack of header requires hacks to find out
the payload's size.

So it's up to you guys: do you want me to drop lzo and/or lzma?
We can also drop them just for vmlinuz but not initrd.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: linux-next: build warnings from Linus' tree

[Adam Borowski]

On Sun, Aug 19, 2018 at 04:21:57PM -0700, Linus Torvalds wrote:
> On Sun, Aug 19, 2018 at 3:13 PM Stephen Rothwell  
> wrote:
> >
> > Today's linux-next build (powerpc ppc64_defconfig) produced these
> > warnings:
> >
> > fs/cifs/cifssmb.c:605:3: warning: 'strncpy' writing 16 bytes into a region 
> > of size 1 overflows the destination [-Wstringop-overflow=]
> >strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
> >
> > Presumably caused by my update to gcc 8.2.0.
> 
> Yeah. There are some patches to mark some arrays as non-strings to get
> rid of these, but we'll see. Maybe we'll just disable the new gcc
> warning if it causes more pain than it is worth.

Every single use of strncpy() for a C string is either a bug, inefficiency,
or both.  In this particular case the code:

count = 0;
for (i = 0; i < CIFS_NUM_PROT; i++) {
strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
count += strlen(protocols[i].name) + 1;
/* null at end of source and target buffers anyway */
}

* pointlessly clears 16 bytes in every iteration
* calculates the string's length twice
* there's no protection against buffer overflow anyway

So what is the strncpy() there for, when an unbounded copy would be just as
good?  For other cases, there's a bunch of better functions: strlcpy(),
snprintf(), even strlen()+memcpy(), etc.

Valid uses of strncpy() do exist (such as SCSI structs), but those deal with
fixed-width fields.  Thus, gcc is right for warning for at least some of
misuse of strncpy() for C strings.  The function wasn't designed for them.

(Skipped analysis why strncpy is always a bad choice for C strings.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: Can we drop upstream Linux x32 support?

[Adam Borowski]

On Thu, Dec 13, 2018 at 10:37:31AM +0100, Sven Hartrumpf wrote:
> Will the proposed patch ("only") remove the possibility to build x32 kernels
> or will it make impossible to compile and run any x32 binaries?

There's no such thing as x32 kernels.  It's an ABI atop amd64 kernels; the
kernel is always 64-bit -- short pointers apply only to userspace and
syscalls.

It could be interesting to block regular amd64 syscalls and leave x32 only
-- for "security through rarity", but the kernel itself would still look the
same.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄ to the city of his birth to die.

drivers by default (was Re: Another HID problem this merge window..)

[Adam Borowski]

On Sat, Oct 27, 2018 at 11:13:17AM -0700, Linus Torvalds wrote:
> Ok, so this is a much smaller issue than the i2c one that cause boot
> problems, but it's annoying.
> 
> We do *not* enable new random drivers by default. And we most
> *definitely* don't do it when they are odd-ball ones that most people
> have never heard of.
> 
> Yet the new "BigBen Interactive" driver that was added this merge
> window did exactly that.
> 
> Just don't do it.

Amen to that.  But, perhaps you could encourage people to do enable drivers
once they become very popular?  For example, I just (72a9c673636) got hit by
USB 3.0 being off in defconfigs, and not having keyboard is not that cool.

So what about a policy that says "almost all drivers are to be =n initially,
but patches enabling popular stuff by default are welcome"?  Preferably if
obsolete crap gets mass-disabled at least once per decade (think of the
floppy driver's importance on day one vs now).

> Yes, yes, every developer always thinks that _their_ driver is so
> special and so magically important that it should be enabled by
> default. But no. When we have thousands of drivers, we don't randomly
> pick one new driver to be enabled by default just because some
> developer thinks it is special. It's not.

We don't know which drivers will become important and which won't, only time
can tell.  But not having such drivers on by default is also a problem.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.

Re: [...] an apology, and a maintainership note

[Adam Borowski]

On Sun, Sep 16, 2018 at 12:22:43PM -0700, Linus Torvalds wrote:
> This is my reality.  I am not an emotionally empathetic kind of person
> and that probably doesn't come as a big surprise to anybody.  Least of
> all me.  The fact that I then misread people and don't realize (for
> years) how badly I've judged a situation and contributed to an
> unprofessional environment is not good.
> 
> This week people in our community confronted me about my lifetime of
> not understanding emotions.  My flippant attacks in emails have been
> both unprofessional and uncalled for.  Especially at times when I made
> it personal.  In my quest for a better patch, this made sense to me.
> I know now this was not OK and I am truly sorry.
> 
> The above is basically a long-winded way to get to the somewhat
> painful personal admission that hey, I need to change some of my
> behavior, and I want to apologize to the people that my personal
> behavior hurt and possibly drove away from kernel development
> entirely.

Despite me being just among bottom-rung popcorn of kernel contributors, let
me says this:

No.  Just no.  You're so successful because you're one of few people who
don't waste time beating around the bush.  You call a spade a spade instead
of polite "professional" bullshit.

You often use rude words, but you don't do so without a reason.  IMO your
most striking quality is not technical ability (pretty high...) but the
ratio of times you open your mouth to the times you're right.  And even
if you're not right, you don't take offense at getting corrected and
immediately admit someone else was right.  

Sure, there are cases when both choices are right, but your approach avoids
wasting time making a decision.  For example: recently, you forced disabling
string truncation warnings despite many people feeling otherwise.  I for one
believe GCC's warnings even though sounding bogus are good for eliminating
strncpy -- what I would have done would be giving it an aliased version
named "fixedfieldncpy" or such that disables the warning, and fixing the
whole rest.  But what you did instead deprioritizes the issue: the kernel
doesn't work any worse than it did with gcc-7, thus there are indeed more
urgent matters elsewhere.  So even if I don't fully agree with you, you are
the boss and as long as your version is acceptable, let's stick to it.

And, it's _you_ who has proven merit, not me.

> I am going to take time off and get some assistance on how to
> understand people’s emotions and respond appropriately.
> 
> Put another way: When asked at conferences, I occasionally talk about
> how the pain-points in kernel development have generally not been
> about the _technical_ issues, but about the inflection points where
> development flow and behavior changed.

Too many projects get detracted by prolonged crap about social things, don't
let this pull you down.  There's a problem when people _without merit_ are
rude -- those indeed need to get a spanking.  A spanking not ADHD meds.
Short and to the point, letting them learn.

But you, you _earned_ the right to be rude to get your point across.
I watched a video about you getting shamed on a DebConf because of breaching
some "code of conduct" by using a naughty word.  I didn't like that and
believe it was you who was right (I don't recall the details though).

> I've talked to Greg to ask him if he'd mind finishing up 4.19 for me, so
> that I can take a break, and try to at least fix my own behavior.
> 
> This is not some kind of "I'm burnt out, I need to just go away"
> break.  I'm not feeling like I don't want to continue maintaining
> Linux. Quite the reverse.  I very much *do* want to continue to do
> this project that I've been working on for almost three decades.
> 
> This is more like the time I got out of kernel development for a while
> because I needed to write a little tool called "git".  I need to take
> a break to get help on how to behave differently and fix some issues
> in my tooling and workflow.

You do deserve a vacation.  By all means, do take a break and let the
community rehearse for "Linus got mauled by a bear".  But we want you back.

> And yes, some of it might be "just" tooling.  Maybe I can get an email
> filter in place so at when I send email with curse-words, they just
> won't go out.  Because hey, I'm a big believer in tools, and at least
> _some_ problems going forward might be improved with simple
> automation.

Please don't.  When you use curse words, they're _warranted_.  They're a
tool which, in my opinion, you don't overuse.

And it's fun to listen to a true master of words.  An example: how many
pages would https://lkml.org/lkml/2016/8/2/2062 take to say politely yet get
the same effect?

> I know when I really look “myself in the mirror” it will be clear it's
> not the only change that has to happen, but hey...  You can send me
> suggestions in email.

When you look yourself in the mirror, I want you to see that guy who codes
in a bathrobe instea

Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll function

[Adam Borowski]

On Tue, Jul 17, 2018 at 09:02:40PM -0400, Nicolas Pitre wrote:
> The nr argument is typically small: most often nr == 1. However this
> could be abused with a very large explicit scroll in a resized screen.
> Make the code scroll lines one at a time in all cases to avoid the VLA.
> Anything smarter is most likely not warranted here.

Even though nr can be 32767 at most, your new version is O(nr*nr) for no
reason.  Instead of O(n) memory or O(n²) time, a variant of the original
that copies values one at a time would be shorter and faster.

> Requested-by: Kees Cook 
> Signed-off-by: Nicolas Pitre 
> ---
>  drivers/tty/vt/vt.c | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> index 2d14bb195d..03e79f7787 100644
> --- a/drivers/tty/vt/vt.c
> +++ b/drivers/tty/vt/vt.c
> @@ -433,20 +433,22 @@ static void vc_uniscr_scroll(struct vc_data *vc, 
> unsigned int t, unsigned int b,
>  
>   if (uniscr) {
>   unsigned int s, d, rescue, clear;
> - char32_t *save[nr];
>  
>   s = clear = t;
> - d = t + nr;
> - rescue = b - nr;
> + d = t + 1;
> + rescue = b - 1;
>   if (dir == SM_UP) {
>   swap(s, d);
>   swap(clear, rescue);
>   }
> - memcpy(save, uniscr->lines + rescue, nr * sizeof(*save));
> - memmove(uniscr->lines + d, uniscr->lines + s,
> - (b - t - nr) * sizeof(*uniscr->lines));
> - memcpy(uniscr->lines + clear, save, nr * sizeof(*save));
> - vc_uniscr_clear_lines(vc, clear, nr);
> + while (nr--) {
> + char32_t *tmp;
> + tmp = uniscr->lines[rescue];
> + memmove(uniscr->lines + d, uniscr->lines + s,
> + (b - t - 1) * sizeof(*uniscr->lines));
> + uniscr->lines[clear] = tmp;
> + vc_uniscr_clear_lines(vc, clear, 1);
> + }
>   }
>  }

What the function does is rotating an array (slice [t..b) here), by nr if
SM_DOWN or by -nr ie (b - t - nr) if SM_UP.  A nice problem that almost every
"code interview questions" book includes :)

Please say if you don't have time for such games, I've just refreshed what's
a good answer. :þ


Meow.
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/3] use unicode for vt mouse paste

[Adam Borowski]

Hi!
Based on Nicolas' nice work (in tty-next), let's avoid corrupting characters
that have been copy+pasted via mouse selection.  The uniscr array holds
their original identity even if they got mangled by glyph conversion.
The glyph conversion lossily turns similar-looking characters into a
representation, and everyone else into a replacement character.

There's no proper handling for CJK (yet?) but anything of wcwidth()==1 will
work fine.

The whole thing doesn't get enabled until something reads from /dev/vcsu for
that console, but let's test this code first before enabling it widely.


Diffstat:
 drivers/tty/vt/selection.c | 48 
+---
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 40 insertions(+), 19 deletions(-)


-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

[PATCH 2/3] vt: selection: handle storing of characters above U+FFFF

[Adam Borowski]

Those above U+10 get replaced with U+FFFD.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 34e7110f310d..69ca337d3220 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,8 +116,8 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* stores the char in UTF8 and returns the number of bytes used (1-3) */
-static int store_utf8(u16 c, char *p)
+/* stores the char in UTF8 and returns the number of bytes used (1-4) */
+static int store_utf8(u32 c, char *p)
 {
if (c < 0x80) {
/*  0*** */
@@ -128,13 +128,26 @@ static int store_utf8(u16 c, char *p)
p[0] = 0xc0 | (c >> 6);
p[1] = 0x80 | (c & 0x3f);
return 2;
-   } else {
+   } else if (c < 0x1) {
/* 1110 10** 10** */
p[0] = 0xe0 | (c >> 12);
p[1] = 0x80 | ((c >> 6) & 0x3f);
p[2] = 0x80 | (c & 0x3f);
return 3;
-   }
+   } else if (c < 0x11) {
+   /* 0*** 10** 10** 10** */
+   p[0] = 0xf0 | (c >> 18);
+   p[1] = 0x80 | ((c >> 12) & 0x3f);
+   p[2] = 0x80 | ((c >> 6) & 0x3f);
+   p[3] = 0x80 | (c & 0x3f);
+   return 4;
+   } else {
+   /* outside Unicode, replace with U+FFFD */
+   p[0] = 0xef;
+   p[1] = 0xbf;
+   p[2] = 0xbd;
+   return 3;
+   }
 }
 
 /**
@@ -273,7 +286,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
sel_end = new_sel_end;
 
/* Allocate a new buffer before freeing the old one ... */
-   multiplier = use_unicode ? 3 : 1;  /* chars can take up to 3 bytes */
+   multiplier = use_unicode ? 4 : 1;  /* chars can take up to 4 bytes */
bp = kmalloc_array((sel_end - sel_start) / 2 + 1, multiplier,
   GFP_KERNEL);
if (!bp) {
-- 
2.18.0

[PATCH 3/3] vt: selection: take screen contents from uniscr if available

[Adam Borowski]

This preserves whatever was written even if we can't currently display the
given glyph.  Mouse paste won't corrupt any character of wcwidth() == 1
anymore.

Note that for now uniscr doesn't get allocated until something reads
/dev/vcsuN for that console, making this code dormant for most users.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 11 +++
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 69ca337d3220..07496c711d7d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -57,11 +57,13 @@ static inline void highlight_pointer(const int where)
complement_pos(sel_cons, where);
 }
 
-static u16
+static u32
 sel_pos(int n)
 {
+   if (use_unicode)
+   return screen_glyph_unicode(sel_cons, n / 2);
return inverse_translate(sel_cons, screen_glyph(sel_cons, n),
-   use_unicode);
+   0);
 }
 
 /**
@@ -90,7 +92,8 @@ static u32 inwordLut[]={
   0x07FE, /* lowercase */
 };
 
-static inline int inword(const u16 c) {
+static inline int inword(const u32 c)
+{
return c > 0x7f || (( inwordLut[c>>5] >> (c & 0x1F) ) & 1);
 }
 
@@ -167,7 +170,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
struct tiocl_selection v;
char *bp, *obp;
int i, ps, pe, multiplier;
-   u16 c;
+   u32 c;
int mode;
 
poke_blanked_console();
diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 7fcb0ff2dccf..19da4c0b4b8e 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -4545,6 +4545,16 @@ u16 screen_glyph(struct vc_data *vc, int offset)
 }
 EXPORT_SYMBOL_GPL(screen_glyph);
 
+u32 screen_glyph_unicode(struct vc_data *vc, int n)
+{
+   struct uni_screen *uniscr = get_vc_uniscr(vc);
+
+   if (uniscr)
+   return uniscr->lines[n / vc->vc_cols][n % vc->vc_cols];
+   return inverse_translate(vc, screen_glyph(vc, n * 2), 1);
+}
+EXPORT_SYMBOL_GPL(screen_glyph_unicode);
+
 /* used by vcs - note the word offset */
 unsigned short *screen_pos(struct vc_data *vc, int w_offset, int viewed)
 {
diff --git a/include/linux/selection.h b/include/linux/selection.h
index 067d2e99c79f..a8f5b97b216f 100644
--- a/include/linux/selection.h
+++ b/include/linux/selection.h
@@ -32,6 +32,7 @@ extern unsigned char default_blu[];
 
 extern unsigned short *screen_pos(struct vc_data *vc, int w_offset, int 
viewed);
 extern u16 screen_glyph(struct vc_data *vc, int offset);
+extern u32 screen_glyph_unicode(struct vc_data *vc, int offset);
 extern void complement_pos(struct vc_data *vc, int offset);
 extern void invert_screen(struct vc_data *vc, int offset, int count, int 
shift);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/3] vt: don't reinvent min()

[Adam Borowski]

All the helper function saved us was a cast.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 90ea1cc52b7a..34e7110f310d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,12 +116,6 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* constrain v such that v <= u */
-static inline unsigned short limit(const unsigned short v, const unsigned 
short u)
-{
-   return (v > u) ? u : v;
-}
-
 /* stores the char in UTF8 and returns the number of bytes used (1-3) */
 static int store_utf8(u16 c, char *p)
 {
@@ -167,10 +161,10 @@ int set_selection(const struct tiocl_selection __user 
*sel, struct tty_struct *t
if (copy_from_user(&v, sel, sizeof(*sel)))
return -EFAULT;
 
-   v.xs = limit(v.xs - 1, vc->vc_cols - 1);
-   v.ys = limit(v.ys - 1, vc->vc_rows - 1);
-   v.xe = limit(v.xe - 1, vc->vc_cols - 1);
-   v.ye = limit(v.ye - 1, vc->vc_rows - 1);
+   v.xs = min_t(u16, v.xs - 1, vc->vc_cols - 1);
+   v.ys = min_t(u16, v.ys - 1, vc->vc_rows - 1);
+   v.xe = min_t(u16, v.xe - 1, vc->vc_cols - 1);
+   v.ye = min_t(u16, v.ye - 1, vc->vc_rows - 1);
ps = v.ys * vc->vc_size_row + (v.xs << 1);
pe = v.ye * vc->vc_size_row + (v.xe << 1);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

Hi!
Here's a patchset with two entangled improvements:

* it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
  allows disabling it, rendering such characters with a bright background
  instead.
* due to my error, 256-color mode uses a much darker palette for conversion,
  resulting in behaving inconsistently with 24-bit mode.

The new code uses bright backgrounds when possible, enabled with \e[100m or
\e[48;m.

Despite the whole idea following a VGA capability, this patchset doesn't
change vgacon yet, just fbcon.  The reason being: ~80% of x86 users have an
nVidia chip, which means nouveau or nvidia-proprietary.  Nouveau implies
fbcon, nvidia-proprietary fails to properly restore text flags (as evidenced
by 512 glyph mode turning to 256 on switch from graphics).  You don't care
about the proprietary driver, but let's not break it pointlessly, and as
both nVidia cards I own work only with nouveau, I don't want to touch what I
can't test.

Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
register 0x10) later.

This fixes the display of catimg and similar tools.


Diffstat:
 drivers/tty/vt/vt.c  | 56 
+---
 drivers/video/fbdev/core/fbcon.c |  1 +
 include/linux/console_struct.h   |  4 ++--
 3 files changed, 32 insertions(+), 29 deletions(-)

-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH 1/6] vt: drop unused struct vt_struct

[Adam Borowski]

Hasn't been ever used within historic (ie, git) times.

Signed-off-by: Adam Borowski 
---
 include/linux/console_struct.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index 2c8d3239899b..fea64f2692a0 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -17,7 +17,6 @@
 #include 
 #include 
 
-struct vt_struct;
 struct uni_pagedir;
 struct uni_screen;
 
@@ -150,7 +149,7 @@ struct vc {
struct vc_data *d;
struct work_struct SAK_work;
 
-   /* might add  scrmem, vt_struct, kbd  at some time,
+   /* might add  scrmem, kbd  at some time,
   to have everything in one place - the disadvantage
   would be that vc_cons etc can no longer be static */
 };
-- 
2.18.0

[PATCH 2/6] vt: add console flag "unblinking"

[Adam Borowski]

Marks consoles that interpret the blink bit by showing bright background
instead.  Doesn't matter if the console can't do either.

For now, turn it on for fbcon when in color mode.  Vgacon can also do so
but requires setting appropriate VGA register (bit 3 of AMCR).  I don't
know other consoles: newport looks like it shows bright bg, sti can't do
either, mda appears to blink, etc -- but confirmation would be needed.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c  | 1 +
 drivers/video/fbdev/core/fbcon.c | 1 +
 include/linux/console_struct.h   | 1 +
 3 files changed, 3 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 846dfedb657d..45057bbf6f74 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -998,6 +998,7 @@ static void visual_init(struct vc_data *vc, int num, int 
init)
vc->vc_hi_font_mask = 0;
vc->vc_complement_mask = 0;
vc->vc_can_do_color = 0;
+   vc->vc_unblinking = 0;
vc->vc_panic_force_write = false;
vc->vc_cur_blink_ms = DEFAULT_CURSOR_BLINK_MS;
vc->vc_sw->con_init(vc, init);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index c910e74d46ff..4c67254f1ec4 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -1092,6 +1092,7 @@ static void fbcon_init(struct vc_data *vc, int init)
 
vc->vc_panic_force_write = !!(info->flags & FBINFO_CAN_FORCE_OUTPUT);
vc->vc_can_do_color = (fb_get_color_depth(&info->var, &info->fix)!=1);
+   vc->vc_unblinking = vc->vc_can_do_color;
vc->vc_complement_mask = vc->vc_can_do_color ? 0x7700 : 0x0800;
if (charcnt == 256) {
vc->vc_hi_font_mask = 0;
diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index fea64f2692a0..f94b28a6bd2d 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -122,6 +122,7 @@ struct vc_data {
unsigned intvc_ques : 1;
unsigned intvc_need_wrap: 1;
unsigned intvc_can_do_color : 1;
+   unsigned intvc_unblinking   : 1;/* shows bright bg for blink */
unsigned intvc_report_mouse : 2;
unsigned char   vc_utf  : 1;/* Unicode UTF-8 encoding */
unsigned char   vc_utf_count;
-- 
2.18.0

[PATCH 5/6] vt: compensate for brightening the 256-color palette

[Adam Borowski]

The algorithm for 256-to-16 conversion was designed with wrong input
palette but actually tuned on mainstream GUI terminals.  This resulted in
something that works well only for data we convert ourselves (ie, 256 not
24-bit).

As the change is non-linear, I did not bother replicating it exactly, thus
there are some differences, among others:
* values very close to black go to 0 (black) rather than 8 (dark grey)
* grayscale ramp is more even

A comparison of the old vs new vs FreeBSD's teken is at:
https://github.com/kilobyte/colorkernel

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 8c61caafdf3c..c777f4c91df0 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1559,17 +1559,17 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
-   if (c->r > max / 2)
+   if (c->r > max / 2 + 32)
hue |= 4;
-   if (c->g > max / 2)
+   if (c->g > max / 2 + 32)
hue |= 2;
-   if (c->b > max / 2)
+   if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x55) {
+   if (hue == 7 && max <= 0x70) {
hue = 0;
vc->vc_intensity = 2;
-   } else if (max > 0xaa)
+   } else if (max > 0xc0)
vc->vc_intensity = 2;
else
vc->vc_intensity = 1;
-- 
2.18.0

[PATCH 4/6] vt: change 256-color palette to match all(?) modern terminals

[Adam Borowski]

Turns out that osso-xterm which I based upon uses something a lot different
from apparently any other terminal -- they all use identical shades, much
brighter than what I copied:

Old: 00 2a 55 7f aa d4
New: 00 5f 87 af d7 ff

This did hardly matter as we immediately shoehorn the colors into only 16
values, but recently 24-bit codes turned from an oddity to something
widespread, thus it's better to handle 256 vs 24-bit consistently.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 4096093c8cd2..8c61caafdf3c 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1545,9 +1545,12 @@ static void rgb_from_256(int i, struct rgb *c)
c->g = i&2 ? 0xff : 0x55;
c->b = i&4 ? 0xff : 0x55;
} else if (i < 232) {   /* 6x6x6 colour cube. */
-   c->r = (i - 16) / 36 * 85 / 2;
-   c->g = (i - 16) / 6 % 6 * 85 / 2;
-   c->b = (i - 16) % 6 * 85 / 2;
+   int r = (i - 16) / 36;
+   int g = (i - 16) / 6 % 6;
+   int b = (i - 16) % 6;
+   c->r = r ? r * 0x28 + 0x37 : 0;
+   c->g = g ? g * 0x28 + 0x37 : 0;
+   c->b = b ? b * 0x28 + 0x37 : 0;
} else  /* Grayscale ramp. */
c->r = c->g = c->b = i * 10 - 2312;
 }
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] vt: support bright backgrounds for \e[48m if unblinking

[Adam Borowski]

This improves schemes used by fancy new programs.  For example, it lets
bright powerline segments match, and fixes images shown by catimg being
striped to the point of unreadability.

Handling of 8-color backgrounds uses stripped 16-color value instead of a
dedicated formula; this works worse for dark and better for bright inputs.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 38 +-
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c777f4c91df0..7fcb0ff2dccf 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1555,7 +1555,7 @@ static void rgb_from_256(int i, struct rgb *c)
c->r = c->g = c->b = i * 10 - 2312;
 }
 
-static void rgb_foreground(struct vc_data *vc, const struct rgb *c)
+static u8 rgb_to_16(const struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
@@ -1566,22 +1566,12 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x70) {
-   hue = 0;
-   vc->vc_intensity = 2;
-   } else if (max > 0xc0)
-   vc->vc_intensity = 2;
+   if (hue == 7 && max <= 0x70)
+   return 8;
+   if (max > 0xc0)
+   return hue | 8;
else
-   vc->vc_intensity = 1;
-
-   vc->vc_color = (vc->vc_color & 0xf0) | hue;
-}
-
-static void rgb_background(struct vc_data *vc, const struct rgb *c)
-{
-   /* For backgrounds, err on the dark side. */
-   vc->vc_color = (vc->vc_color & 0x0f)
-   | (c->r&0x80) >> 1 | (c->g&0x80) >> 2 | (c->b&0x80) >> 3;
+   return hue;
 }
 
 /*
@@ -1593,10 +1583,10 @@ static void rgb_background(struct vc_data *vc, const 
struct rgb *c)
  * Subcommands 3 (CMY) and 4 (CMYK) are so insane there's no point in
  * supporting them.
  */
-static int vc_t416_color(struct vc_data *vc, int i,
-   void(*set_color)(struct vc_data *vc, const struct rgb *c))
+static int vc_t416_color(struct vc_data *vc, int i, int bgshift)
 {
struct rgb c;
+   u8 v;
 
i++;
if (i > vc->vc_npar)
@@ -1615,7 +1605,13 @@ static int vc_t416_color(struct vc_data *vc, int i,
} else
return i;
 
-   set_color(vc, &c);
+   v = rgb_to_16(&c);
+
+   vc->vc_color = (vc->vc_color & (0xf0 >> bgshift)) | v << bgshift;
+   if (!bgshift)
+   vc->vc_intensity = (v & 8 >> 4) + 1;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = v & 8 >> 4;
 
return i;
 }
@@ -1695,10 +1691,10 @@ static void csi_m(struct vc_data *vc)
vc->vc_reverse = 0;
break;
case 38:
-   i = vc_t416_color(vc, i, rgb_foreground);
+   i = vc_t416_color(vc, i, 0);
break;
case 48:
-   i = vc_t416_color(vc, i, rgb_background);
+   i = vc_t416_color(vc, i, 4);
break;
case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) |
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/6] vt: let \e[100m use bright background if unblinking

[Adam Borowski]

Let's keep \e[5m setting this bit, it's a nice way to convey the
information, and it preserves old behaviour.  Some other terminals
that can't or don't want to blink do so as well.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 45057bbf6f74..4096093c8cd2 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1709,6 +1709,8 @@ static void csi_m(struct vc_data *vc)
if (vc->vc_par[i] >= 90 && vc->vc_par[i] <= 107) {
if (vc->vc_par[i] < 100)
vc->vc_intensity = 2;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = 1;
vc->vc_par[i] -= 60;
}
if (vc->vc_par[i] >= 30 && vc->vc_par[i] <= 37)
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to disable Linux kernel self-extraction (KERNEL_GZIP, KERNEL_BZIP2, …)?

[Adam Borowski]

On Tue, Apr 24, 2018 at 11:08:34AM +0200, Paul Menzel wrote:
> On 04/24/18 04:08, Adam Borowski wrote:
> > On Mon, Apr 23, 2018 at 07:02:30PM +0200, Pavel Machek wrote:
> > > > > > > I try to decrease boot time, and my system has an SSD and enough 
> > > > > > > space, so
> > > > > > > loading 18 instead of 12 MB doesn’t make a difference, but the
> > > > > > > self-extraction is noticeable. So, I like to disable it.
> > > > > > 
> > > > > > How long does GZIP extraction take on your hardware?
> > > > > 
> > > > > It’s hard to measure – at least I didn’t find a way to do so –, but 
> > > > > counting
> > > > > from the last GRUB message to the first message of Linux (with `quiet`
> > > > > removed from the the command line), it takes roughly *two* seconds.
> > 
> > I took a somewhat different approach: I recorded the output from grub+kernel
> > to ttyrec over serial line, and rigged ttyrec2ansi to output timestamp
> > difference from the last checkpoint every time an '\e' or '\n' is seen.
> > '\e' is important, as there's no other marking for when grub stops the
> > interactive phase and starts the actual boot.
> 
> Nice work. It’d be great, if you shared these patches, so others and I can
> reproduce it.

ttyrec2ansi.c attached.  To use: save the serial output as ttyrec (via the
eponymous tool, termrec, conversion from a similar format, etc), pipe
through modified ttyrec2ansi to a terminal, "less -R".


userland lz4:
diff --git a/lib/lz4.c b/lib/lz4.c
index 213b085..39d2cff 100644
--- a/lib/lz4.c
+++ b/lib/lz4.c
@@ -499,6 +499,7 @@ LZ4_FORCE_INLINE U32 LZ4_hashPosition(const void* const p, 
tableType_t const tab
 
 static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, 
tableType_t const tableType, const BYTE* srcBase)
 {
+return;
 switch (tableType)
 {
 case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; 
hashTable[h] = p; return; }

Somehow this affects only /usr/bin/lz4 not /usr/bin/lz4c, which I did not
bother to fix but hacked via:

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index f2014876405f..91534a801090 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -332,7 +332,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
 
 quiet_cmd_lz4 = LZ4 $@
 cmd_lz4 = (cat $(filter-out FORCE,$^) | \
-   lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out 
FORCE,$^))) > $@ || \
+   lz4 -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) 
> $@ || \
(rm -f $@ ; false)
 
 # U-Boot mkimage


I have serious doubts this approach is sound, but worked well enough at
least to spot the GRUB read slowness issue.

> > Turns out that, reading from SSD, grub is way way slower than it should take
> > normally.  The machine is old (AMD Phenom II X6 1055T), SSD is Crucial
> > CT240M500SSD1.
> 
> What firmware does the device (mainboard) have? Legacy BIOS or UEFI (or even
> coreboot ;-)). It’s my understanding, that GRUB does not have a native
> driver with the legacy BIOS and UEFI, and just uses the BIOS calls or the
> UEFI equivalent.

An old machine -- real BIOS.

> > I also have the zstd patch applied, which adds another data point.
> > 
> > The two "Loading XXX ..." lines come from grub, those timestamped within []
> > brackets from the kernel, 〈〉are ttyrec timestamps, ⤸ is wrapped lines.
> > 
> > 
> > zstd:
> > 
> > Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.739823〉
> > ^MLoading initial ramdisk ...〈0.402010〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23⤸
> > 10:25:58 CEST 2018^M〈0.785922〉
> > 
> > gzip:
> > 
> > Loading Linux 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ...〈0.724988〉
> > ^MLoading initial ramdisk ...〈0.357951〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
> > 23:15:07 CEST 2018^M〈0.777977〉
> > 
> > lz4:
> > 
> > Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d ...〈0.799969〉
> > ^MLoading initial ramdisk ...〈0.424959〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-lz4-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Tue Apr 24 ⤸
> > 00:34:59 CEST 2018^M〈0.732925〉
> > 
> > zstd again:
> > 
> > Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.728852〉
> > ^MLoading ini

On Tue, Apr 24, 2018 at 11:08:34AM +0200, Paul Menzel wrote:

[Adam Borowski]

'ere you go.  KERNEL_ZSTD is not in mainline yet but knowing its magic
can't hurt -- especially that scripts may be out of sync with an installed
kernel.  LZ4 is in since 3.11.

--8<8<8<8<8<8<8<8<8<8<8<8<--
>From 30886e965e7aeae8d3729c4bacf614a19e103cea Mon Sep 17 00:00:00 2001
From: Adam Borowski 
Date: Wed, 25 Apr 2018 02:29:18 +0200
Subject: [PATCH] scripts: teach extract-vmlinux about LZ4 and ZSTD

Note that the LZ4 signature is different than that of modern LZ4 as we
use the "legacy" format which suffers from some downsides like inability
to disable compression.

Signed-off-by: Adam Borowski 
---
 scripts/extract-vmlinux | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
index 5061abcc2540..e6239f39abad 100755
--- a/scripts/extract-vmlinux
+++ b/scripts/extract-vmlinux
@@ -57,6 +57,8 @@ try_decompress '\3757zXZ\000' abcde unxz
 try_decompress 'BZh'  xybunzip2
 try_decompress '\135\0\0\0'   xxx   unlzma
 try_decompress '\211\114\132' xy'lzop -d'
+try_decompress '\002!L\030'   xxx   'lz4 -d'
+try_decompress '(\265/\375'   xxx   unzstd
 
 # Bail out:
 echo "$me: Cannot find vmlinux." >&2
-- 
2.17.0

Re: How to turn off IPv4 without disabling IPv6

[Adam Borowski]

On Thu, Apr 25, 2019 at 11:32:38AM +0200, Nico Schottelius wrote:
> running some IPv6 only
> networks. The systems in the IPv6 only networks do not need any IPv4
> support anymore and thus for switches/routers we turned the support off.

> Today we tried to turn off IPv4 in the Linux kernel at compile time.
> But it seems that as soon as we turn off CONFIG_INET, CONFIG_IPV6 is
> automatically turned off as well.

Even if you don't want global nor even link-scope IPv4, way too many
programs assume that at least 127.0.0.1 (ie, lo) is working.  They can't be
reconfigured to use ::1 without patching and rebuilding.

> Coming back to my original question: is there a way or how would we turn
> off IPv4 support in the Linux kernel?

I believe this is not worth your time for today.

Just do what IPv6-haters do on stock modern distros: have no routes for the
other IP version configured; non-buggy programs will do the right thing.

This seems to work well.  Heck, I had a busy dev server with broken IPv4, I
didn't notice that for 1.5 years until I tried to pull something directly
from Github (which is still v4 only).

You're a network admin so you know far more than me wrt anything that goes
over the wire -- but as as a distro developer/user, I'd say there's a
considerable cost to have every of tens of thousands programs shipped by a
distro, and many more that are private to a company/university/etc, updated
to autodetect how to access "localhost" on a particular box.

That's an extra moving part where there was none before.  Complexity is bad.
Having the IPv4 stack built just for the lo interface simplifies things.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄

Re: [PATCH RESEND v5 0/5] namei: vfs flags to restrict path resolution

[Adam Borowski]

On Thu, Apr 25, 2019 at 01:38:06AM +1000, Aleksa Sarai wrote:
>  * openat(2) ignores unknown flags, meaning that old kernels will ignore
>new programs trying to use O_THISROOT and might end up causing
>security issues. Yes, it'd be trivial to check whether the new O_*
>flags are supported at start-up, but I think a security feature
>shouldn't have a foot-gun associated with it. In fact, I didn't know
>openat(2) ignored unknown flags until I wrote this patchset -- I
>doubt many other userspace developers do either.

For this reason, I propose every new syscall that has flags to follow a
bitmask scheme, where any flag assigned a bit in the upper half returns
EOPNOTSUPP when called on an old kernel.  That would allow defining which
flags can be safely ignored and which can't.

It otherwise takes major hacks to implement a fail-if-not-supported flag
while keeping compat with old kernels.  For example, for mmap(), MAP_SHARED
has been duplicated as MAP_SHARED_VALIDATE just to allow an unrelated flag
(MAP_SYNC) to fail on old kernels.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven giant trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...

Re: TLS for 5.10

[Adam Borowski]

On Mon, Feb 08, 2021 at 07:08:05AM +, Wadepohl, Wolfram wrote:
> I'm sad to hear that 5.10 has still an EOL of Dec, 2022. We are in
> development of our 1st GNU/Linux based System,  50k devices for IoT.
[...]
> In general a 2 year support for embedded systems in automation is not a
> reasonable thing. Nevertheless the CIP project has commited to 5.10 as the
> next SLTS kernel.
[...]
> What can we do as a very small development team to support an extended LTS
> period of 5.10.?

The two years are a minimal promise from Greg.  Debian is using 5.10 for
upcoming Bullseye, thus even if Greg won't extend (99% chance he will), Ben
Hutchings will continue the support for as long as Debian Bullseye is alive.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [PATCH] tty: Remove dead termiox code

[Adam Borowski]

On Fri, Dec 04, 2020 at 10:10:16AM +0100, Greg Kroah-Hartman wrote:
> On Fri, Dec 04, 2020 at 09:51:07AM +0100, Jiri Slaby wrote:
> > > > > On Fri, Dec 04, 2020 at 08:22:41AM +0100, Jiri Slaby wrote:
> > > > > > On 03. 12. 20, 3:03, Jann Horn wrote:
> > > > > > > Delete this dead code; but leave the definition of struct termiox 
> > > > > > > in the
> > > > > > > UAPI headers intact.
[was snipped]
> > > > > > I am thinking -- can/should we mark the structure as deprecated so 
> > > > > > that 
> > > > > > userspace stops using it eventually?   

> > Note this ^. He is talking about _not_ touching the definition in the
> > UAPI header. Does the rest below makes more sense now?
> 
> No, I'm still confused :)
> 
> We can't touch the UAPI definitions, but the fact that this api never
> did anything still is ok as after this patch it continues to not do
> anything.
> 
> I'm confused as to what you are proposing...

The UAPI definition can't be removed, but it would be nice to issue a
compiler _warning_ if it's ever used.

Like eg. __attribute__ ((deprecated))


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀.--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁# beware of races
⢿⡄⠘⠷⠚⠋⠀all: pillage burn
⠈⠳⣄`

Re: unknown NMI on AMD Rome

[Adam Borowski]

On Tue, Mar 16, 2021 at 04:45:02PM +0100, Jiri Olsa wrote:
> hi,
> when running 'perf top' on AMD Rome (/proc/cpuinfo below)
> with fedora 33 kernel 5.10.22-200.fc33.x86_64
> 
> we got unknown NMI messages:
> 
> [  226.700160] Uhhuh. NMI received for unknown reason 3d on CPU 90.
> [  226.700162] Do you have a strange power saving mode enabled?
> [  226.700163] Dazed and confused, but trying to continue
> 
> also when discussing ths with Borislav, he managed to reproduce easily
> on his AMD Rome machine

Likewise, 3c on Pinnacle Ridge.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

[Adam Borowski]

On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > DAX on btrfs has been attempted[1]. Of course, we could not
> 
> But why?  A completeness fetish?  I don't understand why you decided
> to do this work.

* xfs can shapshot only single files, btrfs entire subvolumes
* btrfs-send|receive
* enumeration of changed parts of a file


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢠⠒⠀⣿⡁ productivity.  You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so.  I recommend Skepticism
⠈⠳⣄ (funeral doom metal).

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

[Adam Borowski]

On Sat, Mar 13, 2021 at 11:24:00AM -0500, Neal Gompa wrote:
> On Sat, Mar 13, 2021 at 8:09 AM Adam Borowski  wrote:
> >
> > On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> > > On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > > > DAX on btrfs has been attempted[1]. Of course, we could not
> > >
> > > But why?  A completeness fetish?  I don't understand why you decided
> > > to do this work.
> >
> > * xfs can shapshot only single files, btrfs entire subvolumes
> > * btrfs-send|receive
> > * enumeration of changed parts of a file
> 
> XFS cannot do snapshots since it lacks metadata COW. XFS reflinking is
> primarily for space efficiency.

A reflink is a single-file snapshot.

My work team really wants this very patchset -- reflinks on DAX allow
backups and/or checkpointing, without stopping the world (there's a single
file, "pool", here).

Besides, you can still get poor-man's whole-subvolume(/directory)
snapshots by manually walking the tree and reflinking everything.
That's not atomic -- but rsync isn't atomic either.  That's enough for
eg. dnf/dpkg purposes.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ NADIE anticipa la inquisición de españa!
⠈⠳⣄

MaxLinear, please maintain your drivers was Re: [PATCH] leds: lgm: fix gpiolib dependency

[Adam Borowski]

On Tue, Mar 09, 2021 at 08:39:10PM +0100, Pavel Machek wrote:
> > I'd like people from Intel to contact me. There's more to fix there,
> > and AFAICT original author went away.
> 
> The following message to  was
> undeliverable.

> : Recipient
> +address rejected: User unknown in virtual mailbox table'

> commit c3987cd2bca34ddfec69027acedb2fae5ffcf7a0
> Author: Amireddy Mallikarjuna reddy 

I asked around, and got told Mallikarjuna has been "sold" to MaxLinear,
together with the rest of the Connected Home Division.  So he most likely
still works on this stuff, just under a different banner.

> If someone knows how to contact the author, that would be welcome.

Alas, no idea about his MaxLinear address.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: 5.10 LTS Kernel: 2 or 6 years?

[Adam Borowski]

On Mon, Jan 25, 2021 at 11:55:11AM -0800, Scott Branden wrote:
> The 5.10 LTS kernel being officially LTS supported for 2 years presents a 
> problem:
> why would anyone select a 5.10 kernel with 2 year LTS when 5.4 kernel has a 6 
> year LTS.
> 
> Yet, various unofficial reports indicate it will be supported for 6
> years.  And AOSP has already declared the use of 5.10 kernel in their
> Android S and T releases.

5.10 will also be used for Debian Bullseye.

-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `

Re: Linux 5.11-rc1

[Adam Borowski]

On Sun, Jan 03, 2021 at 05:45:16PM +, David Laight wrote:
> From: Ilkka Prusi
> > Note that /sbin is now just a symlink to /usr/sbin on Debian since 10 
> > (Buster) as per FHS[1][2].
> > 
> > [1] https://wiki.linuxfoundation.org/lsb/fhs
> > [2] 
> > https://arstechnica.com/information-technology/2019/09/debian-10-playing-catch-up-with-the-rest-
> > of-the-linux-world-thats-a-good-thing/
> 
> Which is exactly 100% backwards :-)

I agree with you that, if all, it's /usr/sbin which should be a symlink,
to reduce typing and to get rid of a vestige of _The_ Unix machine having
/bin spill to a disk that used to have user homes.

But, /sbin is a symlink on only _some_ Debian installations.  There's
currently a discussion whether to go deeper into this scheme, abandon it
or do nothing.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `

[PATCH] vt: detect and ignore OSC codes.

[Adam Borowski]

These can be used to send commands consisting of an arbitrary string to the
terminal, most often used to set a terminal's window title or to redefine
the colour palette.  Our console doesn't use OSC, unlike everything else,
which can lead to junk being displayed if a process sends such a code
unconditionally.

The rules for termination follow established practice rather than Ecma-48.
Ecma-48 requires the string to use only byte values 0x08..0x0D and
0x20..0x7E, terminated with either ESC \ or 0x9C.  This would disallow using
8-bit characters, which are reasonable for example in window titles.
A widespread idiom is to terminate with 0x07.  The behaviour for other
control characters differs between terminal emulators, I followed libvte and
xterm:
* 0x07 and ESC anything terminate
* nothing else terminates, all 8-bit values including 0x9C are considered a
  part of the string

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index b7bde54..3ad0b61 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1592,7 +1592,7 @@ static void restore_cur(struct vc_data *vc)
 
 enum { ESnormal, ESesc, ESsquare, ESgetpars, ESfunckey,
EShash, ESsetG0, ESsetG1, ESpercent, ESignore, ESnonstd,
-   ESpalette };
+   ESpalette, ESosc };
 
 /* console_lock is held (except via vc_init()) */
 static void reset_terminal(struct vc_data *vc, int do_clear)
@@ -1652,11 +1652,15 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
 *  Control characters can be used in the _middle_
 *  of an escape sequence.
 */
+   if (vc->vc_state == ESosc && c>=8 && c<=13) /* ... except for OSC */
+   return;
switch (c) {
case 0:
return;
case 7:
-   if (vc->vc_bell_duration)
+   if (vc->vc_state == ESosc)
+   vc->vc_state = ESnormal;
+   else if (vc->vc_bell_duration)
kd_mksound(vc->vc_bell_pitch, vc->vc_bell_duration);
return;
case 8:
@@ -1767,7 +1771,9 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
} else if (c=='R') {   /* reset palette */
reset_palette(vc);
vc->vc_state = ESnormal;
-   } else
+   } else if (c>='0' && c<='9')
+   vc->vc_state = ESosc;
+   else
vc->vc_state = ESnormal;
return;
case ESpalette:
@@ -2021,6 +2027,8 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
vc->vc_translate = set_translate(vc->vc_G1_charset, vc);
vc->vc_state = ESnormal;
return;
+   case ESosc:
+   return;
default:
vc->vc_state = ESnormal;
}
-- 
1.9.0

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH] vt: detect and ignore OSC codes.

[Adam Borowski]

These can be used to send commands consisting of an arbitrary string to the
terminal, most often used to set a terminal's window title or to redefine
the colour palette.  Our console doesn't use OSC, unlike everything else,
which can lead to junk being displayed if a process sends such a code
unconditionally.

Not following Ecma-48, this commit recognizes 7-bit forms (ESC ] ... 0x07,
ESC ] .. ESC \) but not 8-bit (0x9D ... 0x9C).

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 61b1137..0377c52 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1590,7 +1590,7 @@ static void restore_cur(struct vc_data *vc)
 
 enum { ESnormal, ESesc, ESsquare, ESgetpars, ESgotpars, ESfunckey,
EShash, ESsetG0, ESsetG1, ESpercent, ESignore, ESnonstd,
-   ESpalette };
+   ESpalette, ESosc };
 
 /* console_lock is held (except via vc_init()) */
 static void reset_terminal(struct vc_data *vc, int do_clear)
@@ -1650,11 +1650,15 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
 *  Control characters can be used in the _middle_
 *  of an escape sequence.
 */
+   if (vc->vc_state == ESosc && c>=8 && c<=13) /* ... except for OSC */
+   return;
switch (c) {
case 0:
return;
case 7:
-   if (vc->vc_bell_duration)
+   if (vc->vc_state == ESosc)
+   vc->vc_state = ESnormal;
+   else if (vc->vc_bell_duration)
kd_mksound(vc->vc_bell_pitch, vc->vc_bell_duration);
return;
case 8:
@@ -1765,7 +1769,9 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
} else if (c=='R') {   /* reset palette */
reset_palette(vc);
vc->vc_state = ESnormal;
-   } else
+   } else if (c>='0' && c<='9')
+   vc->vc_state = ESosc;
+   else
vc->vc_state = ESnormal;
return;
case ESpalette:
@@ -2023,6 +2029,8 @@ static void do_con_trol(struct tty_struct *tty, struct 
vc_data *vc, int c)
return;
default:
vc->vc_state = ESnormal;
+   case ESosc:
+   return;
}
 }
 
-- 
1.8.5.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] vt: detect and ignore OSC codes.

[Adam Borowski]

On Thu, Feb 13, 2014 at 10:39:12AM -0800, Greg Kroah-Hartman wrote:
> On Wed, Jan 15, 2014 at 07:21:04AM +0100, Adam Borowski wrote:
> > These can be used to send commands consisting of an arbitrary string to the
> > terminal, most often used to set a terminal's window title or to redefine
> > the colour palette.  Our console doesn't use OSC, unlike everything else,
> > which can lead to junk being displayed if a process sends such a code
> > unconditionally.
> > 
> > Not following Ecma-48, this commit recognizes 7-bit forms (ESC ] ... 0x07,
> > ESC ] .. ESC \) but not 8-bit (0x9D ... 0x9C).
> > 
> > Signed-off-by: Adam Borowski 
> 
> Where is this documented?

It's a mix of Ecma-48 and undocumented practice.  Ecma-48 says:

# 8.3.89
#   OSC - OPERATING SYSTEM COMMAND
#   Notation: (C1)
#   Representation: 0x9D or ESC 0x5D (])
#
#   OSC is used as the opening delimiter of a control string for operating
#   system use.  The command string following may consist of a sequence of
#   bit combinations in the range 0x08 to 0x0D and 0x20 to 0x7E.  The
#   control string is closed by the terminating delimiter STRING TERMINATOR
#   (ST).  The interpretation of the command string depends on the relevant
#   operating system.

# 8.3.143
#   ST - STRING TERMINATOR
#   Notation: (C1)
#   Representation: 0x9C or ESC 0x5C (\)
#
#   ST is used as the closing delimiter of a control string opened by
#   APPLICATION PROGRAM COMMAND (APC), DEVICE CONTROL STRING (DCS),
#   OPERATING SYSTEM COMMAND (OSC), PRIVACY MESSAGE (PM), or START OF STRING
#   (SOS).

... which doesn't define the behaviour for characters 0x00..0x07, 0x0E..0x1F
or 0x7F..0xFF.  Somehow, using 0x07 for termination became a widespread
idiom, used more often than proper ESC \.  For this reason, implementations
I know all recognize 0x07 as a terminator.  The behaviour for other
characters differs, ie, is truly undefined.

As, unlike what Ecma-48 says, using 8-bit characters in a window title is a
reasonable thing to do, I'd allow 0x80..0xFF as non-terminators.  I have no
idea what to do with remaining control characters: the current patch allows
them to be interpreted, as it's usually the case for control characters
inside terminal codes.  I did not research other implementation here.

I did not recognize 0x9C as ST for two reasons: 1. it'd break non-ASCII
characters that happen to include this byte, and 2. Linux already fails to
recognize 8-bit control codes (with one exception: 0x9B stands for ESC [).


Should I put the above explanation somewhere?  As a comment?  In the commit
message?  Or does it need to be elaborated even further?

> > ---
> >  drivers/tty/vt/vt.c | 14 +++---
> >  1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> > index 61b1137..0377c52 100644
> > --- a/drivers/tty/vt/vt.c
> > +++ b/drivers/tty/vt/vt.c
> > @@ -1590,7 +1590,7 @@ static void restore_cur(struct vc_data *vc)
> >  
> >  enum { ESnormal, ESesc, ESsquare, ESgetpars, ESgotpars, ESfunckey,
> > EShash, ESsetG0, ESsetG1, ESpercent, ESignore, ESnonstd,
> > -   ESpalette };
> > +   ESpalette, ESosc };
> >  
> >  /* console_lock is held (except via vc_init()) */
> >  static void reset_terminal(struct vc_data *vc, int do_clear)
> > @@ -1650,11 +1650,15 @@ static void do_con_trol(struct tty_struct *tty, 
> > struct vc_data *vc, int c)
> >  *  Control characters can be used in the _middle_
> >  *  of an escape sequence.
> >  */
> > +   if (vc->vc_state == ESosc && c>=8 && c<=13) /* ... except for OSC */
> > +   return;
> > switch (c) {
> > case 0:
> > return;
> > case 7:
> > -   if (vc->vc_bell_duration)
> > +   if (vc->vc_state == ESosc)
> > +   vc->vc_state = ESnormal;
> > +   else if (vc->vc_bell_duration)
> > kd_mksound(vc->vc_bell_pitch, vc->vc_bell_duration);
> > return;
> > case 8:
> > @@ -1765,7 +1769,9 @@ static void do_con_trol(struct tty_struct *tty, 
> > struct vc_data *vc, int c)
> > } else if (c=='R') {   /* reset palette */
> > reset_palette(vc);
> > vc->vc_state = ESnormal;
> > -   } else
> > +   } else if (c>='0' && c<='9')
> > +   vc->vc_state = ESosc;
> > +   else
> > vc->vc_state = ESnormal;
> > return;
> > case ESpalette:
> > @@ -2023,6 +2029,8 @@ static void do_con_tr

[PATCH] perf: tools: fix missing casts for printf arguments.

[Adam Borowski]

Because of -Werror, they caused build failure at least on x32, as time_t
is of different size than "unsigned long".  In another place, __suseconds_t
is not compatible with "long int".

Signed-off-by: Adam Borowski 
---
 tools/perf/bench/sched-messaging.c | 4 ++--
 tools/perf/bench/sched-pipe.c  | 4 ++--
 tools/perf/builtin-kvm.c   | 2 +-
 tools/perf/builtin-stat.c  | 3 ++-
 4 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/tools/perf/bench/sched-messaging.c 
b/tools/perf/bench/sched-messaging.c
index cc1190a..6595e2f 100644
--- a/tools/perf/bench/sched-messaging.c
+++ b/tools/perf/bench/sched-messaging.c
@@ -318,11 +318,11 @@ int bench_sched_messaging(int argc, const char **argv,
   num_groups, num_groups * 2 * num_fds,
   thread_mode ? "threads" : "processes");
printf(" %14s: %lu.%03lu [sec]\n", "Total time",
-  diff.tv_sec,
+  (unsigned long) diff.tv_sec,
   (unsigned long) (diff.tv_usec/1000));
break;
case BENCH_FORMAT_SIMPLE:
-   printf("%lu.%03lu\n", diff.tv_sec,
+   printf("%lu.%03lu\n", (unsigned long) diff.tv_sec,
   (unsigned long) (diff.tv_usec/1000));
break;
default:
diff --git a/tools/perf/bench/sched-pipe.c b/tools/perf/bench/sched-pipe.c
index 07a8d76..d7c1df4 100644
--- a/tools/perf/bench/sched-pipe.c
+++ b/tools/perf/bench/sched-pipe.c
@@ -157,7 +157,7 @@ int bench_sched_pipe(int argc, const char **argv, const 
char *prefix __maybe_unu
result_usec += diff.tv_usec;
 
printf(" %14s: %lu.%03lu [sec]\n\n", "Total time",
-  diff.tv_sec,
+  (unsigned long) diff.tv_sec,
   (unsigned long) (diff.tv_usec/1000));
 
printf(" %14lf usecs/op\n",
@@ -169,7 +169,7 @@ int bench_sched_pipe(int argc, const char **argv, const 
char *prefix __maybe_unu
 
case BENCH_FORMAT_SIMPLE:
printf("%lu.%03lu\n",
-  diff.tv_sec,
+  (unsigned long) diff.tv_sec,
   (unsigned long) (diff.tv_usec / 1000));
break;
 
diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index a735051..b470f0f 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -730,7 +730,7 @@ static void show_timeofday(void)
gettimeofday(&tv, NULL);
if (localtime_r(&tv.tv_sec, <ime)) {
strftime(date, sizeof(date), "%H:%M:%S", <ime);
-   pr_info("%s.%06ld", date, tv.tv_usec);
+   pr_info("%s.%06ld", date, (long int) tv.tv_usec);
} else
pr_info("00:00:00.00");
 
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 8b0e1c9..26cfc12 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -455,7 +455,8 @@ static void print_interval(void)
 
clock_gettime(CLOCK_MONOTONIC, &ts);
diff_timespec(&rs, &ts, &ref_time);
-   sprintf(prefix, "%6lu.%09lu%s", rs.tv_sec, rs.tv_nsec, csv_sep);
+   sprintf(prefix, "%6lu.%09lu%s", (unsigned long) rs.tv_sec,
+   (unsigned long) rs.tv_nsec, csv_sep);
 
if (num_print_interval == 0 && !csv_output) {
switch (aggr_mode) {
-- 
1.9.0.rc3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH] perf/x86/amd: Explicitly define PERF_COUNT_HW_REF_CPU_CYCLES as undefined.

[Adam Borowski]

filter_events() relies on the value of 0 to remove events that are not
applicable, like this one.

UBSAN: Undefined behaviour in arch/x86/events/amd/core.c:132:30
index 9 is out of range for type 'u64 [9]'
UBSAN: Undefined behaviour in arch/x86/events/amd/core.c:132:9
load of address 81c021c8 with insufficient space
for an object of type 'const u64'

Signed-off-by: Adam Borowski 
---
 arch/x86/events/amd/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index 86a9bec..5fa1b8e 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -125,6 +125,7 @@ static const u64 amd_perfmon_event_map[] =
   [PERF_COUNT_HW_BRANCH_MISSES]= 0x00c3,
   [PERF_COUNT_HW_STALLED_CYCLES_FRONTEND]  = 0x00d0, /* "Decoder empty" 
event */
   [PERF_COUNT_HW_STALLED_CYCLES_BACKEND]   = 0x00d1, /* "Dispatch stalls" 
event */
+  [PERF_COUNT_HW_REF_CPU_CYCLES]   =  0,
 };
 
 static u64 amd_pmu_event_map(int hw_event)
-- 
2.8.1

Re: [PATCH] perf/x86/amd: Explicitly define PERF_COUNT_HW_REF_CPU_CYCLES as undefined.

[Adam Borowski]

On Wed, Apr 27, 2016 at 10:03:45AM +0200, Ingo Molnar wrote:
> * Adam Borowski  wrote:
> > diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
> > index 86a9bec..5fa1b8e 100644
> > --- a/arch/x86/events/amd/core.c
> > +++ b/arch/x86/events/amd/core.c
> > @@ -125,6 +125,7 @@ static const u64 amd_perfmon_event_map[] =
> >[PERF_COUNT_HW_BRANCH_MISSES]= 0x00c3,
> >[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND]  = 0x00d0, /* "Decoder empty" 
> > event */
> >[PERF_COUNT_HW_STALLED_CYCLES_BACKEND]   = 0x00d1, /* "Dispatch stalls" 
> > event */
> > +  [PERF_COUNT_HW_REF_CPU_CYCLES]   =  0,
> >  };
> 
> Hm, I think it would be cleaner and more robust to change this (and all other 
> similar, if any) arrays to [PERF_COUNT_HW_MAX] instead.

Good idea!  Both of Intel's copies (one for p4, one for core+) already set
the size this way.

-- 
A tit a day keeps the vet away.

[PATCH] perf/x86/amd: Set the size of event map array to PERF_COUNT_HW_MAX.

[Adam Borowski]

The entry for PERF_COUNT_HW_REF_CPU_CYCLES is not used on AMD, but is
referenced by filter_events() which expects undefined events to have a
value of 0.

UBSAN: Undefined behaviour in arch/x86/events/amd/core.c:132:30
index 9 is out of range for type 'u64 [9]'
UBSAN: Undefined behaviour in arch/x86/events/amd/core.c:132:9
load of address 81c021c8 with insufficient space
for an object of type 'const u64'

Signed-off-by: Adam Borowski 
---
 arch/x86/events/amd/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
index 86a9bec..bd3e842 100644
--- a/arch/x86/events/amd/core.c
+++ b/arch/x86/events/amd/core.c
@@ -115,7 +115,7 @@ static __initconst const u64 amd_hw_cache_event_ids
 /*
  * AMD Performance Monitor K7 and later.
  */
-static const u64 amd_perfmon_event_map[] =
+static const u64 amd_perfmon_event_map[PERF_COUNT_HW_MAX] =
 {
   [PERF_COUNT_HW_CPU_CYCLES]   = 0x0076,
   [PERF_COUNT_HW_INSTRUCTIONS] = 0x00c0,
-- 
2.8.1

Re: [RFC6 PATCH v6 00/21] ILP32 for ARM64

[Adam Borowski]

On Wed, 6 Apr 2016, Geert Uytterhoeven wrote:
> On Wed, Apr 6, 2016 at 12:08 AM, Yury Norov  wrote:
>>  v6:
>>  - time_t, __kenel_off_t and other types turned to be 32-bit
>>for compatibility reasons (after v5 discussion);

Introducing a new arch today with y2038 problems is not a good idea.
Linus said so with appropriately pointy words in 2011.

> What makes you think these "applications that can’t readily be migrated to 
> LP64
> because they were written assuming an ILP32 data model, and that will never
> become suitable for a LP64 data model and will remain locked into ILP32
> operating environments" are more likely to be fixed for y2038 later, than for
> LP64 now?

Such broken applications already have plenty of bogus architecture detection
code so you need porting anyway...

> We're already closer to the (future) y2038 than to the (past) introduction of
> LP64...
>
> These unfixable legacy applications have been spreading through x32 to
> the shiny new arm64 server architecture (does ppc64el also have an ILP32 mode,
> or is it planned)? Lots of resources are spent on maintaining the status quo,
> instead of on fixing the real problems.
  
As an x32 (userland) porter, I can tell you that time_t!=long _did_ cause
non-trivial amounts of work.  But that work is already done (at least in
Debian), so you might as well benefit from it.


-- 
A tit a day keeps the vet away.

Re: Staging status of speakup

[Adam Borowski]

On Tue, Mar 19, 2019 at 04:31:21PM +, Alan Cox wrote:
> On Sat, 16 Mar 2019 10:35:43 +0100
> Samuel Thibault  wrote:
> > Chris Brannon, le ven. 15 mars 2019 18:19:39 -0700, a ecrit:
> > > What kind of reproducer do you need here?  It's straightforward to
> > > reproduce in casual use, at least with a software synthesizer.  
> > 
> > we need a walk-through of the kind of operation that
> > produces the issue. It does not have to be reproducible each time it is
> > done. Perhaps (I really don't know what that bug is about actually) it
> > is a matter of putting text in the selection buffer, and try to paste it
> > 100 times, and once every 10 times it will be garbled, for instance.
> 
> paste_selection still says
> 
> /* Insert the contents of the selection buffer into the
>  * queue of the tty associated with the current console.
>  * Invoked by ioctl().
>  *
>  * Locking: called without locks. Calls the ldisc wrongly with
>  * unsafe methods,
>  */
> 
> from which I deduce that with everyone using X nobody ever bothered to
> fix it. So before you look too hard at the speakup code you might want to
> review the interaction with selection.c too.

This looks like https://bugs.debian.org/849474 which causes a lockup, and
for which Bill Allombert wrote a nice reproducer.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄

Re: [PATCH v3 0/8] AMD64 EDAC fixes

[Adam Borowski]

On Wed, Aug 21, 2019 at 11:59:53PM +, Ghannam, Yazen wrote:
> I've also added RFC patches to avoid the "ECC disabled" message for
> nodes without memory. I haven't fully tested these, but I wanted to get
> your thoughts. Here's an earlier discussion:
> https://lkml.kernel.org/r/20180321191335.7832-1-yazen.ghan...@amd.com

While you're editing that code, could you please also cut the spam if ECC is
actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
64 copies of:

[8.186164] EDAC amd64: Node 0: DRAM ECC disabled.
[8.188364] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.194762] EDAC amd64: Node 1: DRAM ECC disabled.
[8.196307] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.199840] EDAC amd64: Node 2: DRAM ECC disabled.
[8.200963] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.204326] EDAC amd64: Node 3: DRAM ECC disabled.
[8.205436] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋  The root of a real enemy is an imaginary friend.
⠈⠳⣄

Re: [PATCH v3 0/8] AMD64 EDAC fixes

[Adam Borowski]

On Thu, Aug 22, 2019 at 10:35:48AM +0200, Borislav Petkov wrote:
> On Thu, Aug 22, 2019 at 02:50:20AM +0200, Adam Borowski wrote:
> > While you're editing that code, could you please also cut the spam if ECC is
> > actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
> 
> Patch is in there. I'll give you extra points if you spot it.

Yeah, some of messages are no longer emitted for memory-less nodes (NUMA 1
and 3).  Your patch set also overhauls the messages.

But, the amount of redundant messages I'm complaining about has actually
increased:

dmesg|grep EDAC|cut -c 16-|sort|uniq -c
256 EDAC MC: UMC0 chip selects:
256 EDAC MC: UMC1 chip selects:
  1 EDAC MC: Ver: 3.0.0
128 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: F17h detected (node 0).
 64 EDAC amd64: F17h detected (node 1).
 64 EDAC amd64: F17h detected (node 2).
 64 EDAC amd64: F17h detected (node 3).
512 EDAC amd64: MC: 0: 0MB 1: 0MB
256 EDAC amd64: MC: 2: 0MB 3: 0MB
256 EDAC amd64: MC: 2:  8192MB 3: 0MB
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
256 EDAC amd64: using x4 syndromes.

(Full dmesg at http://ix.io/1T1o)

While on 5.3-rc5 without the patchset I get:

  1 EDAC MC: Ver: 3.0.0
256 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 1: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
 64 EDAC amd64: Node 3: DRAM ECC disabled.

So I wonder if we could deduplicate those.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH] Documentation: sysrq: don't recommend 'S' 'U' before 'B'

[Adam Borowski]

This advice is obsolete and slightly harmful for filesystems from this
millenium: any modern filesystem can handle unexpected crashes without
requiring fsck -- and on the other hand, trying to write to the disk when
the kernel is in a bad state risks introducing corruption.

For ext2, any unsafe shutdown meant widespread breakage, but it's no longer
a reasonable filesystem for any non-special use.

Signed-off-by: Adam Borowski 
---
 Documentation/admin-guide/sysrq.rst | 20 +---
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/Documentation/admin-guide/sysrq.rst 
b/Documentation/admin-guide/sysrq.rst
index 7b9035c01a2e..72b2cfb066f4 100644
--- a/Documentation/admin-guide/sysrq.rst
+++ b/Documentation/admin-guide/sysrq.rst
@@ -171,22 +171,20 @@ It seems others find it useful as (System Attention Key) 
which is
 useful when you want to exit a program that will not let you switch consoles.
 (For example, X or a svgalib program.)
 
-``reboot(b)`` is good when you're unable to shut down. But you should also
-``sync(s)`` and ``umount(u)`` first.
+``reboot(b)`` is good when you're unable to shut down, it is an equivalent
+of pressing the "reset" button.
 
 ``crash(c)`` can be used to manually trigger a crashdump when the system is 
hung.
 Note that this just triggers a crash if there is no dump mechanism available.
 
-``sync(s)`` is great when your system is locked up, it allows you to sync your
-disks and will certainly lessen the chance of data loss and fscking. Note
-that the sync hasn't taken place until you see the "OK" and "Done" appear
-on the screen. (If the kernel is really in strife, you may not ever get the
-OK or Done message...)
+``sync(s)`` is handy before yanking removable medium or after using a rescue
+shell that provides no graceful shutdown -- it will ensure your data is
+safely written to the disk. Note that the sync hasn't taken place until you see
+the "OK" and "Done" appear on the screen.
 
-``umount(u)`` is basically useful in the same ways as ``sync(s)``. I generally
-``sync(s)``, ``umount(u)``, then ``reboot(b)`` when my system locks. It's saved
-me many a fsck. Again, the unmount (remount read-only) hasn't taken place until
-you see the "OK" and "Done" message appear on the screen.
+``umount(u)`` can be used to mark filesystems as properly unmounted. From the
+running system's point of view, they will be remounted read-only. The remount
+isn't complete until you see the "OK" and "Done" message appear on the screen.
 
 The loglevels ``0``-``9`` are useful when your console is being flooded with
 kernel messages you do not want to see. Selecting ``0`` will prevent all but
-- 
2.23.0

Re: Stop breaking the CSRNG

[Adam Borowski]

On Thu, Oct 03, 2019 at 10:13:39AM +, David Laight wrote:
> From: Kurt Roeckx
> > Sent: 02 October 2019 17:56
> > As OpenSSL, we want cryptograhic secure random numbers. Before
> > getrandom(), Linux never provided a good API for that, both
> > /dev/random and /dev/urandom have problems. getrandom() fixed
> > that, so we switched to it were available.
> 
> The fundamental problem is that you can't always get ' cryptograhic secure
> random numbers'. No API changes are ever going to change that.
> 
> The system can either return an error or sleep (possibly indefinitely)
> until some 'reasonably random' numbers are available.
> 
> A RISC-V system running on an FGPA (I've only used Altera NIOS cpu)
> may have absolutely no sources of randomness at boot time.

I'd say this is a hardware security vulnerability; no different from eg.
having no or faulty MMU, speculation that allows exfiltrating data, etc.
We did not understand the seriousness of lacking hardware sources of
randomness, but that's a common thing to many other security
vulnerabilities.

Machines that lack any sources of entropy have their uses, but they're akin
to processors with no MMU.  You should never run a world-accessible ssh
daemon on either of them.

> Saying the architecture must include a random number instruction
> doesn't help!

It won't fix existing systems, and is irrelevant to deeply embedded, but
communicating this requirement to SoC designers sounds like a good idea to
me.  IoTrash appliance makers won't care but their security is already so
atrocious that lack of entropy is nowhere near the easiest way to get in,
while anyone else will at least notice the warning.

Any real-silicon hardware can include an entropy source, and if it doesn't,
shaming the maker is the way to go.  Calling the problem a security
vulnerability (which I say it is) sends a stronger message.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol,
⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month.
⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake,
⠈⠳⣄ etc), let the drink age at least 3-6 months.

Re: boot failure: stack-protector: Kernel stack is corrupted in: start_secondary

[Adam Borowski]

On Tue, Apr 28, 2020 at 09:45:27AM +0200, Borislav Petkov wrote:
> On Tue, Apr 21, 2020 at 03:32:34AM +0200, Adam Borowski wrote:
> > Hi!
> > With kernels compiled with gcc-10, on two different machines (AMD Phenom2,
> > AMD 2990WX) I get the following panic during boot:
> 
> Welcome to the party:
> 
> https://git.kernel.org/tip/f670269a42bfdd2c83a1118cc3d1b475547eac22
> 
> Try this branch to check whether it works for ya:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/build

✓


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: [PATCH] selftests/ftrace: Make the coloring POSIX compliant

[Adam Borowski]

On Wed, Feb 20, 2019 at 09:20:20PM +0100, Juerg Haefliger wrote:
> On Wed, 20 Feb 2019 14:49:34 -0500
> Steven Rostedt  wrote:
> 
> > On Wed, 20 Feb 2019 17:13:33 +0100
> > Juerg Haefliger  wrote:
> > 
> > > echo -e and \e are not POSIX. Depending on what /bin/sh is, we can get
> > > incorrect output like:  
> > 
> > I'm curious to which shell this is.
> 
> Quite frankly I don't know but that's the output we get when we run it in
> Jenkins. I'll try to find out.

The only shell that did not support \e was dash -- I fixed it myself on
2017-01-24; thus I expect any Ubuntus prior to Zesty to require \033.  This
means your Jenkins likely runs Xenial.

\e has been supported since ages in at least: bash zsh mksh sash posh ksh
busybox:sh; also in perl python ruby lua php, gcc clang tcc -- MSVC being
the only other exception I know about.

Indeed POSIX doesn't specify \e, but as it pretends there are charsets other
than ASCII and EBCDIC, it can't.  There's no escape in its "portable
character set".


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ Have you accepted Khorne as your lord and saviour?
⠈⠳⣄

[PATCH] doc: process: GPL -> GPL-compatible

[Adam Borowski]

Drivers under MIT, BSD-17-clause, or uncle-Bob's-newest-take-on-PD are
all fine, not just GPL.

Signed-off-by: Adam Borowski 
---
Not reformatting to fill lines, it'll semi-conflict with another patch
that's been acked but not yet pushed.

 Documentation/process/stable-api-nonsense.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/stable-api-nonsense.rst 
b/Documentation/process/stable-api-nonsense.rst
index 24f5aeecee91..9b69fb08b65e 100644
--- a/Documentation/process/stable-api-nonsense.rst
+++ b/Documentation/process/stable-api-nonsense.rst
@@ -170,7 +170,8 @@ nightmare, and trying to keep up with an ever changing 
kernel interface
 is also a rough job.
 
 Simple, get your kernel driver into the main kernel tree (remember we
-are talking about GPL released drivers here, if your code doesn't fall
+are talking about drivers released under a GPL-compatible license here,
+if your code doesn't fall
 under this category, good luck, you are on your own here, you leech
 .)  If your
 driver is in the tree, and a kernel interface changes, it will be fixed
-- 
2.20.1

Re: GRSec is vital to Linux security

[Adam Borowski]

On Thu, Jan 24, 2019 at 04:31:10PM +0100, Enrico Weigelt, metux IT consult 
wrote:
> On 23.01.19 21:46, Ivan Ivanov wrote:
> 
> > Linux really needs to stop adding new features and
> > refactor itself to a smaller and more secure codebase before going
> > forward. Maybe 1 year break would be nice.
> 
> Do you have some actual proposals / patches ?

Enrico, you're responding to a notorious troll.  If you haven't noticed,
this "Ivan Ivanov" sock puppet is a persona of some bastard who talks to
him/herself while tarnishing the name of our dear friend MikeeUSA (a true
pillar of the community!).  His/her methods evolve, but the gist is the
same.  Expect bringing up a bogus but semi-plausible controversy in order
to start as big a thread as possible, then once people who this bastard
wants to attack have joined, try to equate their position in the public view
with statements such as:

(Excuse the quotation, please wipe your monitor afterwards.)

# But from a man?
#
# Well, goes to show you. White men ain't men. Best they are is 40 year
# old bois. Faggots to say for short in American parlance.
#
# Same reason they won't hold it down when a bunch of fucking cunts CoC
# them. You build the whole edifice, then you let a bunch of do-nothing
# white women rule over the thing you built and you.

And this has been going for quite a while.

Connecting to systemd threads doesn't seem to work any longer, as people on
debian-user vs dng have wisened up.  Same with license rescinsion threads. 
What you read is just a yet another attempt to stir up some excrement.
Don't let any of it spray on you.  Because that's the fake-Mikee's way.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Remember, the S in "IoT" stands for Security, while P stands
⢿⡄⠘⠷⠚⠋⠀ for Privacy.
⠈⠳⣄

Re: Device to write to all (serial) consoles

[Adam Borowski]

On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> On 02.08.19 18:02, Greg Kroah-Hartman wrote:
> > On Fri, Aug 02, 2019 at 03:23:08PM +0200, Paul Menzel wrote:
> > > On a lot of devices, like servers, you have more than one serial console,
> > > and you do not always know, how they are numbered. Therefore, we start a
> > > console on ttyS0 and ttyS1.
> 
> Because the cable is always connected to the port on the back side, and
> sometimes the port in the front has ID 0, and the one in the back 1, and
> other times vice versa. We do not want to track that, and it would be
> convenient to just write to both ports.

Sounds like an XY problem then: what you want is not writing to all ports,
but to have the port assignments stable (see also: disk device reordering).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: Device to write to all (serial) consoles

[Adam Borowski]

On Sat, Aug 03, 2019 at 03:55:37PM +0200, Greg Kroah-Hartman wrote:
> On Sat, Aug 03, 2019 at 03:23:23PM +0200, Adam Borowski wrote:
> > On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> > > Because the cable is always connected to the port on the back side, and
> > > sometimes the port in the front has ID 0, and the one in the back 1, and
> > > other times vice versa. We do not want to track that, and it would be
> > > convenient to just write to both ports.
> > 
> > Sounds like an XY problem then: what you want is not writing to all ports,
> > but to have the port assignments stable (see also: disk device reordering).
> 
> You can get that information from the symlinks in /dev/serial/ which
> udev creates.

Doesn't seem to work for me, for any ttyS0 ttyS1 ttySAC1 device:

Box one, PCIe card + one USB dongle:
07:00.0 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
07:00.1 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
Bus 003 Device 004: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter

/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
/dev/serial/by-path/pci-:0b:00.3-usb-0:4:1.0-port0

Only ttyUSB0 is there.


Box two, on-board + one USB dongle:
[3.404340] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[3.431287] 00:01: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 
16550A
Bus 001 Device 002: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 
UART Bridge Controller [CP210x family]

/dev/serial/by-id/usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_00DB1604-if00-port0
/dev/serial/by-path/pci-:00:14.0-usb-0:1:1.0-port0


Box three: RockPro64, euler + USB dongle, kernel 4.4.
Box four: Pine64, euler.
Box five: Odroid-U2, something GPIOish (ttySAC1), kernel 5.0.


Most are running kernel 5.2, Debian unstable.

And indeed, in /lib/udev/rules.d/60-serial.rules :
# /dev/serial/by-path/, /dev/serial/by-id/ for USB devices
KERNEL!="ttyUSB[0-9]*|ttyACM[0-9]*", GOTO="serial_end"


Like me, Paul is using ttyS0 for server-side.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.

Re: [RFC] remove arch/sh?

[Adam Borowski]

On Tue, Jun 25, 2019 at 11:02:36AM +0200, John Paul Adrian Glaubitz wrote:
> On 6/25/19 10:56 AM, Christoph Hellwig wrote:
> > arch/sh seems pretty much unmaintained these days.  The last time I got
> > any reply to sh patches from the list maintainers, and the last maintainer
> > pull request was over a year ago, and even that has been rather sporadic.
> > 
> > In the meantime we've not really seen any updates for new kernel features
> > and code seems to be bitrotting.
> 
> We're still using sh4 in Debian

I wouldn't call it "used": it has popcon of 1, and despite watching many
Debian channels, I don't recall hearing a word about sh4 in quite a while.

Hardware development is dead: we were promised modern silicon by j-core
after original patents expired, but after J2 nothing happened, there was
silence from their side, and now https://j-core.org is down.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Packager's rule #1: upstream _always_ screws something up.  This
⢿⡄⠘⠷⠚⠋⠀ is true especially if you're packaging your own project.
⠈⠳⣄ 

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Jun 07, 2019 at 07:20:46PM +, Nick Terrell wrote:
> We'd love to get this mainlined as well!
> 
> We're using these patches internally as well. We're seeing an improvement on 
> an
> Intel Atom N3710, where boot time is reduced by one second over using an xz
> compressed kernel. It looks like Ubuntu just switched to a lz4 compressed 
> kernel,
> but zstd is likely a better trade off, because it compresses much better and 
> still has
> excellent decompression speed.
> 
> Since its been nearly a year since I sent these out, I will take some time to 
> rebase
> and retest the patches in case anything changed, and then then resend the 
> patches
> in the next weeks.

Hi!
After the ping, I intended to resend the patch-set (with removals included)
after I return from miniDebconf Hamburg, but you 1. are the author of the
non-trivial part, 2. you have a better test machinery, and 3. I have a
deeply seated preference for effort to be done by people who are not me.

A rebased and working version is at 
https://github.com/kilobyte/linux/tree/nobz2-v3
but there are no real improvements beyond rebases, a typo fix, and Paul Burton's
ACK for mips.

There's an unaddressed comment by Ingo Molnar
https://lore.kernel.org/lkml/20181112042200.ga96...@gmail.com/
for your part of the code.

So what do you suggest?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: 5.0-rc1 KVM inspired "BUG: Bad page state in process" spew

[Adam Borowski]

On Wed, Jan 09, 2019 at 06:38:58AM +0100, Mike Galbraith wrote:
> KVM seems to be busted in master ATM.  All I have to do to have host
> start screaming and maybe exploding (if the guest doesn't do so first)
> is to try to install a (obese in this case) kernel over nfs mount of
> the host in a guest.
> 
> Kernel producing the spew below is 3bd6e94, config attached.

I get same, except that the BUGs were preceded by a bunch of warnings,

> homer: # grep BUG: /netconsole.log
> [ 1531.909703] BUG: Bad page state in process X  pfn:100491
> [ 1531.958141] BUG: Bad page state in process systemd-journal  pfn:100412
> [ 1532.662359] BUG: Bad page state in process X  pfn:10043f
> [ 1532.664033] BUG: Bad page state in process X  pfn:10044d
> [ 1532.686433] BUG: Bad page state in process systemd-journal  pfn:1027b0

the first one being:

Jan  9 00:41:22 umbar kernel: [74122.790461] WARNING: CPU: 2 PID: 26769 at 
arch/x86/kvm/mmu.c:830 mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.799676] Modules linked in: tun dm_mod 
cdc_acm rndis_wlan rndis_host cdc_ether usbnet sha256_generic cfg80211 rfkill 
mii nfnetlink snd_usb_audio snd_usbmidi_lib cp210x usbserial radeon ttm 
pcc_cpufreq
Jan  9 00:41:22 umbar kernel: [74122.817764] CPU: 2 PID: 26769 Comm: 
qemu-system-x86 Not tainted 5.0.0-rc1-debug-00035-gdce22716f1b4 #1
Jan  9 00:41:22 umbar kernel: [74122.827069] Hardware name: System manufacturer 
System Product Name/M4A77T, BIOS 240105/18/2011
Jan  9 00:41:22 umbar kernel: [74122.836025] RIP: 
0010:mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.841528] Code: 48 89 e8 48 ba 00 00 00 00 
00 ea ff ff 48 c1 e0 06 48 01 d0 48 8b 50 08 48 8d 4a ff 83 e2 01 48 0f 45 c1 
8b 40 34 85 c0 75 02 <0f> 0b 48 0f ba e3 3e 73 34 48 85 1d d2 32 26 01 75 5a 48 
d1 eb 83
Jan  9 00:41:22 umbar kernel: [74122.860290] RSP: 0018:c900028634d0 EFLAGS: 
00010246
Jan  9 00:41:22 umbar kernel: [74122.865516] RAX:  RBX: 
00010198bc67 RCX: dead00ff
Jan  9 00:41:22 umbar kernel: [74122.872649] RDX:  RSI: 
 RDI: ea00040662c0
Jan  9 00:41:22 umbar kernel: [74122.879790] RBP: 0010198b R08: 
28f5c28f5c28f5c3 R09: 8882264dd000
Jan  9 00:41:22 umbar kernel: [74122.886912] R10: 88822fffa000 R11: 
88822fffa000 R12: 
Jan  9 00:41:22 umbar kernel: [74122.894046] R13: c90002863580 R14: 
 R15: c90002863580
Jan  9 00:41:22 umbar kernel: [74122.901170] FS:  7f19b9953700() 
GS:888227a8() knlGS:
Jan  9 00:41:22 umbar kernel: [74122.909249] CS:  0010 DS:  ES:  CR0: 
80050033
Jan  9 00:41:22 umbar kernel: [74122.914994] CR2: 102630e4 CR3: 
0001c80ee000 CR4: 06e0
Jan  9 00:41:22 umbar kernel: [74122.922135] Call Trace:
Jan  9 00:41:22 umbar kernel: [74122.924590]  drop_spte+0x1b/0xc0
Jan  9 00:41:22 umbar kernel: [74122.927822]  mmu_page_zap_pte+0xb2/0xe0
Jan  9 00:41:22 umbar kernel: [74122.931661]  
kvm_mmu_prepare_zap_page+0x4f/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.936297]  mmu_shrink_scan+0x199/0x240
Jan  9 00:41:22 umbar kernel: [74122.940223]  do_shrink_slab+0x11e/0x1a0
Jan  9 00:41:22 umbar kernel: [74122.944054]  shrink_slab+0x220/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.947632]  shrink_node+0x168/0x460
Jan  9 00:41:22 umbar kernel: [74122.951203]  do_try_to_free_pages+0xc1/0x370
Jan  9 00:41:22 umbar kernel: [74122.955467]  try_to_free_pages+0xb0/0xe0
Jan  9 00:41:22 umbar kernel: [74122.959385]  __alloc_pages_slowpath+0x37d/0xb60
Jan  9 00:41:22 umbar kernel: [74122.963917]  __alloc_pages_nodemask+0x255/0x270
Jan  9 00:41:22 umbar kernel: [74122.968441]  
do_huge_pmd_anonymous_page+0x13c/0x5d0
Jan  9 00:41:22 umbar kernel: [74122.973322]  __handle_mm_fault+0x984/0xfb0
Jan  9 00:41:22 umbar kernel: [74122.977438]  handle_mm_fault+0xc2/0x200
Jan  9 00:41:22 umbar kernel: [74122.981278]  __get_user_pages+0x258/0x6c0
Jan  9 00:41:22 umbar kernel: [74122.985290]  
get_user_pages_unlocked+0x153/0x1d0
Jan  9 00:41:22 umbar kernel: [74122.989954]  __gfn_to_pfn_memslot+0x149/0x410
Jan  9 00:41:22 umbar kernel: [74122.994320]  try_async_pf+0x96/0x1b0
Jan  9 00:41:22 umbar kernel: [74122.997900]  ? kvm_host_page_size+0x81/0xa0
Jan  9 00:41:22 umbar kernel: [74123.002093]  tdp_page_fault+0x168/0x2b0
Jan  9 00:41:22 umbar kernel: [74123.005927]  ? svm_vcpu_run+0x294/0x680
Jan  9 00:41:22 umbar kernel: [74123.009765]  kvm_mmu_page_fault+0x89/0x3f0
Jan  9 00:41:22 umbar kernel: [74123.013865]  ? kvm_set_cr8.part.87+0xa/0x30
Jan  9 00:41:22 umbar kernel: [74123.018049]  ? svm_vcpu_run+0x4fd/0x680
Jan  9 00:41:22 umbar kernel: [74123.021888]  ? kvm_fast_pio+0x140/0x190
Jan  9 00:41:22 umbar kernel: [74123.025729]  
kvm_arch_vcpu_ioctl_run+0x59e/0x19c0
Jan  9 00:41:22 umbar kernel: [74123.030435]  ? _copy_to_user+0x26/0x30
Jan  9 00:41:22 umbar kernel: [74123.034187]  ? kvm_vm_ioctl+0x69a/0x950
Jan  9 00:41:22 umb

Re: [PATCH] mm/mmu_notifier: mm/rmap.c: Fix a mmu_notifier range bug in try_to_unmap_one

[Adam Borowski]

On Wed, Jan 09, 2019 at 04:51:17PM -0800, Sean Christopherson wrote:
> Manifests as KVM use-after-free WARNINGs and subsequent "BUG: Bad page
> state in process X" errors when reclaiming from a KVM guest due to KVM
> removing the wrong pages from its own mappings.

With your patch, no badness happened so far.  Thanks!

> Reported-by: Adam Borowski 
> Fixes: ac46d4f3c432 ("mm/mmu_notifier: use structure for 
> invalidate_range_start/end calls v2")

> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> - mmu_notifier_range_init(&range, vma->vm_mm, vma->vm_start,
> - min(vma->vm_end, vma->vm_start +
> + mmu_notifier_range_init(&range, vma->vm_mm, address,
> + min(vma->vm_end, address +


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ Hans 1 was born and raised in Johannesburg, then moved to Boston,
⣾⠁⢠⠒⠀⣿⡁ and has just became a naturalized citizen.  Hans 2's grandparents
⢿⡄⠘⠷⠚⠋⠀ came from Melanesia to Düsseldorf, and he hasn't ever been outside
⠈⠳⣄ Germany until yesterday.  Which one is an African-American?

Re: POSIX violation by writeback error

[Adam Borowski]

On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S.  One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a reboot or if the process was killed or exits
> without an explicit close(2).  For networked/remote file systems that
> supported this flag, after the client comes back up after a reboot, it
> could notify the server that all files created previously from that
> client should be unlinked.
> 
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.

Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:

  char path[PATH_MAX];
  fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
  S_IRUSR | S_IWUSR);

  /* File I/O on 'fd'... */

  snprintf(path, PATH_MAX,  "/proc/self/fd/%d", fd);
  linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file",
  AT_SYMLINK_FOLLOW);


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.

Re: [RFC] new SYSCALL_DEFINE/COMPAT_SYSCALL_DEFINE wrappers

[Adam Borowski]

On Fri, Mar 30, 2018 at 12:58:02PM +0200, Ingo Molnar wrote:
> * John Paul Adrian Glaubitz  wrote:
> 
> > On 03/27/2018 12:40 PM, Linus Torvalds wrote:
> > > On Mon, Mar 26, 2018 at 4:37 PM, John Paul Adrian Glaubitz
> > >  wrote:
> > >>
> > >> What about a tarball with a minimal Debian x32 chroot? Then you can
> > >> install interesting packages you would like to test yourself.

> Here's the direct download link:
>   $ wget 
> https://people.debian.org/~glaubitz/chroots/debian-x32-unstable.tar.gz

> Seems to work fine here (on a distro kernel) even if I extract all the files 
> as a 
> non-root user and do:
> 
>   ~/s/debian-x32-unstable> fakechroot /usr/sbin/chroot . /usr/bin/dpkg -l  | 
> tail -2
> 
>   ERROR: ld.so: object 'libfakechroot.so' from LD_PRELOAD cannot be preloaded 
> (cannot open shared object file): ignored.
>   ii  util-linux:x32 2.31.1-0.5   x32  miscellaneous 
> system utilities
>   ii  zlib1g:x32 1:1.2.8.dfsg-5   x32  compression 
> library - runtime

> So that 'dpkg' instance appears to be running inside the chroot environment 
> and is 
> listing x32 installed packages.

> Although I did get this warning:
>   ERROR: ld.so: object 'libfakechroot.so' from LD_PRELOAD cannot be preloaded 
> (cannot open shared object file): ignored.
> Even with that warning, is still still a sufficiently complex test of x32 
> syscall 
> code paths?

Instead of mucking with fakechroot which would require installing its :x32
part inside the guest, or running the test as root, what about using any
random static binary?  For example, a shell like sash or bash-static would
have a decentish syscall coverage even by itself.

I've extracted sash from
http://ftp.ports.debian.org/debian-ports//pool-x32/main/s/sash/sash_3.8-4_x32.deb
and placed at https://angband.pl/tmp/sash.x32

$ sha256sum sash.x32 
de24097c859b313fa422af742b648c9d731de6b33b98cb995658d1da16398456  sash.x32

Obviously, you can compile a static binary that uses whatever syscalls you
want.  Without a native chroot, you can "gcc -mx32" although you'd need some
kind of libc unless your program is stand-alone.


It might be worth mentioning my "arch-test:
https://github.com/kilobyte/arch-test
Because of many toolchain pieces it needs, you want a prebuilt copy:
https://github.com/kilobyte/arch-test/releases/download/v0.10/arch-test_prebuilt_0.10.tar.xz
https://github.com/kilobyte/arch-test/releases/download/v0.10/arch-test_prebuilt_0.10.tar.xz.asc
-- while it has _extremely_ small coverage of syscalls (just write() and
_exit(), enough to check endianness and pointer width), concentrating on
instruction set inadequacies (broken SWP on arm, POWER7 vs POWER8, powerpc
vs powerpcspe, etc), it provides minimal test binaries for a wide range of
architectures.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁ I was born a dumb, ugly and work-loving kid, then I got swapped on
⢿⡄⠘⠷⠚⠋⠀ the maternity ward.
⠈⠳⣄


signature.asc
Description: PGP signature

Re: [PATCH 1/2] lib: vsprintf: Implement %pCOW

[Adam Borowski]

On Sun, Apr 01, 2018 at 10:56:21AM +0200, Richard Weinberger wrote:
> + .cow_lines = {
> + "\\   ^__^",
> + " \\  (oo)\\___",
> + "(__)\\   )\\/\\",
> + "||w |",
> + "|| ||",

Userspace cowsay(6) knows of different cows.  What about using those as a
visual indication of message level?


Meow! err... Moo!
-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ

Re: How to disable Linux kernel self-extraction (KERNEL_GZIP, KERNEL_BZIP2, …)?

[Adam Borowski]

On Mon, Apr 23, 2018 at 07:02:30PM +0200, Pavel Machek wrote:
> > > >>I try to decrease boot time, and my system has an SSD and enough space, 
> > > >>so
> > > >>loading 18 instead of 12 MB doesn’t make a difference, but the
> > > >>self-extraction is noticeable. So, I like to disable it.
> > > >
> > > >How long does GZIP extraction take on your hardware?
> > > 
> > > It’s hard to measure – at least I didn’t find a way to do so –, but 
> > > counting
> > > from the last GRUB message to the first message of Linux (with `quiet`
> > > removed from the the command line), it takes roughly *two* seconds.

I took a somewhat different approach: I recorded the output from grub+kernel
to ttyrec over serial line, and rigged ttyrec2ansi to output timestamp
difference from the last checkpoint every time an '\e' or '\n' is seen.
'\e' is important, as there's no other marking for when grub stops the
interactive phase and starts the actual boot.

Turns out that, reading from SSD, grub is way way slower than it should take
normally.  The machine is old (AMD Phenom II X6 1055T), SSD is Crucial
CT240M500SSD1.

I also have the zstd patch applied, which adds another data point.

The two "Loading XXX ..." lines come from grub, those timestamped within []
brackets from the kernel, 〈〉are ttyrec timestamps, ⤸ is wrapped lines.


zstd:

Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.739823〉
^MLoading initial ramdisk ...〈0.402010〉
^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23⤸
10:25:58 CEST 2018^M〈0.785922〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020199〉

gzip:

Loading Linux 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ...〈0.724988〉
^MLoading initial ramdisk ...〈0.357951〉
^M[0.00] Linux version 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
23:15:07 CEST 2018^M〈0.777977〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-gz-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020117〉

lz4:

Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d ...〈0.799969〉
^MLoading initial ramdisk ...〈0.424959〉
^M[0.00] Linux version 4.17.0-rc2-debug-lz4-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Tue Apr 24 ⤸
00:34:59 CEST 2018^M〈0.732925〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.021019〉

zstd again:

Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.728852〉
^MLoading initial ramdisk ...〈0.399968〉
^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
10:25:58 CEST 2018^M〈0.786964〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020071〉

lz4 rigged for no compression:

Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty ...〈0.479834〉
^MLoading initial ramdisk ...〈2.246985〉
^M[0.00] Linux version 4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #5 SMP Tue Apr 24 ⤸
02:57:18 CEST 2018^M〈0.711949〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.021902〉

Sizes of relevant files:

14826134 initrd.img-4.17.0-rc2-debug-00025-gd426b0ba363d(zstd)
14826352 initrd.img-4.17.0-rc2-debug-gz-00025-gd426b0ba363d
14826909 initrd.img-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d
14826761 initrd.img-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty
 6567408 vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d   (zstd)
 7230960 vmlinuz-4.17.0-rc2-debug-gz-00025-gd426b0ba363d
 8775152 vmlinuz-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d
27821552 vmlinuz-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty
(I did not alter initrd compression, which is zstd in all cases).

> > So yes, looks like uncompressed kernel image may be good idea.

Seems like the time to actually read this far bigger file from the disk
using grub's inefficient way, takes longer than the gains from faster
decompression.  You can eliminate the decompression step altogether by
av

Re: unable to load modules with CONFIG_MODVERSIONS=y after commit 8ab2ae655b

[Adam Borowski]

On Tue, Dec 06, 2016 at 08:31:01AM -0500, Jan Stancek wrote:
> Starting with 4.9-rc8 / commit 8ab2ae655b ("default exported asm symbols to 
> zero")
> I'm running into issue with kernel built with CONFIG_MODVERSIONS=y
> and (older) binutils (binutils-2.25.1-20.base.el7.ppc64le). 
> 
> Modules fail to load, for example:
> 
> [3.163646] Found checksum 0 vs module 4829A47E 
> [3.163787] dm_mod: disagrees about version of symbol memcpy 
> [3.163862] dm_mod: Unknown symbol memcpy (err -22)
> 
> Bisect led me to 8ab2ae655b, reverting it allows boot to
> progress as before.

powerpc happens to be the only arch that actually followed the plan and
implemented asm-prototypes.h (not including Debian which applied my patch to
do so on x86, that patch is not in mainline).

Could you try reverting commits that add exports to that file?

-- 
u-boot problems can be solved with the help of your old SCSI manuals, the
parts that deal with goat termination.  You need a black-handled knife, and
an appropriate set of candles (number and color matters).  Or was it a
silver-handled knife?  Crap, need to look that up.

Re: [PATCH v5 2/2] console: Add persistent scrollback buffers for all VGA consoles

[Adam Borowski]

On Tue, Nov 22, 2016 at 05:56:42PM +0100, Manuel Schölling wrote:
> On Mo, 2016-11-21 at 21:17 +0100, Adam Borowski wrote:
> > On Sun, Nov 20, 2016 at 10:58:08PM +0100, Manuel Schölling wrote:
> > > Add a scrollback buffers for each VGA console. The benefit is that
> > > the scrollback history is not flushed when switching between consoles
> > > but is persistent.
> 
> > But alas, this commit breaks that very \e[3J.  It does only a \e[2J, leaving
> > the scrollback uncleared.  For comparison, both mainline and with just your
> > preparatory commit, \e[3J works as expected.
> Really? All my tests worked fine: I compiled the kernel with the latest 
> patches, started the kernel in QEMU and then did
> 
>   $ openvt /bin/sh
>   $ echo -e '\e[3J' # scrollback buffer was flushed correctly
>   $ chvt 2
>   $ echo -e '\e[3J' # scrollback buffer was flushed correctly
> 
> Can you tell me how you tested it? Maybe I can reproduce the bug.

(Re-tested on v6 of the patch.)

On bare metal: boot, log in on tty1, printf '\e[3J', screen clears but when
I scroll back, it still has bootup messages.  Switching to another tty then
back obviously doesn't clear it either.  Same on any other tty (after
putting something into the scrollback of).  Graphics card is nvidia GT240,
neither proprietary driver nor nouveau loaded; nouveau forces fbdev and your
patch is vgacon specific (hopefully just for now).

But then, I just reproduced this on qemu (-vga qxl) too, so it might be due
to a difference between our setups somehow.  In case it's something related
to .config, mine's at https://angband.pl/tmp/config-4.9.0-rc6-debug2+.xz


Meow!
-- 
An imaginary friend squared is a real enemy.

Re: BUG: 4.9-rc6 Still "no symbol version" on boot

[Adam Borowski]

On Wed, Nov 23, 2016 at 09:08:28PM +0100, Philip Müller wrote:
> > due to following commit it seems the 64bit architecture of linux 4.9-rc
> > is not able to boot at all, as it is unable to find its root device:

> you have to apply following patch also:
> 
> provide-asm-prototypes.h-for-x86.patch:
> https://patchwork.kernel.org/patch/9408985/raw/
> 
> @Adam, Nick: Was this patch not yet sent to Linus?

The patch stewed in a kbuild-targetted thread since the morning after -rc1,
Nick has recently requested that it should go through x86 maintainers
instead.  I've sent it there, lemme ping them, as the regression is severe
and 4.9-final is close.

Apologies if I'm doing something wrong, I'm not a real kernel dev and merely
was the person who came here looking for a fix, saw Nick's instructions
and did the legwork implementing them.

Last version (rewritten description) is at:
https://patchwork.kernel.org/patch/9439501/
(needs s/oeter/peter/ for a typo in Peter Wu's address)


Meow!
-- 
The bill declaring Jesus as the King of Poland fails to specify whether
the addition is at the top or end of the list of kings.  What should the
historians do?

[PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Adam Borowski]

Commit 4efca4ed ("kbuild: modversions for EXPORT_SYMBOL() for asm") adds
modversion support for symbols exported from asm files. Architectures
must include C-style declarations for those symbols in asm/asm-prototypes.h
in order for them to be versioned.

Add these declarations for x86, and an architecture-independent file that
can be used for common symbols.

User impact: kernels may fail to load modules at all when
CONFIG_MODVERSIONS=y.

Signed-off-by: Adam Borowski 
Tested-by: Kalle Valo 
Acked-by: Nicholas Piggin 
Tested-by: Peter Wu 
Tested-by: Oliver Hartkopp 
---
Changes: corrected Peter Wu's address, added Tested-by: Oliver.
This is an unsplit version (x86/include/ and include/ together).

 arch/x86/include/asm/asm-prototypes.h | 12 
 include/asm-generic/asm-prototypes.h  |  7 +++
 2 files changed, 19 insertions(+)
 create mode 100644 arch/x86/include/asm/asm-prototypes.h
 create mode 100644 include/asm-generic/asm-prototypes.h

diff --git a/arch/x86/include/asm/asm-prototypes.h 
b/arch/x86/include/asm/asm-prototypes.h
new file mode 100644
index 000..ae87224
--- /dev/null
+++ b/arch/x86/include/asm/asm-prototypes.h
@@ -0,0 +1,12 @@
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
diff --git a/include/asm-generic/asm-prototypes.h 
b/include/asm-generic/asm-prototypes.h
new file mode 100644
index 000..df13637
--- /dev/null
+++ b/include/asm-generic/asm-prototypes.h
@@ -0,0 +1,7 @@
+#include 
+extern void *__memset(void *, int, __kernel_size_t);
+extern void *__memcpy(void *, const void *, __kernel_size_t);
+extern void *__memmove(void *, const void *, __kernel_size_t);
+extern void *memset(void *, int, __kernel_size_t);
+extern void *memcpy(void *, const void *, __kernel_size_t);
+extern void *memmove(void *, const void *, __kernel_size_t);
-- 
2.10.2

Re: [PATCH v10 3/4] console: Add persistent scrollback buffers for all VGA consoles

[Adam Borowski]

On Thu, Jan 19, 2017 at 05:12:15PM +0100, Manuel Schölling wrote:
> On Thu, 2017-01-19 at 14:23 +0100, Greg KH wrote:
> > On Fri, Jan 13, 2017 at 09:07:57PM +0100, Manuel Schölling wrote:
> > > Add a scrollback buffers for each VGA console.  The benefit is that
> > > the scrollback history is not flushed when switching between consoles
> > > but is persistent.  The buffers are allocated on demand when a new
> > > console is opened.
> > > 
> > > This breaks tools like clear_console that rely on flushing the
> > > scrollback history by switching back and forth between consoles
> > > which is why this feature is disabled by default.
> > > Use the escape sequence \e[3J instead for flushing the buffer.
> > 
> > This issue is the one that makes me the most worried.  Why doesn't
> > clear_console() work anymore?  Why doesn't it use \e[3J ?
> 
> Well, clear_console() just switches from one console to another and
> back again. It just assumes that the scrollback buffer is flushed when
> switching.
> My plan is to make a patch for clear_console() as soon as these patches
> are in the kernel - it's chicken-and-egg problem.

No need to wait, \e[3J is supported since Linux 2.6.39; the problem I
spotted was that a previous version of your patch would break that.

It is also safe to output that sequence to a terminal unconditionally: I've
tested a number of terminals, they all either support it (most X terminals,
our console) or silently ignore it.

We can't, though, rely on terminfo to do so: it knows about this capability
(which it calls "E3") for TERM=linux only since very recently.  The TERM
variable is also unreliable: it fails to carry over a serial link while
blindly printing \e[3J works.

As for patching clear_console, https://bugs.debian.org/845177 has a minimal
fix; although for all setups supported by Debian that program could be be
better replaced with just "printf '\e[3J\e[2J'" which would make it work on
strictly more terminals than current code does.  Same for distributions
which copied clear_console (it originates from Ubuntu, maintained in Debian
since then).


Meow!
-- 
Autotools hint: to do a zx-spectrum build on a pdp11 host, type:
  ./configure --host=zx-spectrum --build=pdp11

Re: [PATCH v10 3/4] console: Add persistent scrollback buffers for all VGA consoles

[Adam Borowski]

On Thu, Jan 19, 2017 at 05:33:14PM +0100, Greg KH wrote:
> On Thu, Jan 19, 2017 at 05:12:15PM +0100, Manuel Schölling wrote:
> > On Thu, 2017-01-19 at 14:23 +0100, Greg KH wrote:
> > > On Fri, Jan 13, 2017 at 09:07:57PM +0100, Manuel Schölling wrote:
> > > > +     This feature might break your tool of choice to flush
> > > > the scrollback
> > > > +     buffer, e.g. clear(1) will work fine but Debian's
> > > > clear_console(1)
> > > > +     will be broken, which might cause security issues.
> > > > +     You can use the escape sequence \e[3J instead if this
> > > > feature is
> > > > +     activated.
> > > 
> > > This issue is the one that makes me the most worried.  Why doesn't
> > > clear_console() work anymore?  Why doesn't it use \e[3J ?
> > 
> > Well, clear_console() just switches from one console to another and
> > back again. It just assumes that the scrollback buffer is flushed when
> > switching.
> > My plan is to make a patch for clear_console() as soon as these patches
> > are in the kernel - it's chicken-and-egg problem.
> 
> I'd recommend that patch get to clear_console() first, having it use the
> new escape sequence, if it isn't supported, shouldn't cause any
> problems, right?

In that case, we need to hurry -- the last day for any non-serious fixes in
Debian is Jan 26, after that it'll be frozen for months, and any subsequent
changes won't get to stable users for around two years.

doko: would you consider, pretty please with a cherry on top, applying the
patch I've sent to this bug?  The privacy/security issue is pretty minor and
applies only to a tiny fraction of users, but I understand why Greg is
reluctant.

Manuel's scrollback changes won't go to 4.9, and won't be enabled by default
for the time being, but using a newer kernel on old userspace is something
really widespread, be it via bpo, containers on an updated host, etc.


Meow!
-- 
Autotools hint: to do a zx-spectrum build on a pdp11 host, type:
  ./configure --host=zx-spectrum --build=pdp11