Re: Broken commandline

[Kacper Gutowski]

On Fri, Jun 04, 2021 at 12:47:31AM +0200, Hans-Peter Sorge wrote:
However, there are "graphically oriented" UTF characters (including 
⍝⊖⍟⍱), that take two character positions when being displayed and take 
one character position when being selected.


So it's not an APL character / font handling problem. It Just happened 
to be the first popup.


This sounds indicative of the kind of font problem where the terminal 
emulator has a different idea about widths of glyphs than what the font 
actually provides, i.e. when the font isn't really monospaced.


You mentioned Noto Sans Mono earlier. If APL characters are displayed 
for you using that font, either because you chose it or because your 
chosen font doesn't have them and Noto Sans Mono happens to be used as 
a fallback, then you surely will have this problem because the mentioned 
characters are twice as wide as other characters in this font.


And it's for no good reason. Unlike Chinese characters, these were never 
a part of any East Asian character set (unless you count a certain 
communist symbol from North Korean character set being informally mapped 
to ⍟ by some sources) and their East Asian Width property in Unicode is 
"neutral" and wcwidth returns 1 for them. So they are expected to take 
only one character cell and in many terminal emulators this leads to the 
behaviour you described when they don't.


This should be considered a bug in Noto Sans Mono. 



-k

Re: Broken commandline

[Hans-Peter Sorge]

Hi Kasper,

the console (Konsole) uses
    /etc/vconsole.conf
    FONT=latarcyrheb-sun16  # 
/usr/lib/kbd/consolefonts/latarcyrheb-sun16.psfu



Noto Sans Mono behaves the same. And it is not listed 
in/usr/lib/kbd/consolefonts/


I tried to find the underlying reason for the bug.

Your hint about wcwidth is helpful.

I hope to find some time digging deeper:-)

btw - Konsole is really 'broken'.

After playing with the different UF8 chars I have to [ENTER] twice to 
execute a command

(need to do reset).

Best Regards
Hans-Peter


Am 04.06.21 um 10:24 schrieb Kacper Gutowski:

On Fri, Jun 04, 2021 at 12:47:31AM +0200, Hans-Peter Sorge wrote:
However, there are "graphically oriented" UTF characters (including 
⍝⊖⍟⍱), that take two character positions when being displayed and 
take one character position when being selected.


So it's not an APL character / font handling problem. It Just 
happened to be the first popup.


This sounds indicative of the kind of font problem where the terminal 
emulator has a different idea about widths of glyphs than what the 
font actually provides, i.e. when the font isn't really monospaced.


You mentioned Noto Sans Mono earlier. If APL characters are displayed 
for you using that font, either because you chose it or because your 
chosen font doesn't have them and Noto Sans Mono happens to be used as 
a fallback, then you surely will have this problem because the 
mentioned characters are twice as wide as other characters in this font.


And it's for no good reason. Unlike Chinese characters, these were 
never a part of any East Asian character set (unless you count a 
certain communist symbol from North Korean character set being 
informally mapped to ⍟ by some sources) and their East Asian Width 
property in Unicode is "neutral" and wcwidth returns 1 for them. So 
they are expected to take only one character cell and in many terminal 
emulators this leads to the behaviour you described when they don't.


This should be considered a bug in Noto Sans Mono.

-k

Re: Broken commandline

[Dr . Jürgen Sauermann]

Hi Kacper,

thanks a lot for your explanation.

I should add that the APL formatting model is heavily based on the 
assumption that
every character has the same size (like it was the case for old APL 
terminals). So
even if a font were able to tell that some (APL-) character takes 2 
screen columns,
then I am afraid that the printout of APL values with rank > 1 would 
still look a little

messy if such characters are being printed.

So fixing the the Noto Sans Mono font bug would not solve the problem 
entirely.
My proposal would be to use a different font where all characters have 
the same size

(my favourite is Courier with Unicode characters).

Best Regards,
Jürgen


On 6/4/21 10:24 AM, Kacper Gutowski wrote:

On Fri, Jun 04, 2021 at 12:47:31AM +0200, Hans-Peter Sorge wrote:
However, there are "graphically oriented" UTF characters (including 
⍝⊖⍟⍱), that take two character positions when being displayed and 
take one character position when being selected.


So it's not an APL character / font handling problem. It Just 
happened to be the first popup.


This sounds indicative of the kind of font problem where the terminal 
emulator has a different idea about widths of glyphs than what the 
font actually provides, i.e. when the font isn't really monospaced.


You mentioned Noto Sans Mono earlier. If APL characters are displayed 
for you using that font, either because you chose it or because your 
chosen font doesn't have them and Noto Sans Mono happens to be used as 
a fallback, then you surely will have this problem because the 
mentioned characters are twice as wide as other characters in this font.


And it's for no good reason. Unlike Chinese characters, these were 
never a part of any East Asian character set (unless you count a 
certain communist symbol from North Korean character set being 
informally mapped to ⍟ by some sources) and their East Asian Width 
property in Unicode is "neutral" and wcwidth returns 1 for them. So 
they are expected to take only one character cell and in many terminal 
emulators this leads to the behaviour you described when they don't.


This should be considered a bug in Noto Sans Mono.

-k

Re: Broken commandline

[Hans-Peter Sorge]

Hi,

I have opened a bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=1967789 



Best Regards
Hans-Peter

Am 02.06.21 um 11:23 schrieb Hans-Peter Sorge:

Hi,

I am on Fedora 34 now (apl SVN: 1198:1472M)

The apl font partially fails to work with keyboard layout apl.

The "over strike characters" ⌽⍉⍟⍝⍱⍲ ,when entered in a terminal, 
appear in half

The left half of the character is being displayed when entered.
The right half appears when space bar is being pressed.
Anny further character entered is being offset by one.

That is  ⌽[space]ABC  -> ends up as ⌽BC where space bar "overlays" 
character A!
The Apl-Session, terminal, emacs, vi are affected too. slikedid, 
Firefox are OK.


If changing the layout via *setxkbmap -layout "apl"* I get  
instead of ⌽⍉⍟⍝.
The � are being displayed the same way as described above: Left half 
character, [SPACEBAR] right half of the character.


I installed apl from the repository to verify the behavior - that 
segfaults.


The font-manager shows
⍝  APL FUNCTIONAL SYMBOL UP SHOE JOT   U+235D
So there is nothing special about this and 'normal' UTF-8 characters.

The terminal font is set to *latarcyrheb-sun16* in */etc/voconsole.conf.*

Changing the terminal font to either Unifont-APL8x16-12.1.02 or 
NotoSansMono did not help.


Emacs:
After some 'fiddling around' font changes with emacs *M-x 
menu-set-font* will display ⍝ in one go,
Char ⍝ will be displayed for different fonts - even if the 
font-manager does not show it.
Char ⍝ with font NotoSansMono selected will be displayed in one shot 
but with additional spacing to the left and right.

Char ⍝ with Unifont selected will be displayed w/o spacing.
( The console fonts and the content of *menu-set-font* are different...)

Hopefully some one has an idea where / how to dig deeper into the problem.

Best Regards
Hans-Peter



Is there / some where an experiment going on to 'improve' character 
composition?