Re: Porter roll call for Debian Stretch

[Christoph Biedl]

John Paul Adrian Glaubitz wrote...

> On 09/20/2016 11:16 PM, Niels Thykier wrote:
> >- powerpc: No porter (RM blocker)
> 
> I'd be happy to pick up powerpc to keep it for Stretch. I'm already
> maintaining powerpcspe which is very similar to powerpc.

For somewhat personal reasons I'm interested in keeping powerpc in
stretch as well. I certainly cannot take the entire role as a porter,
especially since I don't know what amount of work this implies. But I
am willing to help.

There are two powerpc boxes in my collection, used regulary. One runs
on stable, the other on testing. I haven't done d-i tests but
certainly could do.

Christoph



signature.asc
Description: Digital signature

Re: PowerPC roadmap for Stretch

[Christoph Biedl]

Mathieu Malaterre wrote...

> Since there has been some fuss about powerpc recently, let's see if I
> can get some help on issue(s) I'd like to fix for Stretch.

Let's see what I can do (and I'm subscribed to debian-powerpc now, so
no need for Cc: in the future :)

> 1. Secret GRUB option in d-i.
> 
> This is no suprise that we have a situation with yaboot in powerpc (in
> particular partman-ext3 #825110). So I would like to finish the
> initial work to have GRUB be an alternative on PowerMac (famous secret
> option in d-i).
> 
> Who can help me finish:
> - https://bugs.debian.org/610206 (I sent a patch but would appreciate
> feedback) ?
> - https://bugs.debian.org/691207 ?
> - https://bugs.debian.org/691209 ?

I'll give it a try.

> 2. offb/ATY,RockHopper: inverts red and blue in bterm
> 
> I have not been able to make progress on
> https://bugs.debian.org/825840. So if anyone is familar with
> ioctl/FBIOPUTCMAP, I'd appreciate a little help here.

I can confirm it's still there. Saw it when testing d-i for my recent
report and was a bit confused. But isn't this rather some
nice-to-have?

> 3. radeonfb should be able to kick out offb
> 
> In case (2) above cannot be solved in time. We can also fix the color
> inversion another way. Technically some PowerMac uses a radeon based
> graphic card, so we could fix the radeonfb module issue so that it is
> properly loaded during d-i.
> 
> So who can help me with:
> 
> https://bugs.debian.org/826629 ?

Sorry, both my boxes are nvidia. Which causes a lot of trouble, too.

Christoph


signature.asc
Description: Digital signature

CPU requirements for powerpc

[Christoph Biedl]

Hello,

just to make sure I got things right: The 74xx/G4 processors are
to be supported by Debian's powerpc architecture?

Background: On my G4 boxes I encounter SIGILL from several packages,
turns out they were built with compiler options for more recent CPUs,
hence the program abort. The buildds don't notice that as they are
already G5 or whatever.

So before filing grim bug reports for nothing, would they be
justified?

Christoph


signature.asc
Description: Digital signature

Re: CPU requirements for powerpc

[Christoph Biedl]

Rick Thomas wrote...

> Can you give us some more details?

In general I'm somewhat reluctant since first conclusions are usually
wrong and I certainly don't want to create noise at the wrong place.
So I'd rather debug a little longer until I can identify the real
cause, and create a patch to prove I was right. So yesterday I would
wrongly have blamed openssl, now it rather looks like pcre3. Which
still might not be true.

But since I'm rather busy today doing other things, here are the bits
if you want to take over:

> 1) Which debian (stable/testing/sid)?
> 2) Which packages?
> 3) Which G4 boxes?

This is Debian stretch, package is syslog-ng 3.7.3-3, box is a
PowerMac G4 running a "7410, altivec supported" CPU.

Rebuild syslog-ng, you should see four errors in the test suite, the
first being

| PASS: lib/transport/tests/test_aux_data
| ../../test-driver: line 107: 77989 Illegal instruction "$@" > $log_file 
2>&1

This is triggered by ./lib/logproto/tests/test_logproto, started
without any parameters.

Using gdb you will encounter SIGILL in the libcrypt initialization,
that's ok, openssl probes the CPU capabilities (see above). Second
time it's something like

| #0  0xb7faf008 in ?? ()
| #1  0x0fc42a38 in ?? () from /lib/powerpc-linux-gnu/libpcre.so.3
| #2  0x0fc67c20 in ?? () from /lib/powerpc-linux-gnu/libpcre.so.3

where disassemble shows different instructions when tried again, it's
either "mflr" or "mr". Next step would be to identify where the actual
instruction comes from and how the code path is triggered. Also big
question why this doesn't happen more often.

The package used to build with 3.7.3-1 back in July, the changes in
syslog-ng are minimal, and nothing obvious in pcre3 (was 2:8.38-3.1,
now it's 2:8.39-2).

Godspeed, and please share any findings so I can focus on other
issues.

Christoph


signature.asc
Description: Digital signature

Re: CPU requirements for powerpc

[Christoph Biedl]

Mathieu Malaterre wrote...

> Don't forget to set the usertags so that we can track those.

#840354 about the syslog-ng issue (which is actually in src:pcre3 as
assumed). I'm not sure if usertags can be set that way, we'll see.

Christoph


signature.asc
Description: Digital signature

Re: Release Architectures for Debian 9 'Stretch'

[Christoph Biedl]

Lennart Sorensen wrote...

> On Mon, Oct 31, 2016 at 09:09:56AM -0700, Herminio Hernandez Jr.  wrote:
> > Is there even a chance for PowerPC to return as a release architecture for 
> > Debian 10 or is that just wishful thinking. 
> 
> Well as pointed out in the meeting, it does not seem any architecture
> has ever done so.

I bet never before an architecture got kicked out that is ranking as
number 4 according to popcon. The question is, are there enough people
that are willing to support powerpc in Debian? The way downhill began
when nobody stepped forward in the role call.

Christoph

Re: Release Architectures for Debian 9 'Stretch'

[Christoph Biedl]

Herminio Hernandez, Jr. wrote...

> On Mon, Oct 31, 2016 at 4:13 PM, Rick Thomas  wrote:
> 
> > Will there be an LTS (long-term-support) release for powerpc — best would
> > be one based on Jessie?
>
> No there is no lts for ppc only i386, amd64, and arm
> 
> https://wiki.debian.org/LTS/

Perhaps Rick's question was about a future jessie-LTS for powerpc.
That would extend powerpc support beyond around spring 2018. I bet the
LTS folks will not object in general. However somebody, more than one
person, should be willing and able to support this.

It's however more than a year until this becomes an issue.

Christoph


signature.asc
Description: Digital signature

Re: Release Architectures for Debian 9 'Stretch'

[Christoph Biedl]

Herminio Hernandez Jr.  wrote...

> I saw that after I asked the question. It just really stinks. 

Keep it down. From an outsider's point of view the decision is more
than understandable. Their interest is only "are there enough people
willing to support powerpc?", and the indications were pretty bad.
Reading the log I cannot see any bias against powerpc in general,
rather the opposite.

Christoph


signature.asc
Description: Digital signature

Re: Release Architectures for Debian 9 'Stretch'

[Christoph Biedl]

luigi burdo wrote...

> Im thinking to start a petition about ... "dont kill debian penguins on 
> powerpc"

Don't. The release team folks are not politicians who get convinced by
a lot of noise. Unless I completely misunderstand their position, the
only question that matters is: Can we (as the Debian project) be sure
powerpc will have backing until EOL stretch (somewhen 2020)?

The right answer is to create trust for this, not whining around.

The best thing that could happen right now was a company approaching
Debian, emphasizing the importance of the powerpc architecture for
their business, and declaring they will hire two Debian Developers who
will contribute to the project as part of their job. But let's not
hope for a miracle.

A remaining idea was was to appeal and ask for a grace peroid. This
appeal requires substantiation: Several people would have to sign
this, people who are well-established contributors to the Debian
project so it is certain they can do the job and their enthusiasm
will last (I wouldn't expect I would qualify for that). And during
that grace period, some two or three weeks, we, the powerpc people,
show significant progress by fixing all the open issues. The fact
#832931 did not get resolved in due course was one of things that hit
the reputation quite badly.

Even if the appeal is unsuccessful on any stage, a clean backlog will
certainly help to have jessie-LTS for powerpc, and re-entry as well.
So please start cracking.

Christoph


signature.asc
Description: Digital signature

Re: Release Architectures for Debian 9 'Stretch'

[Christoph Biedl]

Herminio Hernandez Jr.  wrote...

> To say that no one stepped forward is not true. Adrian stepped up
> and asked to be the porter for PowerPC.

Digging in the past isn't very helpful, but there one thing I'd like
to understand: Nobody responded to the initial role call in August.
Some folks had done this for jessie some two years ago. Who were they?
Are they still around? Did they silently disappear, or did they sign
off somewhat formally?

I was quite shocked to learn so few people are backing this
architecture. I'm afraid I couldn't spend more time on this but real
life was pretty tough in the past weeks, and my job is slowly killing
me.

And personally I'm cut as well - with powerpc I finally saw a niche
where I could do something useful without interference of the Debian
bullies. Some boxes are on their way to me, I had plans to rebuild the
entire archive on G4 CPU systems since appearently there are more
issues that don't show on the G5 buildds - I'm not sure how much of
this I will still do.

Also I consider big-endian architectures crucial: There still is a
tremendous amount of endianness bugs in the code, powerpc helps to
detect them.

Christoph


signature.asc
Description: Digital signature

Re: diffutils FTBFS on ppc64el

[Christoph Biedl]

Aurelien Jarno wrote...

> > FAIL: colors
> > 
> >
> > diff: standard output: Broken pipe
> > FAIL colors (exit status: 1)

This is the related code:

| mkfifo fifo
| printf '%*s-a' 100 > a
| printf '%*s-b' 100 > b
| head -c 10 < fifo > /dev/null &
+ diff --color=always ---presume-output-tty a b > fifo
| test $? = 141 || fail=1

There is a race condition involved, it fails in about three of
four attempts - but not at all if the test is run under strace.

A "sleep 0.1" before the diff made the test pass in ten of ten
attempts. So the reason might indeed be the head command hasn't opened
fifo yet by the time diff tries to write to it. But this shouldn't be
restricted to ppc64el. Someone got a super-fast amd64 to try there?

Christoph


signature.asc
Description: Digital signature

Re: diffutils FTBFS on ppc64el

[Christoph Biedl]

Santiago Vila wrote...

> Are you sure we need a super-fast computer and not a super-slow one?

There is no lack of super-slow computers so this would have been noticed
earlier ... on the other hand, recent ppc64 boxes can be really, really
fast. Where "fast" has many aspects, and there might be something
different on this arch so this race was never seen anywhere else.

> We would need either the diff line to be fast, or alternatively the head line
> to be slow, right?

My wild guess: The fork system call is faster than usual while execve is
slower. Also, the diff program is already in the caches, while head
perhaps not.

It was interesting to learn what actually happens. I could experiment a
bit but I bet this isn't quite covered by DMUP.

> Anyway, I'll forward this upstream and will add an extremely
> conservative "sleep 1".

Thanks,

Christoph


signature.asc
Description: Digital signature

Re: yaboot problem? OS won't load after install on a Power Mac G5

[Christoph Biedl]

ilko Iliev wrote...

> image  =  /boot/  vmlinux
(...)
>  initrd  =  /boot/  initrd  .  img

Do these files actually exist? When setting up two installations during
the weekend, they were missing. This ought to be a bug in yaboot but I
didn't get around to debug this.

So either: Remove the "/boot" part and re-run ybin, or create according
links in /boot/ that point to vmlinux-, same for initrd.

The first is robust, the second easier to test.

Christoph


signature.asc
Description: Digital signature

Performance difference 32bit/64bit userland

[Christoph Biedl]

Hi there,

while preparing other tests I created two installations on two hosts
with identical hardware (LPARs on IBM POWER):

- powerpc (64 bit kernel, 32 bit userland)
- ppc64 (64 bit kernel, 64 bit userland)

Both are up-to-date sid with systemd held to 232-10 (#852811).

Now the surprise: Using the 32 bit userland, CPU bound operations like
gzip or xz are significantly faster (5 to 10 percent). Comparing to x86
where i386 is 10 to 15 percent slower than amd64. Also running
debootstrap showed a similar pattern. All tests were repeated a few
times to rule out any caching or similar effects.

This is not quite satisfying. Anyone an explanation for this?

Christoph


signature.asc
Description: Digital signature

Re: Updated installer images

[Christoph Biedl]

John Paul Adrian Glaubitz wrote...

> Please test and report back on the individual architecture
> mailing lists

So far, the ride for ppc64 has been *extremely* painful. This is not
necessarly due to your efforts, but it feels a lot like nobody ever has
tried to set up Debian on a G5 using netboot.

So far (might be incomplete, and I'm both tired and fairly upset):

* Any reasonable documentation on this anywhere? No about how to set up
  DHCP/TFTP server, I've done this many time. But what about which files
  are needed, and how to provide a netboot-adjusted yaboot.conf, and
  mostly: How to make yaboot make using it?

* The OF bootloader needs two rounds to load yaboot.

* yaboot should either get a decent on-line help or see bitrot.

* yaboot's "conf /path/to/config" command, when initially using netboot,
  happiliy ignores the file name but retrieves 01-xx-yy-xx-yy-xx-yy
  using TFTP instead.

* After a lot of trickery, the installer's vmlinux now gets loaded. At a
  whopping 6 kbyte/sec (yes: six kilobytes). Just to remind you, kernel
  and initrd take some 35 megabytes, and the G5 has already turned to
  airplane mode. My neighbors will love me.

This isn't getting anywhere useful soon.

Christoph


signature.asc
Description: Digital signature

Re: Updated installer images

[Christoph Biedl]

Frank Scheiner wrote...

> Let's continue this on the corresponding mailing list.

Jupp, that's how it was meant to be.

> >* Any reasonable documentation on this anywhere? No about how to set up
> >   DHCP/TFTP server, I've done this many time. But what about which files
> >   are needed, and how to provide a netboot-adjusted yaboot.conf, and
> >   mostly: How to make yaboot make using it?
>
> I believe the Debian Wheezy installation manual gives hints about how to
> configure DHCP and TFTP servers for netboot, see e.g. [1].
>
> [1]: https://www.debian.org/releases/wheezy/powerpc/ch04s05.html.en

While I think I managed that step, just a note: I cannot find the
mentioned tftpboot.img listed anywhere in

The important bit is to ship the yaboot file AFAICT.

> >[...]
> >* After a lot of trickery, the installer's vmlinux now gets loaded. At a
> >   whopping 6 kbyte/sec (yes: six kilobytes). Just to remind you, kernel
> >   and initrd take some 35 megabytes, and the G5 has already turned to
> >   airplane mode. My neighbors will love me.
>
> I don't see that slow loading of kernel and initrd you describe on my
> POWER5, but haven't yet tested this on my G5 Cluster Node.

This is a PowerMac G5 Dual Core (11,2).

> I have to disagree, the ppc64 port is totally useful for all my 64 bit POWER
> and PowerPC gear, after Debian switched to Little Endian and POWER8 as
> minimum for its POWER architecture. :-)

After another long night where I was supposed to sleep I assume the core
reason for the nightmare is the TFTP client, and I'm not sure whether
this is yaboot or the OpenFirmware - I haven't studied the architecture
of yaboot yet.

In either way, not only TFTP is horribly slow, but also any provided
filename to download is ignored.

After understanding this, I eventually managed to get the installer
running by doing the following steps (a bit exhaustive for the sake of
other people running into that problem):

* Prepare a sata hard disk using mac-fdisk[0]: Initialize the partition
  map, Create the "Apple_Bootstrap" partition (as /dev/sdX2), create and
  format a "standard Linux type" partition as /dev/sdX3, ext2 is fine,
  some 128 Mbyte size, copy vmlinux and initrd.gz from the iso image
  (install/powerpc64/) into this. This partition may later be reused as
  /boot. Umount.

  Background: This will avoid having to load these files using TFTP.

  Not checked: Does yaboot understand MBR partitions? That would make
  things easier. However you'd have to purge the entire partition during
  the installation then, taking your chance for a second attempt.

  Not checked: Can yaboot handle usb? It should but it didn't work for
  me. That would allow installation from USB instead from the disk.
* dd the installer image onto a USB flash drive, either native or
  into a partition big enough to hold it.

  Background: The installer needs the CD-ROM but will accept any block
  device that holds the image. However, in that early stage the disk
  controller will not be visible yet. The USB modules will have been
  loaded, though.
* Configure the DHCP server so it tells the G5 to download the yaboot
  file. If it's in the TFTP server's root, basically 'filename "yaboot";'
  in dhcpd.conf does the trick. (The yaboot file is in the install/
  directory of the CD image.)
* Drop the yaboot configuration[1] in the tftproot as 01-xx-xx-xx-xx-xx-xx
  where xx is the G5's MAC, the actually requested file is shown on the
  G5's screen, and also quite visible when snooping using tcpdump.
* Install the hard disk into the G5, plug in the USB drive.
* Start the G5, force the Open Firmware prompt (on a PC keyboard, that's
  Win-Alt-O-F after the chime until your told to release the keys).
* Trigger netboot by "boot enet:" (or "enet0:"?)
  Gotchas: If the OpenFirmware complains about a missing (ethernet)
  link, put an el-cheapo ethernet switch between your G5 and the actual
  switch. If still no network activity is happening (check using tcpdump
  on the DHCP server's side), try "nvram-reset" and "reset-all".
* After a few moments you should get the yaboot prompt.
* My memory is a bit blurry here. Either the yaboot configuration was
  loaded or it has to be done manually. So hit Tab, if the "install"
  label is shown, skip the next step.
* Load the configuation, "conf foo" should do the trick, or any other
  filename as it's ignored anyway.
* Enter "install", and the installation kernel should be loaded.
* From then on, it's the usual Debian installer. The CD-ROM image
  will be found automatically thanks to the preparation above.

The installation itself was smoothless and brought no surprises. The
manual mirror configuration is still required.

The red background during installation is probably still #825840.

After reboot into the installed system:

The framebuffer console is sometimes distorted, especially when
scrolling backwards, requires m

Spurious lzma decompression error on ppc64

[Christoph Biedl]

Hello,

since a few days ago, I get messages like

| Unpacking libsmartcols1:ppc64 (2.31.1-0.4) over (2.30.2-0.3) ...
| dpkg-deb (subprocess): decompressing archive member: lzma error: compressed 
data
|  is corrupt
| dpkg-deb: error:  subprocess returned error exit status 2
| dpkg: error processing archive 
/var/cache/apt/archives/libsmartcols1_2.31.1-0.4_ppc64.deb (--unpack):
|  cannot copy extracted data for 
'./lib/powerpc64-linux-gnu/libsmartcols.so.1.1.0' to 
'/lib/powerpc64-linux-gnu/libsmartcols.so.1.1.0.dpkg-new': unexpected end of 
file or stream
| Errors were encountered while processing:
|  /var/cache/apt/archives/libsmartcols1_2.31.1-0.4_ppc64.deb

There is no obvious pattern in which packages are affected, and just
repeating the upgrade makes the problem go away. So it's rather not a
problem with the package itself. Wisdom of the net suggest to indeed
just redo the installation step, this is just band-aid and feels wrong.

FWIW, the computer benefits from a local apt-cacher-ng but no other
systems (quite a few, several archs and distributions) connected to it
shows that behaviour.

Rechecking xz-utils and liblzma using the md5sums file showed no errors.
So I'm slightly concerned there's something inside that causes these
errors, and the first idea was some kind of memory corruption, due to
hardware or kernel.

Anybody else seeing something similar?

Christoph


signature.asc
Description: PGP signature

Re: logrotate 3.14.0-2 build failure

[Christoph Biedl]

Christian Göttsche wrote...

> the package logrotate in version 3.14.0-2 fails to build on ppc64el
> due to a test suite failure [1].

That looks familiar. Is the patch

debian/patches/fix-test-pagesize.patch

I created for 3.11.0-0.1 still part of your packaging?

And by the way, thanks for taking care of logrotate.

Christoph



signature.asc
Description: PGP signature

Re: Package libnss3_3.60-1_ppc64.deb requires Power-7 hardware

[Christoph Biedl]

Carsten Jacobi wrote...

> The reason is that in one makefile the compiler options "-mcrypto" and
> "-mvsx" were turned. Hence, there are Power-7 instructions now in
> libfreeblpriv3.so:

Ouch. That could also happen when the build system guesses some possible
additional compile options from the CPU the build runs on. Now I am
wondering whether tools like blhc could be used to detect such a
situation. I bet I wouldn't want to know the result of an archive-wide
build log check.

Christoph


signature.asc
Description: PGP signature