On Tuesday, December 1, 2020 5:16:51 AM PST Bulent Abali wrote: > I don't know anything about VAS page size requirements in the kernel. I > checked the user compression library and saw that we do a sysconf to > get the page size; so the library should be immune to page size by > design. But it wouldn't surprise me if a 64KB constant is inadvertently > hardcoded somewhere else in the library. Giving heads up to Tulio and > Raphael who are owners of the github repo. > > https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922 > > If we got this wrong in the library it might manifest itself as an error > message of the sort "excessive page faults". The library must touch > pages ahead to make them present in the memory; occasional page faults > is acceptable. It will retry.
Hm, good to know. As I said I haven't noticed any problems so far, over a few different days of testing. My change is now in the Void Linux kernel package, and is working for others as well (including the Void maintainer Daniel/q66 who I CC'd initially). > > Bulent > > > > > From: "Sukadev Bhattiprolu" <suka...@linux.ibm.com> > To: "Christophe Leroy" <christophe.le...@csgroup.eu> > Cc: "Will Springer" <skirmis...@protonmail.com>, > linuxppc-dev@lists.ozlabs.org, dan...@octaforge.org, Bulent > Abali/Watson/IBM@IBM, ha...@linux.ibm.com Date: 12/01/2020 12:53 > AM > Subject: Re: CONFIG_PPC_VAS depends on 64k pages...? > > Christophe Leroy [christophe.le...@csgroup.eu] wrote: > > Hi, > > > > Le 19/11/2020 à 11:58, Will Springer a écrit : > > > I learned about the POWER9 gzip accelerator a few months ago when > > > the > > > support hit upstream Linux 5.8. However, for some reason the Kconfig > > > dictates that VAS depends on a 64k page size, which is problematic > > > as I > > > run Void Linux, which uses a 4k-page kernel. > > > > > > Some early poking by others indicated there wasn't an obvious page > > > size > > > dependency in the code, and suggested I try modifying the config to > > > switch it on. I did so, but was stopped by a minor complaint of an > > > "unexpected DT configuration" by the VAS code. I wasn't equipped to > > > figure out exactly what this meant, even after finding the > > > offending condition, so after writing a very drawn-out forum post > > > asking for help, I dropped the subject. > > > > > > Fast forward to today, when I was reminded of the whole thing again, > > > and decided to debug a bit further. Apparently the VAS platform > > > device (derived from the DT node) has 5 resources on my 4k kernel, > > > instead of 4 (which evidently works for others who have had success > > > on 64k kernels). I have no idea what this means in practice (I > > > don't know how to introspect it), but after making a tiny patch[1], > > > everything came up smoothly and I was doing blazing-fast gzip > > > (de)compression in no time. > > > > > > Everything seems to work fine on 4k pages. So, what's up? Are there > > > pitfalls lurking around that I've yet to stumble over? More > > > reasonably, > > > I'm curious as to why the feature supposedly depends on 64k pages, > > > or if there's anything else I should be concerned about. > > Will, > > The reason I put in that config check is because we were only able to > test 64K pages at that point. > > It is interesting that it is working for you. Following code in skiboot > https://github.com/open-power/skiboot/blob/master/hw/vas.cshould > restrict it to 64K pages. IIRC there is also a corresponding change in > some NX registers that should also be configured to allow 4K pages. Huh, that is interesting indeed. As far as the kernel code, the only thing specific to 64k pages I could find was in [1], where VAS_XLATE_LPCR_PAGE_SIZE is set. There is also NX_PAGE_SIZE in drivers/ crypto/nx/nx.h, which is set to 4096, but I don't know if that's related to kernel page size at all. Without a better idea of the code base, I didn't examine more thoroughly. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/powernv/vas-window.c#n293 > static int init_north_ctl(struct proc_chip *chip) > { > uint64_t val = 0ULL; > > val = SETFIELD(VAS_64K_MODE_MASK, val, > true); val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true); val = > SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true); val = > SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true); val = > SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true); > > return vas_scom_write(chip, > VAS_MISC_N_CTL, val); } > > I am copying Bulent Albali and Haren Myneni who have been working with > VAS/NX for their thoughts/experience. Thanks for this and for your input, by the way. > > > Maybe ask Sukadev who did the implementation and is maintaining it ? > > > > > I do have to say I'm quite satisfied with the results of the NX > > > accelerator, though. Being able to shuffle data to a RaptorCS box > > > over gigE and get compressed data back faster than most software > > > gzip could ever hope to achieve is no small feat, let alone the > > > instantaneous results locally.> > > > > :) > > > > > > Cheers, > > > Will Springer [she/her] > > > > > > [1]: > > > https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch > > Christophe Will [she/her]
