Re: [PATCH] drop the deprecated malloc/free hooks in hurd/mach-defpager
Hey :)
Quoting Thomas Schwinge (2016-05-16 18:49:47)
> Hi!
>
> On Tue, 29 Dec 2015 23:09:54 +0100, Flavio Cruz wrote:
> > mach-defpager: Drop the deprecated malloc/free hooks.
>
> (After approval by Samuel, this became commit
> 8c49801c8f7e3f800cabedf8fca8ccec3cf35a22.)
>
> > * mach-defpager/kalloc.c: Define malloc and free directly since those are
> > weak
> > symbols.
>
> The idea here is to just change the mechanism (override glibc's weak
> symbols instead of using glibc's deprecated malloc hooks), while not
> changing any semantics, right?
>
> From a quick look, I have some doubts about this generally (so, not only
> related to your change).
>
> > --- a/mach-defpager/kalloc.c
> > +++ b/mach-defpager/kalloc.c
>
> > -static void *
> > -malloc_hook (size_t size, const void *caller)
> > +void *
> > +malloc (size_t size)
> > {
> >return (void *) kalloc ((vm_size_t) size);
> > }
> >
> > -static void
> > -free_hook (void *ptr, const void *caller)
> > +void
> > +free (void *ptr)
> > {
> >/* Just ignore harmless attempts at cleanliness. */
> >/* panic("free not implemented"); */
Uh, we actually don't free memory here...
> So, here we now only provide the malloc and free symbols. What happens
> if, for example, calloc, realloc, posix_memalign et al. are called?
>
> For example, looking at [glibc]/malloc/malloc.c:__libc_calloc (weak alias
> for calloc). Before your change, it called mach-defpager's malloc_hook,
> and then zeroed that region. After your change, it is implemented using
> the real glibc calloc -- which actually does change the semantics in an
> unintended way?
>
> Backing up one step: what is actually being done here in mach-defpager?
> Again from just a quick look, is the idea to use wired memory for all of
> mach-defpager's process memory?
>
> So, for example, if now some mach-defpager code calls calloc, it will now
> get non-wired "glibc" memory instead of wired "kalloc" memory. The
> problem here is not mach-defpager's own code (which we can easily
> verify/control), but the problems is the libraries it links against,
> which currently are: libihash, libpthread, glibc itself, and any
And indeed libihash uses calloc.
> dependencies these may have. Can we be sure these are not internally
> using calloc, for example?
>
> Now, this is not a new problem that you introduced ;-) -- previously we
> also didn't provide malloc's memalign and realloc hooks, and realloc is
> used quite commonly, I suppose?
>
> If my theories are correct, do we need to use a different mechanism to
> get the whole mach-defpager process into wired memory?
As Richard said in #hurd, implement mlockall and MCL_FUTURE and just
use the default allocator.
I'd suggest reverting that change, or is the removal of that
deprecated interface imminent?
Cheers,
Justus
Re: Build times of hurd and glibc for different versions of gnumach, hurd and glibc
On Thu, 2016-05-05 at 19:05 +0200, Richard Braun wrote: > On Wed, Apr 20, 2016 at 11:40:39AM +0200, Richard Braun wrote: > > > > I wrote an incomplete vm_map_enter in X15 [1] that should be helpful. > > In addition to a free hint pointer, it uses the red-black tree, making > > all allocations at worst O(n). If anyone feels like doing that little > > task, enjoy yourself. Otherwise, I'll probably do it, when I have > > some time, as usual. > I've written some code to back virtual allocations with a red-black > tree [1]. The code includes the removal of flow control in the pageout > daemon because I was hitting swap deadlocks way too often. > > Feedbacks are welcome. Updated table attached. Good news, the latest gnumach version, with the page- cache-policy patches, including the pageout deadlock patch, by Richard is even faster than the reverted one :) build-times.pdf Description: Adobe PDF document
Re: Build times of hurd and glibc for different versions of gnumach, hurd and glibc
Damned: The latest pdf file was shown in xpdf, still not in the attachment when sent! Re-attching the latest version. On Wed, 2016-05-18 at 15:05 +0200, Svante Signell wrote: > On Thu, 2016-05-05 at 19:05 +0200, Richard Braun wrote: > > > > On Wed, Apr 20, 2016 at 11:40:39AM +0200, Richard Braun wrote: > > > > > > > > > I wrote an incomplete vm_map_enter in X15 [1] that should be helpful. > > > In addition to a free hint pointer, it uses the red-black tree, making > > > all allocations at worst O(n). If anyone feels like doing that little > > > task, enjoy yourself. Otherwise, I'll probably do it, when I have > > > some time, as usual. > > I've written some code to back virtual allocations with a red-black > > tree [1]. The code includes the removal of flow control in the pageout > > daemon because I was hitting swap deadlocks way too often. > > > > Feedbacks are welcome. > Updated table attached. Good news, the latest gnumach version, with the page- > cache-policy patches, including the pageout deadlock patch, by Richard is even > faster than the reverted one :) Add to that the small difference in build times when qemu runs with cache=none :43m42s vs cache=writeback: 39m10s, a more realistic case for people with real hardware :) build-times.pdf Description: Adobe PDF document
GNU Hurd 0.8, GNU Mach 1.7, GNU MIG 1.7 released
e Mach 3.0 Interface Generator (MIG). This tool translates Remote Procedure Call (RPC) definition files to C code, and is required to compile any packages that are receiving or invoking RPCs, such as GNU Mach, GNU Hurd, and the GNU C Library (glibc) when compiled for the Hurd. More detailed: documentation. | | glibc-2.19-hurd+libpthread-20160518 | | Based on the glibc 2.19 release, we include additional changes for GNU Hurd support, and bundle GNU Hurd's libpthread. | | Snapshot tarballs may be downloaded from ftp://alpha.gnu.org/gnu/hurd/, http://alpha.gnu.org/gnu/hurd/, or checked out of Git, http://git.savannah.gnu.org/cgit/hurd/glibc.git and http://git.savannah.gnu.org/cgit/hurd/libpthread.git. SHA1 checksums: | | 3722b7f52aac89c66f064e1e6d19ec9b92ffc9e0 glibc-2.19-hurd+libpthread-20160518.tar.bz2 | 69dfe1297013056b4b0d6436a1b3906c1bb67a52 glibc-2.19-hurd+libpthread-20160518.tar.bz2.sig | 2795bacc85c799a028577089c422ff4ef9b876bc glibc-2.19-hurd+libpthread-20160518.tar.gz | a7af1d2500f35413003f08801e258f3666dfcb87 glibc-2.19-hurd+libpthread-20160518.tar.gz.sig | | The GNU C Library (glibc) implements a system's standard library functions (as described by ISO C, and POSIX, for example). An important part of the Hurd actually resides in glibc: here, the system interfaces are implemented on top of the Hurd IPC protocols. This is different to the Linux port, where most simple system interfaces are in fact simply forwarded to/implemented as system calls. | | Many thanks to all the people who are helping! | | If you want to give the Hurd a try, you may easily do so with Debian GNU/Hurd. | | The GNU Hurd system currently runs on 32-bit x86 machines. To compile the Hurd, you need a toolchain configured to target i?86-gnu; you cannot use a toolchain targeting GNU/Linux. | | Please read the FAQ. Bug reports should be sent to bug-hurd or filed on http://savannah.gnu.org/bugs/?group=hurd. Requests for assistance should be sent to help-hurd or filed on http://savannah.gnu.org/support/?group=hurd. You can also find us on the Freenode IRC network in the #hurd channel. For the maintainers Thomas signature.asc Description: PGP signature
