(sorry to follow up to my own email and topposting)
I got the vendor branch bootstrapped: I created vendor/jemalloc
and tagged vendor/jemalloc/5.2.1,
and created a merge commit from that branch to main. I had to
tweak the
FREEBSD-Xlist a little.I've not updated the other two FREEBSD-*
files, but the steps were
documented in the commit messages (the vendor branch one is
especially long and
likely should migrate into the developer handbook).
FREEBSD-update is basically
a shell script to do the same thing that git subtree merge does,
though I'm sure some
tweaks could help the next time (if there is a next time,
jemalloc upstream seems to
have slowed way down of late).
Next up,I'll create 5.3.0 import on the vendor branch, do the
merge and start testing (it will be
Minsoo's pull request, rebased, with any conflicts resolved and
merge commit recorded).
But that will have to be tomorrow or more likely during the work
week. I'm too tired tonight
to get it right at the moment.
And a special thanks to emaste for giving bz the right recipe for
doing the subtree merge
w/o git subtree for his work on the linux wifi drivers in the tree.
Warner
On Sat, Nov 30, 2024 at 9:38 PM Warner Losh <i...@bsdimp.com> wrote:
Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try
to 'bootstrap' the vendor branch.
Then I need to bootstrap it...
I just did the same with edk2 (which had a vendor branch, but
hadn't been updated since svn times).
However, jemalloc doesn't have a vendor branch yet, so I'll
have to create that, but I'll start with the
current version rather than doing full history... So I'll
start there.
I also just did awk and lua, so once I have things
bootstrapped, I'll be able to add 5.3.0 and then layer
Minsoo's on
top of that and then start testing it somehow.
Malloc makes me nervous to touch, honestly, but I'll give it
a go and test boot on my system and
maybe see if we can survive a workload at work w/o
regressions... But I can't do a full test with lots
of machines until after the first of the year (though I can
do a couple for a few days before then).
So my next step is to bootstrap the vendor branch... I'll
give that a try tonight.
Warner
On Sat, Nov 30, 2024 at 8:26 PM Minsoo Choo
<minsoochoo0...@proton.me> wrote:
I have already submitted PR on github
(https://github.com/freebsd/freebsd-src/pull/1337) and
phabricator (https://reviews.freebsd.org/D41421). I don't
have access (commit bit) to freebsd git repo, so there is
nothing I can do at this point since vendor import and
landing patches requires commit bit.
On Saturday, November 30th, 2024 at 1:42 PM, cglogic
<cglo...@protonmail.com> wrote:
I see, it happens.
Maybe another committer will volunteer to do the update.
I hope it will make its way into 15.0 release.
Thanks.
On Friday, November 29th, 2024 at 9:38 PM, Warner Losh
<i...@bsdimp.com> wrote:
I've been swamped. we need to bootstrap the vendor
branch, and the way prior updates were done
isn't so great.
Warner
On Mon, Nov 25, 2024 at 2:21 AM cglogic
<cglo...@protonmail.com> wrote:
Hello guys,
How the update of jemalloc is going? It's November now.
Thanks.
On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo
<minsoochoo0...@proton.me> wrote:
First, sorry for late response.
cglogic, thank you for bringing up this issue
again since I nearly forgot that this issue was
still open.
Warner, as I can't access to my FreeBSD instance
until the end of August, but I can still edit and
push the code through my Arm Mac. This means that
I can't test the updated code on my machine, but I
can join the review process and listen to change
proposals.
I'll open a Github PR in a few hours. (The
phabricator review will stay opened just in case)
On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh
<i...@bsdimp.com> wrote:
On Sun, Jul 21, 2024 at 2:03 PM cglogic
<cglo...@protonmail.com> wrote:
On Sunday, July 21st, 2024 at 6:54 AM, Warner
Losh <i...@bsdimp.com> wrote:
On Sat, Jul 20, 2024 at 1:59 AM cglogic
<cglo...@protonmail.com> wrote:
Hello FreeBSD community,
After Jason Evans stepped aside from
maintaining jemalloc in FreeBSD, it's
not updating in time anymore.
Version 5.3.0 was released May 6, 2022
and FreeBSD still not imported it into
the tree.
There is a pending review
https://reviews.freebsd.org/D41421 from
Aug 11, 2023.
I'm successfully running FreeBSD/amd64
system with D41421 applied for 8 months,
as well as many other people.
Can it be reviewed and committed to CURRENT?
Or, if there is no committers willing to
do it, can commit bit be given to
submitter or another person willing to
do this?
It's very disappointing when users spend
their time to fill such gaps and their
efforts just ignored by the developers.
Every year FreeBSD Community Survey
asking about user experience in
contributing to FreeBSD.
Here you can see an example of such
contributing.
First, thank you for being persistent and
continuing to bring it up. It's important to
do that to make sure this (and your many
other) contribution doesn't fall on the floor.
And to be fair, we're only 3 months since
the last update. Still, quite a bit longer
than you should have to wait, but not nearly
the year the original date suggests.
And this is a perfect storm of "how the
project is bad at accepting contributions":
(1) The original submission was close to the
14 branch creation time. This meant that we
weren't well prepared to look at it since it
is such an invasive change (at least on its
surface). It also slowed the initial response...
(2) There was a number of back and forth
requests for changes, which took time to
sort out...
(3) The size of this is huge, well beyond
the capacity of Phabricator to review
accurately...
(4) It's a vendor import. That means we
can't just drop the Phabricator review into
the tree...
(5) It's phabricator: this is a great tool
for developers, but we have a terrible track
record of using it for intake from new
contributors. We don't have any oversight at
all over this tool, at there's at best tepid
and luke warm attempts to look for drop balls.
All of these things are a terrible
experience. I can only apologize. These
days, we might steer this towards github,
but the 'vendor import' means you really
need someone on the inside, or you need to
be on the inside to make that work.
So, how to move forward? Well, I'd like to
propose the following:
(1) submit all the other Phabricator reviews
you have open (they are mostly good, or
close to good) to github. Github is being
actively managed and will make it faster to
get things it. It's a much better tool for
new contributors (and even frequent
contributors of smallish things).
(2) I should do an vendor import of 5.3.0
from github, and do the merge to a branch
and push that to github. You can then layer
on your changes and those can be reviewed
more closely as a pull request against the
branch I push. I suspect that most of the
issues are sorted out already
(3) I'll land it via that route...
And, if the sum of the other pull requests
and this are good (and I suspect they will
be), then we can talk about commit bits and
such.
It's experiences like this which is why I'm
trying to stand up github pull requests as a
reliable way to get things and and the best
place to send people...
Thanks again for persisting, and also for
expressing this criticism that we
(hopefully) can use to make it better.
Warner
Hello.
I'm not the author of D41421. Just applied
the patch to test it 8 months ago. And
recently discovered that it's still not
committed.
I can't copy your message to Phabricator
because don't have an account. Please, if you
have time, help the author in D41421.
Ah yes. I've been in touch with the author for
other things, and somehow thought it was you....
I'll reach out to him via other means...
Warner
Is this still in progress?
I remember that we had an issue with varnish on
ubuntu where varnish would take way more memory
than configured, with a cache of 512MB, we did
consume almost 12GB of memory. The internet seems
to blame this on the jemalloc version comming with
ubuntu. I had the change to let the traffic go by
a FreeBSD machine with varnish installed, i saw
the same thing happening on FreeBSD also, i did
not have the time to investigate, but now that i
read this thread, it seems FreeBSD also uses 5.2.1
jemalloc. The same version Ubuntu had.
This update to 5.3.0 fixed it on ubuntu, so i gues
this would fix the varnish problem on FreeBSD also.
regards
Johan
