Hi Warner, Several users have been running their systems with the updated jemalloc for a while now. I've been running my systems, a dual-socket NUMA workstation and a laptop (both x86_64), for over a year without issues.
May I suggest importing the updated jemalloc into 15-CURRENT for wider testing? It's better to have it in the tree for a longer period of time before the 15.0 release. Sorry for the noise and thanks. On Sunday, December 8th, 2024 at 7:41 PM, Warner Losh <i...@bsdimp.com> wrote: > Great! I'll take a look at that, as well as do the merge the typical way > (which only takes a few minutes now that I've bootstrapped things). I'll > compare the two to see what diffs there might be (to act as a cross check for > both methods). I'll then build a copy of the Netflix firmware with the change > and put it on a couple machines and see if they can handle the load and if > there are any performance regressions. I don't expect any, since malloc > typically doesn't appear in the flame graphs as "visible", but you never know. > > So, once that's done, and I expect it to be done this week, I'll push it into > main with both the proper vendor branch merge commits as well as an > acknowledgement for this pull request and your work to move it forward (I'm > just verifying the typical process will produce the same results and the > typical process doesn't take a long time, etc). > > Warner > > On Sun, Dec 8, 2024 at 9:03 AM Minsoo Choo <minsoochoo0...@proton.me> wrote: > >> I resolved merge conflict by rebasing main. What's next? >> >> https://github.com/freebsd/freebsd-src/pull/1337 >> On Sunday, December 1st, 2024 at 2:05 AM, Warner Losh <i...@bsdimp.com> >> wrote: >> >>> (sorry to follow up to my own email and topposting) >>> >>> I got the vendor branch bootstrapped: I created vendor/jemalloc and tagged >>> vendor/jemalloc/5.2.1, >>> and created a merge commit from that branch to main. I had to tweak the >>> FREEBSD-Xlist a little.I've not updated the other two FREEBSD-* files, but >>> the steps were >>> documented in the commit messages (the vendor branch one is especially long >>> and >>> likely should migrate into the developer handbook). FREEBSD-update is >>> basically >>> a shell script to do the same thing that git subtree merge does, though I'm >>> sure some >>> tweaks could help the next time (if there is a next time, jemalloc upstream >>> seems to >>> have slowed way down of late). >>> >>> Next up,I'll create 5.3.0 import on the vendor branch, do the merge and >>> start testing (it will be >>> Minsoo's pull request, rebased, with any conflicts resolved and merge >>> commit recorded). >>> But that will have to be tomorrow or more likely during the work week. I'm >>> too tired tonight >>> to get it right at the moment. >>> >>> And a special thanks to emaste for giving bz the right recipe for doing the >>> subtree merge >>> w/o git subtree for his work on the linux wifi drivers in the tree. >>> >>> Warner >>> >>> On Sat, Nov 30, 2024 at 9:38 PM Warner Losh <i...@bsdimp.com> wrote: >>> >>>> Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try to >>>> 'bootstrap' the vendor branch. >>>> Then I need to bootstrap it... >>>> >>>> I just did the same with edk2 (which had a vendor branch, but hadn't been >>>> updated since svn times). >>>> However, jemalloc doesn't have a vendor branch yet, so I'll have to create >>>> that, but I'll start with the >>>> current version rather than doing full history... So I'll start there. >>>> I also just did awk and lua, so once I have things bootstrapped, I'll be >>>> able to add 5.3.0 and then layer Minsoo's on >>>> top of that and then start testing it somehow. >>>> >>>> Malloc makes me nervous to touch, honestly, but I'll give it a go and test >>>> boot on my system and >>>> maybe see if we can survive a workload at work w/o regressions... But I >>>> can't do a full test with lots >>>> of machines until after the first of the year (though I can do a couple >>>> for a few days before then). >>>> >>>> So my next step is to bootstrap the vendor branch... I'll give that a try >>>> tonight. >>>> >>>> Warner >>>> >>>> On Sat, Nov 30, 2024 at 8:26 PM Minsoo Choo <minsoochoo0...@proton.me> >>>> wrote: >>>> >>>>> I have already submitted PR on github >>>>> (https://github.com/freebsd/freebsd-src/pull/1337) and phabricator >>>>> (https://reviews.freebsd.org/D41421). I don't have access (commit bit) to >>>>> freebsd git repo, so there is nothing I can do at this point since vendor >>>>> import and landing patches requires commit bit. >>>>> On Saturday, November 30th, 2024 at 1:42 PM, cglogic >>>>> <cglo...@protonmail.com> wrote: >>>>> >>>>>> I see, it happens. >>>>>> Maybe another committer will volunteer to do the update. >>>>>> I hope it will make its way into 15.0 release. >>>>>> >>>>>> Thanks. >>>>>> On Friday, November 29th, 2024 at 9:38 PM, Warner Losh <i...@bsdimp.com> >>>>>> wrote: >>>>>> >>>>>>> I've been swamped. we need to bootstrap the vendor branch, and the way >>>>>>> prior updates were done >>>>>>> isn't so great. >>>>>>> >>>>>>> Warner >>>>>>> >>>>>>> On Mon, Nov 25, 2024 at 2:21 AM cglogic <cglo...@protonmail.com> wrote: >>>>>>> >>>>>>>> Hello guys, >>>>>>>> >>>>>>>> How the update of jemalloc is going? It's November now. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo >>>>>>>> <minsoochoo0...@proton.me> wrote: >>>>>>>> >>>>>>>>> First, sorry for late response. >>>>>>>>> >>>>>>>>> cglogic, thank you for bringing up this issue again since I nearly >>>>>>>>> forgot that this issue was still open. >>>>>>>>> >>>>>>>>> Warner, as I can't access to my FreeBSD instance until the end of >>>>>>>>> August, but I can still edit and push the code through my Arm Mac. >>>>>>>>> This means that I can't test the updated code on my machine, but I >>>>>>>>> can join the review process and listen to change proposals. >>>>>>>>> >>>>>>>>> I'll open a Github PR in a few hours. (The phabricator review will >>>>>>>>> stay opened just in case) >>>>>>>>> On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh <i...@bsdimp.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> On Sun, Jul 21, 2024 at 2:03 PM cglogic <cglo...@protonmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh >>>>>>>>>>> <i...@bsdimp.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> On Sat, Jul 20, 2024 at 1:59 AM cglogic <cglo...@protonmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello FreeBSD community, >>>>>>>>>>>>> >>>>>>>>>>>>> After Jason Evans stepped aside from maintaining jemalloc in >>>>>>>>>>>>> FreeBSD, it's not updating in time anymore. >>>>>>>>>>>>> Version 5.3.0 was released May 6, 2022 and FreeBSD still not >>>>>>>>>>>>> imported it into the tree. >>>>>>>>>>>>> >>>>>>>>>>>>> There is a pending review https://reviews.freebsd.org/D41421 from >>>>>>>>>>>>> Aug 11, 2023. >>>>>>>>>>>>> I'm successfully running FreeBSD/amd64 system with D41421 applied >>>>>>>>>>>>> for 8 months, as well as many other people. >>>>>>>>>>>>> >>>>>>>>>>>>> Can it be reviewed and committed to CURRENT? >>>>>>>>>>>>> Or, if there is no committers willing to do it, can commit bit be >>>>>>>>>>>>> given to submitter or another person willing to do this? >>>>>>>>>>>>> >>>>>>>>>>>>> It's very disappointing when users spend their time to fill such >>>>>>>>>>>>> gaps and their efforts just ignored by the developers. >>>>>>>>>>>>> >>>>>>>>>>>>> Every year FreeBSD Community Survey asking about user experience >>>>>>>>>>>>> in contributing to FreeBSD. >>>>>>>>>>>>> >>>>>>>>>>>>> Here you can see an example of such contributing. >>>>>>>>>>>> >>>>>>>>>>>> First, thank you for being persistent and continuing to bring it >>>>>>>>>>>> up. It's important to do that to make sure this (and your many >>>>>>>>>>>> other) contribution doesn't fall on the floor. >>>>>>>>>>>> >>>>>>>>>>>> And to be fair, we're only 3 months since the last update. Still, >>>>>>>>>>>> quite a bit longer than you should have to wait, but not nearly >>>>>>>>>>>> the year the original date suggests. >>>>>>>>>>>> >>>>>>>>>>>> And this is a perfect storm of "how the project is bad at >>>>>>>>>>>> accepting contributions": >>>>>>>>>>>> (1) The original submission was close to the 14 branch creation >>>>>>>>>>>> time. This meant that we weren't well prepared to look at it since >>>>>>>>>>>> it is such an invasive change (at least on its surface). It also >>>>>>>>>>>> slowed the initial response... >>>>>>>>>>>> (2) There was a number of back and forth requests for changes, >>>>>>>>>>>> which took time to sort out... >>>>>>>>>>>> (3) The size of this is huge, well beyond the capacity of >>>>>>>>>>>> Phabricator to review accurately... >>>>>>>>>>>> (4) It's a vendor import. That means we can't just drop the >>>>>>>>>>>> Phabricator review into the tree... >>>>>>>>>>>> (5) It's phabricator: this is a great tool for developers, but we >>>>>>>>>>>> have a terrible track record of using it for intake from new >>>>>>>>>>>> contributors. We don't have any oversight at all over this tool, >>>>>>>>>>>> at there's at best tepid and luke warm attempts to look for drop >>>>>>>>>>>> balls. >>>>>>>>>>>> >>>>>>>>>>>> All of these things are a terrible experience. I can only >>>>>>>>>>>> apologize. These days, we might steer this towards github, but the >>>>>>>>>>>> 'vendor import' means you really need someone on the inside, or >>>>>>>>>>>> you need to be on the inside to make that work. >>>>>>>>>>>> >>>>>>>>>>>> So, how to move forward? Well, I'd like to propose the following: >>>>>>>>>>>> (1) submit all the other Phabricator reviews you have open (they >>>>>>>>>>>> are mostly good, or close to good) to github. Github is being >>>>>>>>>>>> actively managed and will make it faster to get things it. It's a >>>>>>>>>>>> much better tool for new contributors (and even frequent >>>>>>>>>>>> contributors of smallish things). >>>>>>>>>>>> (2) I should do an vendor import of 5.3.0 from github, and do the >>>>>>>>>>>> merge to a branch and push that to github. You can then layer on >>>>>>>>>>>> your changes and those can be reviewed more closely as a pull >>>>>>>>>>>> request against the branch I push. I suspect that most of the >>>>>>>>>>>> issues are sorted out already >>>>>>>>>>>> (3) I'll land it via that route... >>>>>>>>>>>> >>>>>>>>>>>> And, if the sum of the other pull requests and this are good (and >>>>>>>>>>>> I suspect they will be), then we can talk about commit bits and >>>>>>>>>>>> such. >>>>>>>>>>>> >>>>>>>>>>>> It's experiences like this which is why I'm trying to stand up >>>>>>>>>>>> github pull requests as a reliable way to get things and and the >>>>>>>>>>>> best place to send people... >>>>>>>>>>>> >>>>>>>>>>>> Thanks again for persisting, and also for expressing this >>>>>>>>>>>> criticism that we (hopefully) can use to make it better. >>>>>>>>>>>> >>>>>>>>>>>> Warner >>>>>>>>>>> >>>>>>>>>>> Hello. >>>>>>>>>>> >>>>>>>>>>> I'm not the author of D41421. Just applied the patch to test it 8 >>>>>>>>>>> months ago. And recently discovered that it's still not committed. >>>>>>>>>>> I can't copy your message to Phabricator because don't have an >>>>>>>>>>> account. Please, if you have time, help the author in D41421. >>>>>>>>>> >>>>>>>>>> Ah yes. I've been in touch with the author for other things, and >>>>>>>>>> somehow thought it was you.... I'll reach out to him via other >>>>>>>>>> means... >>>>>>>>>> >>>>>>>>>> Warner
