[hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Christian Beikov]

I'm moving this discussion from Zulip to the mailing list as Rafael does 
not seem to be on Zulip. Here a little context:

 > Does anyone know if there is a known performance issue with Bytebuddy 
during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes up to 
30% percent longer to boot, even when using a prebuilt Jandex index, 
apparently due to the use of Bytebuddy

 > @Moritz Becker found the issue while trying to find out why our 
Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not with 5.3

 > So AFAIK, the PojoEntityTuplizer builds the proxy class through 
org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy which 
takes significantly longer than the javassist implementation

I haven't tried it yet, but it seems to me that making the 
byteBuddyState in 
org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static 
might help. I'd guess this will help with the test execution time for 
the Hibernate testsuite as well.

Any idea how to improve this @Rafael ?

Regards,

Christian

___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Yoann Rodiere]

I have just one comment:

> I haven't tried it yet, but it seems to me that making the
> byteBuddyState in
> org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static
> might help. I'd guess this will help with the test execution time for
> the Hibernate testsuite as well.

Please don't do this. It was previously static, and that was changed to fix
very real problems:

https://github.com/hibernate/hibernate-orm/commit/d95b36ffb6cea2337d6a0280c86d82912ae585c8

Maybe it's slower than javassist proxying because of that, but it's also
less buggy than javassist proxying because of that?

cc @Guillaume Smet 

Yoann Rodière
Hibernate Team
yo...@hibernate.org


On Tue, 4 Aug 2020 at 17:28, Christian Beikov 
wrote:

> I'm moving this discussion from Zulip to the mailing list as Rafael does
> not seem to be on Zulip. Here a little context:
>
>  > Does anyone know if there is a known performance issue with Bytebuddy
> during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes up to
> 30% percent longer to boot, even when using a prebuilt Jandex index,
> apparently due to the use of Bytebuddy
>
>  > @Moritz Becker found the issue while trying to find out why our
> Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not with 5.3
>
>  > So AFAIK, the PojoEntityTuplizer builds the proxy class through
> org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy which
> takes significantly longer than the javassist implementation
>
> I haven't tried it yet, but it seems to me that making the
> byteBuddyState in
> org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static
> might help. I'd guess this will help with the test execution time for
> the Hibernate testsuite as well.
>
> Any idea how to improve this @Rafael ?
>
> Regards,
>
> Christian
>
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Guillaume Smet]

Hi Christian,

I don't like "apparently" when talking about performances :).

I would advise to use async profiler (
https://github.com/jvm-profiling-tools/async-profiler, also see
https://github.com/quarkusio/quarkus/blob/master/TROUBLESHOOTING.md to
learn more on how to use it) to try to understand what's going wrong and
how we could improve things.

I remember things were very slow with the initial ByteBuddy implementation
Rafael did and we got much better results by caching the ByteBuddy
transformations. In particular the calls to the ByteBuddy DSL are quite
expensive in terms of allocations because they are creating a ton of
objects.

Once we have some profiling results, we can talk.

-- 
Guillaume

On Tue, Aug 4, 2020 at 5:26 PM Christian Beikov 
wrote:

> I'm moving this discussion from Zulip to the mailing list as Rafael does
> not seem to be on Zulip. Here a little context:
>
>  > Does anyone know if there is a known performance issue with Bytebuddy
> during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes up to
> 30% percent longer to boot, even when using a prebuilt Jandex index,
> apparently due to the use of Bytebuddy
>
>  > @Moritz Becker found the issue while trying to find out why our
> Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not with 5.3
>
>  > So AFAIK, the PojoEntityTuplizer builds the proxy class through
> org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy which
> takes significantly longer than the javassist implementation
>
> I haven't tried it yet, but it seems to me that making the
> byteBuddyState in
> org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static
> might help. I'd guess this will help with the test execution time for
> the Hibernate testsuite as well.
>
> Any idea how to improve this @Rafael ?
>
> Regards,
>
> Christian
>
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Christian Beikov]

Hey,

sorry for the confusion. "Apparently" was the wrong word I guess, my 
colleague Moritz Becker ran a few CPU sampling rounds with JFR which is 
how we found out about this, so this is more like "the data suggests" 
that the problem is in ByteBuddy.

I can do some profiling with the async-profiler as well, but I the issue 
definitely is the proxy building code.

@Yoann: It seems the commit you posted is about supporting a 
SecurityManager environment, but I'm not sure how making the state an 
instance variable rather than static helps with this. What kind of 
buggyness are you referring to?

Am 04.08.2020 um 17:56 schrieb Guillaume Smet:
> Hi Christian,
>
> I don't like "apparently" when talking about performances :).
>
> I would advise to use async profiler 
> (https://github.com/jvm-profiling-tools/async-profiler, also see 
> https://github.com/quarkusio/quarkus/blob/master/TROUBLESHOOTING.md to 
> learn more on how to use it) to try to understand what's going wrong 
> and how we could improve things.
>
> I remember things were very slow with the initial ByteBuddy 
> implementation Rafael did and we got much better results by caching 
> the ByteBuddy transformations. In particular the calls to the 
> ByteBuddy DSL are quite expensive in terms of allocations because they 
> are creating a ton of objects.
>
> Once we have some profiling results, we can talk.
>
> -- 
> Guillaume
>
> On Tue, Aug 4, 2020 at 5:26 PM Christian Beikov 
> mailto:christian.bei...@gmail.com>> wrote:
>
> I'm moving this discussion from Zulip to the mailing list as
> Rafael does
> not seem to be on Zulip. Here a little context:
>
>  > Does anyone know if there is a known performance issue with
> Bytebuddy
> during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes
> up to
> 30% percent longer to boot, even when using a prebuilt Jandex index,
> apparently due to the use of Bytebuddy
>
>  > @Moritz Becker found the issue while trying to find out why our
> Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not
> with 5.3
>
>  > So AFAIK, the PojoEntityTuplizer builds the proxy class through
> org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy
> which
> takes significantly longer than the javassist implementation
>
> I haven't tried it yet, but it seems to me that making the
> byteBuddyState in
> org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static
> might help. I'd guess this will help with the test execution time for
> the Hibernate testsuite as well.
>
> Any idea how to improve this @Rafael ?
>
> Regards,
>
> Christian
>
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org 
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Yoann Rodiere]

> @Yoann: It seems the commit you posted is about supporting a
> SecurityManager environment, but I'm not sure how making the state an
> instance variable rather than static helps with this. What kind of
> buggyness are you referring to?

Not sure what the bug was; I just remember there was a problem (multiple
problems?) related to the bytebuddy state being static. Maybe the bytebuddy
state does change from one execution to the other, depending on the
classpath, security manager, or whatever. Maybe it was just a problem when
running tests and changing some classes from one test to another, or
changing the security manager. To be honest I don't remember. Maybe @Guillaume
Smet  remembers, but that was two years ago...

Yoann Rodière
Hibernate Team
yo...@hibernate.org


On Tue, 4 Aug 2020 at 18:17, Christian Beikov 
wrote:

> Hey,
>
> sorry for the confusion. "Apparently" was the wrong word I guess, my
> colleague Moritz Becker ran a few CPU sampling rounds with JFR which is
> how we found out about this, so this is more like "the data suggests"
> that the problem is in ByteBuddy.
>
> I can do some profiling with the async-profiler as well, but I the issue
> definitely is the proxy building code.
>
> @Yoann: It seems the commit you posted is about supporting a
> SecurityManager environment, but I'm not sure how making the state an
> instance variable rather than static helps with this. What kind of
> buggyness are you referring to?
>
> Am 04.08.2020 um 17:56 schrieb Guillaume Smet:
> > Hi Christian,
> >
> > I don't like "apparently" when talking about performances :).
> >
> > I would advise to use async profiler
> > (https://github.com/jvm-profiling-tools/async-profiler, also see
> > https://github.com/quarkusio/quarkus/blob/master/TROUBLESHOOTING.md to
> > learn more on how to use it) to try to understand what's going wrong
> > and how we could improve things.
> >
> > I remember things were very slow with the initial ByteBuddy
> > implementation Rafael did and we got much better results by caching
> > the ByteBuddy transformations. In particular the calls to the
> > ByteBuddy DSL are quite expensive in terms of allocations because they
> > are creating a ton of objects.
> >
> > Once we have some profiling results, we can talk.
> >
> > --
> > Guillaume
> >
> > On Tue, Aug 4, 2020 at 5:26 PM Christian Beikov
> > mailto:christian.bei...@gmail.com>> wrote:
> >
> > I'm moving this discussion from Zulip to the mailing list as
> > Rafael does
> > not seem to be on Zulip. Here a little context:
> >
> >  > Does anyone know if there is a known performance issue with
> > Bytebuddy
> > during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes
> > up to
> > 30% percent longer to boot, even when using a prebuilt Jandex index,
> > apparently due to the use of Bytebuddy
> >
> >  > @Moritz Becker found the issue while trying to find out why our
> > Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not
> > with 5.3
> >
> >  > So AFAIK, the PojoEntityTuplizer builds the proxy class through
> > org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy
> > which
> > takes significantly longer than the javassist implementation
> >
> > I haven't tried it yet, but it seems to me that making the
> > byteBuddyState in
> > org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl static
> > might help. I'd guess this will help with the test execution time for
> > the Hibernate testsuite as well.
> >
> > Any idea how to improve this @Rafael ?
> >
> > Regards,
> >
> > Christian
> >
> > ___
> > hibernate-dev mailing list
> > hibernate-dev@lists.jboss.org 
> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
> >
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev

Re: [hibernate-dev] Bad boot performance since 5.4 switched to ByteBuddy

[Guillaume Smet]

If you can get some SVG profiles from Async Profiler, that would help to
determine what we can do to improve things.

And yes, we don't want the BB state to be static again. We had a hard time
avoiding the static here and I don't remember exactly why but it was for
sure worth it or I wouldn't have spent time on it :).

-- 
Guillaume

On Tue, Aug 4, 2020 at 6:28 PM Yoann Rodiere  wrote:

> > @Yoann: It seems the commit you posted is about supporting a
> > SecurityManager environment, but I'm not sure how making the state an
> > instance variable rather than static helps with this. What kind of
> > buggyness are you referring to?
>
> Not sure what the bug was; I just remember there was a problem (multiple
> problems?) related to the bytebuddy state being static. Maybe the bytebuddy
> state does change from one execution to the other, depending on the
> classpath, security manager, or whatever. Maybe it was just a problem when
> running tests and changing some classes from one test to another, or
> changing the security manager. To be honest I don't remember. Maybe @Guillaume
> Smet  remembers, but that was two years ago...
>
> Yoann Rodière
> Hibernate Team
> yo...@hibernate.org
>
>
> On Tue, 4 Aug 2020 at 18:17, Christian Beikov 
> wrote:
>
>> Hey,
>>
>> sorry for the confusion. "Apparently" was the wrong word I guess, my
>> colleague Moritz Becker ran a few CPU sampling rounds with JFR which is
>> how we found out about this, so this is more like "the data suggests"
>> that the problem is in ByteBuddy.
>>
>> I can do some profiling with the async-profiler as well, but I the issue
>> definitely is the proxy building code.
>>
>> @Yoann: It seems the commit you posted is about supporting a
>> SecurityManager environment, but I'm not sure how making the state an
>> instance variable rather than static helps with this. What kind of
>> buggyness are you referring to?
>>
>> Am 04.08.2020 um 17:56 schrieb Guillaume Smet:
>> > Hi Christian,
>> >
>> > I don't like "apparently" when talking about performances :).
>> >
>> > I would advise to use async profiler
>> > (https://github.com/jvm-profiling-tools/async-profiler, also see
>> > https://github.com/quarkusio/quarkus/blob/master/TROUBLESHOOTING.md to
>> > learn more on how to use it) to try to understand what's going wrong
>> > and how we could improve things.
>> >
>> > I remember things were very slow with the initial ByteBuddy
>> > implementation Rafael did and we got much better results by caching
>> > the ByteBuddy transformations. In particular the calls to the
>> > ByteBuddy DSL are quite expensive in terms of allocations because they
>> > are creating a ton of objects.
>> >
>> > Once we have some profiling results, we can talk.
>> >
>> > --
>> > Guillaume
>> >
>> > On Tue, Aug 4, 2020 at 5:26 PM Christian Beikov
>> > mailto:christian.bei...@gmail.com>> wrote:
>> >
>> > I'm moving this discussion from Zulip to the mailing list as
>> > Rafael does
>> > not seem to be on Zulip. Here a little context:
>> >
>> >  > Does anyone know if there is a known performance issue with
>> > Bytebuddy
>> > during boot in 5.4? I noticed that 5.4 takes about 25%, sometimes
>> > up to
>> > 30% percent longer to boot, even when using a prebuilt Jandex index,
>> > apparently due to the use of Bytebuddy
>> >
>> >  > @Moritz Becker found the issue while trying to find out why our
>> > Blaze-Persistence testsuite runs into CI timeouts with 5.4 but not
>> > with 5.3
>> >
>> >  > So AFAIK, the PojoEntityTuplizer builds the proxy class through
>> > org.hibernate.proxy.pojo.bytebuddy.ByteBuddyProxyHelper#buildProxy
>> > which
>> > takes significantly longer than the javassist implementation
>> >
>> > I haven't tried it yet, but it seems to me that making the
>> > byteBuddyState in
>> > org.hibernate.bytecode.internal.bytebuddy.BytecodeProviderImpl
>> static
>> > might help. I'd guess this will help with the test execution time
>> for
>> > the Hibernate testsuite as well.
>> >
>> > Any idea how to improve this @Rafael ?
>> >
>> > Regards,
>> >
>> > Christian
>> >
>> > ___
>> > hibernate-dev mailing list
>> > hibernate-dev@lists.jboss.org > >
>> > https://lists.jboss.org/mailman/listinfo/hibernate-dev
>> >
>> ___
>> hibernate-dev mailing list
>> hibernate-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev