Re: [hibernate-dev] SessionEventsListener feature (HHH-8654)
Hi,
This sounds very promising.
Regarding the suggested type names, I'd personally prefer SessionEventListener
(without the plural "s") and something like BaseSessionEventListener
instead of EmptySessionEventsListener, as "empty" implies a specific
behavior which a sub-class would not satisfy when overriding methods.
--Gunnar
2013/11/13 Steve Ebersole
> I wanted to highlight a new feature in 4.3 as it came about from
> performance testing efforts. Its a way to hopefully help track down
> potential performance problems in applications that use Hibernate. In
> this way it is similar to statistics, but it operates per-Session
> (though certainly custom impls could role the metrics up to a SF level).
>
> It revolves around the SessionEventsListener[1] interface which
> essentially defines a number of start/end pairs for the interesting
> events (for example starting to prepare a JDBC statement and ending that
> preparation).
>
> Multiple SessionEventsListener instances can be associated with the
> Session simultaneously. You can add them programatically to a Session
> using Session#addEventsListeners(SessionEventsListener...) method. They
> can also be added to the Session up-front via the
> SessionFactory#withOptions API for building Sessions.
>
> Additionally there are 2 settings that allow SessionEventsListener impls
> to be applied to all Sessions created:
>
> * 'hibernate.session.events.auto' allows you to name any arbitrary
> SessionEventsListener class to apply to all Sessions.
> * 'hibernate.session.events.log' refers to a particular built-in
> implementation of SessionEventsListener that applies some timings across
> the start/end pairs
> (org.hibernate.engine.internal.LoggingSessionEventsListener). In fact
> this listener is added by default if (a) stats are enabled and (b) the
> log level (currently INFO) of LoggingSessionEventsListener is enabled.
> Below[2] is some sample output of LoggingSessionEventsListener.
>
> There is also a org.hibernate.EmptySessionEventsListener (no-op) class
> to help develop custom ones.
>
> Anyway, as much as anything I wanted to point it out so people can try
> it out and to get feedback. I think the API covers most of the
> interesting events. If you feel there are any missing, lets discuss
> here or on a Jira issue.
>
>
> [1] https://gist.github.com/sebersole/7438250
>
> [2]
> 14:40:20,017 INFO LoggingSessionEventsListener:275 - Session Metrics {
> 9762 nanoseconds spent acquiring 1 JDBC connections;
> 0 nanoseconds spent releasing 0 JDBC connections;
> 1020726 nanoseconds spent preparing 4 JDBC statements;
> 1442351 nanoseconds spent executing 4 JDBC statements;
> 0 nanoseconds spent executing 0 JDBC batches;
> 0 nanoseconds spent executing 0 L2C puts;
> 0 nanoseconds spent executing 0 L2C hits;
> 0 nanoseconds spent executing 0 L2C misses;
> 2766689 nanoseconds spent executing 1 flushes (flushing a total of
> 3 entities and 1 collections);
> 1096552384585007 nanoseconds spent executing 2 partial-flushes
> (flushing a total of 3 entities and 3 collections)
> }
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] SessionEventsListener feature (HHH-8654)
On 14 November 2013 11:36, Gunnar Morling wrote:
> Hi,
>
> This sounds very promising.
>
> Regarding the suggested type names, I'd personally prefer SessionEventListener
> (without the plural "s") and something like BaseSessionEventListener
> instead of EmptySessionEventsListener, as "empty" implies a specific
> behavior which a sub-class would not satisfy when overriding methods.
+1
+1
Sanne
>
> --Gunnar
>
>
>
>
> 2013/11/13 Steve Ebersole
>
>> I wanted to highlight a new feature in 4.3 as it came about from
>> performance testing efforts. Its a way to hopefully help track down
>> potential performance problems in applications that use Hibernate. In
>> this way it is similar to statistics, but it operates per-Session
>> (though certainly custom impls could role the metrics up to a SF level).
>>
>> It revolves around the SessionEventsListener[1] interface which
>> essentially defines a number of start/end pairs for the interesting
>> events (for example starting to prepare a JDBC statement and ending that
>> preparation).
>>
>> Multiple SessionEventsListener instances can be associated with the
>> Session simultaneously. You can add them programatically to a Session
>> using Session#addEventsListeners(SessionEventsListener...) method. They
>> can also be added to the Session up-front via the
>> SessionFactory#withOptions API for building Sessions.
>>
>> Additionally there are 2 settings that allow SessionEventsListener impls
>> to be applied to all Sessions created:
>>
>> * 'hibernate.session.events.auto' allows you to name any arbitrary
>> SessionEventsListener class to apply to all Sessions.
>> * 'hibernate.session.events.log' refers to a particular built-in
>> implementation of SessionEventsListener that applies some timings across
>> the start/end pairs
>> (org.hibernate.engine.internal.LoggingSessionEventsListener). In fact
>> this listener is added by default if (a) stats are enabled and (b) the
>> log level (currently INFO) of LoggingSessionEventsListener is enabled.
>> Below[2] is some sample output of LoggingSessionEventsListener.
>>
>> There is also a org.hibernate.EmptySessionEventsListener (no-op) class
>> to help develop custom ones.
>>
>> Anyway, as much as anything I wanted to point it out so people can try
>> it out and to get feedback. I think the API covers most of the
>> interesting events. If you feel there are any missing, lets discuss
>> here or on a Jira issue.
>>
>>
>> [1] https://gist.github.com/sebersole/7438250
>>
>> [2]
>> 14:40:20,017 INFO LoggingSessionEventsListener:275 - Session Metrics {
>> 9762 nanoseconds spent acquiring 1 JDBC connections;
>> 0 nanoseconds spent releasing 0 JDBC connections;
>> 1020726 nanoseconds spent preparing 4 JDBC statements;
>> 1442351 nanoseconds spent executing 4 JDBC statements;
>> 0 nanoseconds spent executing 0 JDBC batches;
>> 0 nanoseconds spent executing 0 L2C puts;
>> 0 nanoseconds spent executing 0 L2C hits;
>> 0 nanoseconds spent executing 0 L2C misses;
>> 2766689 nanoseconds spent executing 1 flushes (flushing a total of
>> 3 entities and 1 collections);
>> 1096552384585007 nanoseconds spent executing 2 partial-flushes
>> (flushing a total of 3 entities and 3 collections)
>> }
>> ___
>> hibernate-dev mailing list
>> hibernate-dev@lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>>
> ___
> hibernate-dev mailing list
> hibernate-dev@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] [OGM] public repositories and snapshot version
Hi guys, How often is the snapshot version of ogm published onto the public repositories ? (btw which one is "valid" [1] ? [2] ? other? ). I asking that because I worked with Arun and the Java EE 7 sample to have an OGM integration (and it's working with the Ehcache module) and we need to have access to the current version (which is 4.0.0-SNAPSHOT), I went on the 2 repos mentioned above and on both the release dates back to January and that's quite old. Other question, shouldn't we release a new version like Beta5 specially now we have the Neo4j support ? Last but not least, why the version on the README.md and the pom.xml are not the same (respectively 4.0.0.Beta4 and 4.0.0-SNAPSHOT). Cheers, Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [OGM] public repositories and snapshot version
PS: here are the repo URLS I'm talking about: [1] http://repository.jboss.org/nexus/content/groups/public/ [2] http://repository.jboss.org/nexus/content/groups/public-jboss/ 2013/11/14 Guillaume SCHEIBEL > Hi guys, > > How often is the snapshot version of ogm published onto the public > repositories ? (btw which one is "valid" [1] ? [2] ? other? ). > > I asking that because I worked with Arun and the Java EE 7 sample to have > an OGM integration (and it's working with the Ehcache module) and we need > to have access to the current version (which is 4.0.0-SNAPSHOT), I went on > the 2 repos mentioned above and on both the release dates back to January > and that's quite old. > > Other question, shouldn't we release a new version like Beta5 specially > now we have the Neo4j support ? > > Last but not least, why the version on the README.md and the pom.xml are > not the same (respectively 4.0.0.Beta4 and 4.0.0-SNAPSHOT). > > > Cheers, > Guillaume > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] Tuning ORM memory consumption [HHH-8682]
Hi all,
from some performance tests we learned that a bottleneck for Hibernate
using applications is often identified in the amount of memory we
allocate at runtime, even considering the so called "short lived"
objects which usually are not a threat are actually too high.
Specifically, the highest consumer is the wild high amount of
instances of _org.hibernate.engine.spi.EntityKey_.
I think we have mainly two different strategies to attack this:
1# reduce the amount of EntityKey instances being allocated
2# reduce the size of each EntityKey instance
While 1# seems a wise move, I'll leave that to the ORM experts to
consider for future as it probably requires a broader knowledge of how
all components fit together.
So I'm attacking 2#, especially as I thought I could get some high win
in short time :)
To properly estimate the runtime size of each instance, I could simply
use the various reference tables of how much each pointer and
primitive takes, but there is actually some more complexity related to
the order in which the fields will be organized, and the requirement
of object alignment.
The rules aren't too hard to figure out the cost of a small value
object, but I'm actually using a tool which nicely dumps out a
reasonable estimate: "java object layout".
So this is the output of the current EntityKey implementation:
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Running 64-bit HotSpot VM.
Using compressed references with 3-bit shift.
Objects are 8 bytes aligned.
org.hibernate.engine.spi.EntityKey
offset size type description
012 (assumed to be the object
header + first field alignment)
12 4 int EntityKey.hashCode
16 1 boolean EntityKey.isBatchLoadable
17 3 (alignment/padding gap)
20 4 Serializable EntityKey.identifier
24 4String EntityKey.entityName
28 4String EntityKey.rootEntityName
32 4String EntityKey.tenantId
36 4 Type EntityKey.identifierType
40 4 SessionFactoryImplementor EntityKey.factory
44 4 (loss due to the next object alignment)
48 (object boundary, size estimate)
So that's 48 bytes per instance, and matches my expectations.
In our benchmark, we created about 200GB of such instances in 20
minutes, and the impact is of approximately 10% of the overall memory
pressure of the application.
I have an open pull request [2] which improves things, changing the
code to the following layout:
org.hibernate.engine.spi.EntityKey
offset size type description
012 (assumed to be the object header +
first field alignment)
12 4 int EntityKey.hashCode
16 4 Serializable EntityKey.identifier
20 4 String EntityKey.tenantId
24 4 EntityEssentials EntityKey.persister
28 4 (loss due to the next object alignment)
32(object boundary, size estimate)
So we get from 48 to 32 bytes per instance; assuming the same
allocation that means saving about 33%, so some 60GB of memory
bandwith saved.
I think we could take it as a first step in the right direction, still
we're so very close to actually improve of 50%: there are 4 bytes per
instance "wasted" just because of alignment needs, if we could remove
just one more field we would save 8bytes.
In this example I removed the _tenantId_:
org.hibernate.engine.spi.EntityKey
offset size type description
012 (assumed to be the object header +
first field alignment)
12 4 int EntityKey.hashCode
16 4 Serializable EntityKey.identifier
20 4 EntityEssentials EntityKey.persister
24(object boundary, size estimate)
So I would aim at implementing this, or alternatively if we really
can't remove the tenantId we could remove the (cached) hashCode:
org.hibernate.engine.spi.EntityKey
offset size type description
012 (assumed to be the object header +
first field alignment)
12 4 Serializable EntityKey.identifier
16 4 String EntityKey.tenantId
20 4 EntityEssentials EntityKey.persister
24(object boundary, size estimate)
But that hashCode calculation is really hot so I would love to rather
remove the _tenantId_.
How can we remove the tenantId ?
I was thinking to have the tenandId included in the persister, so we'd
have a new interface:
public interface TenantLocalEntityPersister extends EntityPersister {
String getTenantId();
}
Implementors would have two fields:
EntityPersister ep; //shared across all tenants
Strin
Re: [hibernate-dev] [OGM] public repositories and snapshot version
Hi, 2013/11/14 Guillaume SCHEIBEL > Hi guys, > > How often is the snapshot version of ogm published onto the public > repositories ? (btw which one is "valid" [1] ? [2] ? other? ). > Apparently this doesn't happen on a regular basis atm. I'll have a look into the CI job ( http://ci.hibernate.org/job/hibernate-ogm-master/), so that it publishes a SNAPSHOT whenever something is pushed to master. The artifacts will then be available from https://repository.jboss.org/nexus/content/groups/public/. I'll report back how this works out. > I asking that because I worked with Arun and the Java EE 7 sample to have > an OGM integration (and it's working with the Ehcache module) Awesome. Let us know once you have the sample working or need any help with that. > and we need > to have access to the current version (which is 4.0.0-SNAPSHOT), I went on > the 2 repos mentioned above and on both the release dates back to January > and that's quite old. > > Other question, shouldn't we release a new version like Beta5 specially now > we have the Neo4j support ? > The initial Neo4j support has been released with Beta4 (see http://in.relation.to/Bloggers/HibernateOGM400Beta4IsOut for the details) about two weeks ago. Make sure to post any feedback you have on this :) Regarding Beta5, the two big themes are the option support (OGM-208) and initial support for CouchDB (OGM-262). As always, any help is highly welcome. > > Last but not least, why the version on the README.md and the pom.xml are > not the same (respectively 4.0.0.Beta4 and 4.0.0-SNAPSHOT). > The readme file is only really updated upon releases so it has the right version in the released distribution. I guess one could update to SNAPSHOT after a release. --Gunnar > > Cheers, > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Tuning ORM memory consumption [HHH-8682]
Adding tenantIdentifier to persister is absolutely wrong. Think of persisters as scoped to the SessionFactory while tenancy is per Session. Now because an EntityKey is logically per session as well we may be able to remove from there. Again *logically*. I'll have to look at the ways in which EntityKey#tenantIdentifier is actually used. But I think that should be doable. On Nov 14, 2013 7:37 AM, "Sanne Grinovero" wrote: > Hi all, > from some performance tests we learned that a bottleneck for Hibernate > using applications is often identified in the amount of memory we > allocate at runtime, even considering the so called "short lived" > objects which usually are not a threat are actually too high. > > Specifically, the highest consumer is the wild high amount of > instances of _org.hibernate.engine.spi.EntityKey_. > > I think we have mainly two different strategies to attack this: > 1# reduce the amount of EntityKey instances being allocated > 2# reduce the size of each EntityKey instance > > While 1# seems a wise move, I'll leave that to the ORM experts to > consider for future as it probably requires a broader knowledge of how > all components fit together. > > So I'm attacking 2#, especially as I thought I could get some high win > in short time :) > > To properly estimate the runtime size of each instance, I could simply > use the various reference tables of how much each pointer and > primitive takes, but there is actually some more complexity related to > the order in which the fields will be organized, and the requirement > of object alignment. > > The rules aren't too hard to figure out the cost of a small value > object, but I'm actually using a tool which nicely dumps out a > reasonable estimate: "java object layout". > > > So this is the output of the current EntityKey implementation: > > Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) > Running 64-bit HotSpot VM. > Using compressed references with 3-bit shift. > Objects are 8 bytes aligned. > > org.hibernate.engine.spi.EntityKey > offset size type description > 012 (assumed to be the object > header + first field alignment) > 12 4 int EntityKey.hashCode > 16 1 boolean EntityKey.isBatchLoadable > 17 3 (alignment/padding gap) > 20 4 Serializable EntityKey.identifier > 24 4String EntityKey.entityName > 28 4String EntityKey.rootEntityName > 32 4String EntityKey.tenantId > 36 4 Type EntityKey.identifierType > 40 4 SessionFactoryImplementor EntityKey.factory > 44 4 (loss due to the next object > alignment) > 48 (object boundary, size estimate) > > So that's 48 bytes per instance, and matches my expectations. > In our benchmark, we created about 200GB of such instances in 20 > minutes, and the impact is of approximately 10% of the overall memory > pressure of the application. > > I have an open pull request [2] which improves things, changing the > code to the following layout: > > org.hibernate.engine.spi.EntityKey > offset size type description > 012 (assumed to be the object header + > first field alignment) > 12 4 int EntityKey.hashCode > 16 4 Serializable EntityKey.identifier > 20 4 String EntityKey.tenantId > 24 4 EntityEssentials EntityKey.persister > 28 4 (loss due to the next object alignment) > 32(object boundary, size estimate) > > So we get from 48 to 32 bytes per instance; assuming the same > allocation that means saving about 33%, so some 60GB of memory > bandwith saved. > > I think we could take it as a first step in the right direction, still > we're so very close to actually improve of 50%: there are 4 bytes per > instance "wasted" just because of alignment needs, if we could remove > just one more field we would save 8bytes. > > In this example I removed the _tenantId_: > > org.hibernate.engine.spi.EntityKey > offset size type description > 012 (assumed to be the object header + > first field alignment) > 12 4 int EntityKey.hashCode > 16 4 Serializable EntityKey.identifier > 20 4 EntityEssentials EntityKey.persister > 24(object boundary, size estimate) > > So I would aim at implementing this, or alternatively if we really > can't remove the tenantId we could remove the (cached) hashCode: > > org.hibernate.engine.spi.EntityKey > offset size type description > 012 (assumed to be the object header + > first field alignment) > 12 4
[hibernate-dev] Meeting today
I won't be able to make the irc meeting today ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] Tuning ORM memory consumption [HHH-8682]
On 14 November 2013 14:52, Steve Ebersole wrote: > Adding tenantIdentifier to persister is absolutely wrong. Think of > persisters as scoped to the SessionFactory while tenancy is per Session. I get that, I'm not suggesting to add it to _the_ persister of that entity. I'm proposing to create a new type which includes a pointer to the Persister (the one shared across all tenants) and a pointer to the tenantId. Or as you might prefer, any other solution which "wraps up" multiple frequently needed components in a reusable instance. I'm not sure if the set of tenants can be enumerated (so that we would create all combinations upfront), or if they should be cached and created on demand. In this case it's probably not worth to make a SessionFactory cache but there might be various opportunities for reuse, for example when loading multiple instances of the same type. If this specific combination of data aggregation sounds too crazy, we need to find other similar opportunities. Sanne ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [OGM] public repositories and snapshot version
2013/11/14 Gunnar Morling > Hi, > > 2013/11/14 Guillaume SCHEIBEL > >> Hi guys, >> >> How often is the snapshot version of ogm published onto the public >> repositories ? (btw which one is "valid" [1] ? [2] ? other? ). >> > > Apparently this doesn't happen on a regular basis atm. > > I'll have a look into the CI job ( > http://ci.hibernate.org/job/hibernate-ogm-master/), so that it publishes > a SNAPSHOT whenever something is pushed to master. The artifacts will then > be available from > https://repository.jboss.org/nexus/content/groups/public/. I'll report > back how this works out. > The CI job for master deploys the artifacts now to the Nexus repo, so we'll get fresh artifacts whenever a pull request is merged. Btw. just out of curiosity, what is it you need from 4.0.0-SNAPSHOT which isn't part of Beta4? > > >> I asking that because I worked with Arun and the Java EE 7 sample to have >> an OGM integration (and it's working with the Ehcache module) > > > Awesome. Let us know once you have the sample working or need any help > with that. > > >> and we need >> to have access to the current version (which is 4.0.0-SNAPSHOT), I went on >> the 2 repos mentioned above and on both the release dates back to January >> and that's quite old. >> > > >> Other question, shouldn't we release a new version like Beta5 specially >> now >> we have the Neo4j support ? >> > > The initial Neo4j support has been released with Beta4 (see > http://in.relation.to/Bloggers/HibernateOGM400Beta4IsOut for the details) > about two weeks ago. Make sure to post any feedback you have on this :) > > Regarding Beta5, the two big themes are the option support (OGM-208) and > initial support for CouchDB (OGM-262). As always, any help is highly > welcome. > >> >> Last but not least, why the version on the README.md and the pom.xml are >> not the same (respectively 4.0.0.Beta4 and 4.0.0-SNAPSHOT). >> > > The readme file is only really updated upon releases so it has the right > version in the released distribution. I guess one could update to SNAPSHOT > after a release. > > --Gunnar > > >> >> Cheers, >> Guillaume >> ___ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >> > > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
[hibernate-dev] [OGM] Support of the MongoDB aggregation framework
Hello, I'm (at the moment) listening to the "MongoDB for JPA developers) and the speaker talks about the aggregation framework. Should we plan to support it as well ? I think a eventual way would be by adding a DSL next to the MongoDBDialect ? What do you think ? Guillaume ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
Re: [hibernate-dev] [OGM] Support of the MongoDB aggregation framework
Hey Guillaume, 2013/11/14 Guillaume SCHEIBEL > Hello, > > I'm (at the moment) listening to the "MongoDB for JPA developers) and the > speaker talks about the aggregation framework. Should we plan to support it > as well ? > That's a very interesting topic. I haven't yet really looked into the aggregation framework. Just from a very quick look it might be helpful for mapping certain query constructs (group by). It may be a possible first step to run aggregations as "native queries". Do you have any particular use case or scenario in mind where an integration would be useful and which kind of support you have in mind? > > I think a eventual way would be by adding a DSL next to the MongoDBDialect > ? > What would such a DSL do? Would it be for building aggregation operations in a safe manner? > > What do you think ? > > Guillaume > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > ___ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
