[hibernate-dev] Re: [jbosscache-dev] JBoss Cache Lucene Directory
Sanne, Agreed. Could all involved please make sure we post to both hibernate- dev as well as infinispan-dev (rather than jbosscache-dev) when discussing anything to do with such integration work. As there are parallel efforts which can be brought together. Cheers Manik On 25 May 2009, at 10:53, Sanne Grinovero wrote: Hello, I'm forwarding this email to Emmanuel and Hibernate Search dev, as I believe we should join the discussion. Could we keep both dev-lists (jbosscache-...@lists.jboss.org, hibernate-dev@lists.jboss.org ) on CC ? Sanne 2009/4/29 Manik Surtani : On 27 Apr 2009, at 05:18, Andrew Duckworth wrote: Hello, I have been working on a Lucene Directory provider based on JBoss Cache, my starting point was an implementation Manik had already written which pretty much worked with a few minor tweaks. Our use case was to cluster a Lucene index being used with Hibernate Search in our application, with the requirements that searching needed to be fast, there was no shared file system and it was important that the index was consistent across the cluster in a relatively short time frame. Maniks code used a token node in the cache to implement the distributed lock. During my testing I set up multiple cache copies with multiple threads reading/writing to each cache copy. I was finding a lot of transactions to acquire or release this lock were timing out, not understanding JBC well I modified the distributed lock to use JGroups DistrubutedLockManager. This worked quite well, however the time taken to acquire/release the lock (~100 ms for both) dwarfed the time to process the index update, lowering throughput. Even using Hibernate Search with an async worker thread, there was still a lot of contention for the single lock which seemed to limit the scalability of the solution. I thinkl part of the problem was that our use of HB Search generates a lot of small units of work (remove index entry, add index entry) and each of these UOW acquire a new IndexWriter and new write lock on the underlying Lucene Directory implementation. Out of curiosity, I created an alternative implementation based on the Hibernate Search JMS clustering strategy. Inside JBoss Cache I created a queue node and each slave node in the cluster creates a separate queue underneath where indexing work is written: /queue/slave1/[work0, work1, work2 ] /slave2 /slave3 etc In each cluster member a background thread runs continuously when it wakes up, it decides if it is the master node or not (currently checks if it is the view coordinator, but I'm considering changing it to use a longer lived distributed lock). If it is the master it merges the tasks from each slave queue, and updates the JBCDirectory in one go, it can safely do this with only local VM locking. This approach means that in all the slave nodes they can write to their queue without needing a global lock that any other slave or the master would be using. On the master, it can perform multiple updates in the context of a single Lucene index writer. With a cache loader configured, work that is written into the slave queue is persistent, so it can survive the master node crashing with automatic fail over to a new master meaning that eventually all updates should be applied to the index. Each work element in the queue is time stamped to allow them to be processed in order (requires! time synchronisation across the cluster) by the master. For our workload the master/slave pattern seems to improve the throughput of the system. Currently I'm refining the code and I have a few JBoss Cache questions which I hope you can help me with: 1) I have noticed that under high load I get LockTimeoutExceptions writing to /queue/slave0 when the lock owner is a transaction working on /queue/slave1 , i.e. the same lock seems to be used for 2 unrelated nodes in the cache. I'm assuming this is a result of the lock striping algorithm, if you could give me some insight into how this works that would be very helpful. Bumping up the cache concurrency level from 500 to 2000 seemed to reduce this problem, however I'm not sure if it just reduces the probability of a random event of if there is some level that will be sufficient to eliminate the issue. It could well be the lock striping at work. As of JBoss Cache 3.1.0 you can disable lock striping and have one lock per node. While this is expensive in that if you have a lot of nodes, you end up with a lot of locks, if you have a finite number of nodes this may help you a lot. 2) Is there a reason to use separate nodes for each slave queue ? Will it help with locking, or can each slave safely insert to the same parent node in separate transactions without interfering or blocking each other ? If I can reduce it to a single queue I thin that would be a more elegant solution. I am setting the lockParent
[hibernate-dev] [BV] Support for constraints by groups
Please review. Also we need to implement the // group validation in HV
to avoid to many passes.
Begin forwarded message:
From: Emmanuel Bernard
Date: May 26, 2009 16:43:53 CEDT
To: jsr-303...@jcp.org
Subject: Support for constraints by groups
Reply-To: Java Community Process JSR #303 Expert List >
Hello,
following a subject dear to Gerhard's heart :)
I could think of two approaches, let's beat them to death and if
possible find a better alternative :)
# background
When validating a specific group(s), Bean Validation expends the
group into a sequence of groups to be validated (because a group can
be a group sequence really). Depending on how clever the
implementation is, it could sequencialize all group execution or try
to validate several non sequenced group in parallel.
The concept of discovering which group is validated based on the
requested groups is complex enough so an API at the metadata level
is required.
examples
public interface Minimal {}
@GroupSequence(Minimal.class, Default.class)
public interface Complete {}
#1 define the API at the ElementDescriptor level
BeanDescriptor bd = validator.getConstraintsForClass(Address.class);
PropertyDescriptor pd = bd.getConstraintsForProperty("street1");
List> sequencedConstraints =
pd.getConstraintDescriptorsForGroups(Complete.class);
//all ConstraintDescriptor in a given set can be validated in the
same pass
//two sets of constraints must be validated sequencially. If one
constraint fails, the following sets must be ignored
for (Set subConstraints :
sequencedConstraints) {
if ( validate(subConstraints) == true ) break;
}
The API added is
ElementDescriptor#getConstraintDescriptorsForGroups(Class...
groups)
Optionally we can add
ElementDescriptor#hasConstraints(Class... groups)
Pro: The metadata is never "contextual" hence has less risk of error.
Con: Needs to pass the groups at several levels
#2 define the API at the validator level (more aligned with
Gerhard's proposal but supporting sequences)
List sequencedDescriptors =
validator.getConstraintsForClassAndGroups(Address.class,
Complete.class);
for (BeanDescriptor bd : sequencedDescriptor) {
PropertyDescriptor pd = bd. getConstraintsForProperty("street1");
Set subConstraints =
pd.getConstraintDescriptors();
if ( validate(subConstraints) == true ) break;
}
Pro: The metadata is "contextual" hence people can become confused
Con: once contextualized, the API looks the same as the non
contextual one
Questions:
- should we use List to represent a sequence r should we have a
dedicated object implementing Iterable
- we need to ensure that a given group always return the same
sequencing regardless of the element (bean or property). This is
necessary to ensure that one can validate all bean and property
level constraints (and potentially the associated object) before
going to the next sequence. How can we formalize that?
WDYT of all that?
___
hibernate-dev mailing list
hibernate-dev@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
RE: [hibernate-dev] RE: EntityPersister.initializeLazyProperty: ask for one, initialize all?
Hey Nikita Bad timing as the US had a long weekend, so most of us were not around. I'll take a look at this tomorrow and get back to you. WRT your specific question... AFAIR, the reason cloning would be needed was that you would need to remove the property names as they are initialized. The set of lazy property names are defined per tuplizer which is scoped to the persister which is scoped to the sessionfactory. So, as it stands right now, removing a property name from that set once a lazy property is initialized would remove it "everywhere". Cloning would give each FieldInterceptor (associated to each bytecode enhanced entity) its own set with which to work. In pseudo code think of it as: Set unitializedPropertyNames = ...; // shared instance as of now Entity yourEnhancedEntity = new Entity(); yourEnhancedEntity.injectInterceptor( new Interceptor( unitializedPropertyNames, ... ) ); Entity anotherEnhancedEntityInst = new Entity(); anotherEnhancedEntityInst.injectInterceptor( new Interceptor( unitializedPropertyNames, ... ) ); Notice both entity instances share the same 'unitializedPropertyNames' set reference... On Sun, 2009-05-24 at 19:51 -0700, Nikita Tovstoles wrote: > Submitted a patch to Hibernate Core 3.3.1.GA that fixes the issue for > multiple lazy one-to-one's on a shared PK. I think the only thing(s) left to > do to enable un-batched lazy property initialization for cases other than > one-to-one on a shared PK are: > - generate appropriate sql lazySelectString given property being requested > (in AbstractEntityPersister.initializeLazyPropertiesFromDatastore) > - possibly alter CacheEntry to enumerate uninitialized properties (rather > than simply store a boolean value) > > I've tested my changes but, of course, would appreciate feedback from someone > who knows Hibernate internals better than I do. My patch and further comments > can be found here: > http://opensource.atlassian.com/projects/hibernate/browse/HHH-2309?focusedCommentId=33239#action_33239 > > -nikita > > > -Original Message- > From: hibernate-dev-boun...@lists.jboss.org > [mailto:hibernate-dev-boun...@lists.jboss.org] On Behalf Of Nikita Tovstoles > Sent: Friday, May 22, 2009 8:57 PM > To: hibernate-dev@lists.jboss.org > Subject: [hibernate-dev] RE: EntityPersister.initializeLazyProperty: ask for > one, initialize all? > > Naturally, things are a bit more involved: > -replaced AbstractFieldInterceptor.initializing with a Set tracking that > state on a per-fieldName basis. > > That wasn't enough - TestLazyExecutable is failing - returning a null value - > it almost seems as though the same FilterInterceptor instance is reused by > multiple sessions after my awesome coding. > > I assumed that at most (or exactly?) one AbstractFieldInterceptor impl > instance is created per entity instance per Session and that no > fieldInterceptor is used by multiple sessions. > > The interceptor seems to be injected with lazyProps only in > PojoEntityTuplizer.afterInitialize() where the following comment resides: > > //TODO: if we support multiple fetch groups, we would need > // to clone the set of lazy properties! > FieldInterceptionHelper.injectFieldInterceptor( entity, getEntityName(), > lazyProps, session ); > > > Why is cloning needed? Is it because my assumption about the lifespan of one > is incorrect? Is a new FieldInterceptor created every time a different lazy > property is accessed for the first time on the same entity? > > Thanks > > -nikita > > > > > -Original Message- > > Our app fails to scale sufficiently and I'd traced our problems to eager > loading of all OneToOne relations when any single one is accessed. I would > like to fix that but wanted to get feedback first. I'm referring to Hibernate > Core 3.3.1.GA below: > > Currently in AbstractFieldInterceptor.intercept(): > "uninitializedFields = null; //let's assume that there is only one lazy fetch > group, for now!" > > proposed fix: > -after 'result' is returned call uninitializedFields.remove(fieldname). > Question: should this only be done if result != null? > > And then AbstractEntityPersister.initializeLazyProperties() calls methods > that initialize *all* properties even though a specific fieldname is supplied: > * initializeLazyPropertiesFromDatastore or > * initializeLazyPropertiesFromCache > > Proposed fix: > -In both cases, determine appropriate 'j' value by searching > lazyPropertyNames for 'fieldName' > -only call nullSafeGet, and initializeLazyProperty( fieldName, entity, > session, snapshot, j, propValue ) once. > > > What do folks think? > > Thanks, > -nikita > > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev > > ___ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailm
RE: [hibernate-dev] RE: EntityPersister.initializeLazyProperty: ask for one, initialize all?
Thanks for getting back to me, Steve. Yep, I realized that subsequently and submitted a patch. Please see my latter e-mails with further details. -nikita -Original Message- From: Steve Ebersole [mailto:steven.ebers...@gmail.com] On Behalf Of Steve Ebersole Sent: Tuesday, May 26, 2009 10:17 AM To: Nikita Tovstoles Cc: hibernate-dev@lists.jboss.org Subject: RE: [hibernate-dev] RE: EntityPersister.initializeLazyProperty: ask for one, initialize all? Hey Nikita Bad timing as the US had a long weekend, so most of us were not around. I'll take a look at this tomorrow and get back to you. WRT your specific question... AFAIR, the reason cloning would be needed was that you would need to remove the property names as they are initialized. The set of lazy property names are defined per tuplizer which is scoped to the persister which is scoped to the sessionfactory. So, as it stands right now, removing a property name from that set once a lazy property is initialized would remove it "everywhere". Cloning would give each FieldInterceptor (associated to each bytecode enhanced entity) its own set with which to work. In pseudo code think of it as: Set unitializedPropertyNames = ...; // shared instance as of now Entity yourEnhancedEntity = new Entity(); yourEnhancedEntity.injectInterceptor( new Interceptor( unitializedPropertyNames, ... ) ); Entity anotherEnhancedEntityInst = new Entity(); anotherEnhancedEntityInst.injectInterceptor( new Interceptor( unitializedPropertyNames, ... ) ); Notice both entity instances share the same 'unitializedPropertyNames' set reference... On Sun, 2009-05-24 at 19:51 -0700, Nikita Tovstoles wrote: > Submitted a patch to Hibernate Core 3.3.1.GA that fixes the issue for > multiple lazy one-to-one's on a shared PK. I think the only thing(s) left to > do to enable un-batched lazy property initialization for cases other than > one-to-one on a shared PK are: > - generate appropriate sql lazySelectString given property being requested > (in AbstractEntityPersister.initializeLazyPropertiesFromDatastore) > - possibly alter CacheEntry to enumerate uninitialized properties (rather > than simply store a boolean value) > > I've tested my changes but, of course, would appreciate feedback from someone > who knows Hibernate internals better than I do. My patch and further comments > can be found here: > http://opensource.atlassian.com/projects/hibernate/browse/HHH-2309?focusedCommentId=33239#action_33239 > > -nikita > > > -Original Message- > From: hibernate-dev-boun...@lists.jboss.org > [mailto:hibernate-dev-boun...@lists.jboss.org] On Behalf Of Nikita Tovstoles > Sent: Friday, May 22, 2009 8:57 PM > To: hibernate-dev@lists.jboss.org > Subject: [hibernate-dev] RE: EntityPersister.initializeLazyProperty: ask for > one, initialize all? > > Naturally, things are a bit more involved: > -replaced AbstractFieldInterceptor.initializing with a Set tracking that > state on a per-fieldName basis. > > That wasn't enough - TestLazyExecutable is failing - returning a null value - > it almost seems as though the same FilterInterceptor instance is reused by > multiple sessions after my awesome coding. > > I assumed that at most (or exactly?) one AbstractFieldInterceptor impl > instance is created per entity instance per Session and that no > fieldInterceptor is used by multiple sessions. > > The interceptor seems to be injected with lazyProps only in > PojoEntityTuplizer.afterInitialize() where the following comment resides: > > //TODO: if we support multiple fetch groups, we would need > // to clone the set of lazy properties! > FieldInterceptionHelper.injectFieldInterceptor( entity, getEntityName(), > lazyProps, session ); > > > Why is cloning needed? Is it because my assumption about the lifespan of one > is incorrect? Is a new FieldInterceptor created every time a different lazy > property is accessed for the first time on the same entity? > > Thanks > > -nikita > > > > > -Original Message- > > Our app fails to scale sufficiently and I'd traced our problems to eager > loading of all OneToOne relations when any single one is accessed. I would > like to fix that but wanted to get feedback first. I'm referring to Hibernate > Core 3.3.1.GA below: > > Currently in AbstractFieldInterceptor.intercept(): > "uninitializedFields = null; //let's assume that there is only one lazy fetch > group, for now!" > > proposed fix: > -after 'result' is returned call uninitializedFields.remove(fieldname). > Question: should this only be done if result != null? > > And then AbstractEntityPersister.initializeLazyProperties() calls methods > that initialize *all* properties even though a specific fieldname is supplied: > * initializeLazyPropertiesFromDatastore or > * initializeLazyPropertiesFromCache > > Proposed fix: > -In both cases, determine appropriate 'j' value by searching > lazyPropertyNames for 'fieldName' > -only call nullSafeG
[hibernate-dev] DefaultLoadEventListener.assembleCacheEntry several times slower than fetching state from 2nd level cache
I am about to file a bug but wanted to run this by the experts first.
Let's say there is a User entity class that declares 20 lazy collections much
like so:
@OneToMany( mappedBy = "owner")
@Cascade({CascadeType.REMOVE, CascadeType.DELETE_ORPHAN})
@LazyCollection(LazyCollectionOption.TRUE)
@BatchSize(size = 1000)
@Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL)
public Set getCustomSpaces()
In Hibernate 3.3.1.GA if I load ~1000 User entities from second level cache
(w/o accessing any of the collections), I'll end up spending 5x time assembling
cache entries vs. actually loading de-hydrated state from the cache provider
(profiler snapshot attached). Most of the time is spent eagerly assembling
CollectionTypes, all of which may or may not be needed during a given Session.
So, basically, I'm being penalized for merely declaring OneToMany's. Currently,
the only way to avoid this penalty - as far as I can tell - is to complicate
the domain model by introducing some number of new entities that will be
OneToOne to User; and redistributing collection declarations amongst those. But
even that may not help: what if services using the domain model access a
variety of collections at different times? IMO a much more scalable approach
would be to:
* stash serialized states representing collection properties during
assembleCacheEntry
* call CollectionType.assemble lazily when collection property field is
actually accessed (a la lazy properties)
Fwiw, a post covering the above in a little more detail can be found here:
https://forum.hibernate.org/viewtopic.php?p=2412263#p2412263
-nikita
Merged callees
+-++-++
|
Name
| Time (ms)| Own Time (ms) | Invocation Count |
+-++-++
|
+---org.hibernate.event.def.DefaultLoadEventListener.loadFromSecondLevelCache(LoadEvent,
EntityPersister, LoadEventListener$LoadType) | 2,390
100 % | 0 | 999 |
||
|
| ||
|
+---org.hibernate.event.def.DefaultLoadEventListener.assembleCacheEntry(CacheEntry,
Serializable, EntityPersister, LoadEvent)| 1,796
75 % | 15 | 999 |
|| |
|
| ||
|| +---org.hibernate.cache.entry.CacheEntry.assemble(Object, Serializable,
EntityPersister, Interceptor, EventSource)
| 1,546 65 % | 0 | 999 |
|| | |
|
| ||
|| | +---org.hibernate.cache.entry.CacheEntry.assemble(Serializable[],
Object, Serializable, EntityPersister, Interceptor, EventSource)
| 1,546 65 % | 31 | 999 |
|| | |
|
| ||
|| | +---org.hibernate.type.TypeFactory.assemble(Serializable[], Type[],
SessionImplementor, Object)
| 1,250 52 % | 0 | 999 |
|| | | |
|
| ||
|| | | +---org.hibernate.type.CollectionType.assemble(Serializable,
SessionImplementor, Object)
| 1,109 46 % | 31 |10,998 |
|| | | | |
|
| ||
|| | | | +---org.hibernate.type.CollectionType.resolveKey(Se
