Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Emmanuel Bernard
I don't have much stake in the specialized method vs context object debate as indeed the interface is very specialized and prone to changes. But as Sanne mentioned, there are memory pressure consequences if this call is in the hot path. It is correct that the current use of ForDeletion requires to

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Hardy Ferentschik
On 8 Jan 2013, at 1:38 PM, Sanne Grinovero wrote: > Guys let's put this into perspective. > These arguments I'm hearing against adding a method in a power-user > oriented SPI are way outbalancing the harm they do to the project in > terms of release delays and our very own time, there are defini

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Sanne Grinovero
Guys let's put this into perspective. These arguments I'm hearing against adding a method in a power-user oriented SPI are way outbalancing the harm they do to the project in terms of release delays and our very own time, there are definitely more interesting issues to dedicate our time on. I appr

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Gunnar Morling
2013/10/8 Sanne Grinovero > We'll need to time cap this discussion as we're way too late, of > course this will need to be tagged @experimental. > Having said that, let's try to find the best proposal possible by > lunch time, as one of the approaches needs to be merged: it's very > clear that th

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Hardy Ferentschik
On 8 Jan 2013, at 11:56 AM, Sanne Grinovero wrote: > Having said that, let's try to find the best proposal possible by > lunch time, as one of the approaches needs to be merged: I think we very well could go w/ the current state right now and evolve in future versions. I would like to explore

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Sanne Grinovero
We'll need to time cap this discussion as we're way too late, of course this will need to be tagged @experimental. Having said that, let's try to find the best proposal possible by lunch time, as one of the approaches needs to be merged: it's very clear that there is big practical value for the use

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Hardy Ferentschik
On 8 Jan 2013, at 10:08 AM, Gunnar Morling wrote: > I'm not sure whether it has been considered before, but maybe we could unify > the methods and work with a parameter object as a middle ground: It has been. I suggested before to combine the methods. I think it is a good approach, but Sanne

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-08 Thread Gunnar Morling
Sanne, As you say adding yet another interface makes things even more difficult to grok; So I'd vote for adding the method for the deletion use case to SIP directly. I'm not sure whether it has been considered before, but maybe we could unify the methods and work with a parameter object as a midd

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-07 Thread Sanne Grinovero
I've included an example which represents a good reason to provide the controversial method. Technically the test is crafted as a static sharding approach but is using the new API; you can easily figure the same case for a dynamic sharding case; also considering we're deprecating the older static s

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-07 Thread Sanne Grinovero
Hi Hardy, could you have a look at the following two commits, while I work on a test as you suggested. (documentation will follow depending on which one you like best). In this case I just add the missing method, and I don't think it's bad, actually the name is fine and while I'm sure you might ha

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-07 Thread Hardy Ferentschik
On 7 Jan 2013, at 5:03 PM, Sanne Grinovero wrote: > I've tried hard to find an agreement on this, but it seems we're > wasting time without making progress. > I'm not happy in ignoring a strong recommendation from any of you, > very hard choice :-( In the end it is your call. I tried to give ar

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-07 Thread Sanne Grinovero
On 2 October 2013 14:34, Emmanuel Bernard wrote: > On Tue 2013-09-24 14:30, Sanne Grinovero wrote: >> On 24 September 2013 14:12, Hardy Ferentschik wrote: >> > 2) remove 'String[] getShardIdentifiers(Class entity, Serializable id, >> > String idInString)' from ShardIdentifierProvider >> >> +1 we

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-03 Thread Emmanuel Bernard
On Tue 2013-09-24 14:30, Sanne Grinovero wrote: > On 24 September 2013 14:12, Hardy Ferentschik wrote: > > 2) remove 'String[] getShardIdentifiers(Class entity, Serializable id, > > String idInString)' from ShardIdentifierProvider > > +1 we're automatically assuming a deletion needs to be routed

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-03 Thread Emmanuel Bernard
What does Iterable give you over String[]? On Mon 2013-09-23 23:04, Sanne Grinovero wrote: > Correct me if I'm wrong, but trying to synthesize this discussion I > think that we're fundamentally agreeing that dynamic sharding is a > "better replacement" for static sharding. > Still, let's keep in m

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-10-02 Thread Emmanuel Bernard
On Tue 2013-09-24 10:51, Hardy Ferentschik wrote: > String[] getShardIdentifiers(Class entity, Serializable id, String > idInString); > > all together. Here is my reasoning. AFAIU, the method is there for the > deletion of > documents. In this case we don't have the Lucene document nor the entit

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Sanne Grinovero
On 24 September 2013 14:56, Hardy Ferentschik wrote: > > On 24 Jan 2013, at 3:16 PM, Sanne Grinovero wrote: > >> On 24 September 2013 13:46, Hardy Ferentschik wrote: >>> Cool, we seem to agree on almost everything now :-) >> >> +1 it's hard to get convergence when the thread explodes exponential

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Hardy Ferentschik
On 24 Jan 2013, at 3:16 PM, Sanne Grinovero wrote: > On 24 September 2013 13:46, Hardy Ferentschik wrote: >> Cool, we seem to agree on almost everything now :-) > > +1 it's hard to get convergence when the thread explodes exponentially > on several different subjects but it seems it was worth

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Sanne Grinovero
On 24 September 2013 13:46, Hardy Ferentschik wrote: > Cool, we seem to agree on almost everything now :-) +1 it's hard to get convergence when the thread explodes exponentially on several different subjects but it seems it was worth it. thanks for the huge energy and ideas :-) > > On 24 Jan 201

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Hardy Ferentschik
Cool, we seem to agree on almost everything now :-) On 24 Jan 2013, at 2:30 PM, Sanne Grinovero wrote: >> 5) rename 'getShardIdentifier' to 'getShardIdentifierForAddition' > > Is that needed? I thought that by removing the conflicting method > there would be no further need to clarify the metho

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Sanne Grinovero
On 24 September 2013 14:12, Hardy Ferentschik wrote: > > On 24 Jan 2013, at 12:54 PM, Sanne Grinovero wrote: > >>> We should for sure try to keep API's stable. On the other hand I don't see >>> why we should not be >>> able to change SPI contracts. With this super restrictive behaviour we are >

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Hardy Ferentschik
On 24 Jan 2013, at 12:54 PM, Sanne Grinovero wrote: >> We should for sure try to keep API's stable. On the other hand I don't see >> why we should not be >> able to change SPI contracts. With this super restrictive behaviour we are >> seriously limiting >> our ability to move the software forw

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Sanne Grinovero
On 24 September 2013 09:51, Hardy Ferentschik wrote: > > On 24 Jan 2013, at 12:04 AM, Sanne Grinovero wrote: > >> Correct me if I'm wrong, but trying to synthesize this discussion I >> think that we're fundamentally agreeing that dynamic sharding is a >> "better replacement" for static sharding.

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Gunnar Morling
2013/9/24 Sanne Grinovero > Correct me if I'm wrong, but trying to synthesize this discussion I > think that we're fundamentally agreeing that dynamic sharding is a > "better replacement" for static sharding. Yes, from what I understand I think that's right. To me, the question really is what

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-24 Thread Hardy Ferentschik
On 24 Jan 2013, at 12:04 AM, Sanne Grinovero wrote: > Correct me if I'm wrong, but trying to synthesize this discussion I > think that we're fundamentally agreeing that dynamic sharding is a > "better replacement" for static sharding. It has the potential for a replacement I think, but I don't

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-23 Thread Sanne Grinovero
Correct me if I'm wrong, but trying to synthesize this discussion I think that we're fundamentally agreeing that dynamic sharding is a "better replacement" for static sharding. Still, let's keep in mind that this needs to be a backwards compatible patch, so we're not looking for something disruptiv

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-23 Thread Hardy Ferentschik
On 23 Jan 2013, at 1:55 PM, Sanne Grinovero wrote: >> Or I set a custom sharding strategy which does not care about the number of >> shards? > > I think that's far fetched. The NBR_OF_SHARDS option defines the size > of the array of indexes passed to the IndexShardingStrategy so it's > hard to

Re: [hibernate-dev] [Search] Dynamic sharding configuration

2013-09-23 Thread Sanne Grinovero
On 20 September 2013 11:37, Hardy Ferentschik wrote: > Hi, > > I am currently working on HSEARCH-471 [1] - dynamic sharding. The work is > built on Emmanuel's prototype and you find the current code on my fork [2]. > > Right now I am wondering about how to configure (dynamic) sharding. Here is >

[hibernate-dev] [Search] Dynamic sharding configuration

2013-09-20 Thread Hardy Ferentschik
Hi, I am currently working on HSEARCH-471 [1] - dynamic sharding. The work is built on Emmanuel's prototype and you find the current code on my fork [2]. Right now I am wondering about how to configure (dynamic) sharding. Here is how things worked prior to dynamic sharding. Basically there two