Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Guillaume Smet
Hi, So, I analyzed the dumps yesterday evening. The problem is real, meaning his SessionFactory is consuming more than 1GB of memory for 600+ tables, some with a lot of attributes. So for sure, the model is a big one, but it would be nice to be more gentle with this type of configuration. I don't

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Sanne Grinovero
Hi Guillaume, thanks for investigating. Steve in chat mentioned the option of creating these EntityLoaders lazily: as while they need to be reused, it's possible many of them are never really used. I think that's a very interesting observation, is that what you plan to explore? I see two difficu

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Guillaume Smet
On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero wrote: > thanks for investigating. Steve in chat mentioned the option of > creating these EntityLoaders lazily: as while they need to be reused, > it's possible many of them are never really used. > > I think that's a very interesting observation,

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Gail Badner
I don't think the batching strategies actually changed when we moved to load plans. Is the problem that each loader is taking considerably more memory? On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet wrote: > On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero > wrote: > > > thanks for investigati

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Sanne Grinovero
On 4 May 2018 at 19:45, Gail Badner wrote: > I don't think the batching strategies actually changed when we moved to load > plans. > > Is the problem that each loader is taking considerably more memory? Yes that's right. > > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet > wrote: >> >> On Fri,

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Guillaume Smet
Don't know about the history but with the entity I checked out in the OP's case, each EntityLoader is approximately 200 KB. The entity contains a lot of columns so that's no surprise. What doesn't help is that I don't think the 13 EntityLoaders/entity share a lot of things so they are just adding

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Gail Badner
I *think* that most, if not all, of the loaders for a particular entity could be reworked to share most of the same information. IIRC, the only real differences are the batch size and the actual query that gets generated. I'm happy to look into this, but I don't want to duplicate what others are

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Guillaume Smet
As already mentioned, I plan to take a look at this next week. I also thought we could share most of the info between loaders but Steve was thinking it might not be possible without massive changes. I'll take a look and report back to all of you. I might be way over my head as I don't know the co

[hibernate-dev] Small change to the ORM Travis configuration

2018-05-04 Thread Guillaume Smet
Hi, The Travis cache behaves this way: - try to get a cache specific to the PR - then try to get a cache for master - if none found, start without cache The ORM Travis build was configured to not build the branches so master was never build thus each PR starts with an empty cache (but subsequent

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Gail Badner
OK, that's fine. I did a lot of the work converting to load plans, so feel free to ask if you have questions. On Fri, May 4, 2018 at 1:37 PM, Guillaume Smet wrote: > As already mentioned, I plan to take a look at this next week. > > I also thought we could share most of the info between loaders

Re: [hibernate-dev] SingleTableEntityPersister memory footprint in 5.3

2018-05-04 Thread Steve Ebersole
Loaders were designed to be an encapsulation of the query (the SQL, how to build and execute the PreparedStatement) and the "load plan" (how the ResultSet is processed). 4.2 changed how this last part is represented, but overall the design is the same it always has been. The problem is that Loade