I am glad to hear you have at least a plan of action and you can move on. Good luck
Jacopo On Thu, Jul 21, 2016 at 9:16 AM, Justin Robinson < [email protected]> wrote: > It was a chicken egg case. The transaction manger wasn't the problem. But I > am getting time outs now with tomcat nio exec threads being blocked in > UtilCache and CachedClassloader those are major synchronization bottle > necks but I have already updated then both to ofbiz 14. Anyway I suppose I > should search the mail archive and then try a new thread. > > > > On Wed, Jul 20, 2016 at 6:32 PM, Jacopo Cappellato < > [email protected]> wrote: > > > Is it possible that your code is creating EntityListIterator objects and > > not properly closing them? This may cause connections to stay open until > > the SqlProcessor objects are finalized. > > My guess is that the connection leaks are happening before the > > finalization, and not as a consequence of it, because EntityListIterator > > are not properly closed, and a db transaction is active when the finalize > > method attempts to close the connection. > > Based on the above assumptions, that could be wrong, I don't think that > > there is an interference of transactions for different tenants, but > rather > > data that you would expect to see is rolled back when the connection is > > closed before a commit. > > Please try to investigate in this direction and let me know if you do any > > progress. > > > > Good luck! > > > > Jacopo > > > > On Wed, Jul 20, 2016 at 1:51 PM, Jacopo Cappellato < > > [email protected]> wrote: > > > > > Hi Justin, > > > > > > On Wed, Jul 20, 2016 at 12:37 PM, Justin Robinson < > > > [email protected]> wrote: > > > > > >> I am really just interested in the architecture, it's no so much an > > issue > > >> with any version of ofbiz. (In fact I am working on a system based on > > >> ofbiz, it would be better to replace it with pure ofbiz, but that is > not > > >> my > > >> decision to make). I hope you will still be willing to discuss your > > >> implementation regardless of that fact. > > >> > > > > > > Definitely! > > > > > > > > >> > > >> Yes org.ofbiz.geronimo.GeronimoTransactionFactory > > >> calls ConnectionFactoryLoader.getInstance().getConnection(helperInfo, > > >> datasourceInfo.getInlineJdbc()); > > >> which is implemented > > >> through > > >> > > > org.ofbiz.entity.connection.DBCPConnectionFactory.getConnection(GenericHelperInfo, > > >> Element) which uses a > > >> ConcurrentHashMap to store a connection pool per tenant datasource and > > it > > >> seems to work very well, the correct pool is provided to the > appropriate > > >> tenant threads. > > >> > > >> This is an intermittent issue when testing with 50-100 tenants, > > >> transactions across tenants seem to collide occasionally. At least > that > > >> seems to be the case, I have stepped through the code in a debugger. > > >> > > >> It seem to happen in SqlProcessor finalize method, the gc calls > finalize > > >> and when the connection is closed (returned to the pool) it calls > commit > > >> first and then rolls back because a transaction is in progress as a > > result > > >> the connection is leaked and doesn't return to the pool. This is > clearly > > >> seen on a massive scale in jprofiler on the jdbc probe page. > > >> > > > > > > Hmmm... this is an interesting detail, there could be an issue in the > > > SqlProcessor class or maybe in the way that is used in your framework. > > > > > > I will dig into this more and let you know if I find something > > interesting. > > > > > > Jacopo > > > > > > > > >> > > >> Is it a limitation of the architecture? Is it possible to provide a > > >> TransactionManger per tenant in ThreadLocal or would that not work? > > >> > > >> Or am I missing something here? I realise it's a complex issue, but I > am > > >> trying to stay as close to ofbiz as I can I hope one day we can > replace > > >> the > > >> framework with a pure ofbiz implementation. I'd rather not make a > change > > >> like this which diverges from ofbiz architecture. How would you solve > > this > > >> issue? > > >> > > >> > > >> On Wed, Jul 20, 2016 at 1:17 PM, Jacopo Cappellato < > > >> [email protected]> wrote: > > >> > > >> > Hi Justin, > > >> > > > >> > your understanding of the class implementation is correct and there > > may > > >> be > > >> > a chance that some code may need to be improved, but I am not sure > as > > I > > >> > could only give a cursory look at the code. > > >> > By the way, I see that the code in GeronimoTransactionManager calls > a > > >> > "tenant-aware" method in DBCPConnectionFactory and this should > provide > > >> the > > >> > correct connection to every thread. > > >> > > > >> > Could you please elaborate more on your concern? If you could share > a > > >> > usage/failure scenario to recreate the issue it would be great too. > > >> > > > >> > Thanks, > > >> > > > >> > Jacopo > > >> > > > >> > On Wed, Jul 20, 2016 at 6:38 AM, Justin Robinson < > > >> > [email protected]> wrote: > > >> > > > >> > > I am trying to understand the database connection pooling in ofbiz > > 14, > > >> > > which uses dbcp2. > > >> > > > > >> > > In the class org.ofbiz.entity.connection.DBCPConnectionFactory > there > > >> is a > > >> > > static ConcurrentHashMap which provides a ManagedDataSource for > each > > >> > > tenant. > > >> > > > > >> > > But when it comes to obtaining a transaction manager reference, it > > >> looks > > >> > > like there is one global instance for all tenants. > > >> > > > > >> > > getTransactionManager in > > org.ofbiz.geronimo.GeronimoTransactionFactory > > >> > > returns the static reference to a single > GeronimoTransactionManager. > > >> > > > > >> > > In multi tenant testing overlapping from different tenants seem > like > > >> they > > >> > > interact with eachother. > > >> > > > > >> > > Can you explain how the implementation is supposed to work in with > > >> multi > > >> > > tenancy ? > > >> > > > > >> > > > >> > > > > > > > > >
