Re: Fetching lots of objects

2017-03-08 Thread Andrus Adamchik
Hi Maik, > On Mar 8, 2017, at 7:47 PM, Musall, Maik wrote: > > Well, if I need them all in the same context to work with after this, I would > then need to localObject() them and be back at locking, this time against the > graph manager. Dang. Yes. Unfortunately. > It would be nice if Cayenn

Re: Fetching lots of objects

2017-03-08 Thread Musall, Maik
Whoa. Parallel instantiation down to <2700 ms using multiple threads with a local ObjectContext each. Well, if I need them all in the same context to work with after this, I would then need to localObject() them and be back at locking, this time against the graph manager. Dang. It would be nice

Re: Fetching lots of objects

2017-03-08 Thread John Huss
If parallel is going to have any benefit you have to be using separate object contexts to avoid locking the same DataRow cache. On Wed, Mar 8, 2017 at 5:59 AM Musall, Maik wrote: > > > Am 08.03.2017 um 10:56 schrieb Aristedes Maniatis : > > > > On 8/3/17 6:54pm, Musall, Maik wrote: > > > >> regul

Re: Fetching lots of objects

2017-03-08 Thread Musall, Maik
> Am 08.03.2017 um 10:56 schrieb Aristedes Maniatis : > > On 8/3/17 6:54pm, Musall, Maik wrote: > >> regular SelectQuery: 25888 ms for 1291644 objects >> DataRowQuery alone: 14289 ms for 1291644 rows >> DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum = >> 21167 >> DataRo

Re: Fetching lots of objects

2017-03-08 Thread Aristedes Maniatis
On 8/3/17 6:54pm, Musall, Maik wrote: > regular SelectQuery: 25888 ms for 1291644 objects > DataRowQuery alone: 14289 ms for 1291644 rows > DataRowQuery sequential instantiation: 6878 ms for 1291644 objects, sum = > 21167 > DataRowQuery parallel instantiation: 7351 ms for 1291644 objects, sum = 2

Re: Fetching lots of objects

2017-03-07 Thread Musall, Maik
Hi Ari, > Am 07.03.2017 um 23:14 schrieb Aristedes Maniatis : > > On 7/3/17 8:25am, Musall, Maik wrote: >> Hi all, >> >> I have a number of statistics functions which need to fetch large amounts of >> objects. I need the actual DataObjects because that's where the business >> logic is that I n

Re: Fetching lots of objects

2017-03-07 Thread Aristedes Maniatis
On 7/3/17 8:25am, Musall, Maik wrote: > Hi all, > > I have a number of statistics functions which need to fetch large amounts of > objects. I need the actual DataObjects because that's where the business > logic is that I need for the computations. > > Let's say I need to fetch 300.000 objects.

Re: Fetching lots of objects

2017-03-06 Thread Musall, Maik
Hi Marcel, I know how to do the actual computation in parallel. My question is how to fetch and instantiate the DataObjects in parallel before I can start the computations. An iterator would only slow down the fetch because of the added roundtrips. Iterators are about reducing memory footprint,

Re: Fetching lots of objects

2017-03-06 Thread Markus Reich
Hi Maik, maybe you can use the new iterator and split the iterator for parallel computation? public static Stream asStream(Iterator sourceIterator, boolean parallel) { Iterable iterable = () -> sourceIterator; return StreamSupport.stream(iterable.spliterator(), parallel); } found at http:

Fetching lots of objects

2017-03-06 Thread Musall, Maik
Hi all, I have a number of statistics functions which need to fetch large amounts of objects. I need the actual DataObjects because that's where the business logic is that I need for the computations. Let's say I need to fetch 300.000 objects. Let's also assume the database sits on a fast SSD