Any help here? I think that the problem is that the JobManager creates the
executionContext of the scheduler with
val executionContext = ExecutionContext.fromExecutor(new
ForkJoinPool())
and thus the number of concurrently running threads is limited to the
number of cores (using the default constructor of the ForkJoinPool).
What do you think?
On Wed, Mar 23, 2016 at 6:55 PM, Stefano Bortoli <s.bort...@gmail.com>
wrote:
> Hi guys,
>
> I am trying to test a job that should run a number of tasks to read from a
> RDBMS using an improved JDBC connector. The connection and the reading run
> smoothly, but I cannot seem to be able to move above the limit of 8
> concurrent threads running. 8 is of course the number of cores of my
> machine.
>
> I have tried working around configurations and settings, but the Executor
> within the ExecutionContext keeps on having a parallelism of 8. Although,
> of course, the parallelism of the execution environment is much higher (in
> fact I have many more tasks to be allocated).
>
> I feel it may be an issue of the LocalMiniCluster configuration that may
> just override/neglect my wish for higher degree of parallelism. Is there a
> way for me to work around this issue?
>
> please let me know. Thanks a lot for you help! :-)
>
> saluti,
> Stefano
>
