2016-05-09 14:56 GMT+02:00 Simone Robutti :
> >- You wrote you'd like to "instantiate a H2O's node in every task
> manager". This reads a bit like you want to start H2O in the TM's JVM , but
> I would assume that a H2O node runs as a separate process. So should it be
> started inside the TM JVM or
>- You wrote you'd like to "instantiate a H2O's node in every task
manager". This reads a bit like you want to start H2O in the TM's JVM , but
I would assume that a H2O node runs as a separate process. So should it be
started inside the TM JVM or as an external process next to each TM. Also,
do you
Hi Simone,
sorry for the delayed answer. I have a few questions regarding your
requirements and a some ideas that might be helpful (depending on the
requirements).
1) Starting / stopping of H2O nodes from Flink
- You wrote you'd like to "instantiate a H2O's node in every task manager".
This reads
I'm not sure this is the right way to do it but we were exploring all the
possibilities and this one is the more obvious. We also spent some time to
study how to do it to achieve a better understanding of Flink's internals.
What we want to do though is to integrate Flink with another distributed
s
Hi Simone,
you are right, the interfaces you extend are not considered to be public,
user-facing API.
Adding custom operators to the DataSet API touches many parts of the system
and is not straightforward.
The DataStream API has better support for custom operators.
Can you explain what kind of op
Hello Fabian,
we delved more moving from the input you gave us but a question arised. We
always assumed that runtime operators were open for extension without
modifying anything inside Flink but it looks like this is not the case and
the documentation assumes that the developer is working to a con
Hi Simone,
the GraphCreatingVisitor transforms the common operator plan into a
representation that is translated by the optimizer.
You have to implement an OptimizerNode and OperatorDescriptor to describe
the operator.
Depending on the semantics of the operator, there are a few more places to
make
Hello,
I'm trying to create a custom operator to explore the internals of Flink.
Actually the one I'm working on is rather similar to Union and I'm trying
to mimick it for now. When I run my job though, this error arise:
Exception in thread "main" java.lang.IllegalArgumentException: Unknown
opera
