Yes, that sounds very reasonable.
On Jan 20, 2015 6:40 AM, "Stephan Ewen" wrote:
> Yes, I agree that the Avro serializer should be available by default. That
> is one case of a typical type that should work out of the box, given that
> we support Avro file formats.
>
> Let me summarize how I unde
I think we are talking about the Operator I/O types, as types used
internally only by the UDFs should not be serialized.
2015-01-20 12:27 GMT+01:00 Timo Walther :
> Are we talking about the types for the input/output of operators or also
> types that are used inside UDFs?
> Operator I/O type clas
Are we talking about the types for the input/output of operators or also
types that are used inside UDFs?
Operator I/O type classes are known, so we don't need static code
analysis for that. For types inside UDFs I can add that requirement to
FLINK-1319.
On 20.01.2015 11:51, Alexander Alexand
On 20 Jan 2015, at 11:51, Alexander Alexandrov
wrote:
> +1 for program analysis from me too...
>
> Should be doable also on a lower level (e.g. analysis of compiled *.class
> files) with some off-the-shelf libraries, right?
Yes. There was a prototypical implementation using byte code analysis
+1 for program analysis from me too...
Should be doable also on a lower level (e.g. analysis of compiled *.class
files) with some off-the-shelf libraries, right?
2015-01-20 11:39 GMT+01:00 Till Rohrmann :
> I like the idea to automatically figure out which types are used by a
> program and to re
I like the idea to automatically figure out which types are used by a
program and to register them at Kryo. Thus, +1 for this idea.
On Tue, Jan 20, 2015 at 11:34 AM, Robert Metzger
wrote:
> @Stephan: Yes, you are summarizing it correctly.
> I'll assign FLINK-1417 to myself and implement it as di
@Stephan: Yes, you are summarizing it correctly.
I'll assign FLINK-1417 to myself and implement it as discussed here (once I
have resolved the other issues assigned to me)
There is one additional point we forgot in the discussion so far: We are
initializing Kryo with twitter/chill's "ScalaKryoInst
An alternative way that may not be applicable in your case:
For Sopremo, all types implemented a common interface. When a package is
loaded, the Sopremo package manager scans the jar and looks for classes
implementing the interfaces (quite fast, because not the entire class must
be loaded). All ty
Yes, I agree that the Avro serializer should be available by default. That
is one case of a typical type that should work out of the box, given that
we support Avro file formats.
Let me summarize how I understood that suggestion:
- We make Avro available by default by registering a default seria
Hi,
thank you for putting our discussion to the mailing list. This is indeed
where such discussions belong. For the others, we started discussing here:
https://github.com/apache/flink/pull/304
I think there is one additional approach, which is probably close to (1):
We only register those seriali
Hi all!
We have various pending pull requests that add support for certain types by
adding extra kryo serializers.
I think we need to decide how we want to handle the support for extra
types, because more are certainly to come.
As I understand it, we have three broad options:
(1)
Add as many se
11 matches
Mail list logo
