Yes, you can transform the broadcast set when it is accessed with
RuntimeContext.getBroadcastVariableWithInitializer() and a
BroadcastVariableInitializer.
2016-05-06 14:07 GMT+02:00 Flavio Pompermaier :
> That was more or less what I was thinking. The only thing I'm not sure is
> the usage of the
That was more or less what I was thinking. The only thing I'm not sure is
the usage of the broadcasted dataset, since I'd need to access tot the
MetaData dataset by sourceId (so I'd need an Map.
Probably I'd do:
Map meta = ...;//preparing metadata lookUp table
...
ds.map(MetaMapFunctionWrapper(new
Hi Flavio,
I'll open a JIRA for de/serializing TableSource to textual JSON.
Would something like this work for you?
main() {
ExecutionEnvironment env = ...
TableEnvironment tEnv = ...
// accessing an external catalog
YourTableSource ts = Catalog.getTableSource("someIdentifier");
tEnv.
HI Fabian,
thanks for your detailed answer, as usual ;)
I think that an external service it's ok,actually I wasn't aware of the
TableSource interface.
As you said, an utility to serialize and deserialize them would be very
helpful and will ease this thing.
However, registering metadata for a table
Hi Flavio,
I thought a bit about your proposal. I am not sure if it is actually
necessary to integrate a central source repository into Flink. It should be
possible to offer this as an external service which is based on the
recently added TableSource interface. TableSources could be extended to be
