Re: After OutOfMemoryError State can not be readed

2018-09-07 Thread Edward Alexander Rojas Clavijo
Stream.readUTF(DataInputStream.java:618) >> at java.io.DataInputStream.readUTF(DataInputStream.java:573) >> at >> >> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:381) >> at >> >> org.apache.flink.contrib.streaming.state.RocksDBValueState.value(RocksDBValueState.java:87) >> ... 12 more >> >> >> Thanks in advance for any help >> >> >> >> >> -- >> Sent from: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >> > > -- *Edward Alexander Rojas Clavijo* *Software EngineerHybrid CloudIBM France*

Re: BucketingSink vs StreamingFileSink

2018-11-21 Thread Edward Alexander Rojas Clavijo
o I guess for this case it's > > recommended to use that option(?). > > Is it possible to use the Parquet writers even when the schema of the > data > > may evolve ? > > > > Thanks in advance for your help. > > (Sorry if I put too many questions in the same message) > > > > > > > > -- > > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ > > -- *Edward Alexander Rojas Clavijo* *Software EngineerHybrid CloudIBM France*

Re: How to migrate Kafka Producer ?

2018-12-19 Thread Edward Alexander Rojas Clavijo
Hi Dawid, Piotr, I see that for the kafka consumer base there are some migration tests here: https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-base/src/test/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBaseMigrationTest.java As the kafka cons

Re: Can checkpoints be used to migrate jobs between Flink versions ?

2019-01-09 Thread Edward Alexander Rojas Clavijo
flink run -s" parameter. > > > > Could you help to clarify if this is advised (and supported) or we should > > stick to the use of savepoints for this kind of manipulations ? > > > > > > Thanks in advance for your help. > > > > Edward > > > > > > > > -- > > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ > > -- *Edward Alexander Rojas Clavijo* *Software EngineerHybrid CloudIBM France*

SSL config on Kubernetes - Dynamic IP

2018-03-27 Thread Edward Alexander Rojas Clavijo
Hi all, Currently I have a Flink 1.4 cluster running on kubernetes and with SSL configuration based on https://ci.apache.org/projects/flink/flink-docs- master/ops/security-ssl.html. However, as the IP of the nodes are dynamic (from the nature of kubernetes), we are using only the DNS which we can

Re: SSL config on Kubernetes - Dynamic IP

2018-03-28 Thread Edward Alexander Rojas Clavijo
68, transport >>> type: NIO, number of server threads: 2 (manual), number of client threads: >>> 2 (manual), server connect backlog: 0 (use Netty's default), client connect >>> timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's >>> default)] >>> ... >>> org.apache.flink.runtime.taskmanager.TaskManager - TaskManager data >>> connection information: bf4a9b50e57c99c17049adb66d65f685 @ >>> flink-taskmanager-1.flink-taskmanager-svc.default.svc.cluster.local >>> (dataPort=6121) >>> >>> >>> >>> But even with that, it seems like the taskmanagers are using the IP >>> communicate between them and the SSL validation fails. >>> >>> Do you know if it's possible to make the taskmanagers to use the >>> hostname to communicate instead of the IP ? >>> or >>> Do you have any advice to get the SSL configuration to work on this >>> environment ? >>> >>> Thanks in advance. >>> >>> Regards, >>> Edward >>> >> >> >> >> -- >> Christophe >> > > -- *Edward Alexander Rojas Clavijo* *Software EngineerHybrid CloudIBM France*

Re: SSL config on Kubernetes - Dynamic IP

2018-03-29 Thread Edward Alexander Rojas Clavijo
, Edward 2018-03-28 17:22 GMT+02:00 Edward Alexander Rojas Clavijo < edward.roja...@gmail.com>: > Hi Till, > > I just created the JIRA ticket: https://issues.apache.org/ > jira/browse/FLINK-9103 > > I added the JobManager and TaskManager logs, Hope this helps to resolve

Re: Regression: On Flink 1.5 CLI -m,--jobmanager option not working

2018-04-27 Thread Edward Alexander Rojas Clavijo
t;> >> I did a /git bisect/ and it seems like the commit introducing the >> regression >> is 47909f4 >> <https://github.com/apache/flink/commit/47909f466b9c9ee1f4ca >> f94e9f6862a21b628817> >> >> I created a JIRA ticket for this: >> https://i

Re: Dynamically deleting kafka topics does not remove partitions from kafkaConsumer

2018-05-07 Thread Edward Alexander Rojas Clavijo
Hello, I've being working on a fix for this, I posted more details on the JIRA ticket. Regards, Edward 2018-05-07 5:51 GMT+02:00 Tzu-Li (Gordon) Tai : > Ah, correct, sorry for the incorrect link. > Thanks Ted! > > > On 7 May 2018 at 11:43:12 AM, Ted Yu (yuzhih...@gmail.com) wrote: > > It seems t