Re: Flink 1.13 Native Kubernetes - Custom Pod Templates

2021-07-11 Thread Yang Wang
Hi Hemant, Thanks for trying the native Kubernetes integration and share your feedback. Pod template is a supplementary feature for those advanced features(e.g. volume mounts, sidecar container, init container, etc.), which are not supported by Flink config options[1]. By default, Flink is using

Re: Custom type for env.readCsvFile

2021-07-11 Thread Caizhi Weng
Hi! If your custom type is a POJO then readCsvFile should work, otherwise as far as I know there is no way to let the parser directly parse a custom type for you. You can of course implement your own csv input format by extending GenericCsvInputFormat but I think it would be better to just read th

Custom type for env.readCsvFile

2021-07-11 Thread Mason Chan
Hi, I am using the scala api and I want to deserialize a column in a CSV to a custom object type. Is this possible? Here's a code snippet of something I want to do: val rows = env.readCsvFile[(String, CustomType)]("./test.csv", includedFields = Array(0, 1), ignoreFirstLine = true) When I do this

Re: Stream processing into single sink to multiple DB Schemas

2021-07-11 Thread Tamir Sagi
Hey Community, I'd like to update that the solution was eventually simple. We achieved the desired results; each stream has been saved info into corresponded schema in specific table. the connection was handled by JDBC pool. 1. We have created X Schemas(With known format, i.e. employee_ ; u

Flink 1.13 Native Kubernetes - Custom Pod Templates

2021-07-11 Thread bat man
Hi, I am running a POC to run flink 1.13 on Native Kubernetes. Per documentation [1] this supports custom pod templates. Are there any references for sample pod templates which can be used as a baseline to build upon. Could not find any references, documentation[1] has one sample for pod-template.