Thanks. This is exactly what I needed.
Niels On Wed, Oct 26, 2016 at 11:03 AM, Robert Metzger <rmetz...@apache.org> wrote: > Hi Niels, > > Sorry for the late response. > > you can launch a Kafka Broker within a JVM and use it for testing purposes. > Flink's Kafka connector is using that a lot for integration tests. Here is > the code starting the Kafka server: https://github.com/apache/flink/blob/ > 770f2f83a81b2810aff171b2f56390ef686f725a/flink-streaming- > connectors/flink-connector-kafka-0.8/src/test/java/org/ > apache/flink/streaming/connectors/kafka/KafkaTestEnvironmentImpl.java#L358 > > > In our tests we also start a Flink mini cluster, to submit jobs to. > Here is an example: https://github.com/apache/flink/blob/ > f46ca39188dce1764ee6615eb6697588fdc04a2a/flink-streaming- > connectors/flink-connector-kafka-base/src/test/java/org/ > apache/flink/streaming/connectors/kafka/KafkaConsumerTestBase.java#L1408 > > I hope you can find the right code lines to copy for your purposes. > > Regards, > Robert > > > On Fri, Oct 21, 2016 at 4:00 PM, Niels Basjes <ni...@basjes.nl> wrote: > >> Hi, >> >> In addition to having unit tests for the individual components (map, >> flatmap, reduce, etc) of my application I would like to write unit tests >> for the entire flow of my Flink application. >> >> My application reads from Kafka, does various processing and writes out >> put to both kafka and files. >> >> This means I need a controlled (mock) Kafka broker where I can insert a >> specific sequence of messages, my application that reads from that and then >> writes the output somewhere else. >> >> What is the recommended way of doing this? >> What tools/example projects are available for building such tests? >> >> Thanks. >> >> -- >> Best regards / Met vriendelijke groeten, >> >> Niels Basjes >> > > -- Best regards / Met vriendelijke groeten, Niels Basjes
