Hi Arjun, welcome to the Flink community :-)
On Thu, Jan 7, 2016 at 5:40 AM, Arjun Rao <sporty.ar...@gmail.com> wrote: > Hi, > > I am new to Apache Flink and I really like the look of the API. I have been > working with Storm for the past year and have questions about the > DataStream API among others. > > 1. What are the interactions of the actor system in the flink ecosystem? > Where can I find more information? > Actors are used internally by the system for the communication between the different distributed components (Client, JobManager and TaskManager). Thus, they are used to submit jobs, send job updates and delivering the execution result for example. However, the actual data transfer of your job is done using Netty. > 2. Is there a Flink web UI in embedded mode? ( Not on cluster ) > What do you mean with embedded mode? When you start Flink locally, then you will also have a web UI. > 3. Why is there a poll time min of 50ms for the Datastream API? > Where do you see the poll time of 50 ms in the DataStream API? In order to make the data transfer more efficient, data records are grouped together and shipped together. The size of this group is defined by the network buffer size. Whenever such a buffer is full, it will be sent. However, in order to avoid that records are never sent, there is also a buffer timeout after which the elements of the buffer are sent even if the buffer is not yet full. > 4. Is it possible to log each datastream to a separate file?( i.e. not all in the same task manager log ). > What do you mean by logging different DataStreams to separate files? At the moment, all the logging which happens on the TaskManager is written to the log file of this task manager. > 5. I checked out the master from git, but I am unable to build the project > in Intellij, due to compilation errors in flink-staging. The root cause is > lack of "generated" files of avro such as > > import org.apache.flink.api.io.avro.generated.Address; > import org.apache.flink.api.io.avro.generated.Colors; > import org.apache.flink.api.io.avro.generated.Fixed16; > import org.apache.flink.api.io.avro.generated.User; > > These files are not being found. I have the avro plugin installed for > Intellij. What else do I need to do to make the project build/compile? > I think the first time, you have to execute `mvn clean package -DskipTests -Dmaven.javadoc.skip=true` on the command line in order to generate the missing files. > > Appreciate the help! > > Best, > > Arjun > Cheers, Till
