as reading a big file would be
> a
> big problem in terms of memory.
>
>
>
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-first-operator-tp6377p6451.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
ld be a
big problem in terms of memory.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-first-operator-tp6377p6451.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
Hi Biplop,
you can also implement a generic IF that wraps another IF (such as a
CsvInputFormat).
The wrapping IF forwards all calls to the wrapped IF and in addition counts
how many records were emitted (how often InputFormat.nextRecord() was
called).
Once the count arrives at the threshold, it re
Hey Biplob,
Yes, the file source will read all input. The first operator will add
a combiner to the source for pre-aggregation and then shuffle
everything to a single reduce instance, which emits the N first
elements. Keep in mind that there is no strict order in which the
records will be emitted.
Hi,
It might be a naive question but I was concerned as I am trying to read
from a file.
My question is if I have a file with n lines and i want m lines out of that
where n << m, would the first operator process only the first m lines or
would it go through the entire file?
If it does go through
