Hi Stephan,
thank you for your explanation.
I thought I will be getting just 100MB of results after the Cross. This
is why I used it.
I will try something else then, most possibly a Map on the input.
Best,
Mihail
On 16.06.2015 04:27, Stephan Ewen wrote:
Cross is a quadratic operation. As such
Cross is a quadratic operation. As such, it produces very large results on
moderate inputs, which can easily exceed memory and disk space, if the
subsequent operation requires to gather all data (such as for the sort in
your case).
If you use on both inputs 10 MB of 100 byte elements (100K element
Hi,
I get the following *"No space left on device" IOException* when using
the following Cross operator.
The inputs for the operator are each just *10MB* in size (same input for
IN1 and IN2; 1000 tuples) and I get the exception after Flink manages to
fill *50GB* of SSD space and the partition
