You're welcome. I'm glad I could help out :)
Cheers,
Max
On Thu, Jul 2, 2015 at 9:17 PM, Mihail Vieru
wrote:
> I've implemented the alternating 2 files solution and everything works
> now.
>
> Thanks a lot! You saved my day :)
>
> Cheers,
> Mihail
>
>
> On 02.07.2015 12:37, Maximilian Michels
I've implemented the alternating 2 files solution and everything works now.
Thanks a lot! You saved my day :)
Cheers,
Mihail
On 02.07.2015 12:37, Maximilian Michels wrote:
The problem is that your input and output path are the same. Because
Flink executes in a pipelined fashion, all the operat
The problem is that your input and output path are the same. Because Flink
executes in a pipelined fashion, all the operators will come up at once.
When you set WriteMode.OVERWRITE for the sink, it will delete the path
before writing anything. That means that when your DataSource reads the
input, t
Hi Mihail,
Thanks for the code. I'm trying to reproduce the problem now.
On Wed, Jul 1, 2015 at 8:30 PM, Mihail Vieru
wrote:
> Hi Max,
>
> thank you for your reply. I wanted to revise and dismiss all other factors
> before writing back. I've attached you my code and sample input data.
>
> I ru
HI Mihail,
Thank you for your question. Do you have a short example that reproduces
the problem? It is hard to find the cause without an error message or some
example code.
I wonder how your loop works without WriteMode.OVERWRITE because it should
throw an exception in this case. Or do you change
I think my problem is related to a loop in my job.
Before the loop, the writeAsCsv method works fine, even in overwrite mode.
In the loop, in the first iteration, it writes an empty folder
containing empty files to HDFS. Even though the DataSet it is supposed
to write contains elements.
Need
Hi Till,
thank you for your reply.
I have the following code snippet:
/intermediateGraph.getVertices().writeAsCsv(tempGraphOutputPath, "\n",
";", WriteMode.OVERWRITE);/
When I remove the WriteMode parameter, it works. So I can reason that
the DataSet contains data elements.
Cheers,
Mihail
Hi Mihail,
have you checked that the DataSet you want to write to HDFS actually
contains data elements? You can try calling collect which retrieves the
data to your client to see what’s in there.
Cheers,
Till
On Tue, Jun 30, 2015 at 12:01 PM, Mihail Vieru <
vi...@informatik.hu-berlin.de> wrote
Hi,
the writeAsCsv method is not writing anything to HDFS (version 1.2.1)
when the WriteMode is set to OVERWRITE.
A file is created but it's empty. And no trace of errors in the Flink or
Hadoop logs on all nodes in the cluster.
What could cause this issue? I really really need this feature..
