Datastream reset variable in every-time window in map function

the way to achieve this on every next time window? Best Regards, Subash Basnet

Re: how to get rid of duplicate rows group by in DataStream

plements ReduceFunction> { @Override public Tuple4 reduce(Tuple4 val1, Tuple4 val2) { return new Tuple4(val1.f0, val1.f1, val1.f2, val1.f3 + val2.f3); } } Regards, Subash Basnet On Mon, Aug 22, 2016 at 6:34 PM, Kostas Kloudas wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This messa

how to get rid of duplicate rows group by in DataStream

: Needed Output: (1,3) (2,2).. What could be the way to achieve this. Regards, Subash Basnet

counting elements in datastream

Hello all, If anyone had idea, what could be the probable way to count the elements of a current instance of the datastream. Is it possible? DataStream> pointsWithGridCoordinates; Regards, Subash Basnet

Re: Sorting in datastream

-) > That's why the operation does not exist on DataStream. > > Stephan > > > On Wed, Aug 17, 2016 at 6:22 PM, subash basnet wrote: > >> Hello all, >> >> I found the *sortPartition()* function in dataset for ordering the >> dataset elements as below:

Sorting in datastream

ta; DataStream> partitionedData =data.?? What could be the way to achieve sorting in datastream elements. Best Regards, Subash Basnet

Re: counting elements in datastream

Correction: dataset* On Wed, Aug 17, 2016 at 4:39 PM, subash basnet wrote: > Hello all, > > There is *count()* function in database to count the number of elements > in the dataset. But it's not there in case of datastream. What could be the > way to count the number of

counting elements in datastream

Hello all, There is *count()* function in database to count the number of elements in the dataset. But it's not there in case of datastream. What could be the way to count the number of elements in case of datastream. Best Regards, Subash Basnet

applying where after group by in dataset

Hello all, In the following dataset: DataSet> distancePoints; I wanted to count the number of *distancePoints* where boolean value is either true of false. distancePoints.groupBy(1). didn't find how to apply there 'where' clause here. Best Regards, Subash Basnet

DataStreamUtils conversion problem, showing varied results for same code

and the program ends abruptly. Why so much variation in result on executing the same code. Now, in case of centroid id as '-1' in case 1, I would not be able to perform operations later on as all the *clusteredPoints* have the same centroid id '-1' which should have been rather timest

Re: DataStreamUtils not working properly

print all of them. And as exception occurs and program ends, how will I be able to use this *testCentroids* in operations that I need to perform in the later stage of the program. Regards, Subash Basnet On Wed, Jul 20, 2016 at 4:00 PM, Maximilian Michels wrote: > Everything works as expec

Re: DataStreamUtils not working properly

nvironment.getStreamGraph(StreamExecutionEnvironment.java:1195) at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:86) at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1170) Regards, Subash Basn

Re: DataStreamUtils not working properly

ume(); return (new Tuple2(value.getId(), columns)); } }); Regards, Subash Basnet On Wed, Jul 20, 2016 at 12:20 PM, Maximilian Michels wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (m...@apache.org) Add cleanup rule > <http

Understanding iteration error message

the error message. Could you explain it more in depth the error message in relation to above code. Best Regards, Subash Basnet

Re: DataStreamUtils not working properly

collection of datastream via DataStreamUtils is giving empty result. Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 4:52 PM, Till Rohrmann wrote: > It depends if you have a log4j.properties file specified in your > classpath. If you see log output on the console, then it should also print &g

Re: Unable to get the value of datatype in datastream

ollect the clusteredPoints. What could be the solution to achieve this. Regards, Subash Basnet On Tue, Jul 19, 2016 at 12:20 PM, Aljoscha Krettek wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (aljos...@apache.org) Add cleanup rule &g

Re: DataStreamUtils not working properly

pse. Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 2:49 PM, Till Rohrmann wrote: > Have you checked your logs whether they contain some problems? In general > it is not recommended collecting the streaming result back to your client. > It might also be a problem with `Data

Re: DataStreamUtils not working properly

(Mon Jul 18 17:42:59 CEST 2016) (Mon Jul 18 17:34:01 CEST 2016) (Mon Jul 18 17:52:00 CEST 2016) (Mon Jul 18 17:40:58 CEST 2016) for centroidId.print(), but no output for System.out.println(c); Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 10:48 AM, subash basnet wrote: > Hello all, >

DataStreamUtils not working properly

2016 119.535 119.54 119.445 119.455 152900.0 But the next *System.out.println(c) *within the for loop prints nothing. What could be the problem. My maven has following configuration for dataStreamUtils: org.apache.flink flink-streaming-contrib_2.10 ${flink.version} Best Regards, Subash Basnet

Unable to get the value of datatype in datastream

ggest what could be the probable problem or solution to the case. Best Regards, Subash Basnet

Re: how to get rid of null pointer exception in collection in DataStream

Hello Robert, Yup thank you, I used array in place of collection to rid of that error. Best Regards, Subash Basnet On Thu, Jul 14, 2016 at 4:03 PM, Robert Metzger wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (rmetz..

how to get rid of null pointer exception in collection in DataStream

* new Collection(); Best Regards, Subash Basnet

Streaming Exception error message Explanation

at java.net.Socket.connect(Socket.java:538) at java.net.Socket.(Socket.java:434) at java.net.Socket.(Socket.java:244) at org.apache.flink.contrib.streaming.CollectSink.initializeConnection(CollectSink.java:69) Best Regards, Subash Basnet

closewith(...) not working in DataStream error, but works in DataSet

er the computation has terminated.---->For DataStream Best Regards, Subash Basnet

Broadcast and read broadcast variable in DataStream

ecute.run(DataStreamUtils.java:80) Could you please suggest me possible cause and solution to this exception, as I am not able to see any other option beside to use global variable in absence of broadcast of variable in datastream. Best Regards, Subash Basnet

Re: Unable to understand datastream error message

eansOutlierDetection.main(StockAnalysisKMeansOutlierDetection.java:98) Could you please suggest me where I am wrong here? Best Regards, Subash Basnet On Mon, May 16, 2016 at 6:11 PM, Aljoscha Krettek wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (aljos...@

Re: Unable to understand datastream error message

ava:75) at wikiedits.StockAnalysis.main(StockAnalysis.java:64) Best Regards, Subash Basnet On Sat, May 14, 2016 at 4:26 PM, subash basnet wrote: > Hello Aljoscha, > > Below is the shorted version of StockAnalysis class which is a datastream > adapation of the *KMeans.java* dataset

Re: Unable to understand datastream error message

ce=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001> > > Could you please post your code. > > On Sat, 7 May 2016 at 19:16 subash basnet wrote: > >> Hello all, >> >> I am getting the below error on execute of StreamExecutionEnvironment

Unable to understand datastream error message

o that I could solve it. Best Regards, Subash Basnet

Re: how to convert datastream to collection

given in maven repository: org.apache.flink flink-streaming-contrib 0.10.2 Best Regards, Subash Basnet On Wed, May 4, 2016 at 12:12 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > I think you have to explicitly import contrib packages as a dependency in > Maven/SBT. >

Re: how to convert datastream to collection

Hello there, Thank you! But I couldn't find DataStreamUtils. Where is it located? I am using *1.0.0*, The given, org.apache.flink.*contrib*.streaming.DataStreamUtils doesn't work as there is no *contrib *package within org.apache.flink. Best Regards, Subash Basnet On Tue, May 3, 20

how to convert datastream to collection

Hello all, Suppose I have the datastream as: DataStream> *newCentroids*; How to get collection of *newCentroids * to be able to loop as below: private Collection> *centroids*; for (Centroid cent : *centroids*) { } Best Regards, Subash Basnet

Re: How to perform Broadcast and groupBy in DataStream like DataSet

tends MapFunctions<>{ private Collection *centroids*; public void open(Configuration parameters) throws Exception { this.*centroids* = getRuntimeContext().getBroadcastVariable("centroids"); } for (Centroid cent : *centroids*) { } } Best Regards, Subash Basnet On Tue, May 3, 201

How to perform Broadcast and groupBy in DataStream like DataSet

ccumulator()) .map(new CentroidAverager()); DataStream newCentroids = points.map(new SelectNearestCenter()).??? Best Regards, Subash Basnet

first() function in DataStream

Hello all, In DataSet *first(n)* function can be called to get 'n' no. of elements in the DataSet, how could similar operations be done in DataStream to get 'n' no. of elements from the current DataStream. Best Regards, Subash Basnet

Re: Unable to write stream as csv

eturn acc; } }); *result.writeAsText(.); --> It is working. **result.writeAsCsv(.); ---> It is not working. * Best Regards, Subash Basnet On Wed, Apr 27, 2016 at 4:14 PM, subash basnet wrot

Unable to write stream as csv

0.0/kmeans/streaming/result", FileSystem.WriteMode.OVERWRITE); *-> works *result.writeAsCsv("file:///home/softwares/flink-1.0.0/kmeans/streaming/result", FileSystem.WriteMode.OVERWRITE);* --> doesn't work Why is data getting written

adding source not serializable exception in streaming implementation

tStream.java:348) at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:300) at org.apache.flink.api.java.ClosureCleaner.ensureSerializable(ClosureCleaner.java:97) ... 6 more I have attached Stock.java which is just a model with getters and setters. Not sure what am I doing wrong. Best Regards, Subash Basnet packag

Getting elements from DataStream

Regards, Subash Basnet

override file in flink

(Thread.java:745) How to set the necessary override property? Best Regards, Subash Basnet

Re: Unable to run the batch examples after running stream examples

rty was defined. Yeah it's working when I used the original pom of batch, but I am confused why the modified one is not working. Best Regards, Subash Basnet On Mon, Mar 21, 2016 at 5:14 PM, Chesnay Schepler wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is

Unable to run the batch examples after running stream examples

ation.main(PiEstimation.java:63) Previously before running the Kafka examples, the batch examples were working fine, now not. Now only the streaming examples are working. Best Regards, Subash Basnet

define no. of nodes via source code

Hello all, Using the flink-webclient we have the options to define no. of parallelism and the same no. i.e. taskmanager.numberOfTaskSlots is given in flink-conf.yaml. But, where can I define this no. of parallel task when running the examples from IDE. Best Regards, Subash Basnet

Re: Unable to Read from Kafka [ zookeeper.connect error]

Hello Stefano, Thank you, it's working :). Regards, Subash Basnet On Wed, Mar 16, 2016 at 3:24 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Hi Subash, > > you just have to add the following parameter: --zookeeper.connect > localhost:2181 > Let

Unable to Read from Kafka [ zookeeper.connect error]

'zookeeper.connect', or what am I doing wrong? Best Regards, Subash Basnet

how to convert DataStream to DataSet

n, Tuple1> { @Override public Tuple1 map(Tuple2 value) { return new Tuple1(value.f1 + ""); } } But there is type mismatch, with error as: *Type mismatch: cannot convert from SingleOutputStreamOperator> to DataSet>* Best Regards, Subash Basnet

KMeans result folder not created

*writeAsCsv*? Or after *print *statement writes to outputstream of JVM, the *writeAsCsv *doesn't get executed? Best Regards, Subash Basnet

Re: Retrieve elements from the Dataset without using collect

Hello Konstantin, Yup thanks. Best Regards, Subash Basnet On Sun, Mar 6, 2016 at 7:20 PM, Konstantin Knauf < konstantin.kn...@tngtech.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (konstantin.kn...@tngtech.com

Retrieve elements from the Dataset without using collect

y to retrieve individual elements of DataSet without using list via collect? Best Regards, Subash Basnet

Re: Multi-dimensional[more than 2] input for KMeans Clustering inApache flink

Hello Fabian, Thanks! Is KMeans only the clustering implementation currently existing in flink. Best Regards, Subash Basnet On Tue, Mar 1, 2016 at 5:22 PM, Fabian Hueske wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cl

Multi-dimensional[more than 2] input for KMeans Clustering in Apache flink

dimensional data as input implemented in flink? Best Regards, Subash Basnet

How to iterate over DataSet elements without converting it to List

necessary operations } //add elements } //return elements list I need to iterate over dataSet without converting it to List. Best Regards, Subash Basnet

Re: Master Thesis [Apache-flink paper references]

Hello Matthias, Thank you very much :) Best Regards, Subash Basnet On Fri, Feb 12, 2016 at 8:22 PM, Matthias J. Sax wrote: > You might want to check out the Stratosphere project web site: > http://stratosphere.eu/project/publications/ > > -Matthias > > On 02/12/2016 05:52

Master Thesis [Apache-flink paper references]

rials found in internet and flink/data-artisan materials provided. Could you please suggest me. Best Regards, Subash Basnet

Re: How to convert List to flink DataSet

initial element, it prints the same value as initial set value. for(){ newElement = new Tuple3(); dataSetElement.join(env.fromElements(newElement)); dataSetElement.print(); } Unsure if I am using right function or using join in a wrong manner. Best Regards, Subash Basnet On Wed, Feb 10, 2016 at

Re: How to convert List to flink DataSet

ar not able to write via flink web submission client. Best Regards, Subash Basnet On Wed, Feb 10, 2016 at 1:58 PM, Fabian Hueske wrote: > Hi Subash, > > I would not fetch the data to the client, do the computation there, and > send it back, just for the purpose of writing it to a file. &

Re: How to convert List to flink DataSet

it to the console via *fElements.print();* Does it have something to do with *env.exectue("")*, which must be set somewhere in the previous case but not in my case. Best Regards, Subash Basnet On Tue, Feb 9, 2016 at 6:29 PM, Stefano Baghino < stefano.bagh...@radicalbit.io>

How to convert List to flink DataSet

ArrayList(); Following is the datatype of the elements added to the list: Tuple3 newElement = new Tuple3(); finalElements.add(newElement); Now I am stuck on how to convert this 'finalElements' to DataSet> fElements, so that I could use fElements.writeAsCsv(outputPath, "\n"," "); Best Regards, Subash Basnet

Re: Not able to import MBoxParser in MailCount.java

ean install. You can find more info about it here: > http://dataartisans.github.io/flink-training/devSetup/handsOn.html#clone-and-build-the-flink-training-exercises-project > > Hope I helped. > > Best, > Stefano > > On Mon, Feb 1, 2016 at 12:03 PM, subash basnet wrote

Not able to import MBoxParser in MailCount.java

Hello, I copied the plugin configuration for MBoxParser to my pom.xml file from plugin_link but there is no addition of library. So there is error: the import com.dataartisans cannot be resolved. Below is my pom.xml fil