Re: FlinkML 'DenseVector' object has no attribute 'get_fields_by_names'

user@flink.apache.org Тема: Re: FlinkML 'DenseVector' object has no attribute 'get_fields_by_names' Hi Evgeniy, Yes, the reason of the exception is that you are returning an incorrect data type. Flink ML doesn’t have a data type for `DenseVector` but it pr

Re: FlinkML 'DenseVector' object has no attribute 'get_fields_by_names'

Hi Evgeniy, Yes, the reason of the exception is that you are returning an incorrect data type. Flink ML doesn’t have a data type for `DenseVector` but it provides a function called `pyflink.ml.functions.array_to_vector` which returns an `Expression`. So maybe you can modify your UDF to union mu

Re: FlinkML status

Hi Mohamed, the development of FlinkML has been stopped in favour of a new machine learning library which you can find here [1]. Be aware that this library is still under development. [1] https://github.com/apache/flink/tree/master/flink-ml-parent Cheers, Till On Sat, Aug 1, 2020 at 10:35 AM Mo

Re: FlinkML

Hi, You could try to plug MOA/Weka library too. I did some preliminary work with that: https://moa.cms.waikato.ac.nz/moa-with-apache-flink/ but then it is not anymore FlinkML algorithms. Best regards, Christophe 2018-04-18 21:13 GMT+02:00 shashank734 : > There are no active discussions or gui

Re: FlinkML

There are no active discussions or guide on that. But I found this example in the repo : https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/ml/IncrementalLearningSkeleton.java

Re: FlinkML

Szymon, The short answer is no. See: http://mail-archives.apache.org/mod_mbox/flink-user/201802.mbox/%3ccaadrtt39ciiec1uzwthzgnbkjxs-_h5yfzowhzph_zbidux...@mail.gmail.com%3E On Mon, Apr 16, 2018 at 11:25 PM, Szymon Szczypiński wrote: > Hi, > > i wonder if there are possibility to build FlinkML

Re: FlinkML ALS is taking too long to run

I don't think you need to employ a distributed system for working with this dataset. An SGD implementation on a single machine should easily handle the job. Best, Sebastian 2017-07-12 9:26 GMT+02:00 Andrea Spina : > Dear Ziyad, > > Yep, I had encountered same very long runtimes with ALS as well

Re: FlinkML ALS is taking too long to run

Dear Ziyad, Yep, I had encountered same very long runtimes with ALS as well at the time and I recorded improvements by increasing the number of blocks / decreasing #TSs/TM like you've stated out. Cheers, Andrea -- View this message in context: http://apache-flink-user-mailing-list-archiv

Re: FlinkML ALS is taking too long to run

Dear Ziyad, could you kindly share some additional info about your environment (local/cluster, nodes, machines' configuration)? What does exactly you mean by "indefinitely"? How much time the job is hanging? Hope to help you, then. Cheers, Andrea -- View this message in context: http://apach

Re: FlinkML and DataStream API

Thanks for mentioning it, Theo. Here it is: https://github.com/streamline-eu/ML-Pipelines/tree/stream-ml Look at these examples: https://github.com/streamline-eu/ML-Pipelines/commit/314e3d940f1f1ac7b762ba96067e13d806476f57 On Wed, Dec 21, 2016 at 9:38 PM, wrote: > I'm interested in that code y

Re: FlinkML and DataStream API

I'm interested in that code you mentioned too, I hope you can find it. Regards, Matt > On Dec 21, 2016, at 17:12, Theodore Vasiloudis > wrote: > > Hello Mäki, > > I think what you would like to do is train a model using batch, and use the > Flink streaming API as a way to serve your model a

Re: FlinkML and DataStream API

Hello Mäki, I think what you would like to do is train a model using batch, and use the Flink streaming API as a way to serve your model and make predictions. While we don't have an integrated way to do that in FlinkML currently, I definitely think that's possible. I know Marton Balassi has been

Re: FlinkML - Fail to execute QuickStart example

Hi, No problem I'm going to create a JIRA. Regards Thomas 2016-10-17 21:34 GMT+02:00 Theodore Vasiloudis < theodoros.vasilou...@gmail.com>: > That is my bad, I must have been testing against a private branch when > writing the guide, the SVM as it stands only has a predict operation for > Vecto

Re: FlinkML - Fail to execute QuickStart example

That is my bad, I must have been testing against a private branch when writing the guide, the SVM as it stands only has a predict operation for Vector not LabeledVector. IMHO I would like to have a predict operator for LabeledVector for all predictors (that would just call the existing Vector pred

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi Andrea, the exception says that you don't have enough heap memory available to keep a factors block in memory. You always have to create an object on the heap when the user function is called. You can try the following out to solve the problem. 1. Further decrease the taskmanager.memory.fract

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Ok, I'm still struggling with ALS. Now I'm running with a dataset of 2M users, 250K items, 700 rates per users (1,4B ratings). 50 latent factors, 400 numOfBlocks, 400 DOP. Somehow I got the error, from the JM log I catch the previous mentioned exception: 09/06/2016 19:30:18 CoGroup (CoGroup

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi Stefan, Thank you so much for the answer. Ok, I'll do it asap. For the sake of argument, could the issue be related to the low number of blocks? I noticed the Flink implementation, as default, set the number of blocks to the input count (which is actually a lot). So with a low cardinality and bi

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi, unfortunately, the log does not contain the required information for this case. It seems like a sender to the SortMerger failed. The best way to find this problem is to take a look to the exceptions that are reported in the web front-end for the failing job. Could you check if you find any

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Sure. Here you can find the complete logs file. Still can not run through the issue. Thank you for your help. 2016-08-31 18:15 GMT+02:00 Flavio Pompermaier : > I don't know whether my usual error is related to this one but is very >

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

I don't know whether my usual error is related to this one but is very similar and it happens randomly...I still have to figure out the root cause of the error: java.lang.Exception: The data preparation for task 'CHAIN GroupReduce (GroupReduce at createResult(IndexMappingExecutor.java:43)) -> Map

Re: FlinkML ALS matrix factorization: java.io.IOException: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi, could you provide the log outputs for your job (ideally with debug logging enabled)? Best, Stefan > Am 31.08.2016 um 14:40 schrieb ANDREA SPINA <74...@studenti.unimore.it>: > > Hi everyone. > I'm running the FlinkML ALS matrix factorization and I bumped into the > following exception: >

Re: FlinkML 0.10.1 - Using SparseVectors with MLR does not work

Hi Sourigna, it turned out to be a bug in the GradientDescent implementation which cannot handle sparse gradients. That is not so problematic by itself, because the sum of gradient vectors is usually dense even if the individual gradient vectors are sparse. We simply forgot to initialize the initi

Re: FlinkML 0.10.1 - Using SparseVectors with MLR does not work

Hi Gna, Thanks for reporting the problem. Because level 1 operation in FlinkML BLAS library doesn’t support SparseVector, SparseVector is not supported currently. I’ve filed this to JIRA [1]. Maybe I can send a patch to solve this in few days. [1]: https://issues.apache.org/jira/browse/FLINK-3