Re: a typical ML algorithm flow

2016-03-29 Thread Shannon Quinn
Apologies for hijacking, but this thread hits right at my last message to this list (looking to implement native iterations in the PyFlink API). I'm particularly interested in custom convergence criteria, often centered around measuring some sort of squared loss and checking if it falls below

Native iterations in PyFlink

2016-03-24 Thread Shannon Quinn
Hi all, I'm looking at Flink for highly iterative ALS-like distributed computations, and the concept of native iteration support was very attractive. However, I notice that the Python API is missing this item. I'd absolutely be interested in adding that component if someone could point me in

[jira] [Created] (FLINK-3626) zipWithIndex in Python API

2016-03-19 Thread Shannon Quinn (JIRA)
Shannon Quinn created FLINK-3626: Summary: zipWithIndex in Python API Key: FLINK-3626 URL: https://issues.apache.org/jira/browse/FLINK-3626 Project: Flink Issue Type: New Feature

Re: zipWithIndex in Python API

2016-03-14 Thread Shannon Quinn
ontext constructor, add a getIndexOfThisSubtask() method and you're set. Feel free to open a JIRA for this. On 11.03.2016 18:15, Shannon Quinn wrote: Hi all, I'm interested in getting involved the Python API development. The first use-case I've encountered in my work is that of zipWithIn

zipWithIndex in Python API

2016-03-11 Thread Shannon Quinn
Hi all, I'm interested in getting involved the Python API development. The first use-case I've encountered in my work is that of zipWithIndex, so I started looking into how to go about implementing that. It looks like the core of it involves being able to uniquely identify what worker you're