[jira] [Created] (FLINK-4517) scala code refactoring
shijinkui created FLINK-4517:
Summary: scala code refactoring
Key: FLINK-4517
URL: https://issues.apache.org/jira/browse/FLINK-4517
Project: Flink
Issue Type: Improvement
Reporter: shijinkui
Priority: Minor
such as:
1. case class does't need `new`
2. case block does't need `{ ... }`
3. match-case instead of isInstanceOf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink issue #2414: [FLINK-4341] Let idle consumer subtasks emit max value wa...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2414 To include the missing case @rmetzger mentioned, it turns out the fix is actually more complicated than I expected due to correct state determination after every reshard, and requires a bit of rework on our current shard discovery mechanism to get it right. Heads-up notice that this will probably need a re-review. Sorry for the delay, I'm currently still on it, hopefully will update the PR by the end of today ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-4341) Kinesis connector does not emit maximum watermark properly
[
https://issues.apache.org/jira/browse/FLINK-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441113#comment-15441113
]
ASF GitHub Bot commented on FLINK-4341:
---
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/2414
To include the missing case @rmetzger mentioned, it turns out the fix is
actually more complicated than I expected due to correct state determination
after every reshard, and requires a bit of rework on our current shard
discovery mechanism to get it right.
Heads-up notice that this will probably need a re-review. Sorry for the
delay, I'm currently still on it, hopefully will update the PR by the end of
today ;)
> Kinesis connector does not emit maximum watermark properly
> --
>
> Key: FLINK-4341
> URL: https://issues.apache.org/jira/browse/FLINK-4341
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
>Affects Versions: 1.1.0, 1.1.1
>Reporter: Scott Kidder
>Assignee: Robert Metzger
>Priority: Blocker
> Fix For: 1.2.0, 1.1.2
>
>
> **Prevously reported as "Checkpoint state size grows unbounded when task
> parallelism not uniform"**
> This issue was first encountered with Flink release 1.1.0 (commit 45f7825). I
> was previously using a 1.1.0 snapshot (commit 18995c8) which performed as
> expected. This issue was introduced somewhere between those commits.
> I've got a Flink application that uses the Kinesis Stream Consumer to read
> from a Kinesis stream with 2 shards. I've got 2 task managers with 2 slots
> each, providing a total of 4 slots. When running the application with a
> parallelism of 4, the Kinesis consumer uses 2 slots (one per Kinesis shard)
> and 4 slots for subsequent tasks that process the Kinesis stream data. I use
> an in-memory store for checkpoint data.
> Yesterday I upgraded to Flink 1.1.0 (45f7825) and noticed that checkpoint
> states were growing unbounded when running with a parallelism of 4,
> checkpoint interval of 10 seconds:
> {code}
> ID State Size
> 1 11.3 MB
> 220.9 MB
> 3 30.6 MB
> 4 41.4 MB
> 5 52.6 MB
> 6 62.5 MB
> 7 71.5 MB
> 8 83.3 MB
> 9 93.5 MB
> {code}
> The first 4 checkpoints generally succeed, but then fail with an exception
> like the following:
> {code}
> java.lang.RuntimeException: Error triggering a checkpoint as the result of
> receiving checkpoint barrier at
> org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:768)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:758)
> at
> org.apache.flink.streaming.runtime.io.BarrierBuffer.processBarrier(BarrierBuffer.java:203)
> at
> org.apache.flink.streaming.runtime.io.BarrierBuffer.getNextNonBlocked(BarrierBuffer.java:129)
> at
> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:183)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:66)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:266)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Size of the state is larger than the maximum
> permitted memory-backed state. Size=12105407 , maxSize=5242880 . Consider
> using a different state backend, like the File System State backend.
> at
> org.apache.flink.runtime.state.memory.MemoryStateBackend.checkSize(MemoryStateBackend.java:146)
> at
> org.apache.flink.runtime.state.memory.MemoryStateBackend$MemoryCheckpointOutputStream.closeAndGetBytes(MemoryStateBackend.java:200)
> at
> org.apache.flink.runtime.state.memory.MemoryStateBackend$MemoryCheckpointOutputStream.closeAndGetHandle(MemoryStateBackend.java:190)
> at
> org.apache.flink.runtime.state.AbstractStateBackend$CheckpointStateOutputView.closeAndGetHandle(AbstractStateBackend.java:447)
> at
> org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.snapshotOperatorState(WindowOperator.java:879)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:598)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$2.onEvent(StreamTask.java:762)
> ... 8 more
> {code}
> Or:
> {code}
> 2016-08-09 17:44:43,626 INFO
> org.apache.flink.streaming.runtime.tasks.StreamTask - Restoring
> checkpointed state to task Fold: property_id, player -> 10-minute
> Sliding-Window Percentile Aggregation -> Sink: InfluxDB (2/4)
> 2016-08-09 17:44:51,236 ERROR akka.remote.EndpointWriter-
> Transient association error (association remains live)
> akka.remote.OversizedPayloadException: Discarding oversized payload sen
[GitHub] flink pull request #2428: [FLINK-4517] scala code refactoring
GitHub user shijinkui opened a pull request:
https://github.com/apache/flink/pull/2428
[FLINK-4517] scala code refactoring
the refacter type:
1. case class does't need `new`
2. case block does't need `{ ... }`
3. match-case instead of isInstanceOf
references the related [FLINK-4517]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shijinkui/flink scala-refactoring
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2428.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2428
commit b02bef1f23943b08908f408e03f75fb8c5220a28
Author: shijinkui
Date: 2016-08-27T09:17:04Z
[FLINK-4517] scala code refactoring
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-4517) scala code refactoring
[
https://issues.apache.org/jira/browse/FLINK-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441203#comment-15441203
]
ASF GitHub Bot commented on FLINK-4517:
---
GitHub user shijinkui opened a pull request:
https://github.com/apache/flink/pull/2428
[FLINK-4517] scala code refactoring
the refacter type:
1. case class does't need `new`
2. case block does't need `{ ... }`
3. match-case instead of isInstanceOf
references the related [FLINK-4517]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shijinkui/flink scala-refactoring
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2428.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2428
commit b02bef1f23943b08908f408e03f75fb8c5220a28
Author: shijinkui
Date: 2016-08-27T09:17:04Z
[FLINK-4517] scala code refactoring
> scala code refactoring
> --
>
> Key: FLINK-4517
> URL: https://issues.apache.org/jira/browse/FLINK-4517
> Project: Flink
> Issue Type: Improvement
>Reporter: shijinkui
>Priority: Minor
>
> such as:
> 1. case class does't need `new`
> 2. case block does't need `{ ... }`
> 3. match-case instead of isInstanceOf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (FLINK-4518) maxLineLength change to 120
shijinkui created FLINK-4518: Summary: maxLineLength change to 120 Key: FLINK-4518 URL: https://issues.apache.org/jira/browse/FLINK-4518 Project: Flink Issue Type: Improvement Reporter: shijinkui scala checkstyle maxLineLength increase to 120 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-4519) scala maxLineLength increased to 120
shijinkui created FLINK-4519: Summary: scala maxLineLength increased to 120 Key: FLINK-4519 URL: https://issues.apache.org/jira/browse/FLINK-4519 Project: Flink Issue Type: Improvement Reporter: shijinkui `tools/maven/scalastyle-config.xml` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request #2429: [FLINK-4519] scala maxLineLength increased to 120
GitHub user shijinkui opened a pull request: https://github.com/apache/flink/pull/2429 [FLINK-4519] scala maxLineLength increased to 120 Because Scala function's parameter is more long than Java code, We can set the maxLineLength to 120. The pull request references the related JIRA issue FLINK-4519 You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/flink maxLineLength_120 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2429.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2429 commit 583ca8d40228ade306f1b52be898ac5ffc8ce230 Author: shijinkui Date: 2016-08-27T10:37:03Z [FLINK-4519] scala maxLineLength increased to 120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-4519) scala maxLineLength increased to 120
[ https://issues.apache.org/jira/browse/FLINK-4519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441232#comment-15441232 ] ASF GitHub Bot commented on FLINK-4519: --- GitHub user shijinkui opened a pull request: https://github.com/apache/flink/pull/2429 [FLINK-4519] scala maxLineLength increased to 120 Because Scala function's parameter is more long than Java code, We can set the maxLineLength to 120. The pull request references the related JIRA issue FLINK-4519 You can merge this pull request into a Git repository by running: $ git pull https://github.com/shijinkui/flink maxLineLength_120 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/2429.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2429 commit 583ca8d40228ade306f1b52be898ac5ffc8ce230 Author: shijinkui Date: 2016-08-27T10:37:03Z [FLINK-4519] scala maxLineLength increased to 120 > scala maxLineLength increased to 120 > -- > > Key: FLINK-4519 > URL: https://issues.apache.org/jira/browse/FLINK-4519 > Project: Flink > Issue Type: Improvement >Reporter: shijinkui > > `tools/maven/scalastyle-config.xml` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (FLINK-4518) maxLineLength change to 120
[ https://issues.apache.org/jira/browse/FLINK-4518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shijinkui closed FLINK-4518. Resolution: Duplicate > maxLineLength change to 120 > --- > > Key: FLINK-4518 > URL: https://issues.apache.org/jira/browse/FLINK-4518 > Project: Flink > Issue Type: Improvement >Reporter: shijinkui > > scala checkstyle maxLineLength increase to 120 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2254) Add Bipartite Graph Support for Gelly
[ https://issues.apache.org/jira/browse/FLINK-2254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441673#comment-15441673 ] Ivan Mushketyk commented on FLINK-2254: --- [~vkalavri] I went through the paper listed in this issue, but I still have some questions. If I understand correctly bipartite graph should be analyzed differently than a regular (one-mode) graph. For example in case of movies/actors graph, we may consider that two actors are connected if they have a movie in common, a conclusion we can come to only if we know that the graph is bipartite. I wonder if we should treat bipartite graph is a special case of graph? In other words does all operations (methods and algorithms) defined for the Graph class make sense for a bipartite graph? Also the issue description mentions that "We would like to support efficient operations for this type of graphs". What are these operations? Are there any operations additional operations that we need for bipartiate graph? > Add Bipartite Graph Support for Gelly > - > > Key: FLINK-2254 > URL: https://issues.apache.org/jira/browse/FLINK-2254 > Project: Flink > Issue Type: New Feature > Components: Gelly >Affects Versions: 0.10.0 >Reporter: Andra Lungu >Assignee: Ivan Mushketyk > Labels: requires-design-doc > > A bipartite graph is a graph for which the set of vertices can be divided > into two disjoint sets such that each edge having a source vertex in the > first set, will have a target vertex in the second set. We would like to > support efficient operations for this type of graphs along with a set of > metrics(http://jponnela.com/web_documents/twomode.pdf). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4440) Make API for edge/vertex creation less verbose
[
https://issues.apache.org/jira/browse/FLINK-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442325#comment-15442325
]
ASF GitHub Bot commented on FLINK-4440:
---
Github user mushketyk closed the pull request at:
https://github.com/apache/flink/pull/2398
> Make API for edge/vertex creation less verbose
> --
>
> Key: FLINK-4440
> URL: https://issues.apache.org/jira/browse/FLINK-4440
> Project: Flink
> Issue Type: Improvement
> Components: Gelly
>Reporter: Ivan Mushketyk
>Assignee: Ivan Mushketyk
>Priority: Trivial
>
> It would be better if one could create vertex/edges like this:
> {code:java}
> Vertex v = Vertex.create(42);
> Edge e = Edge.create(5, 6);
> {code}
> Instead of this:
> {code:java}
> Vertex v = new Vertex(42,
> NullValue.getInstance());
> Edge e = new Edge NullValue>(5, 6, NullValue.getInstance());
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink issue #2398: [FLINK-4440][gelly] Add Vertex.create and Edge.create hel...
Github user mushketyk commented on the issue: https://github.com/apache/flink/pull/2398 Closing according to discussion in JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-4440) Make API for edge/vertex creation less verbose
[
https://issues.apache.org/jira/browse/FLINK-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442326#comment-15442326
]
ASF GitHub Bot commented on FLINK-4440:
---
Github user mushketyk commented on the issue:
https://github.com/apache/flink/pull/2398
Closing according to discussion in JIRA.
> Make API for edge/vertex creation less verbose
> --
>
> Key: FLINK-4440
> URL: https://issues.apache.org/jira/browse/FLINK-4440
> Project: Flink
> Issue Type: Improvement
> Components: Gelly
>Reporter: Ivan Mushketyk
>Assignee: Ivan Mushketyk
>Priority: Trivial
>
> It would be better if one could create vertex/edges like this:
> {code:java}
> Vertex v = Vertex.create(42);
> Edge e = Edge.create(5, 6);
> {code}
> Instead of this:
> {code:java}
> Vertex v = new Vertex(42,
> NullValue.getInstance());
> Edge e = new Edge NullValue>(5, 6, NullValue.getInstance());
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] flink pull request #2398: [FLINK-4440][gelly] Add Vertex.create and Edge.cre...
Github user mushketyk closed the pull request at: https://github.com/apache/flink/pull/2398 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #2430: [FLINK-3874] Rewrite Kafka JSON Table sink tests
GitHub user mushketyk opened a pull request:
https://github.com/apache/flink/pull/2430
[FLINK-3874] Rewrite Kafka JSON Table sink tests
Turned Kafka JSON Table sink tests into unit tests as discussed here:
https://issues.apache.org/jira/browse/FLINK-3874
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [x] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mushketyk/flink fix-kafka-tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2430.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2430
commit bdb52d2164d3cea83a9cc9121dc3663e50e50aa9
Author: Ivan Mushketyk
Date: 2016-08-27T22:24:21Z
[FLINK-3874] Rewrite Kafka JSON Table sink tests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (FLINK-3874) Add a Kafka TableSink with JSON serialization
[
https://issues.apache.org/jira/browse/FLINK-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442329#comment-15442329
]
ASF GitHub Bot commented on FLINK-3874:
---
GitHub user mushketyk opened a pull request:
https://github.com/apache/flink/pull/2430
[FLINK-3874] Rewrite Kafka JSON Table sink tests
Turned Kafka JSON Table sink tests into unit tests as discussed here:
https://issues.apache.org/jira/browse/FLINK-3874
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [x] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mushketyk/flink fix-kafka-tests
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2430.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2430
commit bdb52d2164d3cea83a9cc9121dc3663e50e50aa9
Author: Ivan Mushketyk
Date: 2016-08-27T22:24:21Z
[FLINK-3874] Rewrite Kafka JSON Table sink tests
> Add a Kafka TableSink with JSON serialization
> -
>
> Key: FLINK-3874
> URL: https://issues.apache.org/jira/browse/FLINK-3874
> Project: Flink
> Issue Type: New Feature
> Components: Table API & SQL
>Reporter: Fabian Hueske
>Assignee: Ivan Mushketyk
>Priority: Minor
>
> Add a TableSink that writes JSON serialized data to Kafka.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1707) Add an Affinity Propagation Library Method
[ https://issues.apache.org/jira/browse/FLINK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442548#comment-15442548 ] Josep Rubió commented on FLINK-1707: Hi [~vkalavri] I've pushed a new version of AP with following changes: https://github.com/joseprupi/flink/blob/vertexcentric/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/AffinityPropagation.java - I've changed the model to the vertex centric avoiding having the values in the the vertices - I've added the option of no damping. When a 0 factor of damping is passed to the constructor the convergence condition is no changes on the exemplars on a certain number of iterations. This number of iterations is the last parameter of the constructor. If a damping factor different to 0 is used it keep working as before, having to hold the old sent values in the vertex. To do: - I have not changed the initialization of the graph yet. I posted a question in dev thread with no much luck. Maybe I will implement an initialization with a similarity matrix for now and will see how I can do it using gelly functionality later - I will try to change where are the weight values to be in the edges instead of vertices (I've created a new version of the design document with the diagrams too). This way vertices will only have the old values in case the damping factor has to be used. > Add an Affinity Propagation Library Method > -- > > Key: FLINK-1707 > URL: https://issues.apache.org/jira/browse/FLINK-1707 > Project: Flink > Issue Type: New Feature > Components: Gelly >Reporter: Vasia Kalavri >Assignee: Josep Rubió >Priority: Minor > Labels: requires-design-doc > Attachments: Binary_Affinity_Propagation_in_Flink_design_doc.pdf > > > This issue proposes adding the an implementation of the Affinity Propagation > algorithm as a Gelly library method and a corresponding example. > The algorithm is described in paper [1] and a description of a vertex-centric > implementation can be found is [2]. > [1]: http://www.psi.toronto.edu/affinitypropagation/FreyDueckScience07.pdf > [2]: http://event.cwi.nl/grades2014/00-ching-slides.pdf > Design doc: > https://docs.google.com/document/d/1QULalzPqMVICi8jRVs3S0n39pell2ZVc7RNemz_SGA4/edit?usp=sharing > Example spreadsheet: > https://docs.google.com/spreadsheets/d/1CurZCBP6dPb1IYQQIgUHVjQdyLxK0JDGZwlSXCzBcvA/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (FLINK-1707) Add an Affinity Propagation Library Method
[ https://issues.apache.org/jira/browse/FLINK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442548#comment-15442548 ] Josep Rubió edited comment on FLINK-1707 at 8/28/16 1:47 AM: - Hi [~vkalavri] I've pushed a new version of AP with following changes: https://github.com/joseprupi/flink/blob/vertexcentric/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/AffinityPropagation.java - I've changed the model to the vertex centric avoiding having the values in the the vertices - I've added the option of no damping. When a 0 factor of damping is passed to the constructor the convergence condition is no changes on the exemplars on a certain number of iterations, avoiding to have the old values in vertices. This number of iterations is the last parameter of the constructor. If a damping factor different to 0 is used it keeps working as before, having to hold the old sent values in the vertex. To do: - I have not changed the initialization of the graph yet. I posted a question in dev thread with no much luck. Maybe I will implement an initialization with a similarity matrix for now and will see how I can do it using gelly functionality later - I will try to change where are the weight values to be in the edges instead of vertices (I've created a new version of the design document with the diagrams too). This way vertices will only have the old values in case the damping factor has to be used. was (Author: joseprupi): Hi [~vkalavri] I've pushed a new version of AP with following changes: https://github.com/joseprupi/flink/blob/vertexcentric/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/AffinityPropagation.java - I've changed the model to the vertex centric avoiding having the values in the the vertices - I've added the option of no damping. When a 0 factor of damping is passed to the constructor the convergence condition is no changes on the exemplars on a certain number of iterations. This number of iterations is the last parameter of the constructor. If a damping factor different to 0 is used it keep working as before, having to hold the old sent values in the vertex. To do: - I have not changed the initialization of the graph yet. I posted a question in dev thread with no much luck. Maybe I will implement an initialization with a similarity matrix for now and will see how I can do it using gelly functionality later - I will try to change where are the weight values to be in the edges instead of vertices (I've created a new version of the design document with the diagrams too). This way vertices will only have the old values in case the damping factor has to be used. > Add an Affinity Propagation Library Method > -- > > Key: FLINK-1707 > URL: https://issues.apache.org/jira/browse/FLINK-1707 > Project: Flink > Issue Type: New Feature > Components: Gelly >Reporter: Vasia Kalavri >Assignee: Josep Rubió >Priority: Minor > Labels: requires-design-doc > Attachments: Binary_Affinity_Propagation_in_Flink_design_doc.pdf > > > This issue proposes adding the an implementation of the Affinity Propagation > algorithm as a Gelly library method and a corresponding example. > The algorithm is described in paper [1] and a description of a vertex-centric > implementation can be found is [2]. > [1]: http://www.psi.toronto.edu/affinitypropagation/FreyDueckScience07.pdf > [2]: http://event.cwi.nl/grades2014/00-ching-slides.pdf > Design doc: > https://docs.google.com/document/d/1QULalzPqMVICi8jRVs3S0n39pell2ZVc7RNemz_SGA4/edit?usp=sharing > Example spreadsheet: > https://docs.google.com/spreadsheets/d/1CurZCBP6dPb1IYQQIgUHVjQdyLxK0JDGZwlSXCzBcvA/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (FLINK-1707) Add an Affinity Propagation Library Method
[ https://issues.apache.org/jira/browse/FLINK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442548#comment-15442548 ] Josep Rubió edited comment on FLINK-1707 at 8/28/16 1:48 AM: - Hi [~vkalavri], I've pushed a new version of AP with following changes: https://github.com/joseprupi/flink/blob/vertexcentric/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/AffinityPropagation.java - I've changed the model to the vertex centric avoiding having the values in the the vertices - I've added the option of no damping. When a 0 factor of damping is passed to the constructor the convergence condition is no changes on the exemplars on a certain number of iterations, avoiding to have the old values in vertices. This number of iterations is the last parameter of the constructor. If a damping factor different to 0 is used it keeps working as before, having to hold the old sent values in the vertex. To do: - I have not changed the initialization of the graph yet. I posted a question in dev thread with no much luck. Maybe I will implement an initialization with a similarity matrix for now and will see how I can do it using gelly functionality later - I will try to change where are the weight values to be in the edges instead of vertices (I've created a new version of the design document with the diagrams too). This way vertices will only have the old values in case the damping factor has to be used. Thanks!! was (Author: joseprupi): Hi [~vkalavri] I've pushed a new version of AP with following changes: https://github.com/joseprupi/flink/blob/vertexcentric/flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/AffinityPropagation.java - I've changed the model to the vertex centric avoiding having the values in the the vertices - I've added the option of no damping. When a 0 factor of damping is passed to the constructor the convergence condition is no changes on the exemplars on a certain number of iterations, avoiding to have the old values in vertices. This number of iterations is the last parameter of the constructor. If a damping factor different to 0 is used it keeps working as before, having to hold the old sent values in the vertex. To do: - I have not changed the initialization of the graph yet. I posted a question in dev thread with no much luck. Maybe I will implement an initialization with a similarity matrix for now and will see how I can do it using gelly functionality later - I will try to change where are the weight values to be in the edges instead of vertices (I've created a new version of the design document with the diagrams too). This way vertices will only have the old values in case the damping factor has to be used. > Add an Affinity Propagation Library Method > -- > > Key: FLINK-1707 > URL: https://issues.apache.org/jira/browse/FLINK-1707 > Project: Flink > Issue Type: New Feature > Components: Gelly >Reporter: Vasia Kalavri >Assignee: Josep Rubió >Priority: Minor > Labels: requires-design-doc > Attachments: Binary_Affinity_Propagation_in_Flink_design_doc.pdf > > > This issue proposes adding the an implementation of the Affinity Propagation > algorithm as a Gelly library method and a corresponding example. > The algorithm is described in paper [1] and a description of a vertex-centric > implementation can be found is [2]. > [1]: http://www.psi.toronto.edu/affinitypropagation/FreyDueckScience07.pdf > [2]: http://event.cwi.nl/grades2014/00-ching-slides.pdf > Design doc: > https://docs.google.com/document/d/1QULalzPqMVICi8jRVs3S0n39pell2ZVc7RNemz_SGA4/edit?usp=sharing > Example spreadsheet: > https://docs.google.com/spreadsheets/d/1CurZCBP6dPb1IYQQIgUHVjQdyLxK0JDGZwlSXCzBcvA/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
Hao Chen created FLINK-4520: --- Summary: Integrate Siddhi as a lightweight CEP Library Key: FLINK-4520 URL: https://issues.apache.org/jira/browse/FLINK-4520 Project: Flink Issue Type: New Feature Components: CEP Affects Versions: 1.2.0 Reporter: Hao Chen Siddhi CEP is a lightweight, easy-to-use Open Source Complex Event Processing Engine (CEP) released as a Java Library under Apache Software License v2.0. Siddhi CEP processes events which are generated by various event sources, analyses them and notifies appropriate complex events according to the user specified queries. * http://wso2.com/products/complex-event-processor/ * https://github.com/wso2/siddhi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
