gaoyunhaii opened a new pull request #7: URL: https://github.com/apache/flink-ml/pull/7
This PR implements the feedback mechanism and the controller operators. For feedback mechanism, we will rely on the feedback edge implemented in the Flink Statefun project, which wrap the feedback records via mails. Since processing mail have higher priority compared to processing normal record, it ensures we always first process the feedback records. The feedback edge currently does not support spilling to disk, we would add this support in the future versions. The controller operators include 1. Input operator: wrap user records into IterationRecord, which contains the round information. 2. Output operator: unwrap the IterationRecord into the user records, and discard the events inside the iteration body. 3. Tail operator: the operator increment the round by 1 and emit the record into the feedback edges. 4. Head operator: union the records from the normal input and feedback, and synchronized the epoch watermark message. Since an iteration might have multiple head operators, the OperatorCoordinator would do the global synchronized via a static variable. Also note that the criteria stream would also have the corresponding head operator. Finally, the head operator would be used for all kinds of iterations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
