I don't think there is a truly sane way to do this.

I could envision a separate application triggering samples via the REST API, writing the results into kafka which your operator can read. This is probably the most reasonable solution I can come up with.

Any attempt at accessing the TaskExecutor or metrics from within the operator are inadvisable; you'd be encroaching into truly hacky territory.

You could also do your own backpressure sampling within your operator (separate thread within the operator executing the same sampling logic), but I don't know how easy it would be to re-use Flink code.


On 06/11/2019 13:40, Felipe Gutierrez wrote:
Does anyone know in which metric I can rely on to know if a given operator is activating the backpressure? Or how can I call the same java object that the Flink UI calls to give me the ratio of backpressure?

Thanks,
Felipe

*--*
*-- Felipe Gutierrez*
*-- skype: felipe.o.gutierrez
*
*--****_https://felipeogutierrez.blogspot.com_*


On Tue, Nov 5, 2019 at 4:15 PM Felipe Gutierrez <[email protected] <mailto:[email protected]>> wrote:

    Hi Zhijiang,

    thanks for your reply. Yes, you understood correctly.
    The fact that I cannot get
    "Shuffle.Netty.Input.Buffers.inputQueueLength" on the operator
    might be because of the way Flink runtime architecture was
    designed. But I was wondering what kind of signal I can get. I
    guess some backpressure message I could get because backpressure
    works to slow down the upstream operators.

    For example, I can see the ratio per sub-task on the web interface
    [1]. It means the physical operators. Is there any message flowing
    backward that I can get? Is there anything that makes me able to
    not rely on some external storage?

    [1]
    
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/back_pressure.html#sampling-threads
    *--*
    *-- Felipe Gutierrez*
    *-- skype: felipe.o.gutierrez
    *
    *--****_https://felipeogutierrez.blogspot.com_*


    On Tue, Nov 5, 2019 at 12:23 PM Zhijiang
    <[email protected] <mailto:[email protected]>>
    wrote:

        Hi Felipe,

        That is an interesting idea to control the upstream's output
        based on downstream's input.

        If I understood correctly, the preAggregate operator would
        trigger flush output while the reduce operator is idle/hungry.
        In contrast, the preAggregate would continue aggregating data
        in the case of back pressure.

        I think this requirement is valid, but unfortunately I guess
        you can not get the back pressure signal from the operator
        level. AIK only the upper task level can get the input/output
        state to decide whether to process or not.

        If you want to get the reduce's metric of
        `Shuffle.Netty.Input.Buffers.inputQueueLength` on preAggregate
        side, you might rely on some external metric reporter to query
        it if possible.

        Best,
        Zhijiang

            ------------------------------------------------------------------
            From:Felipe Gutierrez <[email protected]
            <mailto:[email protected]>>
            Send Time:2019 Nov. 5 (Tue.) 16:58
            To:user <[email protected] <mailto:[email protected]>>
            Subject:How can I get the backpressure signals inside my
            function or operator?

            Hi all,

            let's say that I have a "source -> map .> preAggregrate ->
            keyBy -> reduce -> sink" job and the reducer is sending
            backpressure signals to the preAggregate, map and source
            operator. How do I get those signals inside my operator's
            implementation?
            I guess inside the function is not possible. But if I have
            my own operator implemented (preAggregate) can I get those
            backpressure signals?

            I want to get the messages
            "Shuffle.Netty.Input.Buffers.inputQueueLength" [1] on my
            preAggregate operator in order to decide when I stop the
            pre-aggregation and flush tuples or when I keep pre
            aggregating. It is something like the "credit based
            control on the network stack" [2].

            [1]
            
https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/metrics.html#default-shuffle-service
            [2] https://www.youtube.com/watch?v=AbqatHF3tZI

            Thanks!
            Felipe
            *--*
            *-- Felipe Gutierrez*
            *-- skype: felipe.o.gutierrez
            *
            *--***_https://felipeogutierrez.blogspot.com_



Reply via email to