Hi Steve,
This normally shouldn’t happen, unless there simply is two copies of the data.
What is the source of the topology? Also, this might be obvious, but if you
have broadcasted your input stream to the sink, then each sink instance would
then get all records in the input stream.
Cheers,
Gordon
On March 24, 2017 at 9:11:35 AM, Steve Jerman (st...@kloudspot.com) wrote:
Hi,
I have a sink writing data to InfluxDB. I’ve noticed that the sink gets
multiple copies of upstream records..
Why does this happen, and how can I avoid it… ?
Below is a trace …showing 2 records (I have a parallelism of two) for each
record in the ‘.printToError’ for the same stream.
Any help/suggestions appreciated.
Steve
1> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=2F:A4:BD:56:EC:4D, sessionTime=15000}]
2> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=17:8E:FC:7E:F7:20, sessionTime=20000}]
1> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=40:07:2D:CB:41:07, sessionTime=5000}]
2> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=28:24:0B:B6:42:CA, sessionTime=5000}]
1> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=8F:13:AC:4A:DA:93, sessionTime=5000}]
2> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=43:7D:8A:D4:7D:D7, sessionTime=5000}]
2> Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=77:CD:BD:48:EE:D8, sessionTime=5000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=2F:A4:BD:56:EC:4D, sessionTime=15000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=2F:A4:BD:56:EC:4D, sessionTime=15000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=17:8E:FC:7E:F7:20, sessionTime=20000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=17:8E:FC:7E:F7:20, sessionTime=20000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=40:07:2D:CB:41:07, sessionTime=5000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=40:07:2D:CB:41:07, sessionTime=5000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=28:24:0B:B6:42:CA, sessionTime=5000}]
18:41:27,351 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=28:24:0B:B6:42:CA, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=8F:13:AC:4A:DA:93, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=8F:13:AC:4A:DA:93, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=43:7D:8A:D4:7D:D7, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=43:7D:8A:D4:7D:D7, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=77:CD:BD:48:EE:D8, sessionTime=5000}]
18:41:27,352 INFO com.kloudspot.flink.sink.InfluxSink
- Point [name=Presence, time=1490316086000, tags={ap=1OPUA7IKN9,
associated=false, dwellTime=less_than_fifteen_mins, location=1OPUA7IKN9,
returned=false, sessionEnd=false}, precision=MILLISECONDS,
fields={client=77:CD:BD:48:EE:D8, sessionTime=5000}]
