[
https://issues.apache.org/jira/browse/KAFKA-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17352924#comment-17352924
]
Juan C. Gonzalez-Zurita commented on KAFKA-12718:
-------------------------------------------------
Hello [~mjsax]. Sorry for this late follow up but I've mostly fixed the issue.
I've corrected the two test cases that were giving issues due to being tuned to
accept erroneous results wherein the gap time was previously absent. Them being
located in
org.apache.kafka.streams.kstream.internals.KStreamSessionWindowAggregateProcessorTest.
But now I have an issue with shouldSupportFinalResultsForSessionWindows in
org.apache.kafka.streams.kstream.internals.SuppressScenarioTest that I am
unsure of how to fix. I do not understand why the test case has been modeled
that way. The error result given is as follows:
{code:java}
java.lang.AssertionError: [
TestRecord[key=[k1@0/0], value=1, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00Z]
TestRecord[key=[k1@0/0], value=null, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00Z]
TestRecord[key=[k1@0/5], value=2, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.005Z]
TestRecord[key=[k1@0/5], value=null, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.005Z]
TestRecord[key=[k1@0/5], value=3, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.005Z]
TestRecord[key=[k2@6/6], value=1, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.006Z]
TestRecord[key=[k1@0/5], value=null, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.005Z]
TestRecord[key=[k1@0/5], value=4, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.005Z]
TestRecord[key=[k1@30/30], value=1, headers=RecordHeaders(headers = [],
isReadOnly = false), recordTime=1970-01-01T00:00:00.030Z]
] != [KeyValueTimestamp{key=[k1@0/0], value=1, timestamp=0},
KeyValueTimestamp{key=[k1@0/0], value=null, timestamp=0},
KeyValueTimestamp{key=[k1@0/5], value=2, timestamp=5},
KeyValueTimestamp{key=[k1@0/5], value=null, timestamp=5},
KeyValueTimestamp{key=[k1@0/5], value=3, timestamp=5},
KeyValueTimestamp{key=[k2@6/6], value=1, timestamp=6},
KeyValueTimestamp{key=[k1@30/30], value=1, timestamp=30}]
{code}
I can't seem to make sense of what k1@0/5 means or where value is derived from
as the original only shows "v1" in its value argument to pipeInput or why the
original has 6 values but output-raw has 7. If you could offer some assistance
it would be greatly appreciated. I will be back tomorrow sometime in the
afternoon to check back on this again. In the meantime, if you need me to
upload my work so far or require anything else to make it easier on you to help
simply suggest it.
> SessionWindows are closed too early
> -----------------------------------
>
> Key: KAFKA-12718
> URL: https://issues.apache.org/jira/browse/KAFKA-12718
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Juan C. Gonzalez-Zurita
> Priority: Major
> Labels: beginner, easy-fix, newbie
> Fix For: 3.0.0
>
>
> SessionWindows are defined based on a {{gap}} parameter, and also support an
> additional {{grace-period}} configuration to handle out-of-order data.
> To incorporate the session-gap a session window should only be closed at
> {{window-end + gap}} and to incorporate grace-period, the close time should
> be pushed out further to {{window-end + gap + grace}}.
> However, atm we compute the window close time as {{window-end + grace}}
> omitting the {{gap}} parameter.
> Because default grace-period is 24h most users might not notice this issues.
> Even if they set a grace period explicitly (eg, when using suppress()), they
> would most likely set a grace-period larger than gap-time not hitting the
> issue (or maybe only realize it when inspecting the behavior closely).
> However, if a user wants to disable the grace-period and sets it to zero (on
> any other value smaller than gap-time), sessions might be close too early and
> user might notice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
