Hello,
I'm facing an issue in one of our Kafka Streams applications using
GlobalKTable. The idea is to have a GlobalKTable over compacted topic and
be able to re-read it on startup. We had a consumer group and topic
sometime ago, recently I've recreated a topic, requiring consumer offsets
to be reset and consumed from beginning. But application started to fail
with OffsetOutOfRangeException and message "Offsets out of range with no
configured reset policy for partitions..". I do have auto.offset.reset set
in my configuration to "latest", but it is overridden to "none" for global
and restore consumers of Streams application.
This exception is resulting in a shutdown loop, and requiring investigation
to understand what is going on.
This is the line where it is happening
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L1244
So the question is this behavior of overriding offset reset policy is
intended? If not, please confirm it is a bug, and will submit a patch.
See the Streams output for detailed configs, traceback and exceptions in
attachment.
Thanks,
A.
[2020-05-06 13:03:19,360: INFO/main] (AbstractConfig.java:347) - StreamsConfig
values:
application.id = *-green-staging
application.server =
bootstrap.servers = [*:9092]
buffered.records.per.partition = 1000
cache.max.bytes.buffering = 10485760
client.id = *-green-staging
commit.interval.ms = 30000
connections.max.idle.ms = 540000
default.deserialization.exception.handler = class
org.apache.kafka.streams.errors.LogAndContinueExceptionHandler
default.key.serde = class
org.apache.kafka.common.serialization.Serdes$StringSerde
default.production.exception.handler = class
com.test.LogAndContinueProductionExceptionHandler
default.timestamp.extractor = class
org.apache.kafka.streams.processor.FailOnInvalidTimestamp
default.value.serde = class
io.confluent.kafka.streams.serdes.avro.SpecificAvroSerde
max.task.idle.ms = 0
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
num.standby.replicas = 0
num.stream.threads = 1
partition.grouper = class
org.apache.kafka.streams.processor.DefaultPartitionGrouper
poll.ms = 100
processing.guarantee = at_least_once
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
replication.factor = 1
request.timeout.ms = 20000
retries = 2147483647
retry.backoff.ms = 500
rocksdb.config.setter = null
security.protocol = SASL_SSL
send.buffer.bytes = 131072
state.cleanup.delay.ms = 600000
state.dir = /mnt/storage
topology.optimization = none
upgrade.from = null
windowstore.changelog.additional.retention.ms = 86400000
[2020-05-06 13:03:19,386: INFO/main] (KafkaStreams.java:686) - stream-client
[*-green-staging] Kafka Streams version: 2.4.0
[2020-05-06 13:03:19,386: INFO/main] (KafkaStreams.java:687) - stream-client
[*-green-staging] Kafka Streams commit ID: 77a89fcf8d7fa018
[2020-05-06 13:03:21,446: INFO/main] (AbstractConfig.java:347) - ConsumerConfig
values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = none
bootstrap.servers = [*:9092]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = *-green-staging-global-consumer
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = null
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = false
isolation.level = read_uncommitted
key.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class
org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 20000
retry.backoff.ms = 500
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
[2020-05-06 13:05:21,115: INFO/main] (AbstractLogin.java:61) - Successfully
logged in.
[2020-05-06 13:05:21,455: WARN/main] (AbstractConfig.java:355) - The
configuration 'schema.registry.url' was supplied but isn't a known config.
[2020-05-06 13:05:21,457: INFO/main] (AppInfoParser.java:117) - Kafka version:
5.4.0-ccs
[2020-05-06 13:05:21,457: INFO/main] (AppInfoParser.java:118) - Kafka commitId:
f4201a82bea68cc7
[2020-05-06 13:05:21,457: INFO/main] (AppInfoParser.java:119) - Kafka
startTimeMs: 1588770321455
[2020-05-06 13:05:21,465: INFO/main] (AbstractConfig.java:347) -
AdminClientConfig values:
bootstrap.servers = [*:9092]
client.dns.lookup = use_all_dns_ips
client.id = *-green-staging-admin
connections.max.idle.ms = 300000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 20000
retries = 2147483647
retry.backoff.ms = 500
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
[2020-05-06 13:05:21,493: WARN/main] (AbstractConfig.java:355) - The
configuration 'schema.registry.url' was supplied but isn't a known config.
[2020-05-06 13:05:21,493: INFO/main] (AppInfoParser.java:117) - Kafka version:
5.4.0-ccs
[2020-05-06 13:05:21,493: INFO/main] (AppInfoParser.java:118) - Kafka commitId:
f4201a82bea68cc7
[2020-05-06 13:05:21,493: INFO/main] (AppInfoParser.java:119) - Kafka
startTimeMs: 1588770321493
[2020-05-06 13:05:21,494: INFO/main] (StreamThread.java:495) - stream-thread
[*-green-staging-StreamThread-1] Creating restore consumer client
[2020-05-06 13:05:21,496: INFO/main] (AbstractConfig.java:347) - ConsumerConfig
values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = none
bootstrap.servers = [*:9092]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = *-green-staging-StreamThread-1-restore-consumer
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = null
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = false
isolation.level = read_uncommitted
key.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class
org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 20000
retry.backoff.ms = 500
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
[2020-05-06 13:05:33,372: WARN/main] (AbstractConfig.java:355) - The
configuration 'schema.registry.url' was supplied but isn't a known config.
[2020-05-06 13:05:33,372: INFO/main] (AppInfoParser.java:117) - Kafka version:
5.4.0-ccs
[2020-05-06 13:05:33,372: INFO/main] (AppInfoParser.java:118) - Kafka commitId:
f4201a82bea68cc7
[2020-05-06 13:05:33,372: INFO/main] (AppInfoParser.java:119) - Kafka
startTimeMs: 1588770333372
[2020-05-06 13:05:33,375: INFO/main] (StreamThread.java:505) - stream-thread
[*-green-staging-StreamThread-1] Creating shared producer client
[2020-05-06 13:05:33,381: INFO/main] (AbstractConfig.java:347) - ProducerConfig
values:
acks = 1
batch.size = 16384
bootstrap.servers = [*:9092]
buffer.memory = 33554432
client.dns.lookup = use_all_dns_ips
client.id = *-green-staging-StreamThread-1-producer
compression.type = none
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class
org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 100
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class
org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 20000
retries = 2147483647
retry.backoff.ms = 500
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class
org.apache.kafka.common.serialization.ByteArraySerializer
[2020-05-06 13:05:33,419: WARN/main] (AbstractConfig.java:355) - The
configuration 'schema.registry.url' was supplied but isn't a known config.
[2020-05-06 13:05:33,420: INFO/main] (AppInfoParser.java:117) - Kafka version:
5.4.0-ccs
[2020-05-06 13:05:33,420: INFO/main] (AppInfoParser.java:118) - Kafka commitId:
f4201a82bea68cc7
[2020-05-06 13:05:33,420: INFO/main] (AppInfoParser.java:119) - Kafka
startTimeMs: 1588770333419
[2020-05-06 13:05:33,432: INFO/main] (StreamThread.java:544) - stream-thread
[*-green-staging-StreamThread-1] Creating consumer client
[2020-05-06 13:05:33,435: INFO/main] (AbstractConfig.java:347) - ConsumerConfig
values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [*:9092]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = *-green-staging-StreamThread-1-consumer
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = *-green-staging
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = false
isolation.level = read_uncommitted
key.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy =
[org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 20000
retry.backoff.ms = 500
sasl.client.callback.handler.class = null
sasl.jaas.config = [hidden]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = PLAIN
security.protocol = SASL_SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class
org.apache.kafka.common.serialization.ByteArrayDeserializer
[2020-05-06 13:06:40,376: INFO/main] (AssignorConfiguration.java:156) -
stream-thread [*-green-staging-StreamThread-1-consumer] Cooperative rebalancing
enabled now
[2020-05-06 13:06:40,443: WARN/main] (AbstractConfig.java:355) - The
configuration 'schema.registry.url' was supplied but isn't a known config.
[2020-05-06 13:06:40,444: WARN/main] (AbstractConfig.java:355) - The
configuration 'admin.retry.backoff.ms' was supplied but isn't a known config.
[2020-05-06 13:06:40,444: WARN/main] (AbstractConfig.java:355) - The
configuration 'admin.retries' was supplied but isn't a known config.
[2020-05-06 13:06:40,444: INFO/main] (AppInfoParser.java:117) - Kafka version:
5.4.0-ccs
[2020-05-06 13:06:40,445: INFO/main] (AppInfoParser.java:118) - Kafka commitId:
f4201a82bea68cc7
[2020-05-06 13:06:40,445: INFO/main] (AppInfoParser.java:119) - Kafka
startTimeMs: 1588770400444
=== starting HTTP server
[2020-05-06 13:06:40,579: INFO/main] (Log.java:193) - Logging initialized
@692866ms to org.eclipse.jetty.util.log.Slf4jLog
[2020-05-06 13:06:40,924: INFO/kafka-admin-client-thread |
*-green-staging-admin] (AdminMetadataManager.java:238) - [AdminClient
clientId=*-green-staging-admin] Metadata update failed
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the
call.
[2020-05-06 13:06:40,995: INFO/main] (Server.java:370) - jetty-9.4.z-SNAPSHOT;
built: 2019-06-10T16:30:51.723Z; git: afcf563148970e98786327af5e07c261fda175d3;
jvm 11.0.7+10
[2020-05-06 13:06:41,199: INFO/main] (StandardDescriptorProcessor.java:283) -
NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
[2020-05-06 13:06:41,211: INFO/main] (DefaultSessionIdManager.java:365) -
DefaultSessionIdManager workerName=node0
[2020-05-06 13:06:41,211: INFO/main] (DefaultSessionIdManager.java:370) - No
SessionScavenger set, using defaults
[2020-05-06 13:06:41,214: INFO/main] (HouseKeeper.java:149) - node0 Scavenging
every 660000ms
[2020-05-06 13:06:41,285: INFO/main] (ContextHandler.java:857) - Started
o.e.j.w.WebAppContext@6579c3d9{/,file:///tmp/,AVAILABLE}
[2020-05-06 13:06:41,296: INFO/main] (AbstractConnector.java:292) - Started
ServerConnector@2970a5bc{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
[2020-05-06 13:06:41,296: INFO/main] (Server.java:410) - Started @693583ms
=== starting Kafka Streams server
[2020-05-06 13:06:41,298: INFO/main] (KafkaStreams.java:276) - stream-client
[*-green-staging] State transition from CREATED to REBALANCING
[2020-05-06 13:06:41,336: INFO/*-green-staging-GlobalStreamThread]
(AbstractConfig.java:179) - KafkaAvroSerializerConfig values:
bearer.auth.token = [hidden]
schema.registry.ssl.truststore.type = JKS
schema.reflection = false
auto.register.schemas = true
basic.auth.credentials.source = URL
schema.registry.ssl.keystore.password = [hidden]
schema.registry.ssl.provider =
schema.registry.ssl.endpoint.identification.algorithm = https
schema.registry.ssl.truststore.location =
value.subject.name.strategy = class
io.confluent.kafka.serializers.subject.TopicNameStrategy
schema.registry.url =
[http://base-cp-schema-registry.kube-system.svc.cluster.local:8081]
schema.registry.ssl.keystore.location =
schema.registry.ssl.trustmanager.algorithm = PKIX
schema.registry.ssl.key.password = [hidden]
schema.registry.ssl.keystore.type = JKS
proxy.port = -1
schema.registry.ssl.secure.random.implementation =
schema.registry.ssl.cipher.suites = []
max.schemas.per.subject = 1000
schema.registry.ssl.truststore.password = [hidden]
basic.auth.user.info = [hidden]
proxy.host =
schema.registry.ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
schema.registry.ssl.protocol = TLS
schema.registry.basic.auth.user.info = [hidden]
bearer.auth.credentials.source = STATIC_TOKEN
schema.registry.ssl.keymanager.algorithm = SunX509
key.subject.name.strategy = class
io.confluent.kafka.serializers.subject.TopicNameStrategy
[2020-05-06 13:06:41,438: INFO/kafka-producer-network-thread |
*-green-staging-StreamThread-1-producer] (Metadata.java:261) - [Producer
clientId=*-green-staging-StreamThread-1-producer] Cluster ID: lkc-43m2m
[2020-05-06 13:06:41,554: INFO/*-green-staging-GlobalStreamThread]
(AbstractConfig.java:179) - KafkaAvroDeserializerConfig values:
bearer.auth.token = [hidden]
schema.registry.ssl.truststore.type = JKS
schema.reflection = false
auto.register.schemas = true
basic.auth.credentials.source = URL
schema.registry.ssl.keystore.password = [hidden]
schema.registry.ssl.provider =
schema.registry.ssl.endpoint.identification.algorithm = https
schema.registry.ssl.truststore.location =
specific.avro.reader = true
value.subject.name.strategy = class
io.confluent.kafka.serializers.subject.TopicNameStrategy
schema.registry.url =
[http://base-cp-schema-registry.kube-system.svc.cluster.local:8081]
schema.registry.ssl.keystore.location =
schema.registry.ssl.trustmanager.algorithm = PKIX
schema.registry.ssl.key.password = [hidden]
schema.registry.ssl.keystore.type = JKS
proxy.port = -1
schema.registry.ssl.secure.random.implementation =
schema.registry.ssl.cipher.suites = []
max.schemas.per.subject = 1000
schema.registry.ssl.truststore.password = [hidden]
basic.auth.user.info = [hidden]
proxy.host =
schema.registry.ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
schema.registry.ssl.protocol = TLS
schema.registry.basic.auth.user.info = [hidden]
bearer.auth.credentials.source = STATIC_TOKEN
schema.registry.ssl.keymanager.algorithm = SunX509
key.subject.name.strategy = class
io.confluent.kafka.serializers.subject.TopicNameStrategy
[2020-05-06 13:06:41,576: INFO/*-green-staging-GlobalStreamThread]
(GlobalStateManagerImpl.java:209) - global-stream-thread
[*-green-staging-GlobalStreamThread] Restoring state for global store
pow-availability
[2020-05-06 13:06:41,659: INFO/*-green-staging-GlobalStreamThread]
(Metadata.java:261) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Cluster ID: lkc-43m2m
[2020-05-06 13:06:41,982: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-0
[2020-05-06 13:06:41,987: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-0
[2020-05-06 13:06:42,008: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-0 to offset 0.
[2020-05-06 13:06:42,011: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-5
[2020-05-06 13:06:42,012: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-5
[2020-05-06 13:06:42,015: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-5 to offset 0.
[2020-05-06 13:06:42,015: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-4
[2020-05-06 13:06:42,016: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-4
[2020-05-06 13:06:42,019: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-4 to offset 0.
[2020-05-06 13:06:42,019: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-1
[2020-05-06 13:06:42,019: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-1
[2020-05-06 13:06:42,022: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-1 to offset 0.
[2020-05-06 13:06:42,145: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-2
[2020-05-06 13:06:42,146: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-2
[2020-05-06 13:06:42,148: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-2 to offset 0.
[2020-05-06 13:06:42,149: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-pow-availability-changelog-green-staging-3
[2020-05-06 13:06:42,149: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:568) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Seeking to EARLIEST
offset of partition xod-pow-availability-changelog-green-staging-3
[2020-05-06 13:06:42,154: INFO/*-green-staging-GlobalStreamThread]
(SubscriptionState.java:385) - [Consumer
clientId=*-green-staging-global-consumer, groupId=null] Resetting offset for
partition xod-pow-availability-changelog-green-staging-3 to offset 0.
[2020-05-06 13:06:42,155: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1077) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Unsubscribed all topics or patterns and assigned partitions
[2020-05-06 13:06:42,310: INFO/*-green-staging-GlobalStreamThread]
(RocksDBTimestampedStore.java:100) - Opening store
xod-domain-config-merged-green-staging-STATE-STORE-0000000007 in regular mode
[2020-05-06 13:06:42,312: INFO/*-green-staging-GlobalStreamThread]
(GlobalStateManagerImpl.java:209) - global-stream-thread
[*-green-staging-GlobalStreamThread] Restoring state for global store
xod-domain-config-merged-green-staging-STATE-STORE-0000000007
[2020-05-06 13:06:42,647: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-0
[2020-05-06 13:06:42,647: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 3 for partition
xod-domain-config-merged-green-staging-0
[2020-05-06 13:06:42,648: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-5
[2020-05-06 13:06:42,648: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-5
[2020-05-06 13:06:42,649: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-10
[2020-05-06 13:06:42,649: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 27 for partition
xod-domain-config-merged-green-staging-10
[2020-05-06 13:06:42,649: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-8
[2020-05-06 13:06:42,649: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-8
[2020-05-06 13:06:42,649: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-2
[2020-05-06 13:06:42,650: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-2
[2020-05-06 13:06:42,650: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-9
[2020-05-06 13:06:42,650: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-9
[2020-05-06 13:06:42,677: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-11
[2020-05-06 13:06:42,678: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-11
[2020-05-06 13:06:42,678: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-4
[2020-05-06 13:06:42,678: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 1 for partition
xod-domain-config-merged-green-staging-4
[2020-05-06 13:06:42,678: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-1
[2020-05-06 13:06:42,678: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-1
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-6
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-6
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-7
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-7
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-3
[2020-05-06 13:06:42,679: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-3
[2020-05-06 13:06:42,680: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1077) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Unsubscribed all topics or patterns and assigned partitions
[2020-05-06 13:06:42,692: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1123) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Subscribed to partition(s):
xod-domain-config-merged-green-staging-0,
xod-domain-config-merged-green-staging-1,
xod-pow-availability-changelog-green-staging-5,
xod-domain-config-merged-green-staging-2,
xod-domain-config-merged-green-staging-3,
xod-pow-availability-changelog-green-staging-2,
xod-pow-availability-changelog-green-staging-1,
xod-pow-availability-changelog-green-staging-4,
xod-pow-availability-changelog-green-staging-3,
xod-pow-availability-changelog-green-staging-0,
xod-domain-config-merged-green-staging-4,
xod-domain-config-merged-green-staging-5,
xod-domain-config-merged-green-staging-6,
xod-domain-config-merged-green-staging-7,
xod-domain-config-merged-green-staging-8,
xod-domain-config-merged-green-staging-9,
xod-domain-config-merged-green-staging-10,
xod-domain-config-merged-green-staging-11
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 3 for partition
xod-domain-config-merged-green-staging-0
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-1
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-pow-availability-changelog-green-staging-5
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-2
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-3
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-pow-availability-changelog-green-staging-2
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 162 for partition
xod-pow-availability-changelog-green-staging-1
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-pow-availability-changelog-green-staging-4
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-pow-availability-changelog-green-staging-3
[2020-05-06 13:06:42,693: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-pow-availability-changelog-green-staging-0
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 1 for partition
xod-domain-config-merged-green-staging-4
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-5
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-6
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-7
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-8
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 1 for partition
xod-domain-config-merged-green-staging-9
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 27 for partition
xod-domain-config-merged-green-staging-10
[2020-05-06 13:06:42,694: INFO/*-green-staging-GlobalStreamThread]
(KafkaConsumer.java:1564) - [Consumer clientId=*-green-staging-global-consumer,
groupId=null] Seeking to offset 0 for partition
xod-domain-config-merged-green-staging-11
[2020-05-06 13:06:42,695: INFO/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:159) - global-stream-thread
[*-green-staging-GlobalStreamThread] State transition from CREATED to RUNNING
[2020-05-06 13:06:42,697: INFO/*-green-staging-StreamThread-1]
(StreamThread.java:664) - stream-thread [*-green-staging-StreamThread-1]
Starting
[2020-05-06 13:06:42,697: INFO/*-green-staging-StreamThread-1]
(StreamThread.java:219) - stream-thread [*-green-staging-StreamThread-1] State
transition from CREATED to STARTING
[2020-05-06 13:06:42,700: INFO/*-green-staging-StreamThread-1]
(KafkaConsumer.java:1032) - [Consumer
clientId=*-green-staging-StreamThread-1-consumer, groupId=*-green-staging]
Subscribed to pattern:
'xod-incoming-green-staging|xod-kube-events-green-staging'
[2020-05-06 13:06:42,715: ERROR/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:248) - global-stream-thread
[*-green-staging-GlobalStreamThread] Updating global state failed. You can
restart KafkaStreams to recover from this error.
org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of
range with no configured reset policy for partitions:
{xod-domain-config-merged-green-staging-4=1}
at
org.apache.kafka.clients.consumer.internals.Fetcher.initializeCompletedFetch(Fetcher.java:1259)
at
org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:610)
at
org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1313)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1240)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1211)
at
org.apache.kafka.streams.processor.internals.GlobalStreamThread$StateConsumer.pollAndUpdate(GlobalStreamThread.java:238)
at
org.apache.kafka.streams.processor.internals.GlobalStreamThread.run(GlobalStreamThread.java:289)
[2020-05-06 13:06:42,716: INFO/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:159) - global-stream-thread
[*-green-staging-GlobalStreamThread] State transition from RUNNING to
PENDING_SHUTDOWN
[2020-05-06 13:06:42,716: INFO/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:297) - global-stream-thread
[*-green-staging-GlobalStreamThread] Shutting down
[2020-05-06 13:06:42,746: INFO/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:159) - global-stream-thread
[*-green-staging-GlobalStreamThread] State transition from PENDING_SHUTDOWN to
DEAD
[2020-05-06 13:06:42,746: INFO/*-green-staging-GlobalStreamThread]
(KafkaStreams.java:276) - stream-client [*-green-staging] State transition from
REBALANCING to ERROR
[2020-05-06 13:06:42,747: ERROR/*-green-staging-GlobalStreamThread]
(KafkaStreams.java:486) - stream-client [*-green-staging] Global thread has
died. The instance will be in error state and should be closed.
[2020-05-06 13:06:42,747: INFO/*-green-staging-GlobalStreamThread]
(GlobalStreamThread.java:309) - global-stream-thread
[*-green-staging-GlobalStreamThread] Shutdown complete
[2020-05-06 13:06:42,751: INFO/shutdownHook1] (KafkaStreams.java:276) -
stream-client [*-green-staging] State transition from ERROR to PENDING_SHUTDOWN
[2020-05-06 13:06:42,753: INFO/kafka-streams-close-thread]
(StreamThread.java:1098) - stream-thread [*-green-staging-StreamThread-1]
Informed to shut down
[2020-05-06 13:06:42,754: INFO/kafka-streams-close-thread]
(StreamThread.java:219) - stream-thread [*-green-staging-StreamThread-1] State
transition from STARTING to PENDING_SHUTDOWN
[2020-05-06 13:06:42,805: INFO/*-green-staging-StreamThread-1]
(StreamThread.java:1112) - stream-thread [*-green-staging-StreamThread-1]
Shutting down
[2020-05-06 13:06:42,805: INFO/*-green-staging-StreamThread-1]
(KafkaConsumer.java:1077) - [Consumer
clientId=*-green-staging-StreamThread-1-restore-consumer, groupId=null]
Unsubscribed all topics or patterns and assigned partitions
[2020-05-06 13:06:42,805: INFO/*-green-staging-StreamThread-1]
(KafkaProducer.java:1183) - [Producer
clientId=*-green-staging-StreamThread-1-producer] Closing the Kafka producer
with timeoutMillis = 9223372036854775807 ms.
[2020-05-06 13:06:42,820: INFO/*-green-staging-StreamThread-1]
(StreamThread.java:219) - stream-thread [*-green-staging-StreamThread-1] State
transition from PENDING_SHUTDOWN to DEAD
[2020-05-06 13:06:42,820: INFO/*-green-staging-StreamThread-1]
(StreamThread.java:1132) - stream-thread [*-green-staging-StreamThread-1]
Shutdown complete
