[
https://issues.apache.org/jira/browse/FLINK-22436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Perez updated FLINK-22436:
--------------------------------
Description:
Sorry for selecting Connectors/Common, Twitter didn't show up in the Connectors
/ "X" list, I'm not sure why that is.
in addition to item FLINK-22435 I found this exception as well when the recent
version (1.12) twitter connector just hangs.
basically I am following the example here:
[https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/twitter/TwitterExample.java]
which just seems to run indefinitely but not have any data throughput/results
or exceptions.
task manager stdout (not logs) looks something like this:
{code:java}
WARNING: Please consider reporting this to the maintainers of
org.apache.flink.shaded.akka.org.jboss.netty.util.internal.ByteBufferUtilWARNING:
Use --illegal-access=warn to enable warnings of further illegal reflective
access operationsWARNING: All illegal access operations will be denied in a
future releaseMay 01, 2021 8:37:04 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
May 01, 2021 8:39:06 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
May 01, 2021 8:49:48 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
{code}
So the part that stood out to me was "CRLF expected at end of chunk", then
stacktrace.
I'm not sure which is the cause of twitter connector not working between this
and FLINK-22435 but I'm reporting both just in case.
I can provide more info if needed but my setup is really basic/simple.
was:
Sorry for selecting Connectors/Common, Twitter didn't show up in the Connectors
/ "X" list, I'm not sure why that is.
in addition to item FLINK-22435 I found this exception as well when the recent
version (1.12) twitter connector just hangs.
basically I am following the example here:
[https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/twitter/TwitterExample.java]
which just seems to run indefinitely but not have any data throughput/results
or exceptions.
task manager stdout (not logs) looks something like this:
{code:java}
WARNING: Please consider reporting this to the maintainers of
org.apache.flink.shaded.akka.org.jboss.netty.util.internal.ByteBufferUtilWARNING:
Use --illegal-access=warn to enable warnings of further illegal reflective
access operationsWARNING: All illegal access operations will be denied in a
future releaseMay 01, 2021 8:37:04 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
May 01, 2021 8:39:06 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
May 01, 2021 8:49:48 AM
org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
IOException thrown while closing
Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
of chunk at
org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
at
org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
at
org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
at
org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
at
org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
at
org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at
org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at
java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
at java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136)
at
org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
at
org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
{code}
and this just seems to repeat in the log.
I can provide more info if needed but my setup is really basic/simple.
> twitter datastream connector hangs, CRLF expected at end of chunk
> -----------------------------------------------------------------
>
> Key: FLINK-22436
> URL: https://issues.apache.org/jira/browse/FLINK-22436
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, Connectors / Common
> Environment: My environment is just simple local flink env, using the
> twitter connector example.
> Reporter: Jason Perez
> Priority: Major
>
> Sorry for selecting Connectors/Common, Twitter didn't show up in the
> Connectors / "X" list, I'm not sure why that is.
>
> in addition to item FLINK-22435 I found this exception as well when the
> recent version (1.12) twitter connector just hangs.
>
> basically I am following the example here:
> [https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/twitter/TwitterExample.java]
>
> which just seems to run indefinitely but not have any data throughput/results
> or exceptions.
>
> task manager stdout (not logs) looks something like this:
>
> {code:java}
> WARNING: Please consider reporting this to the maintainers of
> org.apache.flink.shaded.akka.org.jboss.netty.util.internal.ByteBufferUtilWARNING:
> Use --illegal-access=warn to enable warnings of further illegal reflective
> access operationsWARNING: All illegal access operations will be denied in a
> future releaseMay 01, 2021 8:37:04 AM
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
> IOException thrown while closing
> Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
> of chunk at
> org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
> at
> org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
> at
> org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
> at
> org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
> at
> org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
> at
> org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
> at
> java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
> at
> java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136) at
> org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
> at
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
> at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
> com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
> at java.base/java.lang.Thread.run(Thread.java:832)
> May 01, 2021 8:39:06 AM
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
> IOException thrown while closing
> Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
> of chunk at
> org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
> at
> org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
> at
> org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
> at
> org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
> at
> org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
> at
> org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
> at
> java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
> at
> java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136) at
> org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
> at
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
> at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
> com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
> at java.base/java.lang.Thread.run(Thread.java:832)
> May 01, 2021 8:49:48 AM
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables closeWARNING:
> IOException thrown while closing
> Closeable.org.apache.http.MalformedChunkCodingException: CRLF expected at end
> of chunk at
> org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:253)
> at
> org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184)
> at
> org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:213)
> at
> org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:315)
> at
> org.apache.http.conn.BasicManagedEntity.streamClosed(BasicManagedEntity.java:166)
> at
> org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
> at
> org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
> at
> java.base/java.util.zip.InflaterInputStream.close(InflaterInputStream.java:231)
> at
> java.base/java.util.zip.GZIPInputStream.close(GZIPInputStream.java:136) at
> org.apache.http.client.entity.LazyDecompressingInputStream.close(LazyDecompressingInputStream.java:94)
> at
> org.apache.flink.twitter.shaded.com.google.common.io.Closeables.close(Closeables.java:77)
> at com.twitter.hbc.httpclient.Connection.close(Connection.java:64) at
> com.twitter.hbc.httpclient.ClientBase.run(ClientBase.java:148) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
> at java.base/java.lang.Thread.run(Thread.java:832)
> {code}
>
> So the part that stood out to me was "CRLF expected at end of chunk", then
> stacktrace.
> I'm not sure which is the cause of twitter connector not working between this
> and FLINK-22435 but I'm reporting both just in case.
>
> I can provide more info if needed but my setup is really basic/simple.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
