Hi Hang,
When I testing the 2.8.1 RC2, looks like the PulsarSQL is not working for
now, here are the logs from the presto server:
```
2021-09-02T16:02:07.629+0800 INFO SplitRunner-23-124
org.apache.pulsar.sql.presto.PulsarRecordCursor Closing cursor record
2021-09-02T16:02:07.630+0800 INFO 20210902_080207_00010_znjjs.1.0-1-110
org.apache.pulsar.sql.presto.PulsarRecordCursor Closing cursor record
2021-09-02T16:02:07.653+0800 ERROR remote-task-callback-6
io.prestosql.execution.StageStateMachine Stage
20210902_080207_00010_znjjs.1 failed
com.google.common.util.concurrent.UncheckedExecutionException:
java.nio.BufferUnderflowException
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
at com.google.common.cache.LocalCache.get(LocalCache.java:3951)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4935)
at
org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider.getSchemaByVersion(PulsarSqlSchemaInfoProvider.java:76)
at
org.apache.pulsar.sql.presto.PulsarRecordCursor.advanceNextPosition(PulsarRecordCursor.java:485)
at
io.prestosql.spi.connector.RecordPageSource.getNextPage(RecordPageSource.java:90)
at
io.prestosql.operator.TableScanOperator.getOutput(TableScanOperator.java:302)
at io.prestosql.operator.Driver.processInternal(Driver.java:379)
at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
at io.prestosql.operator.Driver.processFor(Driver.java:276)
at
io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
at
io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
at
io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
at
io.prestosql.$gen.Presto_332__testversion____20210902_075418_2.run(Unknown
Source)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:532)
at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:417)
at
org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider.loadSchema(PulsarSqlSchemaInfoProvider.java:106)
at
org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider.access$000(PulsarSqlSchemaInfoProvider.java:49)
at
org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider$1.load(PulsarSqlSchemaInfoProvider.java:61)
at
org.apache.pulsar.sql.presto.PulsarSqlSchemaInfoProvider$1.load(PulsarSqlSchemaInfoProvider.java:58)
at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
```
I have tried both JDK8 and JDK11
```
~ » java -version
java version "1.8.0_281"
Java(TM) SE Runtime Environment (build 1.8.0_281-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.281-b09, mixed mode)
```
```
~ » java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9 (build 11.0.11+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed
mode)
```
The reproduce steps:
1. Start standalone (bin/pulsar standalone)
2. Start presto (bin/pulsar sql-worker run)
3. Publish some messages to a topic (bin/pulsar-client produce -m
"test-messages-`date`" -n 10 test_result)
4. Query data by SQL (select * from pulsar."public/default".test_result;)
5. See error:
```
presto> select * from pulsar."public/default".test_result;
Query 20210902_080204_00009_znjjs, FAILED, 1 node
Splits: 18 total, 0 done (0.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
Query 20210902_080204_00009_znjjs failed: java.nio.BufferUnderflowException
```
Here is the schema info of the topic:
```
~/Downloads/2.8.1_release/apache-pulsar-2.8.1 » bin/pulsar-admin schemas
get test_result
{
"version": 0,
"schemaInfo": {
"name": "test_result",
"schema": "",
"type": "STRING",
"properties": {}
}
}
```
After checking 2.8.0, I can query the data from the topic and have no
errors from the presto server, It might be a regression in 2.8.1.
Vote my -1 for RC2
Thanks for the work Hang, If you can reproduce the problem please consider
add https://github.com/apache/pulsar/pull/11852 that @enrico mentioned to
the 2.8.1 release.
Penghui.
On Thu, Sep 2, 2021 at 2:59 PM PengHui Li <peng...@apache.org> wrote:
> Hi @enrico, any reason for https://github.com/apache/pulsar/pull/11852 be
> a blocker for 2.8.1? It's not a regression introduced in 2.8.1 right?
>
> Thanks,
> Penghui
>
> On Thu, Sep 2, 2021 at 2:36 PM Enrico Olivelli <eolive...@gmail.com>
> wrote:
>
>> Hang,
>> I am not able to reproduce Massimiliano's problem
>>
>> These commands pass on the stages sources unpacked by the source tarball,
>> on Ubuntu:
>> mvn clean install -DskipTests
>> mvn package -Pdocker -f docker/pom-xml
>>
>> I have performed other tests and everything passed (apart from some flaky
>> test), but given this recent finding
>> https://github.com/apache/pulsar/pull/11852
>> I believe it is better to add that commit to 2.8.x branch and prepare a
>> new
>> release candidate
>>
>> The final decision to CANCEL this RC is up to you Hang
>>
>> Enrico
>>
>> Il giorno mer 1 set 2021 alle ore 14:21 Massimiliano Mirelli <
>> massimilianomirelli...@gmail.com> ha scritto:
>>
>> > The chmod command solved the previously reported issue.
>> >
>> > However, I got a failure in one of the fallout tests which I would still
>> > need time to better investigate. The test creates a topic, spawns a
>> > function injecting 10000 messages from a data-generator into the topic
>> > which is then read by a consumer. There seems to be some issues at the
>> > function level, but I still need to look into it.
>> >
>> >
>> >
>> > On Wed, 1 Sept 2021 at 11:50, Massimiliano Mirelli <
>> > massimilianomirelli...@gmail.com> wrote:
>> >
>> > > Found at least one problem: pulsar docker image doesn't seem to be
>> > > deployed properly due to a permission problem.
>> > >
>> > > Running:
>> > >
>> > > chmod +x docker/pulsar/scripts/*
>> > >
>> > > Might solve it, I am verifying that now.
>> > >
>> > > On Wed, 1 Sept 2021 at 11:08, Massimiliano Mirelli <
>> > > massimilianomirelli...@gmail.com> wrote:
>> > >
>> > >> Thank you for sharing this rc.
>> > >>
>> > >> I am running a suite of fallout tests on it, let's see how it goes!
>> > >>
>> > >> Massimiliano
>> > >>
>> > >> On Tue, 31 Aug 2021 at 18:33, Hang Chen <chenh...@apache.org> wrote:
>> > >>
>> > >>> This is the second release candidate for Apache Pulsar, version
>> 2.8.1.
>> > >>>
>> > >>> It fixes the following issues:
>> > >>>
>> > >>>
>> >
>> https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.8.1+is%3Aclosed
>> > >>>
>> > >>> *** Please download, test and vote on this release. This vote will
>> stay
>> > >>> open
>> > >>> for at least 72 hours ***
>> > >>>
>> > >>> Note that we are voting upon the source (tag), binaries are provided
>> > for
>> > >>> convenience.
>> > >>>
>> > >>> Source and binary files:
>> > >>>
>> > https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.8.1-candidate-2/
>> > >>>
>> > >>> SHA-512 checksums:
>> > >>> 91feb8885f82c2e76f61679eb15f1ebf7a8b5ad4
>> > apache-pulsar-2.8.1-src.tar.gz
>> > >>> 55af5d767ddc208d49f7cf02a054fe1af0b9120d
>> > apache-pulsar-2.8.1-bin.tar.gz
>> > >>>
>> > >>> Maven staging repo:
>> > >>>
>> > https://repository.apache.org/content/repositories/orgapachepulsar-1098
>> > >>>
>> > >>> The tag to be voted upon:
>> > >>> v2.8.1-candidate-2 (6bc1e0d330524235ac83d55ccfecf680c7da0503)
>> > >>> https://github.com/apache/pulsar/releases/tag/v2.8.1-candidate-2
>> > >>>
>> > >>> Pulsar's KEYS file containing PGP keys we use to sign the release:
>> > >>> https://dist.apache.org/repos/dist/dev/pulsar/KEYS
>> > >>>
>> > >>> Please download the source package, and follow the README to build
>> > >>> and run the Pulsar standalone service.
>> > >>>
>> > >>
>> >
>>
>
