I guess the initial purpose was to reduce the RPC calls and
the workload of the schema registry (each operation will perform
at Znode update). And I agree with saving the BYTES schema is
more clear than saving noting.
For now, the challenge is the client side will skip uploading the BYTES
schema,
Treating the BYTES schema differently (i.e. forbid uploading it) than other
schemas will be confusing to users.
If the BYTES schema is the default schema for a new topic, then how about
saving a BYTES schema in the registry instead of saving nothing?
On Mon, Apr 17, 2023 at 8:42 PM PengHui Li
I have pushed out a PR to revert this change first.
https://github.com/apache/pulsar/pull/20123
Please help review.
Thanks,
Penghui
On Tue, Apr 18, 2023 at 11:36 AM PengHui Li wrote:
> > Flink uses the schema to store some kv based properties. If we can
> expose all the operations of the topi
> Flink uses the schema to store some kv based properties. If we can
expose all the operations of the topic level metadata store. We can
truly drop the use of uploading the BYTES schema in the Flink
connector.
After 3.0.0, Pulsar will provide the ability to set properties for a topic
https://githu
>I support reverting the PR first and then looking for a long-term solution.
Flink uses the schema to store some kv based properties. If we can
expose all the operations of the topic level metadata store. We can
truly drop the use of uploading the BYTES schema in the Flink
connector.
On Mon, Apr
I'm sorry. I have provided the wrong description of the changes from the PR.
The PR has changed the server side, so it's hard for users to ask to
upgrade
the Flink connector if the pulsar server is upgraded.
I support reverting the PR first and then looking for a long-term solution.
Best,
Penghui
> Did we consider
making a call to upload a Bytes schema a no-op?
It was a BUG that the PR fixed.
You will not be able to get the uploaded schema as expected.
Please take a look at the details from the GitHub issue.
What is the challenge for the Flink connector now?
The changes only take effect o
I think the primary point is that unless there is a strict need, we
shouldn't introduce breaking changes to the implementation. Why did we
choose to forbid users from uploading a Bytes schema? Did we consider
making a call to upload a Bytes schema a no-op?
Thanks,
Michael
On Fri, Apr 14, 2023 at
1. I don't know much about flink, but what I see here is that you need to
save a `ResolvedCatalogTable`, which I see has `CatalogTable`, so it is
used to record the metadata information of the table.
**In
org.apache.flink.connector.pulsar.table.catalog.impl.PulsarCatalogSupport#createTable**
```jav
Il giorno ven 14 apr 2023 alle ore 16:48 Christophe Bornet
ha scritto:
>
> The change was not reverted for 3.0.0-RC so in the current state the
> Flink Pulsar catalog won't work with Pulsar 3.0.
> To give more details, Flink Pulsar stores Flink Table metadata in the
> metadata of BYTES schema topi
The change was not reverted for 3.0.0-RC so in the current state the
Flink Pulsar catalog won't work with Pulsar 3.0.
To give more details, Flink Pulsar stores Flink Table metadata in the
metadata of BYTES schema topics (those topics don't have messages,
only the schema metadata of these topics is
Hi SiNan,
In the flink world, we don't always rely on the schema information
provided by Pulsar or other connector systems. Flink application has
its own (de)serialization schema logic, which treats the messages only
in a binary format like a byte array.
In flink-connector-pulsar, we only use the
Hi yufan.
Can you describe a bit the usage scenario of byte schema in
flink-connector-pulsa?
Thanks,
sinan
Yufan Sheng 于 2023年3月28日周二 上午9:53写道：
> As the flink-connector-pulsar developer, I don't want to disable the
> BYTES schema upload. In my opinion, using BYTES schema means the users
> want
As the flink-connector-pulsar developer, I don't want to disable the
BYTES schema upload. In my opinion, using BYTES schema means the users
want to bypass the schema check and handle the schema validation by
themselves.
On Tue, Mar 28, 2023 at 8:58 AM SiNan Liu wrote:
>
> Hi, everyone.
> When a u
Hi, everyone.
When a user uploads bytes schema. We can warn the user and skip uploading
bytes schema.
Also check to see if the topic has a schema other than bytes.
1. If yes, warn the user that it is not necessary to upload bytes schema.
You can subscribe to a topic using bytes schema.
2. If there
This change broke the Flink SQL Pulsar connector:
https://github.com/streamnative/flink/issues/270
So I propose to revert it.
Le ven. 9 déc. 2022 à 11:57, labuladong a écrit :
>
> Hi pulsar community,
>
>
> I'd like to discuss the behavior of schema uploading, for more context see
> https://gith
Hi Xiangying Meng,
For the Pulsar client, even if users set BYTES schema, it won’t store anything
in schema registry and we don’t need any special compatibility check. The key
point here is, users can use admin api to upload a BYTES schema and it store a
NONE type of schema in schema registry,
+1
We should keep the behavior of the Admin API consistent with the Pulsar
Client.
> I think we should think about why we can not add a new schema of BYTES.
> The users use the byte schema to send or consume messages. But the schema
> registry has never stored the BYTES schema and we always need
Hi, Donglai:
Thanks for your discussion.
I think we should think about why we can not add a new schema of BYTES.
The users use the byte schema to send or consume messages. But the schema
registry has never stored the BYTES schema and we always need some special
compatibility checks for the BYTES sc
Hi, Donglai:
Thanks for your discussion. I have a question:
After the user uploads the `BYTES` schema, will users using other schemas to
connect to the topic do compatibility checks?
Thanks,
Baodi Shi
> 2022年12月13日 12:43，丛搏 写道：
>
> Hi, Donglai:
>
> I agree to forbid users to upload `BYTES`.
Hi, Donglai:
I agree to forbid users to upload `BYTES`. Otherwise, It will cause
inconsistent behavior with the pulsar client. Users uploading bytes
schema does not have any effect, on the contrary, it will bring a lot
of confusion to users.
Thanks,
bo
labuladong 于2022年12月9日周五 18:57写道：
>
> Hi p
21 matches
Mail list logo
