I had to verify myself, I wrote an example that uses a key that is not in
the payload and I did where I specify the KEY and now it is a field. Notice
the state down below
CREATE STREAM my_avro_orders (total BIGINT, shipping VARCHAR, state VARCHAR
KEY, discount DOUBLE, gender VARCHAR)
WITH (kafka_topic='my_avro_orders', value_format='AVRO');
This came out as the following with a describe:
ksql> describe my_avro_orders;
Name : MY_AVRO_ORDERS
Field | Type
-----------------------------------
TOTAL | BIGINT
SHIPPING | VARCHAR(STRING)
STATE | VARCHAR(STRING) (key)
DISCOUNT | DOUBLE
GENDER | VARCHAR(STRING)
-----------------------------------
Then querying it, I get:
ksql> select * from my_avro_orders emit changes;
+---------------------+---------------------+---------------------+---------------------+---------------------+
|STATE |TOTAL |SHIPPING |DISCOUNT
|GENDER |
+---------------------+---------------------+---------------------+---------------------+---------------------+
|ME |48025 |TWO_DAY
|0.019999999552965164 |FEMALE |
|MD |32813 |NEXT_DAY
|0.10999999940395355 |FEMALE |
|SC |99302 |TWO_DAY
|0.09000000357627869 |MALE |
|SD |35621 |TWO_DAY
|0.17000000178813934 |FEMALE |
|SD |5698 |NEXT_DAY
|0.12999999523162842 |MALE |
|VT |8567 |NEXT_DAY
|0.05000000074505806 |FEMALE |
|WA |13737 |NEXT_DAY
|0.07999999821186066 |MALE |
|NH |60216 |TWO_DAY
|0.05000000074505806 |MALE |
|UT |47470 |TWO_DAY
|0.05000000074505806 |MALE |
|IN |69220 |TWO_DAY
|0.1899999976158142 |FEMALE |
On Tue, Aug 10, 2021 at 12:11 PM Daniel Hinojosa <
[email protected]> wrote:
> Ah, nevermind, just saw the title that this is KSQLDB.
>
> On Tue, Aug 10, 2021 at 12:09 PM Daniel Hinojosa <
> [email protected]> wrote:
>
>> The keys are already part of the stream. When you run builder.stream or
>> builder.table it returns a Stream<K,V> or a Table<K,V>. From there every
>> operation has a lambda that accepts both key and value. You can use map for
>> example to accept the key and do something with that. Let me know if you
>> have any other questions or if I didn't understand correctly.
>>
>> On Tue, Aug 10, 2021 at 10:58 AM Greer, Andrew C <
>> [email protected]> wrote:
>>
>>> Hello, I am trying to create a Stream that will accept the data from my
>>> topic and be able to use the message keys in the stream as unique
>>> identifiers for the sensors the data originated from. The data in the
>>> messages does not have anything that would be able to identify which sensor
>>> it came from, hence trying to get the keys used in the topic. I have seen
>>> various posts online about a ROWKEY column, but that doesn't appear to be
>>> an option on my version at least. I have had no success finding a way to
>>> incorporate the keys into my Stream and was hoping I would be able to find
>>> some help here.
>>>
>>> Thank you.
>>>
>>> Kafka version 6.0.1
>>>
>>> Andrew
>>>
>>>
