Hi Matthew, If you can give more details about the course objectives that would help me understand where your questions are coming from but I think they have to do with the mixup between message formats and communication mechanisms.
I think you are mixing up the communication mechanisms with the message formats. REST and gRPC are communication mechanisms while JSON and Protocol Buffers are message encoding formats. Kafka has its own binary protocol that clients use when interacting with the brokers but the message can be encoded in protocol buffer, JSON, Avro, CSV or any binary format of your preference. So when you say gRPC message, I assume you are referring to messages from the producers encoded in protocol buffer format. The Kafka connect framework and certain Kafka clients (producers, consumers, streams) support using protocol buffers alongside the Confluent Schema Registry) so you can use it in your apps in language this is supported. The console scripts are typically used when you want to demonstrate client/broker interactions without using code so I can see how this could confuse you especially for a binary message format like protocol buffer. I think you should try to learn how to interact with the console producers/consumers using simple data types like integers and strings and then when you have mastered the concept you can move on to using python code or Java code to interact with the brokers using protocol buffer, Avro or any other complex message format. I hope that helps. Israel Ekpo Lead Instructor, IzzyAcademy.com https://izzyacademy.com/ On Sat, Oct 30, 2021 at 9:37 AM Glassman, Matthew S <glass...@hsrc.unc.edu> wrote: > I am involved with a course whose project has introduced REST, Kafka and > gRPC on its topic of message passing. However, in the project phase of > this section, we are tasked with passing gRPC messages through Kafka to > handle large amounts of location data. Has anyone used these two items > together? The course teaches them separately and uses the kafka console > scripts and so its a bit confusing trying to put the two together. Should > I utilize Kafka's python implementation? > > > Matthew Glassman > > > >
