I tried both. First tried 3.0.0. That didn't work so I tried 3.1.0.
On Wed, Jan 13, 2021 at 11:35 AM Jungtaek Lim <kabhwan.opensou...@gmail.com>
wrote:
> Which exact Spark version did you use? Did you make sure the version for
> Spark and the version for spark-sql-kafka artifact are the same? (I asked
> this because you've said you've used Spark 3.0 but spark-sql-kafka
> dependency pointed to 3.1.0.)
>
> On Tue, Jan 12, 2021 at 11:04 PM Eric Beabes <mailinglist...@gmail.com>
> wrote:
>
>> org.apache.spark.sql.streaming.StreamingQueryException: Data source v2
>> streaming sinks does not support Update mode. === Streaming Query ===
>> Identifier: [id = 1f342043-29de-4381-bc48-1c6eef99232e, runId =
>> 62410f05-db59-4026-83fc-439a79b01c29] Current Committed Offsets: {} Current
>> Available Offsets: {} Current State: INITIALIZING Thread State: RUNNABLE at
>> org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:353)
>> at
>> org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:244)
>> Caused by: java.lang.IllegalArgumentException: Data source v2 streaming
>> sinks does not support Update mode. at
>> org.apache.spark.sql.execution.streaming.StreamExecution.createStreamingWrite(StreamExecution.scala:635)
>> at
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.logicalPlan$lzycompute(MicroBatchExecution.scala:130)
>> at
>> org.apache.spark.sql.execution.streaming.MicroBatchExecution.logicalPlan(MicroBatchExecution.scala:61)
>> at
>> org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:320)
>> ... 1 more
>>
>>
>> *Please see the attached image for more information.*
>>
>>
>> On Tue, Jan 12, 2021 at 6:01 PM Jacek Laskowski <ja...@japila.pl> wrote:
>>
>>> Hi,
>>>
>>> Can you post the whole message? I'm trying to find what might be causing
>>> it. A small reproducible example would be of help too. Thank you.
>>>
>>> Pozdrawiam,
>>> Jacek Laskowski
>>> ----
>>> https://about.me/JacekLaskowski
>>> "The Internals Of" Online Books <https://books.japila.pl/>
>>> Follow me on https://twitter.com/jaceklaskowski
>>>
>>> <https://twitter.com/jaceklaskowski>
>>>
>>>
>>> On Tue, Jan 12, 2021 at 6:35 AM Eric Beabes <mailinglist...@gmail.com>
>>> wrote:
>>>
>>>> Trying to port my Spark 2.4 based (Structured) streaming application to
>>>> Spark 3.0. I compiled it using the dependency given below:
>>>>
>>>> <dependency>
>>>> <groupId>org.apache.spark</groupId>
>>>> <artifactId>spark-sql-kafka-0-10_${scala.binary.version}</artifactId>
>>>> <version>3.1.0</version>
>>>> </dependency>
>>>>
>>>>
>>>> Every time I run it under Spark 3.0, I get this message: *Data source
>>>> v2 streaming sinks does not support Update mode*
>>>>
>>>> I am using '*mapGroupsWithState*' so as per this link (
>>>> https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#output-modes),
>>>> the only supported Output mode is "*Update*".
>>>>
>>>> My Sink is a Kafka topic so I am using this:
>>>>
>>>> .writeStream
>>>> .format("kafka")
>>>>
>>>>
>>>> What am I missing?
>>>>
>>>>
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>
