e.org/POM/4.0.0
>>>>>>>>> http://maven.apache.org/maven-v4_0_0.xsd";>
>>>>>>>>> 4.0.0
>>>>>>>>> com.myorg
>>>>>>>>> spark-3-conversion
>>>>>>>>> jar
>>>>>&
t;>>>>>>> true
>>>>>>>> 1.5
>>>>>>>> 1.5
>>>>>>>> UTF-8
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>
t;>
>>>>>>>
>>>>>>> org.apache.spark
>>>>>>>
>>>>>>> spark-streaming-kafka-0-10_${scala.binary.version}
>>>>>>> ${spark.version
gt; slf4j-log4j12
>>>>>> 1.7.7
>>>>>> runtime
>>>>>>
>>>>>>
>>>>>> log4j
>>>>>> log4j
>>>>>>
>>>>
>>>>>
>>>>>
>>>>> org.apache.maven.plugins
>>>>> maven-shade-plugin
>>>>> 3.0.0
>>>>>
>>>>>
>>>>&g
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> org.apache.maven.plugins
>>>> maven-shade-plugin
>>>> 3.0.0
>>>>
>>>>
&
>>>
>>>
>>> org.codehaus.mojo
>>> build-helper-maven-plugin
>>> 1.7
>>>
>>>
>>>
>>> add-source
>>> generate-sources
>>
gt; org.scalastyle
>> scalastyle-maven-plugin
>> 1.0.0
>>
>> false
>> true
>> true
>> false
>>
>> ${project.basedir}/s
fig.xml
>
> ${project.basedir}/target/scalastyle-output.xml
> UTF-8
>
>
>
>
> check
>
>
>
>
>
> org.sonarsource.scanner.mav
>>> wrote:
>>>
>>>> I tried both. First tried 3.0.0. That didn't work so I tried 3.1.0.
>>>>
>>>> On Wed, Jan 13, 2021 at 11:35 AM Jungtaek Lim <
>>>> kabhwan.opensou...@gmail.com> wrote:
>>>>
>>>&g
>>> asked this because you've said you've used Spark 3.0 but spark-sql-kafka
>>>> dependency pointed to 3.1.0.)
>>>>
>>>> On Tue, Jan 12, 2021 at 11:04 PM Eric Beabes
>>>> wrote:
>>>>
>>>>> org.apache.spark.sql.streaming.StreamingQueryException: D
act are the same? (I asked
>>> this because you've said you've used Spark 3.0 but spark-sql-kafka
>>> dependency pointed to 3.1.0.)
>>>
>>> On Tue, Jan 12, 2021 at 11:04 PM Eric Beabes
>>> wrote:
>>>
>>>> org.apache.spark.sql
ecause you've said you've used Spark 3.0 but spark-sql-kafka
>> dependency pointed to 3.1.0.)
>>
>> On Tue, Jan 12, 2021 at 11:04 PM Eric Beabes
>> wrote:
>>
>>> org.apache.spark.sql.streaming.StreamingQueryException: Data source v2
>>> str
because you've said you've used Spark 3.0 but spark-sql-kafka
> dependency pointed to 3.1.0.)
>
> On Tue, Jan 12, 2021 at 11:04 PM Eric Beabes
> wrote:
>
>> org.apache.spark.sql.streaming.StreamingQueryException: Data source v2
>> streaming sinks does not support
bes
wrote:
> org.apache.spark.sql.streaming.StreamingQueryException: Data source v2
> streaming sinks does not support Update mode. === Streaming Query ===
> Identifier: [id = 1f342043-29de-4381-bc48-1c6eef99232e, runId =
> 62410f05-db59-4026-83fc-439a79b01c29] Current Committ
org.apache.spark
> spark-sql-kafka-0-10_${scala.binary.version}
> 3.1.0
>
>
>
> Every time I run it under Spark 3.0, I get this message: *Data source v2
> streaming sinks does not support Update mode*
>
> I am using '*mapGroupsWithState*' so as per this
sinks does not support Update mode*
I am using '*mapGroupsWithState*' so as per this link (
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#output-modes),
the only supported Output mode is "*Update*".
My Sink is a Kafka topic so I am using
17 matches
Mail list logo
