sorry my mistake i gave wrong id
here is correct one
https://issues.apache.org/jira/browse/SPARK-15183
On Wed, May 18, 2016 at 11:19 AM, Todd <bit1...@163.com> wrote:
> Hi Sachin,
>
> Could you please give the url of jira-15146? Thanks!
>
>
>
>
>
> At 2016-05-18 13:33:47, "Sachin Aggarwal" <different.sac...@gmail.com>
> wrote:
>
> Hi, there is some code I have added in jira-15146 please have a look at
> it, I have not finished it. U can use the same code in ur example as of now
> On 18-May-2016 10:46 AM, "Saisai Shao" <sai.sai.s...@gmail.com> wrote:
>
>> > .mode(SaveMode.Overwrite)
>>
>> From my understanding mode is not supported in continuous query.
>>
>> def mode(saveMode: SaveMode): DataFrameWriter = {
>> // mode() is used for non-continuous queries
>> // outputMode() is used for continuous queries
>> assertNotStreaming("mode() can only be called on non-continuous queries")
>> this.mode = saveMode
>> this
>> }
>>
>>
>> On Wed, May 18, 2016 at 12:25 PM, Todd <bit1...@163.com> wrote:
>>
>>> Thanks Ted.
>>>
>>> I didn't try, but I think SaveMode and OuputMode are different things.
>>> Currently, the spark code contain two output mode, Append and Update.
>>> Append is the default mode,but looks that there is no way to change to
>>> Update.
>>>
>>> Take a look at DataFrameWriter#startQuery
>>>
>>> Thanks.
>>>
>>>
>>>
>>>
>>>
>>>
>>> At 2016-05-18 12:10:11, "Ted Yu" <yuzhih...@gmail.com> wrote:
>>>
>>> Have you tried adding:
>>>
>>> .mode(SaveMode.Overwrite)
>>>
>>> On Tue, May 17, 2016 at 8:55 PM, Todd <bit1...@163.com> wrote:
>>>
>>>> scala> records.groupBy("name").count().write.trigger(ProcessingTime("30
>>>> seconds")).option("checkpointLocation",
>>>> "file:///home/hadoop/jsoncheckpoint").startStream("file:///home/hadoop/jsonresult")
>>>> org.apache.spark.sql.AnalysisException: Aggregations are not supported
>>>> on streaming DataFrames/Datasets in Append output mode. Consider changing
>>>> output mode to Update.;
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.UnsupportedOperationChecker$.org$apache$spark$sql$catalyst$analysis$UnsupportedOperationChecker$$throwError(UnsupportedOperationChecker.scala:142)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.UnsupportedOperationChecker$$anonfun$checkForStreaming$1.apply(UnsupportedOperationChecker.scala:59)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.UnsupportedOperationChecker$$anonfun$checkForStreaming$1.apply(UnsupportedOperationChecker.scala:46)
>>>> at
>>>> org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:125)
>>>> at
>>>> org.apache.spark.sql.catalyst.analysis.UnsupportedOperationChecker$.checkForStreaming(UnsupportedOperationChecker.scala:46)
>>>> at
>>>> org.apache.spark.sql.ContinuousQueryManager.startQuery(ContinuousQueryManager.scala:190)
>>>> at
>>>> org.apache.spark.sql.DataFrameWriter.startStream(DataFrameWriter.scala:351)
>>>> at
>>>> org.apache.spark.sql.DataFrameWriter.startStream(DataFrameWriter.scala:279)
>>>>
>>>>
>>>> I brief the spark code, looks like there is no way to change output
>>>> mode to Update?
>>>>
>>>
>>>
>>
--
Thanks & Regards
Sachin Aggarwal
7760502772
