Yes, I switched to query.explain, though it did't fail, I still can't make
it print executed plan, is there any example?
I first try to placed query.explain() exactly after start, but spark
reports no data.
Hera are my codes:
while (query.awaitTermination(20000)) { // 1
query.explain() // 2
}
query.explain() // 3
The interesting part is query.awaitTermination(20000) will terminate
stream engine, so query.explain() at line 2 never be executed.
On Tue, Nov 21, 2017 at 1:49 PM, Liang-Chi Hsieh <[email protected]> wrote:
>
> wordCounts.explain() -> query.explain()?
>
>
> Chang Chen wrote
> > Hi Guys
> >
> > I modified StructuredNetworkWordCount to see what the executed plan is,
> > here are my codes:
> >
> > val wordCounts = words.groupBy("value").count()
> >
> > // Start running the query that prints the running counts to the console
> > val query = wordCounts.writeStream
> > .outputMode("complete")
> > .format("console")
> > .start()
> >
> > wordCounts.explain() // additional codes
> >
> >
> > But it failed with “AnalysisException: Queries with streaming sources
> must
> > be executed with writeStream.start()”?
> >
> >
> > Thanks
> > Chang
>
>
>
>
>
> -----
> Liang-Chi Hsieh | @viirya
> Spark Technology Center
> http://www.spark.tc/
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: [email protected]
>
>
