Thanks for the clarification. rxin [via Apache Spark Developers List] < ml-node+s1001551n20462...@n3.nabble.com> schrieb am Mi. 4. Jan. 2017 um 23:37:
> Your understanding is correct - it is indeed slower due to extra > serialization. In some cases we can get rid of the serialization if the > value is already deserialized. > > On Wed, Jan 4, 2017 at 7:19 AM, geoHeil <[hidden email] > <http:///user/SendEmail.jtp?type=node&node=20462&i=0>> wrote: > > Hi I would like to know more about typeface aggregations in spark. > > > http://stackoverflow.com/questions/40596638/inquiries-about-spark-2-0-dataset/40602882?noredirect=1#comment70139481_40602882 > An example of these is > https://blog.codecentric.de/en/2016/07/spark-2-0-datasets-case-classes/ > ds.groupByKey(body => body.color) > > does > "myDataSet.map(foo.someVal) is type safe but as any Dataset operation uses > RDD and compared to DataFrame operations there is a significant overhead. > Let's take a look at a simple example:" > hold true e.g. will type safe aggregation require the deserialisation of > the > full objects as displayed for > ds.map(_.foo).explain ? > > Kind regards, > Georg > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/Clarification-about-typesafe-aggregations-tp20459.html > Sent from the Apache Spark Developers List mailing list archive at > Nabble.com. > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [hidden email] > <http:///user/SendEmail.jtp?type=node&node=20462&i=1> > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-developers-list.1001551.n3.nabble.com/Clarification-about-typesafe-aggregations-tp20459p20462.html > To unsubscribe from Clarification about typesafe aggregations, click here > <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=20459&code=Z2Vvcmcua2YuaGVpbGVyQGdtYWlsLmNvbXwyMDQ1OXwtMTgzMzc4NTU4MQ==> > . > NAML > <http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Clarification-about-typesafe-aggregations-tp20459p20463.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
