he-spark-developers-list.1001551.n3.nabble.com/PySpark-DataFrame-When-a-Row-is-not-a-Row-tp12210p13153.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail:
?
Best Regards,
Jerry
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/PySpark-DataFrame-When-a-Row-is-not-a-Row-tp12210p13153.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com
Is there some way around this? For example, can Row just be an
implementation of namedtuple throughout?
from collections import namedtuple
class Row(namedtuple):
...
>From a user perspective, it’s confusing that there are 2 different
implementations of the Row class with the same name.
In my
The class (called Row) for rows from Spark SQL is created on the fly, is
different from pyspark.sql.Row (is an public API to create Row by users).
The reason we done it in this way is that we want to have better performance
when accessing the columns. Basically, the rows are just named tuples
In Row#equals():
while (i < len) {
if (apply(i) != that.apply(i)) {
'!=' should be !apply(i).equals(that.apply(i)) ?
Cheers
On Mon, May 11, 2015 at 1:49 PM, Nicholas Chammas <
nicholas.cham...@gmail.com> wrote:
> This is really strange.
>
> >>> # Spark 1.3.1
> >>> print type(resu
This is really strange.
>>> # Spark 1.3.1
>>> print type(results)
>>> a = results.take(1)[0]
>>> print type(a)
>>> print pyspark.sql.types.Row
>>> print type(a) == pyspark.sql.types.Row
False
>>> print isinstance(a, pyspark.sql.types.Row)
False
If I set a as follows, then the type checks p
