ng parameter
>
> spark.sql.autoBroadcastJoinThreshold to 10
>
>
> Caused by: java.lang.IllegalArgumentException: Can't zip RDDs with unequal
> numbers of partitions
> at
> org.apache.spark.rdd.ZippedPartitionsBaseRDD.getPartitions(ZippedPartitionsRDD.scala:57)
>
t 10:03 AM, Jiří Syrový
> wrote:
> > Hi,
> >
> > any idea what could be causing this issue? It started appearing after
> > changing parameter
> >
> > spark.sql.autoBroadcastJoinThreshold to 100000
> >
> >
> > Caused by: java.lang.Illegal
Hi,
any idea what could be causing this issue? It started appearing after
changing parameter
*spark.sql.autoBroadcastJoinThreshold to 10*
Caused by: java.lang.IllegalArgumentException: Can't zip RDDs with unequal
numbers of partitions
partition number, GraphX jobs will throw:
java.lang.IllegalArgumentException: Can't zip RDDs with unequal numbers of
partitions"
So my quick fix is to repartition the EdgeRDD to exactly the number of
parallelism. But I think this would lead to much network communication.
So is th
Hi All,
Finally I found that the problem occured when I called the graphx lib:
"
Exception in thread "main" java.lang.IllegalArgumentException: Can't zip RDDs
with unequal numbers of partitions
at
org.apache.spark.rdd.ZippedPartitionsBaseRDD.getPartitions(ZippedPa
Hi All,
I met the titled error. This exception occured in line 223, as shown below:
212 // read files
213 val lines =
sc.textFile(path_edges).map(line=>line.split(",")).map(line=>((line(0),
line(1)), line(2).toDouble)).reduceByKey(_+
_).cache
214
215 val lines_ver
