Hi Sharad,
what's your vocabulary size and vector length for Word2Vec?
Regards,
Yuhao
2016-06-13 20:04 GMT+08:00 sharad82 :
> Is this the right forum to post Spark related issues ? I have tried this
> forum along with StackOverflow but not seeing any response.
>
>
>
> --
> View this message in
Is this the right forum to post Spark related issues ? I have tried this
forum along with StackOverflow but not seeing any response.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/OutOfMemoryError-When-saving-Word2Vec-tp27142p27151.html
Sent from the Apache
Hi Sharad.
The array size you (or the serializer) tries to allocate is just too big
for the JVM.
You can also split your input further by increasing parallelism.
Following is good explanintion
https://plumbr.eu/outofmemoryerror/requested-array-size-exceeds-vm-limit
regards,
Vaquar khan
On Sun
