Re: Pickling Keras models for use in UDFs

2018-05-07 Thread erp12
Great idea! That works perfectly, thank you! -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Pickling Keras models for use in UDFs

2018-05-04 Thread Khaled Zaouk
Why don't you try to encapsulate your keras model within a wrapper class (an estimator let's say), and you implement inside this wrapper class the two functions: __getstate__ and __setstate__ On Thu, May 3, 2018 at 5:27 PM erp12 wrote: > I would like to create a Spark UDF which returns the a pre

Pickling Keras models for use in UDFs

2018-05-03 Thread erp12
I would like to create a Spark UDF which returns the a prediction made with a trained Keras model. Keras models are not typically pickle-able, however I have used the monkey patch approach to making Keras models pickle-able, as described here: http://zachmoshe.com/2017/04/03/pickling-keras-models.h