Thank you Jeff! I would certainly give it a try.
Best,
Rahul
On 2021/09/26 22:49:03, Jeff Zhang wrote:
> Hi kumar,
>
> You can try Zeppelin which support the udf sharing across languages
>
> http://zeppelin.apache.org/
>
>
>
>
> rahul kumar 于2021年9月27日周一 上午4:20写道：
>
> > I'm trying to u
Thanks Sean,
- I have tried executing it without wrapping into DataFrame constructor, but
got the same error "
AttributeError: 'StructField' object has no attribute '_get_object_id'
- I have also tried using udf
scala udf:
class PythonUtil() extends UDF5[DataFrame, String,String, StructType
You can also call a Scala UDF from Python in Spark - this doesn't need
Zeppelin or relate to the front-end.
This may indeed be much easier as a proper UDF; depends on what this
function does.
However I think the issue may be that you're trying to wrap the resulting
DataFrame in a DataFrame or somet
You can first try it via docker
http://zeppelin.apache.org/download.html#using-the-official-docker-image
Jeff Zhang 于2021年9月27日周一 上午6:49写道：
> Hi kumar,
>
> You can try Zeppelin which support the udf sharing across languages
>
> http://zeppelin.apache.org/
>
>
>
>
> rahul kumar 于2021年9月27日周一 上午
Hi kumar,
You can try Zeppelin which support the udf sharing across languages
http://zeppelin.apache.org/
rahul kumar 于2021年9月27日周一 上午4:20写道：
> I'm trying to use a function defined in scala jar in pyspark ( spark
> 3.0.2).
>
> --scala ---
>
> Object PythonUtil {
>
> def customedf(dataFrame:
I'm trying to use a function defined in scala jar in pyspark ( spark 3.0.2).
--scala ---
Object PythonUtil {
def customedf(dataFrame: DataFrame,
keyCol: String,
table: String,
outputSchema: StructType,
database: String): DataF
