Error in pyspark
Hi,
I'm puzzled - I have zeppelin 0.10 with spark 3.1.2.
The driver has Anaconda with python 3.6.5 installed.
Running pyspark paragraph, I'm getting some weird behavior. Paragraph runs
successfully on first attempt, but then failing on successive attempts
(until interpreter restarted).
Error is:
---
AttributeError Traceback (most recent call last)
in () > 1 sc.setJobGroup(
'zeppelin|admin|2GAWSQT96|paragraph_1629208863735_110330802', 'Started by:
admin') AttributeError: 'SparkSession' object has no attribute 'setJobGroup'
Fail to setJobGroup
For instance, this paragraph:
%spark.pyspark
import pandas as pd
from pyspark.sql.types import StringType
import numpy as np
def np_sqrt(v):
return np.__path__
spark.udf.register("np_sqrt", np_sqrt, StringType())
df = spark.range(10).createOrReplaceTempView("d")
spark.sql("select np_sqrt(id) as arr from d").show(truncate=False)
BTW, I can run different pyspark paragraphs, each will succeed at the first
attempt. Once re-running a paragraph, every pyspark paragraph will fail.
Any idea what may cause it?
Thanks,
Lior
CORS problem
Evewrithing works in the if zeppelin and my webapp lives in the same origin,
but NOT when I'm trying to connecto to zeppelin server via CORS,
the login step works,
but others as "notebook list"
jquery cannot set JSESSIONID so Zeppelin respond with redirect to /api/login
throwing an exception because authencation params missing.
$('#ab_test1').on('click', function (event) {
event.preventDefault();
$.ajax({
url: "http://192.168.56.105:9090/api/notebook";,
type: 'GET',
cache: false,
processData: false,
timeout: 3000,
xhrFields: {withCredentials: true},
crossDomain: true,
}).done(function (data) {
console.log(data)
}).fail(reason => {
console.log(reason);
});
})
In the same origin trial jquery set cookie correctly,
I tried to access to response headers "Set-Cookie" to set manually the cookie
but only "Content-Type" and "Content-Length" are available.
I've configured zeppelin-site.xml to allow my origin (example: ALLOW FROM
http://192.168.56.107)
Any suggestion?
does exists a way to bypass JSESSIONID passing ticket got with api login?
Regards,
Alessandro
Re: CORS problem
Do you embed zeppelin as an iframe in your app ?
Alessandro Busato 于2021年10月17日周日 下午6:11写道：
> Evewrithing works in the if zeppelin and my webapp lives in the same
> origin,
> but NOT when I'm trying to connecto to zeppelin server via CORS,
> the login step works,
> but others as "notebook list"
> jquery cannot set JSESSIONID so Zeppelin respond with redirect to
> /api/login
> throwing an exception because authencation params missing.
>
> $('#ab_test1').on('click', function (event) {
> event.preventDefault();
>
> $.ajax({
> url: "http://192.168.56.105:9090/api/notebook";,
> type: 'GET',
> cache: false,
> processData: false,
> timeout: 3000,
> xhrFields: {withCredentials: true},
> crossDomain: true,
> }).done(function (data) {
> console.log(data)
> }).fail(reason => {
> console.log(reason);
> });
> })
>
> In the same origin trial jquery set cookie correctly,
> I tried to access to response headers "Set-Cookie" to set manually the
> cookie but only "Content-Type" and "Content-Length" are available.
>
> I've configured zeppelin-site.xml to allow my origin (example: ALLOW FROM
> http://192.168.56.107)
> Any suggestion?
> does exists a way to bypass JSESSIONID passing ticket got with api login?
>
> Regards,
> Alessandro
>
>
--
Best Regards
Jeff Zhang
Re: CORS problem
On 2021/10/17 11:11:21, Jeff Zhang wrote:
> Do you embed zeppelin as an iframe in your app ?
No, it isn't.
Best regards,
Alessandro
>
> Alessandro Busato 于2021年10月17日周日 下午6:11写道：
>
> > Evewrithing works in the if zeppelin and my webapp lives in the same
> > origin,
> > but NOT when I'm trying to connecto to zeppelin server via CORS,
> > the login step works,
> > but others as "notebook list"
> > jquery cannot set JSESSIONID so Zeppelin respond with redirect to
> > /api/login
> > throwing an exception because authencation params missing.
> >
> > $('#ab_test1').on('click', function (event) {
> > event.preventDefault();
> >
> > $.ajax({
> > url: "http://192.168.56.105:9090/api/notebook";,
> > type: 'GET',
> > cache: false,
> > processData: false,
> > timeout: 3000,
> > xhrFields: {withCredentials: true},
> > crossDomain: true,
> > }).done(function (data) {
> > console.log(data)
> > }).fail(reason => {
> > console.log(reason);
> > });
> > })
> >
> > In the same origin trial jquery set cookie correctly,
> > I tried to access to response headers "Set-Cookie" to set manually the
> > cookie but only "Content-Type" and "Content-Length" are available.
> >
> > I've configured zeppelin-site.xml to allow my origin (example: ALLOW FROM
> > http://192.168.56.107)
> > Any suggestion?
> > does exists a way to bypass JSESSIONID passing ticket got with api login?
> >
> > Regards,
> > Alessandro
> >
> >
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: CORS problem
I solved in this way,
I exploited the BasicAuthentication of shiro,
I edited conf/shiro.ini and changed /** = authc into /** = authcBasic
I think this "workaround" is less secure than others but
in this way I can make CORS request even if the jquery doen't set JSESSIONID
$.ajax({
url: "http://192.168.56.105:9090/zeppelin/api/notebook";,
type: 'GET',
cache: false,
xhrFields: {withCredentials: true},
crossDomain: true,
"async": true,
"headers": {
"Authorization":"Basic YWRtaW46cGFzc3dvcmQx"
}
}).done(function (data) {
console.log(data)
}).fail(reason => {
console.log(reason);
});
Best regards,
Alessandro
On 2021/10/17 11:16:12, Alessandro Busato wrote:
>
>
> On 2021/10/17 11:11:21, Jeff Zhang wrote:
> > Do you embed zeppelin as an iframe in your app ?
>
> No, it isn't.
>
> Best regards,
> Alessandro
>
> >
> > Alessandro Busato 于2021年10月17日周日 下午6:11写道：
> >
> > > Evewrithing works in the if zeppelin and my webapp lives in the same
> > > origin,
> > > but NOT when I'm trying to connecto to zeppelin server via CORS,
> > > the login step works,
> > > but others as "notebook list"
> > > jquery cannot set JSESSIONID so Zeppelin respond with redirect to
> > > /api/login
> > > throwing an exception because authencation params missing.
> > >
> > > $('#ab_test1').on('click', function (event) {
> > > event.preventDefault();
> > >
> > > $.ajax({
> > > url: "http://192.168.56.105:9090/api/notebook";,
> > > type: 'GET',
> > > cache: false,
> > > processData: false,
> > > timeout: 3000,
> > > xhrFields: {withCredentials: true},
> > > crossDomain: true,
> > > }).done(function (data) {
> > > console.log(data)
> > > }).fail(reason => {
> > > console.log(reason);
> > > });
> > > })
> > >
> > > In the same origin trial jquery set cookie correctly,
> > > I tried to access to response headers "Set-Cookie" to set manually the
> > > cookie but only "Content-Type" and "Content-Length" are available.
> > >
> > > I've configured zeppelin-site.xml to allow my origin (example: ALLOW FROM
> > > http://192.168.56.107)
> > > Any suggestion?
> > > does exists a way to bypass JSESSIONID passing ticket got with api login?
> > >
> > > Regards,
> > > Alessandro
> > >
> > >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
