OK, I've made it work.
As Jonathan stated above, on emr 4.1 the notebook APIs does not work at
all. On both, emr 4.2 and emr 4.3 the /api/notebook works but the query for
a specific notebook not.
Deploying vanilla Zeppelin (master branch) on the same cluster worked as a
charm.
It probably origin by Zeppelin versions (0.5.5 on emr vs. 0.6.0 the latest).
Now I'm encountering different error that, as far as I can see, does not
interfere Zeppelin work:
ERROR [2016-01-14 09:55:38,499] ({qtp1505370540-32}
NotebookServer.java[onMessage]:176) - Can't handle message
java.lang.NullPointerException
Many of those.
If anyone knows what does it mean....
On Thu, Jan 14, 2016 at 12:23 AM, Jonathan Kelly <jonathaka...@gmail.com>
wrote:
> Interesting... I confirmed that removing all of the extra classpath
> entries added by EMR in zeppelin-env.sh does not fix this issue. This must
> mean that the issue is caused by the custom EMR builds of either Spark or
> Hadoop, which we use when building Zeppelin.
>
> On Wed, Jan 13, 2016 at 2:15 PM Jonathan Kelly <jonathaka...@gmail.com>
> wrote:
>
>> Ah, never mind. I had incorrectly been excluding the jars in
>> /usr/lib/zeppelin/interpreter, and I just noticed that jersey-core-1.9.jar
>> appears in two places underneath that directory (in the hive and phoenix
>> interpreter libs), and those jars are included in the Zeppelin server
>> classpath. This older version of jersey-core conflicts with
>> jersey-core-1.13.jar from /usr/lib/zeppelin/lib, so this probably doesn't
>> help. However, removing just these two jars did not help by itself.
>>
>> Haven't tried removing classpath entries from zeppelin-env.sh yet though.
>>
>> On Wed, Jan 13, 2016 at 2:07 PM Jonathan Kelly <jonathaka...@gmail.com>
>> wrote:
>>
>>> Yeah, I'm sure you're right that it must be due to something that EMR is
>>> putting in the classpath, but the thing is that I didn't see anything else
>>> in the classpath that includes the javax.ws.rs.core.Response class.
>>>
>>> And yes, I agree that trying to remove these two jars from the Zeppelin
>>> classpath was a futile attempt and would only have helped if somehow these
>>> jars were near duplicates of each other but different versions of the same
>>> thing. Besides, if removing one of the jars had helped, it would have shown
>>> that this was probably not a problem with EMR specifically but with
>>> Zeppelin itself, which of course would have been doubtful.
>>>
>>> Anyway, I'll try removing the EMR-supplied extra classpath entries from
>>> zeppelin-env.sh.
>>>
>>> On Wed, Jan 13, 2016 at 1:48 PM Ophir Cohen <oph...@gmail.com> wrote:
>>>
>>>> Interesting
>>>>
>>>> I'm using emr 4.1.0, I'll try 4.2.0 tomorrow.
>>>>
>>>> If I need to guess, I don't think that the mis-version dependency comes
>>>> from the zeppelin jars as Zeppelin api works when not in emr.
>>>> It most likely that it comes from the emr classpath.
>>>> I would have removed all the additional paths (set in zeppelin-env.sh)
>>>> and check again.
>>>>
>>>> BTW
>>>> Removing the one of the jars from Zeppelin as you did won't work as
>>>> Zeppelin needs these jars.
>>>> If you really want to test it you need to build zeppelin and instructe
>>>> maven to exclude the problematic dependency fron one of these jars.
>>>> In other words, maven should bring the jar, but without the duplicate
>>>> dependency.
>>>> Having said that, and as I stated above, I think that the problematic
>>>> dependency comes from the additional classpath and not from Zeppelin core
>>>> jars.
>>>> On Jan 13, 2016 11:08 PM, "Jonathan Kelly" <jonathaka...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi, Ophir, (Jonathan from EMR here)
>>>>>
>>>>> Are you using emr-4.1.0 or emr-4.2.0? I just tried this with emr-4.2.0
>>>>> and found that http://<my-server>:8890/api/notebook works for me, but
>>>>> http://<my-server>:8890/api/notebook/2A94M5J1Z fails. For some reason
>>>>> it doesn't quite fail with the same exception that you're seeing, but it
>>>>> definitely seems like a similar cause anyway. Here's the exception I get:
>>>>>
>>>>> WARN [2016-01-13 20:34:52,279] ({qtp508683864-37}
>>>>> ServletHandler.java[doHandle]:590) - Error for /api/notebook/2A94M5J1Z
>>>>> java.lang.AbstractMethodError:
>>>>> javax.ws.rs.core.Response.getStatusInfo()Ljavax/ws/rs/core/Response$StatusType;
>>>>> at
>>>>> javax.ws.rs.WebApplicationException.validate(WebApplicationException.java:186)
>>>>> at
>>>>> javax.ws.rs.ClientErrorException.<init>(ClientErrorException.java:88)
>>>>> at
>>>>> org.apache.cxf.jaxrs.utils.JAXRSUtils.findTargetMethod(JAXRSUtils.java:503)
>>>>> at
>>>>> org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:207)
>>>>>
>>>>> I found that the Zeppelin classpath includes two different jars that
>>>>> contain the javax/ws/rs/core/Response class:
>>>>> /usr/lib/zeppelin/lib/javax.ws.rs-api-2.0-m10.jar
>>>>> and /usr/lib/zeppelin/lib/jersey-core-1.13.jar
>>>>>
>>>>> What's weird though is that if I remove either of these from the
>>>>> classpath, the Zeppelin server fails to start up for different reasons. If
>>>>> I remove /usr/lib/zeppelin/lib/jersey-core-1.13.jar, I get a
>>>>> ClassNotFoundException for com.sun.jersey.core.util.FeaturesAndProperties,
>>>>> and if I remove /usr/lib/zeppelin/lib/javax.ws.rs-api-2.0-m10.jar, I
>>>>> get a ClassNotFoundException for javax.ws.rs.NotFoundException.
>>>>>
>>>>> According to a `mvn ... dependency:tree` in the zeppelin-server
>>>>> submodule, /usr/lib/zeppelin/lib/jersey-core-1.13.jar comes indirectly
>>>>> from
>>>>> zeppelin-server's direct dependency
>>>>> on com.sun.jersey:jersey-servlet:jar:1.13:compile, and
>>>>> /usr/lib/zeppelin/lib/javax.ws.rs-api-2.0-m10.jar
>>>>> comes from zeppelin-server's direct dependency on
>>>>> javax.ws.rs:javax.ws.rs-api:jar:2.0-m10:compile.
>>>>> Both of these dependencies seem to have been added a long time ago, and
>>>>> both seem to be used by the websocket API rather than the REST API (at
>>>>> least based upon the Git commit descriptions where they were added), so
>>>>> this confuses me even more.
>>>>>
>>>>> Does anybody from the Zeppelin side have any idea what is going on
>>>>> here?
>>>>>
>>>>> Thanks,
>>>>> Jonathan
>>>>>
>>>>> On Wed, Jan 13, 2016 at 5:34 AM Ophir Cohen <oph...@gmail.com> wrote:
>>>>>
>>>>>> Hi All,
>>>>>> We migrated our Zeppelin to use EMR Zeppelin. It's straight forward
>>>>>> and we were happy with the migration till we found out that something
>>>>>> isn't
>>>>>> working well with the rest API.
>>>>>>
>>>>>> Calling to the interpreter API:
>>>>>> http://<my-server>:8890/api/interpreter
>>>>>> and this:
>>>>>> http://<my-server>:8890/api/interpreter/setting
>>>>>> Returns results as expected.
>>>>>>
>>>>>> When trying to access the notebooks:
>>>>>> http://<my-server>:8890/api/notebook
>>>>>> Or a specific notebook:
>>>>>> http://<my-server>:8890/api/notebook/2A94M5J1Z
>>>>>> It fails with HTTP 500 error.
>>>>>>
>>>>>> The error I can see in the logs is NoSuchMethodError (see below)
>>>>>> which suggests we might have here 'jar-hell' and the wrong (probably old)
>>>>>> jar loaded instead of the need one - but I can't figure that out.
>>>>>> Thanks!
>>>>>>
>>>>>> The exception:
>>>>>> WARN [2016-01-13 07:47:11,690] ({qtp716961517-38}
>>>>>> ServletHandler.java[doHandle]:590) - Error for /api/notebook/2A94M5J1Z
>>>>>> java.lang.NoSuchMethodError:
>>>>>> javax.ws.rs.ClientErrorException.validate(Ljavax/ws/rs/core/Response;Ljavax/ws/rs/core/Response$Status$Family;)Ljavax/ws/rs/core/Response;
>>>>>> at
>>>>>> javax.ws.rs.ClientErrorException.<init>(ClientErrorException.java:88)
>>>>>> at
>>>>>> org.apache.cxf.jaxrs.utils.JAXRSUtils.findTargetMethod(JAXRSUtils.java:503)
>>>>>> at
>>>>>> org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:207)
>>>>>>
>>>>>
