HI,
I updated the design document and submitted it to the hadoop community.
https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#
<https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit#>
If you have any questions, you can ask them in the email or directly in
the document. Thank you!
I developed the submarine interpreter module in zeppelin. Below is the
interface for the system to run.
zeppelin submarine interpreter properties
System administrators can configure multiple interpreters, and different
interpreters can have different resource configurations.
Allow different users to use different interpreters for resource allocation and
management.
zeppelin submarine Tensorflow interpreter
Users use their own interpreter, First through the %submarine.tensorflow
paragraph, To write python code for tensorflow.
After the user writes the python code of tensorflow, Click the [RUN] button and
zeppelin will upload the python code to the specified HDFS directory. The
submarine is loaded into the docker container when it is running.
zeppelin submarine interpreter
The user sets the call parameter value of the tensorflow python code, Then
enter the job run command.
The zeppelin submarine interpreter first checks that all parameters of the
submarine run are set completely. After the check is passed, the implementation
of YARN is performed via submarine.jar.
The progress and log information of the submarine will be displayed in
zeppelin's note.
You can enter submarine's other commands in the submarine interpreter, view all
the jobs in the submarine, etc.
> 在 2018年10月21日，上午6:26，Felix Cheung <[email protected]> 写道：
>
> Very cool!
>
>
> ________________________________
> From: Jeff Zhang <[email protected]>
> Sent: Friday, October 19, 2018 7:14 AM
> To: [email protected]
> Subject: Re: Zeppelin add hadoop submarine(machine learning framework)
> interpreter
>
> Thanks xun. This would be a great addon for zeppelin to support deep
> learning. I will check the design later.
>
>
>
> liu xun <[email protected]>于2018年10月19日周五 下午3:56写道：
>
>> Hi,
>> Hadoop Submarine is the latest machine learning framework subproject in
>> the Hadoop 3.2 release. It allows Hadoop to support Tensorflow,
>> MXNet,Caffe, Spark, etc. A variety of deep learning frameworks provide a
>> full-featured system framework for machine learning algorithm development,
>> distributed model training, model management, and model publishing,
>> combined with hadoop's intrinsic data storage and data processing
>> capabilities to enable data scientists to Good mining and the value of the
>> data.
>>
>>
>> I was involved in the development of the hadoop submarine project. So I
>> plan to add the interpreter module of hadoop submarine to zeppelin.
>> Let zeppeline increase the development of deep learning. This is my design
>> document, let's see if there is any opinion, you can put it directly in the
>> document, thank you!
>>
>>
>> https://docs.google.com/document/d/16YN8Kjmxt1Ym3clx5pDnGNXGajUT36hzQxjaik1cP4A/edit?ts=5bc6bfdd
>>
>>
>>
