Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Weiwei Yang
Thanks for proposing this Wangda, my +1 as well. It is amazing to see the progress made in Submarine last year, the community grows fast and quiet collaborative. I can see the reasons to get it release faster in its own cycle. And at the same time, the Ozone way works very well. — Weiwei On Feb

[jira] [Created] (HDDS-1040) Add blockade Tests for client failures

2019-01-31 Thread Nilotpal Nandi (JIRA)
Nilotpal Nandi created HDDS-1040: Summary: Add blockade Tests for client failures Key: HDDS-1040 URL: https://issues.apache.org/jira/browse/HDDS-1040 Project: Hadoop Distributed Data Store Is

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Xun Liu
+1 Hello everyone, I am Xun Liu, the head of the machine learning team at Netease Research Institute. I quite agree with Wangda. Our team is very grateful for getting Submarine machine learning engine from the community. We are heavy users of Submarine. Because Submarine fits into the dire

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Yang Jiandan
+1. I'm from DiDi, and we plan to deploy Submarine. Making Submarine to different release is more agile, and we'd like to join the develop with community. runlin zhang 于2019年2月1日周五 上午10:31写道: > +1, It is very necessary to use Submarine on older Hadoop. What's more, > the development of deep l

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread runlin zhang
+1, It is very necessary to use Submarine on older Hadoop. What's more, the development of deep learning is too fast, and Submarine must keep faster release iterate . > 在 2019年2月1日,上午2:53,Wangda Tan 写道: > > Hi devs, > > Since we started submarine-related effort last year, we received a

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Sunil G
+1 from me on this. ML/DL is one of the fast growing areas and a runtime on YARN helps customers to have ML/DL workloads to run on same cluster where the ETL or other traditional big data workloads ingest or mine data. Faster release cadence can pace up the development for Submarine and more agile

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Rohith Sharma K S
+1, Few interested ML/DL folks from Banglore asked about Submarine release for trying out TensorFlow on YARN. We told them wait for release since they were not ready to use trunk. I see agile release cycle for Submarine brings lot of added value. -Rohith Sharma K S On Fri, 1 Feb 2019 at 00:34, Wa

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Arun Suresh
Thanks for bringing this up Wangda. +1 Makes a lot of sense to have Submarine follow its own release cadence - for all the reasons you outlined. I would one up this proposal to ask why shouldn't we allow YARN to have its own releases as well - but that is for a separate thread :) Cheers -Arun On

[jira] [Created] (HDDS-1039) OzoneManager fails to connect with secure SCM

2019-01-31 Thread Ajay Kumar (JIRA)
Ajay Kumar created HDDS-1039: Summary: OzoneManager fails to connect with secure SCM Key: HDDS-1039 URL: https://issues.apache.org/jira/browse/HDDS-1039 Project: Hadoop Distributed Data Store Iss

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Anu Engineer
>> I propose to adopt Ozone model: which is the same master branch, different >> release cycle, and different release branch. It is a great example to show >> agile release we can do (2 Ozone releases after Oct 2018) with less >> overhead to setup CI, projects, etc. I second this, especially this

[jira] [Created] (HDDS-1038) Datanode fails to connect with secure SCM

2019-01-31 Thread Ajay Kumar (JIRA)
Ajay Kumar created HDDS-1038: Summary: Datanode fails to connect with secure SCM Key: HDDS-1038 URL: https://issues.apache.org/jira/browse/HDDS-1038 Project: Hadoop Distributed Data Store Issue T

Re: [DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Jonathan Hung
+1. This is important for improving the deep learning on hadoop story. There's recently a lot of momentum for this, and decoupling submarine/hadoop will help it continue. Jonathan Hung On Thu, Jan 31, 2019 at 11:04 AM Wangda Tan wrote: > Hi devs, > > Since we started submarine-related effort l

[jira] [Created] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

2019-01-31 Thread Chao Sun (JIRA)
Chao Sun created HDFS-14250: --- Summary: [Standby Reads] msync should sync with active NameNode to fetch the latest stateID Key: HDFS-14250 URL: https://issues.apache.org/jira/browse/HDFS-14250 Project: Hadoo

[DISCUSS] Making submarine to different release model like Ozone

2019-01-31 Thread Wangda Tan
Hi devs, Since we started submarine-related effort last year, we received a lot of feedbacks, several companies (such as Netease, China Mobile, etc.) are trying to deploy Submarine to their Hadoop cluster along with big data workloads. Linkedin also has big interests to contribute a Submarine Ton

Re: proposed new repository for hadoop/ozone docker images (+update on docker works)

2019-01-31 Thread Eric Yang
1, 3. There are 38 Apache projects hosting docker images on Docker hub using Apache Organization. By browsing Apache github mirror. There are only 7 projects using a separate repository for docker image build. Popular projects official images are not from Apache organization, such as zookeepe

[jira] [Created] (HDFS-14249) RBF: Tooling to identify the subcluster location of a file

2019-01-31 Thread JIRA
Íñigo Goiri created HDFS-14249: -- Summary: RBF: Tooling to identify the subcluster location of a file Key: HDFS-14249 URL: https://issues.apache.org/jira/browse/HDFS-14249 Project: Hadoop HDFS Is

Re: proposed new repository for hadoop/ozone docker images (+update on docker works)

2019-01-31 Thread Elek, Marton
Hi Eric, Thanks for the answers 1. > Hadoop-docker-ozone.git source tree naming seems to create a unique process for Ozone. Not at all. We would like to follow the existing practice which is established in HADOOP-14898. In HDDS-851 we discussed why we need two separated repositories for hadoo

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2019-01-31 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1033/ [Jan 30, 2019 2:29:56 AM] (aajisaka) HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira [Jan 30, 2019 5:56:28 AM] (yqlin) HDDS-1024. Handle DeleteContainerCommand in the [Jan 30, 2019 1

[jira] [Created] (HDDS-1037) Fix the block discard logic in Ozone client

2019-01-31 Thread Shashikant Banerjee (JIRA)
Shashikant Banerjee created HDDS-1037: - Summary: Fix the block discard logic in Ozone client Key: HDDS-1037 URL: https://issues.apache.org/jira/browse/HDDS-1037 Project: Hadoop Distributed Data Sto