Re: HIVE building on ARM

2020-07-20 Thread Chinna Rao Lalam
Hi Ashutosh, I want to migrate https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/ this job to the new system https://ci-hadoop.apache.org. Please grant me rights to create jobs on the new Jenkins https://ci-builds.apache.org. My LDAP userid : chinnaraol. I have requested the

Re: HIVE building on ARM

2020-06-18 Thread Stamatis Zampetakis
Hello Chinna, The hudson-jobadmin privilege can be granted by PMC chairs. I don't know if there is any particular policy in Hive on who should have this privilege so I guess you should request it from Ashutosh. Best, Stamatis On Thu, Jun 18, 2020 at 12:05 PM Zoltan Haindrich wrote: > Hey Chinn

Re: HIVE building on ARM

2020-06-18 Thread Zoltan Haindrich
Hey Chinna! On 6/18/20 11:43 AM, Chinna Rao Lalam wrote: As you said, migrating this job to the new ci-hadoop instance looks good as Hadoop also shares the same armN slaves. Sounds great! I am able to login the new ci-hadoop instance with Apache LDAP credentials, but i am not able to see the

Re: HIVE building on ARM

2020-06-18 Thread Chinna Rao Lalam
Hi Zoltan, As you said, migrating this job to the new ci-hadoop instance looks good as Hadoop also shares the same armN slaves. I am able to login the new ci-hadoop instance with Apache LDAP credentials, but i am not able to see the job creation option. Should I request access or the process for c

Re: HIVE building on ARM

2020-06-16 Thread Zhenyu Zheng
Hi Zoltan, Thanks alot for the information, so looks like one possible solution is as you suggest, move the current ARM2 and ARM3 (those two were donate to builds.apache.org by us) to the new ci-hadoop cluster and set up the jobs just as what has been done in current jenkins. I will also ask our

Re: HIVE building on ARM

2020-06-16 Thread Zoltan Haindrich
Hey, There is an effort by the Apache Infra to change the way Jenkins stuff is organized; a couple months ago Gavin wrote an email about it: http://mail-archives.apache.org/mod_mbox/tez-dev/202004.mbox/%3ccan0gg1dodepzatjz9bofe-2ver7qg7h0hmvyjmsldgjr8_r...@mail.gmail.com%3E The resources for run

Re: HIVE building on ARM

2020-06-15 Thread Zhenyu Zheng
Hi Zoltan, Chinna, Thanks for the info, I wonder if where does the resource of ci-hadoop and hive-test-kube come from? Do they include ARM resources? Can you provide some more information about how the new hive-test-kube is running? BR, Kevin Zheng On Tue, Jun 16, 2020 at 12:41 PM Chinna Rao Lal

Re: HIVE building on ARM

2020-06-15 Thread Chinna Rao Lalam
Hi Zoltan, Thanks for the update. Current https://builds.apache.org/job/Hive-linux-ARM-trunk/ job is targeting to run hive tests daily on "arm" slaves, it is using 2 arm slaves. To find any potential issues with "arm" and fix the issues. So please suggest a way which direction we can move and can

Re: HIVE building on ARM

2020-06-15 Thread Zoltan Haindrich
Hey all, In an ticket (INFRA-20416) Gavin asked me if we are completely off builds.apache.org - when I went over the jobs I've saw that https://builds.apache.org/job/Hive-linux-ARM-trunk/ is running there once a day. Since builds.apache.org will be shut down in sometime in the future - we shoul

Re: HIVE building on ARM

2020-03-12 Thread Zhenyu Zheng
Hi Chinna, Thanks alot for the reply, I uploaded a patch and also a github PR for https://issues.apache.org/jira/browse/HIVE-21939 . In the patch, I bumped the protobuf used in standalone-metadata to 2.6.1 and added a new profile, this profile will identify the hardware architecture and if it is A

Re: HIVE building on ARM

2020-03-12 Thread Chinna Rao Lalam
Hi Zhenyu, Until HBase dependency resolved, without effecting the existing code on X86 i suggest create a separate profile with "os72" repo. Down the line we should have common version for both X86 and ARM. Hope It Helps, Chinna On Wed, Mar 11, 2020 at 8:39 AM Zhenyu Zheng wrote: > Hi Chinna,

Re: HIVE building on ARM

2020-03-10 Thread Zhenyu Zheng
Hi Chinna, David and others might interested, Thanks for bring this up, we are currently working on improving enabling big-data software on the ARM platform, we have already done fixes and providing CIs to some of the well-know projects like: 1. Hadoop: https://builds.apache.org/view/H-L/view/Hado

Re: HIVE building on ARM

2020-03-10 Thread Chinna Rao Lalam
Thanks David for the quick reply. Yes currently HBase protobuf version mismatch is the issue. We are investigating any common version of protobuf is there to support for HBase an HIVE. On Tue, Mar 10, 2020 at 6:08 PM David Lavati wrote: > Hi, > > I recently investigated this in > https://iss

Re: HIVE building on ARM

2020-03-10 Thread David Lavati
Hi, I recently investigated this in https://issues.apache.org/jira/browse/HIVE-20359 and it didn't look feasible, as protobuf has to be in sync across the whole stack, but the external protobuf version for hbase is on 2.5.0 for hbase-2.x, see my comment on the jira and https://github.com/apache/hb

HIVE building on ARM

2020-03-10 Thread Chinna Rao Lalam
Hi All, We are working on HIVE building and running on ARM architecture. As part of this i want to add one periodic HIVE JENKINS JOB on ARM machines. Which helps to identity gaps for ARM, If any issues identified we will keep on working to fix those issues. Currently to run HIVE on aarch64, we ne