Re: [External Sender] Re: JDK17 support for Hadoop

What’s the latest status on Java 17 runtime support for Hadoop? Do we have a roadmap when and what the next Hadoop release will support it? Thanks, Jason From: slfan1989 Date: Wednesday, February 7, 2024 at 12:22 AM To: Xiaoqiao He , Ayush Saxena , ste...@cloudera.com , Takanobu Asanuma , Had

Protobuf-java compatibilty question

Hi team, Hadooop 3.2.x release is using protobuf-java 2.5.0 version lib. It looks like protobuf-java 3.x is binary backward compatible with 2.5.0 version. Does that mean Hadoop 3.2.x can work correctly if we replace protobuf-java 2.5.0 lib with 3.x version at runtime? Thanks, Jason

Re: [External Sender] Re: Is there a 3.3.4 release planned?

How about 3.4 release? -Jason From: Ashutosh Gupta Date: Monday, July 25, 2022 at 8:51 AM To: Lei Zhang Cc: Hdfs-dev Subject: [External Sender] Re: Is there a 3.3.4 release planned? Hi Lei Releasing 3.3.4 is already in process. - Ashutosh On Mon, 25 Jul, 2022, 4:37 pm Lei Zhang, wrote: >

3.2.x upgrade to 3.3

Hi All, I see a significant change on 3.3 is using protobuf 3.7.1 * HADOOP-16557 | Major | [pb-upgrade] Upgrade protobuf.version to 3.7.1 Will this cause any compatibility issue when upgrading from Hadoop 3.2.x to 3.3? And will rolling upg

Re: [External Sender] Re: [DISCUSS] Hadoop 3.2.3 release

Hi Steve, Which aws-java-sdk-bundle version did you test and encounter the following test failures? Thanks, Jason On 8/4/21, 7:51 AM, "Steve Loughran" wrote: it didn't take, so don't worry about it. Test failures java.lang.IllegalArgumentException: Input is expected to be encoded in

Re: Both NN crashes due to JN timeout (Hadoop 3)

Hi Wei Chiu, We also observed same issue when NN replays large editlogs from JN. It looks like in jetty 6 the default max idle timeout is 200 seconds. public abstract class AbstractConnector extends AbstractBuffers implements Connector { protected int _maxIdleTime=20;

HDFS upgrade with downtime questions

Hi, In https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Upgrade_with_Downtime, it says following: In a non-HA cluster, there are a NameNode (NN), a SecondaryNameNode (SNN) and many DataNodes (DNs). The procedure for upgrading a non-HA cluster is sim

Re: FSImage size issue

File object? Thanks, Jason On 12/1/20, 7:23 PM, "Jason Wen" wrote: I also suspect it might be caused by snapshots. We don't have lots of xattr or ALCs on our files. How can I confirm if it is really caused by snapshots? Thanks, Jason On 12/1/20

Re: FSImage size issue

les has lots of xattr or ACLs. On Tue, Dec 1, 2020 at 3:52 PM Jason Wen wrote: > Hi, > > We are encountering some odd FSImage size issue in one of our Hadoop > clusters. The Namenode only has about 3M files/blocks, but the FSImage size > is about 55GB.

FSImage size issue

Hi, We are encountering some odd FSImage size issue in one of our Hadoop clusters. The Namenode only has about 3M files/blocks, but the FSImage size is about 55GB. We have never seen this kind of gap between number of files/blocks vs FSImage size. As a comparison, we have another similar cluster