[jira] [Created] (HDFS-16930) Update the wrapper for fuse-dfs
Chao-Heng Lee created HDFS-16930:
Summary: Update the wrapper for fuse-dfs
Key: HDFS-16930
URL: https://issues.apache.org/jira/browse/HDFS-16930
Project: Hadoop HDFS
Issue Type: Bug
Components: fuse-dfs
Reporter: Chao-Heng Lee
The fuse_dfs_wrapper.sh hasn't been updated for quite a long time.
Although the documentation mentions that the script may not work out of the
box, it would be clearer to update any outdated paths.
for example, from
{code:java}
export
LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/usr/local/lib"{code}
to
{code:java}
export
LIBHDFS_PATH="$HADOOP_HOME/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/target/usr/local/lib"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-16931) Observer nn delete blocks asynchronously when tail OP_DELETE editlog
kinit created HDFS-16931: Summary: Observer nn delete blocks asynchronously when tail OP_DELETE editlog Key: HDFS-16931 URL: https://issues.apache.org/jira/browse/HDFS-16931 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: kinit -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/ [Feb 21, 2023, 11:18:49 AM] (github) HADOOP-18622. Upgrade ant to 1.10.13 (#5360). Contributed by Aleksandr Nikolaev. [Feb 21, 2023, 2:39:32 PM] (github) YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. (#5288) -1 overall The following subsystems voted -1: blanks hadolint pathlen spotbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-common-project/hadoop-common/src/test/resources/xml/external-dtd.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml spotbugs : module:hadoop-mapreduce-project/hadoop-mapreduce-client Write to static field org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:[line 120] spotbugs : module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core Write to static field org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:[line 120] spotbugs : module:hadoop-mapreduce-project Write to static field org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:[line 120] spotbugs : module:root Write to static field org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:from instance method new org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, ExceptionReporter, SecretKey) At Fetcher.java:[line 120] Failed junit tests : hadoop.mapred.TestShuffleHandler hadoop.mapreduce.v2.hs.TestJobHistoryParsing hadoop.mapreduce.v2.hs.TestJobHistoryEvents hadoop.mapreduce.v2.hs.TestJobHistoryServer hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/artifact/out/results-compile-cc-root.txt [96K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/artifact/out/results-compile-javac-root.txt [528K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/artifact/out/blanks-eol.txt [14M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1144/artifac
[jira] [Resolved] (HDFS-16901) RBF: Routers should propagate the real user in the UGI via the caller context
[ https://issues.apache.org/jira/browse/HDFS-16901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved HDFS-16901. -- Fix Version/s: 3.4.0 3.3.6 Resolution: Fixed Thanks, Simba! > RBF: Routers should propagate the real user in the UGI via the caller context > - > > Key: HDFS-16901 > URL: https://issues.apache.org/jira/browse/HDFS-16901 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Simbarashe Dzinamarira >Assignee: Simbarashe Dzinamarira >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.6 > > > If the router receives an operation from a proxyUser, it drops the realUser > in the UGI and makes the routerUser the realUser for the operation that goes > to the namenode. > In the namenode UGI logs, we'd like the ability to know the original realUser. > The router should propagate the realUser from the client call as part of the > callerContext. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/946/ No changes ERROR: File 'out/email-report.txt' does not exist - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-16932) Mockito causing ClassCastException
Steve Vaughan created HDFS-16932:
Summary: Mockito causing ClassCastException
Key: HDFS-16932
URL: https://issues.apache.org/jira/browse/HDFS-16932
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.4.0
Environment: Running in the Hadoop development environment in docker,
running mvn.
Reporter: Steve Vaughan
Running tests in TestBalancerRPCDelay fails because of ClassCastExceptions
introduced by Mockito. As an example, in this stack trace note that the
RedundancyMonitor is calling "isRunning" but incorrectly ends up being routed
to getBlocks (which returns BlocksWithLocations) via TestBalancer and a Mockito
Spy. This ultimately is reported as a failure during the shutdown process.
{{Exception in thread "RedundancyMonitor" java.lang.ClassCastException:
java.lang.Boolean cannot be cast to
org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations}}{{ at
org.apache.hadoop.hdfs.server.balancer.TestBalancer$2.answer(TestBalancer.java:1865)}}{{
at
org.apache.hadoop.hdfs.server.balancer.TestBalancer$2.answer(TestBalancer.java:1858)}}{{
at
org.mockito.internal.stubbing.StubbedInvocationMatcher.answer(StubbedInvocationMatcher.java:39)}}{{
at
org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:96)}}{{
at
org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29)}}{{
at
org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:35)}}{{
at
org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:61)}}{{
at
org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:49)}}{{
at
org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptSuperCallable(MockMethodInterceptor.java:108)}}{{
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem$MockitoMock$1070381809.isRunning(Unknown
Source)}}{{ at
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$RedundancyMonitor.run(BlockManager.java:5155)}}{{
at java.lang.Thread.run(Thread.java:750)}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
