[jira] [Resolved] (HDFS-15949) Fix integer overflow
[
https://issues.apache.org/jira/browse/HDFS-15949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka resolved HDFS-15949.
--
Fix Version/s: 3.2.3
3.4.0
3.3.1
Hadoop Flags: Reviewed
Resolution: Fixed
Committed to trunk, branch-3.3, and branch-3.2. Thanks [~gautham] for the
contribution and thanks [~inigoiri] for the review.
> Fix integer overflow
>
>
> Key: HDFS-15949
> URL: https://issues.apache.org/jira/browse/HDFS-15949
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs++
>Affects Versions: 3.4.0
>Reporter: Gautham Banasandra
>Assignee: Gautham Banasandra
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.1, 3.4.0, 3.2.3
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> There are some instances where integer overflow warnings are reported. Need
> to fix them.
> {code}
> [ 63%] Building CXX object
> main/native/libhdfspp/tests/CMakeFiles/hdfs_ext_hdfspp_test_shim_static.dir/hdfs_ext_test.cc.o
> In file included from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googletest/include/gtest/gtest.h:375,
> from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/internal/gmock-internal-utils.h:47,
> from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/gmock-actions.h:51,
> from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/out/build/WSL-GCC-Debug/main/native/libhdfspp/googletest-src/googlemock/include/gmock/gmock.h:59,
> from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfspp_mini_dfs.h:24,
> from
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:19:
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:
> In member function ‘virtual void
> hdfs::HdfsExtTest_TestHosts_Test::TestBody()’:
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:456:95:
> warning: integer overflow in expression of type ‘long int’ results in
> ‘-9223372036854775808’ [-Woverflow]
> 456 | EXPECT_EQ(nullptr, hdfsGetHosts(fs, filename.c_str(), 0,
> std::numeric_limits::max()+1));
> |
> ~~~^~
> /mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc:460:92:
> warning: integer overflow in expression of type ‘long int’ results in
> ‘-9223372036854775808’ [-Woverflow]
> 460 | EXPECT_EQ(nullptr, hdfsGetHosts(fs, filename.c_str(),
> std::numeric_limits::max()+1, std::numeric_limits::max()));
> |
> ~~~^~
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-15953) DUMMY_CLIENT_ID's verification may be invalid
Xiaoxing Wei created HDFS-15953:
---
Summary: DUMMY_CLIENT_ID's verification may be invalid
Key: HDFS-15953
URL: https://issues.apache.org/jira/browse/HDFS-15953
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Xiaoxing Wei
*_rpcClientId != RpcConstants.DUMMY_CLIENT_ID_* in *_FSEditLogOp#hasRpcIds()_*
{code:java}
// code placeholder
public boolean hasRpcIds() {
return rpcClientId != RpcConstants.DUMMY_CLIENT_ID
&& rpcCallId != RpcConstants.INVALID_CALL_ID;
}
{code}
determine that two arrays are equal should use "Arrays.equals"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen spotbugs unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.fs.TestTrash hadoop.fs.TestFileUtil hadoop.crypto.key.kms.server.TestKMS hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.datanode.TestBlockRecovery hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.yarn.server.nodemanager.TestNodeStatusUpdater hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.mapreduce.task.reduce.TestFetcher hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.tools.TestDistCpSystem hadoop.yarn.sls.TestSLSRunner hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-compile-javac-root.txt [496K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/patch-mvnsite-root.txt [580K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-patch-pylint.txt [48K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-patch-shellcheck.txt [56K] shelldocs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-patch-shelldocs.txt [48K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/diff-javadoc-javadoc-root.txt [20K] spotbugs: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-build-tools.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-project.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-annotations.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-project-dist.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-assemblies.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-maven-plugins.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-minikdc.txt [24K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-auth.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/260/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-auth-examples.txt [24K] https://ci
[jira] [Resolved] (HDFS-15909) Make fnmatch cross platform
[ https://issues.apache.org/jira/browse/HDFS-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved HDFS-15909. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Make fnmatch cross platform > --- > > Key: HDFS-15909 > URL: https://issues.apache.org/jira/browse/HDFS-15909 > Project: Hadoop HDFS > Issue Type: Improvement > Components: libhdfs++ >Affects Versions: 3.2.2 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The function fnmatch isn't available in Visual C++. Need to make this cross > platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (HDFS-15954) Pass correct function for SASL callback
Gautham Banasandra created HDFS-15954:
-
Summary: Pass correct function for SASL callback
Key: HDFS-15954
URL: https://issues.apache.org/jira/browse/HDFS-15954
Project: Hadoop HDFS
Issue Type: Bug
Components: libhdfs++
Affects Versions: 3.4.0
Reporter: Gautham Banasandra
Assignee: Gautham Banasandra
It seems like an incorrect function type is passed as the callback for SASL. We
get the following warnings during compilation -
{code}
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:
In constructor ‘hdfs::CySaslEngine::CySaslEngine()’:
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:124:46:
warning: cast between incompatible function types from ‘int (*)(void*, int,
const char**, unsigned int*)’ to ‘hdfs::sasl_callback_ft’ {aka ‘int (*)()’}
[-Wcast-function-type]
124 | { SASL_CB_USER, (sasl_callback_ft) & get_name, this}, // userid
for authZ
| ^~~~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:125:46:
warning: cast between incompatible function types from ‘int (*)(void*, int,
const char**, unsigned int*)’ to ‘hdfs::sasl_callback_ft’ {aka ‘int (*)()’}
[-Wcast-function-type]
125 | { SASL_CB_AUTHNAME, (sasl_callback_ft) & get_name, this}, // authid
for authT
| ^~~~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:126:46:
warning: cast between incompatible function types from ‘int (*)(void*, int,
const char**, const char**)’ to ‘hdfs::sasl_callback_ft’ {aka ‘int (*)()’}
[-Wcast-function-type]
126 | { SASL_CB_GETREALM, (sasl_callback_ft) & getrealm, this}, //
krb/gssapi realm
| ^~~~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:
At global scope:
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:423:41:
warning: cast between incompatible function types from ‘int (*)(void*, int,
const char*)’ to ‘hdfs::sasl_callback_ft’ {aka ‘int (*)()’}
[-Wcast-function-type]
423 | { SASL_CB_LOG, (sasl_callback_ft) & sasl_my_log, NULL},
| ^~~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:424:44:
warning: cast between incompatible function types from ‘int (*)(void*, const
char*, const char*, const char**, unsigned int*)’ to ‘hdfs::sasl_callback_ft’
{aka ‘int (*)()’} [-Wcast-function-type]
424 | { SASL_CB_GETOPT, (sasl_callback_ft) & sasl_getopt, NULL},
|^~~
/mnt/d/projects/apache/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/rpc/cyrus_sasl_engine.cc:425:45:
warning: cast between incompatible function types from ‘int (*)(void*, const
char**)’ to ‘hdfs::sasl_callback_ft’ {aka ‘int (*)()’} [-Wcast-function-type]
425 | { SASL_CB_GETPATH, (sasl_callback_ft) & get_path, NULL}, // to find
th mechanisms
| ^~~~
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/ [Apr 5, 2021 2:42:05 AM] (noreply) HDFS-15950. Remove unused hdfs.proto import (#2858) [Apr 5, 2021 8:47:10 AM] (noreply) HDFS-15951. Remove unused parameters in NameNodeProxiesClient (#2859) [Apr 5, 2021 4:52:48 PM] (noreply) HDFS-15947. Replace deprecated protobuf APIs (#2856) [Apr 5, 2021 6:49:21 PM] (noreply) HDFS-15940. Fixing and refactoring tests specific to Block recovery.(#2844). Contributed by Viraj Jasani [Apr 6, 2021 1:16:14 AM] (Eric Badger) YARN-10702. Add cluster metric for amount of CPU used by RM Event Processor. -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots hadoop.hdfs.TestDFSShell hadoop.hdfs.server.datanode.TestBlockRecovery2 hadoop.hdfs.server.datanode.TestDirectoryScanner hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks hadoop.yarn.client.api.impl.TestTimelineClientV2Impl hadoop.mapred.TestJobEndNotifier hadoop.tools.fedbalance.procedure.TestBalanceProcedureScheduler hadoop.tools.fedbalance.TestDistCpProcedure hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination hadoop.fs.contract.router.web.TestRouterWebHDFSContractCreate hadoop.tools.dynamometer.TestDynamometerInfra hadoop.tools.dynamometer.TestDynamometerInfra hadoop.yarn.sls.TestSLSRunner hadoop.yarn.sls.appmaster.TestAMSimulator cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-compile-cc-root.txt [100K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-compile-javac-root.txt [388K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/blanks-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-checkstyle-root.txt [16M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-pathlen.txt [16K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-shellcheck.txt [28K] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/results-javadoc-javadoc-root.txt [1.1M] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/patch-unit-hadoop-common-project_hadoop-kms.txt [16K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [520K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt [48K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [76K] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/469/artifact/out/patch-unit-hadoop-
