every week or so I do a -Pdist build of hadoop, so I can have an up to date command line. Today I kicked off the build with -Pdist -DskipTests -Dmaven.javadoc.skip=true
And it took half an hour to complete -with 1/3 of that time just shading the minicluster [INFO] Apache Hadoop Main ................................. SUCCESS [ 1.065 s] [INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 3.606 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [ 1.374 s] [INFO] Apache Hadoop Annotations .......................... SUCCESS [ 1.209 s] [INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.221 s] [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 1.096 s] [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 4.158 s] [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 0.710 s] [INFO] Apache Hadoop Auth ................................. SUCCESS [ 2.969 s] [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 1.591 s] [INFO] Apache Hadoop Common ............................... SUCCESS [ 49.545 s] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 1.647 s] [INFO] Apache Hadoop KMS .................................. SUCCESS [ 3.279 s] [INFO] Apache Hadoop Registry ............................. SUCCESS [ 4.420 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.118 s] [INFO] Apache Hadoop HDFS Client .......................... SUCCESS [ 32.316 s] [INFO] Apache Hadoop HDFS ................................. SUCCESS [01:19 min] [INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [ 1.336 s] [INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 4.519 s] [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 2.166 s] [INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [ 13.665 s] [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.103 s] [INFO] Apache Hadoop YARN ................................. SUCCESS [ 0.097 s] [INFO] Apache Hadoop YARN API ............................. SUCCESS [ 23.601 s] [INFO] Apache Hadoop YARN Common .......................... SUCCESS [ 19.536 s] [INFO] Apache Hadoop YARN Server .......................... SUCCESS [ 0.080 s] [INFO] Apache Hadoop YARN Server Common ................... SUCCESS [ 11.128 s] [INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [ 16.997 s] [INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [ 1.623 s] [INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [ 4.663 s] [INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [ 2.247 s] [INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 26.548 s] [INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [ 2.960 s] [INFO] Apache Hadoop YARN Client .......................... SUCCESS [ 4.913 s] [INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [ 1.732 s] [INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [ 1.867 s] [INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 0.123 s] [INFO] Apache Hadoop YARN TimelineService HBase Common .... SUCCESS [ 3.153 s] [INFO] Apache Hadoop YARN TimelineService HBase Client .... SUCCESS [ 3.503 s] [INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [ 0.088 s] [INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SUCCESS [ 4.038 s] [INFO] Apache Hadoop YARN TimelineService HBase tests ..... SUCCESS [ 3.494 s] [INFO] Apache Hadoop YARN Router .......................... SUCCESS [ 2.660 s] [INFO] Apache Hadoop YARN TimelineService DocumentStore ... SUCCESS [ 1.713 s] [INFO] Apache Hadoop YARN Applications .................... SUCCESS [ 0.091 s] [INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [ 2.062 s] [INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [ 1.544 s] [INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [ 0.601 s] [INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [ 6.887 s] [INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [ 6.869 s] [INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [ 3.005 s] [INFO] Apache Hadoop MapReduce App ........................ SUCCESS [ 5.289 s] [INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [ 4.543 s] [INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [ 5.421 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 1.516 s] [INFO] Apache Hadoop YARN Services ........................ SUCCESS [ 0.077 s] [INFO] Apache Hadoop YARN Services Core ................... SUCCESS [ 7.382 s] [INFO] Apache Hadoop YARN Services API .................... SUCCESS [ 2.310 s] [INFO] Apache Hadoop YARN Application Catalog ............. SUCCESS [ 0.086 s] [INFO] Apache Hadoop YARN Application Catalog Webapp ...... SUCCESS [ 24.634 s] [INFO] Apache Hadoop YARN Application Catalog Docker Image SUCCESS [ 0.183 s] [INFO] Apache Hadoop YARN Application MaWo ................ SUCCESS [ 0.128 s] [INFO] Apache Hadoop YARN Application MaWo Core ........... SUCCESS [ 1.844 s] [INFO] Apache Hadoop YARN Site ............................ SUCCESS [ 0.109 s] [INFO] Apache Hadoop YARN Registry ........................ SUCCESS [ 1.188 s] [INFO] Apache Hadoop YARN UI .............................. SUCCESS [ 0.123 s] [INFO] Apache Hadoop YARN CSI ............................. SUCCESS [ 9.718 s] [INFO] Apache Hadoop YARN Project ......................... SUCCESS [ 18.364 s] [INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [ 1.582 s] [INFO] Apache Hadoop MapReduce NativeTask ................. SUCCESS [ 2.293 s] [INFO] Apache Hadoop MapReduce Uploader ................... SUCCESS [ 1.526 s] [INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 2.003 s] [INFO] Apache Hadoop MapReduce ............................ SUCCESS [ 4.660 s] [INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 1.782 s] [INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 2.608 s] [INFO] Apache Hadoop Client Aggregator .................... SUCCESS [ 2.700 s] [INFO] Apache Hadoop Dynamometer Workload Simulator ....... SUCCESS [ 2.079 s] [INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SUCCESS [ 2.756 s] [INFO] Apache Hadoop Dynamometer Block Listing Generator .. SUCCESS [ 1.692 s] [INFO] Apache Hadoop Dynamometer Dist ..................... SUCCESS [ 4.494 s] [INFO] Apache Hadoop Dynamometer .......................... SUCCESS [ 0.085 s] [INFO] Apache Hadoop Archives ............................. SUCCESS [ 1.052 s] [INFO] Apache Hadoop Archive Logs ......................... SUCCESS [ 1.228 s] [INFO] Apache Hadoop Rumen ................................ SUCCESS [ 2.137 s] [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 1.913 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [ 1.181 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [ 1.364 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 0.079 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 1.598 s] [INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [ 6.910 s] [INFO] Apache Hadoop Kafka Library support ................ SUCCESS [ 1.256 s] [INFO] Apache Hadoop Azure support ........................ SUCCESS [ 4.236 s] [INFO] Apache Hadoop Aliyun OSS support ................... SUCCESS [ 1.051 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 2.416 s] [INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [ 2.483 s] [INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [ 1.284 s] [INFO] Apache Hadoop Image Generation Tool ................ SUCCESS [ 1.575 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 5.312 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.074 s] [INFO] Apache Hadoop Client API ........................... SUCCESS [03:17 min] [INFO] Apache Hadoop Client Runtime ....................... SUCCESS [02:59 min] [INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [ 1.076 s] *[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [10:32 min]* [INFO] Apache Hadoop Client Packaging Invariants for Test . SUCCESS [ 2.611 s] [INFO] Apache Hadoop Client Packaging Integration Tests ... SUCCESS [ 0.533 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [02:55 min] [INFO] Apache Hadoop Client Modules ....................... SUCCESS [ 0.168 s] [INFO] Apache Hadoop Tencent COS Support .................. SUCCESS [ 2.697 s] [INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [ 2.792 s] it used to be that -DskipShade would save the shading, but last time I tried to skip the shading something failed. 1. has anyone else tried a full dist build recently -and how long did it take? 2. could they skip the shading?
