from:"Amir Ayupov via cfe\-commits"

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/2] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bol

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits


aaupov wrote:

@petrhosek – thank you for a review!

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/3] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bol

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-11-07 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 1ca9ce9ef3e90e3d24aea3945ae2d5129682621d Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH 1/3] [Clang][CMake] Support perf, LBR, and Instrument
 CLANG_BOLT options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bol

[llvm] [clang] [libcxx] [clang-tools-extra] [mlir] [lldb] [libc] [BOLT][NFC] Print BAT section size (PR #76897)

2024-01-04 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/76897

>From 5106f9e921aa426cf3272277c4aee4bdf76215e8 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 3 Jan 2024 21:25:02 -0800
Subject: [PATCH] =?UTF-8?q?[=F0=9D=98=80=F0=9D=97=BD=F0=9D=97=BF]=20initia?=
 =?UTF-8?q?l=20version?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Created using spr 1.3.4
---
 bolt/lib/Rewrite/RewriteInstance.cpp| 1 +
 bolt/test/X86/bolt-address-translation.test | 1 +
 2 files changed, 2 insertions(+)

diff --git a/bolt/lib/Rewrite/RewriteInstance.cpp 
b/bolt/lib/Rewrite/RewriteInstance.cpp
index a95b1650753cfd..f5a8a5b7168745 100644
--- a/bolt/lib/Rewrite/RewriteInstance.cpp
+++ b/bolt/lib/Rewrite/RewriteInstance.cpp
@@ -4112,6 +4112,7 @@ void RewriteInstance::encodeBATSection() {
   copyByteArray(BoltInfo), BoltInfo.size(),
   /*Alignment=*/1,
   /*IsReadOnly=*/true, ELF::SHT_NOTE);
+  outs() << "BOLT-INFO: BAT section size (bytes): " << BoltInfo.size() << '\n';
 }
 
 template 
diff --git a/bolt/test/X86/bolt-address-translation.test 
b/bolt/test/X86/bolt-address-translation.test
index f68a8f7e9bcb7f..c5813b411d87ba 100644
--- a/bolt/test/X86/bolt-address-translation.test
+++ b/bolt/test/X86/bolt-address-translation.test
@@ -37,6 +37,7 @@
 # CHECK:  BOLT: 3 out of 7 functions were overwritten.
 # CHECK:  BOLT-INFO: Wrote 6 BAT maps
 # CHECK:  BOLT-INFO: Wrote 3 BAT cold-to-hot entries
+# CHECK:  BOLT-INFO: BAT section size (bytes): 1436
 #
 # usqrt mappings (hot part). We match against any key (left side containing
 # the bolted binary offsets) because BOLT may change where it puts instructions

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [libcxx] [clang-tools-extra] [mlir] [lldb] [libc] [BOLT][NFC] Print BAT section size (PR #76897)

2024-01-04 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/76897

>From 5106f9e921aa426cf3272277c4aee4bdf76215e8 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 3 Jan 2024 21:25:02 -0800
Subject: [PATCH] =?UTF-8?q?[=F0=9D=98=80=F0=9D=97=BD=F0=9D=97=BF]=20initia?=
 =?UTF-8?q?l=20version?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Created using spr 1.3.4
---
 bolt/lib/Rewrite/RewriteInstance.cpp| 1 +
 bolt/test/X86/bolt-address-translation.test | 1 +
 2 files changed, 2 insertions(+)

diff --git a/bolt/lib/Rewrite/RewriteInstance.cpp 
b/bolt/lib/Rewrite/RewriteInstance.cpp
index a95b1650753cfd..f5a8a5b7168745 100644
--- a/bolt/lib/Rewrite/RewriteInstance.cpp
+++ b/bolt/lib/Rewrite/RewriteInstance.cpp
@@ -4112,6 +4112,7 @@ void RewriteInstance::encodeBATSection() {
   copyByteArray(BoltInfo), BoltInfo.size(),
   /*Alignment=*/1,
   /*IsReadOnly=*/true, ELF::SHT_NOTE);
+  outs() << "BOLT-INFO: BAT section size (bytes): " << BoltInfo.size() << '\n';
 }
 
 template 
diff --git a/bolt/test/X86/bolt-address-translation.test 
b/bolt/test/X86/bolt-address-translation.test
index f68a8f7e9bcb7f..c5813b411d87ba 100644
--- a/bolt/test/X86/bolt-address-translation.test
+++ b/bolt/test/X86/bolt-address-translation.test
@@ -37,6 +37,7 @@
 # CHECK:  BOLT: 3 out of 7 functions were overwritten.
 # CHECK:  BOLT-INFO: Wrote 6 BAT maps
 # CHECK:  BOLT-INFO: Wrote 3 BAT cold-to-hot entries
+# CHECK:  BOLT-INFO: BAT section size (bytes): 1436
 #
 # usqrt mappings (hot part). We match against any key (left side containing
 # the bolted binary offsets) because BOLT may change where it puts instructions

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[libcxx] [libc] [clang-tools-extra] [llvm] [lldb] [clang] [mlir] [BOLT][NFC] Print BAT section size (PR #76897)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76897
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-30 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From 90686f2cd5e210f9ca974c10f8c1224a825c1315 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH 1/3] [Clang][CMake] Add CSSPGO support to
 LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 32 -
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 54 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 ++-
 8 files changed, 123 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 601f40902fa34..489a097cd49c8 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -14,8 +14,33 @@ if(LLVM_BUILD_INSTRUMENTED)
 DEPENDS clang clear-profraw ${CLANG_PERF_TRAINING_DEPS}
 )
 
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" 
uppercase_LLVM_BUILD_INSTRUMENTED)
+  if (uppercase_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR}
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
+
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -26,9 +51,12 @@ if(LLVM_BUILD_INSTRUMENTED)
 message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to 
llvm-profdata")
   else()
 add_custom_target(generate-profdata
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR}
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA}
+  ${CMAKE_CURRENT_BINARY_D

[llvm] [clang] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)

2024-01-30 Thread Amir Ayupov via cfe-commits


aaupov wrote:

I'm in favor of this change especially in context of CSSPGO profiling 
(https://github.com/llvm/llvm-project/pull/79942) and BOLT LBR profiling 
(https://github.com/llvm/llvm-project/pull/69133).

https://github.com/llvm/llvm-project/pull/78879
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)

2024-01-30 Thread Amir Ayupov via cfe-commits



@@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "")
 set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "")
 
 set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "")
-set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "")
+set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING 
"")

aaupov wrote:

Makes sense to avoid touching to prevent conflicts with 
https://github.com/llvm/llvm-project/pull/78869.

https://github.com/llvm/llvm-project/pull/78879
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)

2024-01-30 Thread Amir Ayupov via cfe-commits



@@ -11,11 +15,11 @@ if(LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-profraw "Generating clang PGO data"
 ${CMAKE_CURRENT_BINARY_DIR}/pgo-data/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang clear-profraw ${CLANG_PERF_TRAINING_DEPS}
+DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS}
 )
 
   add_custom_target(clear-profraw
-COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw

aaupov wrote:

Where does this new path (`${CMAKE_BINARY_DIR}/profiles/`) come from?

https://github.com/llvm/llvm-project/pull/78879
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)

2024-02-02 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov approved this pull request.

LGTM

https://github.com/llvm/llvm-project/pull/78879
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-02-02 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From 63904124dc6c6cd9b855046fac3ad1a5e72d60b7 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH 1/3] [Clang][CMake] Add CSSPGO support to
 LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 32 -
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 55 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 124 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 61df987da7139..b8da611ad4506 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -18,8 +18,33 @@ if(LLVM_BUILD_INSTRUMENTED)
 DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS}
 )
 
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" 
uppercase_LLVM_BUILD_INSTRUMENTED)
+  if (uppercase_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR}
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
+
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -30,9 +55,12 @@ if(LLVM_BUILD_INSTRUMENTED)
 message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to 
llvm-profdata")
   else()
 add_custom_target(generate-profdata
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helpe

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-02-02 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From 63904124dc6c6cd9b855046fac3ad1a5e72d60b7 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH 1/4] [Clang][CMake] Add CSSPGO support to
 LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 32 -
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 55 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 124 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 61df987da7139..b8da611ad4506 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -18,8 +18,33 @@ if(LLVM_BUILD_INSTRUMENTED)
 DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS}
 )
 
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" 
uppercase_LLVM_BUILD_INSTRUMENTED)
+  if (uppercase_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR}
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
+
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -30,9 +55,12 @@ if(LLVM_BUILD_INSTRUMENTED)
 message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to 
llvm-profdata")
   else()
 add_custom_target(generate-profdata
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helpe

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-02-03 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From cb724ca7d7740e828934c6527ca188cfbbb78c0c Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 51 
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 56 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 137 insertions(+), 21 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 61df987da7139..a136753f7b2ff 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -12,14 +12,49 @@ if(LLVM_BUILD_INSTRUMENTED)
 ${CMAKE_CURRENT_BINARY_DIR}/pgo-data/lit.site.cfg
 )
 
-  add_lit_testsuite(generate-profraw "Generating clang PGO data"
+  # Dependencies of generate-profraw-* targets
+  add_custom_target(generate-profraw-deps
+DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS})
+  add_lit_testsuite(generate-profraw-intree "Generating clang PGO data"
 ${CMAKE_CURRENT_BINARY_DIR}/pgo-data/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS}
+DEPENDS generate-profraw-deps
 )
+  add_custom_target(generate-profraw DEPENDS generate-profraw-intree)
+  if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR)
+llvm_ExternalProject_Add(generate-profraw-external 
${CLANG_PGO_TRAINING_DATA_SOURCE_DIR}
+USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL
+DEPENDS generate-profraw-deps)
+add_dependencies(generate-profraw generate-profraw-external)
+  endif()
+
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
+  if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
 
   add_cus

[libcxx] [llvm] [clang-tools-extra] [clang] [lldb] [flang] [openmp] [libc] [mlir] [BOLT] Deduplicate equal offsets in BAT (PR #76905)

2024-01-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76905
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[libcxx] [llvm] [clang-tools-extra] [clang] [lldb] [flang] [openmp] [libc] [mlir] [BOLT] Deduplicate equal offsets in BAT (PR #76905)

2024-01-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76905
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 5a5ce01 - Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (#69133)

2024-01-27 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2024-01-27T19:01:40-08:00
New Revision: 5a5ce012e4de627c52efa1638993fd23c5e74afa

URL: 
https://github.com/llvm/llvm-project/commit/5a5ce012e4de627c52efa1638993fd23c5e74afa
DIFF: 
https://github.com/llvm/llvm-project/commit/5a5ce012e4de627c52efa1638993fd23c5e74afa.diff

LOG: Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options 
(#69133)

This reverts commit 6953b367027e4234607a6718a0a1d57eb52ef57e.

Couldn't reproduce cmake issues from any of the affected buildbots:
https://lab.llvm.org/buildbot/#/builders/139/builds/57628
https://lab.llvm.org/buildbot/#/builders/163/builds/50270
https://lab.llvm.org/buildbot/#/builders/223/builds/34823
https://lab.llvm.org/buildbot/#/builders/225/builds/29950
https://lab.llvm.org/buildbot/#/builders/245/builds/19468

A common trait is that these builders don't clean build directory.
Push again and monitor these builders.

Added: 


Modified: 
clang/CMakeLists.txt
clang/cmake/caches/BOLT.cmake
clang/utils/perf-training/CMakeLists.txt
clang/utils/perf-training/bolt.lit.cfg
clang/utils/perf-training/bolt.lit.site.cfg.in
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 5f2b7f064da437..47fc2e4886cfc2 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -876,23 +876,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT OFF CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -906,6 +921,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM

diff  --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac..eba2346b2f4ca1 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")

diff  --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c..601f40902fa34e 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fda

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-27 Thread Amir Ayupov via cfe-commits


aaupov wrote:

These are all annotated builders. I tried triggering rebuilds with clean build 
dir through buildbot UI but it didn't have an effect. I don't think it's a 
reasonable expectation that every change must configure cleanly from an 
existing build directory.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Status of the builders that reported breakage:
- [reverse-iteration](https://lab.llvm.org/buildbot/#/builders/54): 
https://lab.llvm.org/buildbot/#/builders/54/builds/8671 broken, 
https://lab.llvm.org/buildbot/#/builders/54/builds/8672 green, breakage was 
unrelated (unresolved tests)
- 
[sanitizer-aarch64-linux-bootstrap-msan](https://lab.llvm.org/buildbot/#/builders/237)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/237/builds/6271 
broken, https://lab.llvm.org/buildbot/#/builders/237/builds/6272 green after 
cleanup step
- 
[sanitizer-x86_64-linux-bootstrap-ubsan](https://lab.llvm.org/buildbot/#/builders/85)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/85/builds/21896 
broken, https://lab.llvm.org/buildbot/#/builders/85/builds/21897 green after 
cleanup step
- 
[sanitizer-aarch64-linux-bootstrap-ubsan](https://lab.llvm.org/buildbot/#/builders/238)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/238/builds/7702 
broken, https://lab.llvm.org/buildbot/#/builders/238/builds/7703 green after 
cleanup
- [sanitizer-x86_64-linux](https://lab.llvm.org/buildbot/#/builders/37) - 
annotated builder: https://lab.llvm.org/buildbot/#/builders/37/builds/30519 
broken, https://lab.llvm.org/buildbot/#/builders/37/builds/30521 green after 
two builds
- 
[sanitizer-aarch64-linux-bootstrap-asan](https://lab.llvm.org/buildbot/#/builders/239)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/239/builds/5498 
broken due to merge-fdata target (related), 
https://lab.llvm.org/buildbot/#/builders/239/builds/5499 broken due to some 
other unrelated cause. Cleanup step resolved cmake issue.
- [premerge-monolithic-windows](https://lab.llvm.org/buildbot/#/builders/271): 
not annotated builder, https://lab.llvm.org/buildbot/#/builders/271/builds/3444 
- broken as clean-build-dir step failed, 
https://lab.llvm.org/buildbot/#/builders/271/builds/3446 - green after 
successful cleanup.
- 
[libc-x86_64-debian-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/163)
 - annotated builder: https://lab.llvm.org/buildbot/#/builders/163/builds/50669 
- broken, still broken because of a **missing** cleanup step.
- 
[libc-x86_64-debian-dbg-runtimes-build](https://lab.llvm.org/buildbot/#/builders/225)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/225/builds/30309 
broken, missing cleanup step.
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165) - annotated 
builder, https://lab.llvm.org/buildbot/#/builders/165/builds/48023 broken, 
missing cleanup step.
- 
[libc-x86_64-debian-gcc-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/250)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/250/builds/17603 
broken, missing cleanup step.
- 
[libc-aarch64-ubuntu-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/223)
 - annotated builder, https://lab.llvm.org/buildbot/#/builders/223/builds/35258 
broken, missing cleanup step.
- [sanitizer-windows](https://lab.llvm.org/buildbot/#/builders/127) - annotated 
builder, https://lab.llvm.org/buildbot/#/builders/127/builds/61496 broken, 
missing cleanup step.

In short, this diff is a red herring for the reported breakages. The real cause 
is annotated builders not handling build dir cleanup properly. Still, I'm 
reaching out to all of the broken buildbot owners asking to clean the build 
directory manually.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-28 Thread Amir Ayupov via cfe-commits


aaupov wrote:

This issue with incremental builds is explicitly mentioned in 
[HowToAddABuilder](https://llvm.org/docs/HowToAddABuilder.html):

> Use CCache and NOT incremental builds
Using ccache materially improves average build times. Incremental builds can be 
slightly faster, but introduce the risk of build corruption due to e.g. state 
changes, etc… At this point, the recommendation is not to use incremental 
builds and instead use ccache as the latter captures the majority of the 
benefit with less risk of false positives.

I've sent an email to builders owners. Still, sorry about landing it on a 
weekend. I expected that builders are configured according to the guidelines so 
the issues will clear automatically. Apologies for those depending on these 
broken builders.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-29 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Status update:
- [sanitizer-windows](https://lab.llvm.org/buildbot/#/builders/127): back to 
green starting with https://lab.llvm.org/buildbot/#/builders/127/builds/61505
- 
[libc-aarch64-ubuntu-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/223):
 green https://lab.llvm.org/buildbot/#/builders/223/builds/35300
- 
[libc-x86_64-debian-gcc-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/250):
 green https://lab.llvm.org/buildbot/#/builders/250/builds/17643
- 
[libc-x86_64-debian-dbg-runtimes-build](https://lab.llvm.org/buildbot/#/builders/225):
 green https://lab.llvm.org/buildbot/#/builders/225/builds/30352
- 
[libc-x86_64-debian-fullbuild-dbg](https://lab.llvm.org/buildbot/#/builders/163):
 green https://lab.llvm.org/buildbot/#/builders/163/builds/50708
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165): red


https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-29 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Final update:
- [clang-hip-vega20](https://lab.llvm.org/buildbot/#/builders/165): green 
https://lab.llvm.org/buildbot/#/builders/165/builds/48052

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov created 
https://github.com/llvm/llvm-project/pull/79942

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419

Test Plan:
Added CSSPGO.cmake with same use as PGO.cmake, e.g. for bootstrapped 
ThinLTO+CSSPGO:
```
cmake -B csspgo -S /path/to/llvm-project/llvm \
-DLLVM_ENABLE_LLD=ON -DBOOTSTRAP_LLVM_ENABLE_LLD=ON  \
-DBOOTSTRAP_BOOTSTRAP_LLVM_ENABLE_LLD=ON  \
-DPGO_INSTRUMENT_LTO=Thin \
-GNinja  -C/path/to/llvm-project/clang/cmake/caches/CSSPGO.cmake
ninja stage2
```


>From 90686f2cd5e210f9ca974c10f8c1224a825c1315 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 32 -
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 54 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 ++-
 8 files changed, 123 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 601f40902fa34..489a097cd49c8 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -14,8 +14,33 @@ if(LLVM_BUILD_INSTRUMENTED)
 DEPENDS clang clear-profraw ${CLANG_PERF_TRAINING_DEPS}
 )
 
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" 
uppercase_LLVM_BUILD_INSTRUMENTED)
+  if (uppercase_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR}
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
+
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old p

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From 90686f2cd5e210f9ca974c10f8c1224a825c1315 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH 1/2] [Clang][CMake] Add CSSPGO support to
 LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 32 -
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 54 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 ++-
 8 files changed, 123 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc..5d16442ac7bc3 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -741,11 +741,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 0..34159068d5ea3
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 601f40902fa34..489a097cd49c8 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -14,8 +14,33 @@ if(LLVM_BUILD_INSTRUMENTED)
 DEPENDS clang clear-profraw ${CLANG_PERF_TRAINING_DEPS}
 )
 
+  add_custom_target(generate-profdata-deps)
+  string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" 
uppercase_LLVM_BUILD_INSTRUMENTED)
+  if (uppercase_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+set(PROFDATA_SAMPLE "--sample")
+if(NOT LLVM_PROFGEN)
+  find_program(LLVM_PROFGEN llvm-profgen)
+endif()
+
+if(NOT LLVM_PROFGEN)
+  message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has to 
point to llvm-profgen")
+endif()
+
+# Convert perf profiles into profraw
+add_custom_target(convert-perf-profraw
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN}
+  $ ${CMAKE_CURRENT_BINARY_DIR}
+  COMMENT "Converting perf profiles into profraw"
+  DEPENDS generate-profraw)
+add_dependencies(generate-profdata-deps convert-perf-profraw)
+  else()
+add_dependencies(generate-profdata-deps generate-profraw)
+  endif()
+
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -26,9 +51,12 @@ if(LLVM_BUILD_INSTRUMENTED)
 message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to 
llvm-profdata")
   else()
 add_custom_target(generate-profdata
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR}
+  COMMAND "${Python3_EXECUTABLE}"
+  ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA}
+  ${CMAKE_CURRENT_BINARY_D

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits

aaupov wrote:

> Do you have benchmarks on how well this performs relative to standard IRPGO
> (or on top of it)?

Couple of things to untangle here:
1. In general, CSSPGO is meant as a more practical alternative for IRPGO thanks
to i) the use of sampled profile, ii) context-sensitivity. However, IRPGO is
still expected to provide better performance in all cases where it's applicable
thanks to accurate profile information.
2. Due to that, I don't think it makes sense to apply CSSPGO on top of IRPGO,
and CMake automation would not permit mixing the two (both implemented as
LLVM_BUILD_INSTRUMENTED exclusive options).
3. [CSSPGO
RFC](https://lists.llvm.org/pipermail/llvm-dev/2020-August/144101.html) has
direct perf comparison of early implementation of CSSPGO vs IRPGO on Spec06:
https://github.com/llvm/llvm-project/assets/876514/a921e324-6b5d-4660-9928-0fb0c994649a";>
4. With CMake stuff implemented in this diff, it's expected that the amount of
profile information collected would be miniscule (in-tree perf-training only
has a single hello world source file) and inadequate for getting much perf
boost with CSSPGO (it even prints the warning that it needs 6985000.0x more
profile).
5. But I'll kick out a perf run anyway to see what's the starting point. If
it's zero, I'll try building different targets (in-tree or from LLVM Test
Suite) to identify a viable profiling workload, but it's outside of scope of
this diff.

https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-01-29 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Linking the preprint of the paper accepted to CGO'24 which has a CSSPGO 
performance comparison with AutoFDO and Instr PGO (Fig. 6): 
https://htyu.github.io/paper/Revamping_Sampling_based_PGO_with_Context_sensitivity_and_Pseudo_instrumentation_preprint.pdf

https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[mlir] [lldb] [clang] [flang] [libc] [llvm] [clang-tools-extra] [libcxx] [BOLT] Delta-encode offsets in BAT (PR #76900)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76900
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[mlir] [lldb] [clang] [flang] [libc] [llvm] [clang-tools-extra] [libcxx] [BOLT] Delta-encode offsets in BAT (PR #76900)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76900
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lldb] [libc] [flang] [llvm] [libcxx] [mlir] [clang] [clang-tools-extra] [BOLT] Delta-encode function start addresses in BAT (PR #76902)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76902
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [lldb] [clang] [libc] [libcxx] [mlir] [openmp] [flang] [BOLT] Delta-encode function start addresses in BAT (PR #76902)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76902
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [lldb] [clang] [libc] [libcxx] [mlir] [openmp] [flang] [BOLT] Delta-encode function start addresses in BAT (PR #76902)

2024-01-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76902
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lldb] [clang-tools-extra] [flang] [openmp] [libcxx] [clang] [libc] [llvm] [mlir] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)

2024-01-12 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76903
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lldb] [clang-tools-extra] [flang] [openmp] [libcxx] [clang] [libc] [llvm] [mlir] [BOLT] Embed cold mapping info into function entry in BAT (PR #76903)

2024-01-12 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76903
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[openmp] [clang-tools-extra] [libc] [flang] [llvm] [libcxx] [clang] [mlir] [lldb] [BOLT] Use continuous output addresses in delta encoding in BAT (PR #76904)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76904
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [libcxx] [mlir] [openmp] [clang-tools-extra] [flang] [libc] [llvm] [BOLT] Use continuous output addresses in delta encoding in BAT (PR #76904)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/76904
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [flang] [clang] [libcxx] [mlir] [llvm] [openmp] [libc] [lldb] [BOLT] Use continuous output addresses in delta encoding in BAT (PR #76904)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/76904

>From 2027bc7dc00395884c3bd4da21bbb79d079293fc Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 3 Jan 2024 21:25:27 -0800
Subject: [PATCH 1/2] =?UTF-8?q?[=F0=9D=98=80=F0=9D=97=BD=F0=9D=97=BF]=20ch?=
 =?UTF-8?q?anges=20to=20main=20this=20commit=20is=20based=20on?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Created using spr 1.3.4

[skip ci]
---
 bolt/docs/BAT.md  |  91 
 .../bolt/Profile/BoltAddressTranslation.h |  14 +-
 bolt/lib/Profile/BoltAddressTranslation.cpp   | 138 ++
 bolt/lib/Rewrite/RewriteInstance.cpp  |   1 +
 bolt/test/X86/bolt-address-translation.test   |   2 +-
 5 files changed, 180 insertions(+), 66 deletions(-)
 create mode 100644 bolt/docs/BAT.md

diff --git a/bolt/docs/BAT.md b/bolt/docs/BAT.md
new file mode 100644
index 00..5a257a46955607
--- /dev/null
+++ b/bolt/docs/BAT.md
@@ -0,0 +1,91 @@
+# BOLT Address Translation (BAT)
+# Purpose
+A regular profile collection for BOLT involves collecting samples from
+unoptimized binary. BOLT Address Translation allows collecting profile
+from BOLT-optimized binary and using it for optimizing the input (pre-BOLT)
+binary.
+
+# Overview
+BOLT Address Translation is an extra section (`.note.bolt_bat`) inserted by 
BOLT
+into the output binary containing translation tables and split functions 
linkage
+information. This information enables mapping the profile back from optimized
+binary onto the original binary.
+
+# Usage
+`--enable-bat` flag controls the generation of BAT section. Sampled profile 
+needs to be passed along with the optimized binary containing BAT section to
+`perf2bolt` which reads BAT section and produces fdata profile for the original
+binary. Note that YAML profile generation is not supported since BAT doesn't
+contain the metadata for input functions.
+
+# Internals
+## Section contents
+The section is organized as follows:
+- Hot functions table
+  - Address translation tables
+- Cold functions table
+
+## Construction and parsing
+BAT section is created from `BoltAddressTranslation` class which captures
+address translation information provided by BOLT linker. It is then encoded as 
a
+note section in the output binary.
+
+During profile conversion when BAT-enabled binary is passed to perf2bolt, 
+`BoltAddressTranslation` class is populated from BAT section. The class is then
+queried by `DataAggregator` during sample processing to reconstruct addresses/
+offsets in the input binary.
+
+## Encoding format
+The encoding is specified in bolt/include/bolt/Profile/BoltAddressTranslation.h
+and bolt/lib/Profile/BoltAddressTranslation.cpp.
+
+### Layout
+The general layout is as follows:
+```
+Hot functions table header
+|--|
+|  Function entry  |
+| |--| |
+| | OutOff InOff | |
+| |--| |
+
+
+Cold functions table header
+|--|
+|  Function entry  |
+| |--| |
+| | OutOff InOff | |
+| |--| |
+
+```
+
+### Functions table
+Hot and cold functions tables share the encoding except difference marked 
below.
+Header:
+| Entry  | Encoding | Description |
+| -- | - | --- |
+| `NumFuncs` | ULEB128 | Number of functions in the functions table |
+
+The header is followed by Functions table with `NumFuncs` entries.
+Output binary addresses are delta encoded, meaning that only the difference 
with
+the previous output address is stored. Addresses implicitly start at zero.
+Hot indices are delta encoded, implicitly starting at zero.
+| Entry  | Encoding | Description |
+| -- | --| --- |
+| `Address` | Delta, ULEB128 | Function address in the output binary |
+| `HotIndex` | Delta, ULEB128 | Cold functions only: index of corresponding 
hot function in hot functions table |
+| `NumEntries` | ULEB128 | Number of address translation entries for a 
function |
+Function header is followed by `NumEntries` pairs of offsets for current
+function.
+
+### Address translation table
+Delta encoding means that only the difference with the previous corresponding
+entry is encoded. Offsets implicitly start at zero.
+| Entry  | Encoding | Description |
+| -- | --| --- |
+| `OutputAddr` | Delta, ULEB128 | Function offset in output binary |
+| `InputAddr` | Delta, SLEB128 | Function offset in input binary with 
`BRANCHENTRY` LSB bit |
+
+`BRANCHENTRY` bit denotes whether a given offset pair is a control flow source
+(branch or call instruction). If not set, it signifies a control flow target
+(basic block offset).
diff --git a/bolt/include/bolt/Profile/BoltAddressTranslation.h 
b/bolt/include/bolt/Profile/BoltAddressTranslation.h
index 07e4b283211c69..feeda2ca1871be 100644
--- a/bolt/include/bolt/Profile/BoltAddressTranslation.h
+++ b/bolt/include/bolt/Profile/BoltAddressTranslation.h
@@

[clang-tools-extra] [flang] [clang] [libcxx] [mlir] [llvm] [openmp] [libc] [lldb] [BOLT] Use continuous output addresses in delta encoding in BAT (PR #76904)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/76904
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lldb] [clang] [llvm] [BOLT] Fix unconditional output of boltedcollection in merge-fdata (PR #78653)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/78653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lldb] [clang] [llvm] [BOLT] Fix unconditional output of boltedcollection in merge-fdata (PR #78653)

2024-01-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/78653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 9fec33a - Revert "[BOLT] Fix unconditional output of boltedcollection in merge-fdata (#78653)"

2024-01-18 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2024-01-18T19:59:09-08:00
New Revision: 9fec33aadc56c8c4ad3778a92dc0aaa3201a63ae

URL: 
https://github.com/llvm/llvm-project/commit/9fec33aadc56c8c4ad3778a92dc0aaa3201a63ae
DIFF: 
https://github.com/llvm/llvm-project/commit/9fec33aadc56c8c4ad3778a92dc0aaa3201a63ae.diff

LOG: Revert "[BOLT] Fix unconditional output of boltedcollection in merge-fdata 
(#78653)"

This reverts commit 82bc33ea3f1a539be50ed46919dc53fc6b685da9.

Accidentally pushed unrelated changes.

Added: 


Modified: 
bolt/lib/Core/BinaryEmitter.cpp
bolt/test/X86/merge-fdata-bat-mode.test
bolt/tools/merge-fdata/merge-fdata.cpp
clang/include/clang/Driver/Options.td
clang/lib/Driver/ToolChains/Gnu.cpp
cross-project-tests/lit.cfg.py
cross-project-tests/lit.site.cfg.py.in
lldb/test/API/lit.cfg.py
lldb/test/API/lit.site.cfg.py.in
lldb/test/Shell/helper/toolchain.py
lldb/test/Shell/lit.site.cfg.py.in
llvm/CMakeLists.txt
llvm/include/llvm/MC/MCFragment.h
llvm/include/llvm/MC/MCObjectStreamer.h
llvm/include/llvm/MC/MCStreamer.h
llvm/lib/MC/MCAssembler.cpp
llvm/lib/MC/MCFragment.cpp
llvm/lib/MC/MCObjectStreamer.cpp
llvm/lib/MC/MCStreamer.cpp
llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

Removed: 
bolt/test/X86/merge-fdata-nobat-mode.test
llvm/test/MC/X86/directive-avoid_end_align.s



diff  --git a/bolt/lib/Core/BinaryEmitter.cpp b/bolt/lib/Core/BinaryEmitter.cpp
index 190c700e2c303a..3bff3125a57a86 100644
--- a/bolt/lib/Core/BinaryEmitter.cpp
+++ b/bolt/lib/Core/BinaryEmitter.cpp
@@ -485,7 +485,6 @@ void BinaryEmitter::emitFunctionBody(BinaryFunction &BF, 
FunctionFragment &FF,
 // This assumes the second instruction in the macro-op pair will get
 // assigned to its own MCRelaxableFragment. Since all JCC instructions
 // are relaxable, we should be safe.
-Streamer.emitNeverAlignCodeAtEnd(/*Alignment to avoid=*/64, *BC.STI);
   }
 
   if (!EmitCodeOnly) {

diff  --git a/bolt/test/X86/merge-fdata-bat-mode.test 
b/bolt/test/X86/merge-fdata-bat-mode.test
index 41738e196b5d38..ea3e4c3e4afccd 100644
--- a/bolt/test/X86/merge-fdata-bat-mode.test
+++ b/bolt/test/X86/merge-fdata-bat-mode.test
@@ -4,5 +4,6 @@ RUN: merge-fdata %S/Inputs/bat_profile_1.fdata \
 RUN: %S/Inputs/bat_profile_2.fdata \
 RUN:   | FileCheck %s --check-prefix=CHECK-FDATA
 
-CHECK-FDATA: boltedcollection
 CHECK-FDATA: 1 main 451 1 SolveCubic 0 0 302
+
+

diff  --git a/bolt/test/X86/merge-fdata-nobat-mode.test 
b/bolt/test/X86/merge-fdata-nobat-mode.test
deleted file mode 100644
index 870d9f880e2866..00
--- a/bolt/test/X86/merge-fdata-nobat-mode.test
+++ /dev/null
@@ -1,6 +0,0 @@
-# Check that merge-fdata tool doesn't spuriously print boltedcollection
-
-RUN: merge-fdata %S/Inputs/blarge.fdata %S/Inputs/blarge.fdata \
-RUN:   | FileCheck %s --check-prefix=CHECK-FDATA
-
-CHECK-FDATA-NOT: boltedcollection

diff  --git a/bolt/tools/merge-fdata/merge-fdata.cpp 
b/bolt/tools/merge-fdata/merge-fdata.cpp
index 104991dabc5f07..e21aeba73aca29 100644
--- a/bolt/tools/merge-fdata/merge-fdata.cpp
+++ b/bolt/tools/merge-fdata/merge-fdata.cpp
@@ -329,7 +329,7 @@ void mergeLegacyProfiles(const SmallVectorImpl 
&Filenames) {
   MergedProfile.insert_or_assign(Key, Count);
 }
 
-  if (BoltedCollection.value_or(false))
+  if (BoltedCollection)
 output() << "boltedcollection\n";
   for (const auto &[Key, Value] : MergedProfile)
 output() << Key << " " << Value << "\n";

diff  --git a/clang/include/clang/Driver/Options.td 
b/clang/include/clang/Driver/Options.td
index a6da4b9160b3cf..d2e6c3ff721c27 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -5256,10 +5256,6 @@ def pg : Flag<["-"], "pg">, HelpText<"Enable mcount 
instrumentation">,
   MarshallingInfoFlag>;
 def pipe : Flag<["-", "--"], "pipe">,
   HelpText<"Use pipes between commands, when possible">;
-// Facebook T92898286
-def post_link_optimize : Flag<["--"], "post-link-optimize">,
-  HelpText<"Apply post-link optimizations using BOLT">;
-// End Facebook T92898286
 def prebind__all__twolevel__modules : Flag<["-"], 
"prebind_all_twolevel_modules">;
 def prebind : Flag<["-"], "prebind">;
 def preload : Flag<["-"], "preload">;

diff  --git a/clang/lib/Driver/ToolChains/Gnu.cpp 
b/clang/lib/Driver/ToolChains/Gnu.cpp
index 8f30b5f7da2372..e5e1b1d772697e 100644
--- a/clang/lib/Driver/ToolChains/Gnu.cpp
+++ b/clang/lib/Driver/ToolChains/Gnu.cpp
@@ -668,41 +668,12 @@ void tools::gnutools::Linker::ConstructJob(Compilation 
&C, const JobAction &JA,
 }
   }
 
-  // Facebook T92898286
-  if (Args.hasArg(options::OPT_post_link_optimize))
-CmdArgs.push_back("-q");
-  // End Facebook T92898286
-
   Args.AddAllArgs(CmdArgs, options::OPT_T);
 
   const char *Exec = Args.MakeArgString(ToolChain.GetLinkerPath());
   C.addCommand(std::make_

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-21 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] b83b8d3 - Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (#69133)

2024-01-22 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2024-01-22T14:47:42-08:00
New Revision: b83b8d3fd17885438b0ea154e07088d877d293a8

URL: 
https://github.com/llvm/llvm-project/commit/b83b8d3fd17885438b0ea154e07088d877d293a8
DIFF: 
https://github.com/llvm/llvm-project/commit/b83b8d3fd17885438b0ea154e07088d877d293a8.diff

LOG: Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options 
(#69133)

This reverts commit 6c47419703acfcd7dcca9e30ab9dba6a7a42f977.

Default to CLANG_BOLT=OFF

Test Plan:
Build a regular Clang build.

Added: 


Modified: 
clang/CMakeLists.txt
clang/cmake/caches/BOLT.cmake
clang/utils/perf-training/CMakeLists.txt
clang/utils/perf-training/bolt.lit.cfg
clang/utils/perf-training/bolt.lit.site.cfg.in
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 5f2b7f064da437..47fc2e4886cfc2 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -876,23 +876,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT OFF CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -906,6 +921,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM

diff  --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac..eba2346b2f4ca1 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")

diff  --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c..601f40902fa34e 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bolt-fdata clear-perf-data
 )
 
   add_custom_target(clea

[clang] 6953b36 - Revert "Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (#69133)"

2024-01-22 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2024-01-22T14:56:43-08:00
New Revision: 6953b367027e4234607a6718a0a1d57eb52ef57e

URL: 
https://github.com/llvm/llvm-project/commit/6953b367027e4234607a6718a0a1d57eb52ef57e
DIFF: 
https://github.com/llvm/llvm-project/commit/6953b367027e4234607a6718a0a1d57eb52ef57e.diff

LOG: Revert "Reland [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT 
options (#69133)"

This reverts commit b83b8d3fd17885438b0ea154e07088d877d293a8.

Breaks buildbots e.g.
https://lab.llvm.org/buildbot/#/builders/225/builds/29950

Added: 


Modified: 
clang/CMakeLists.txt
clang/cmake/caches/BOLT.cmake
clang/utils/perf-training/CMakeLists.txt
clang/utils/perf-training/bolt.lit.cfg
clang/utils/perf-training/bolt.lit.site.cfg.in
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 47fc2e4886cfc28..5f2b7f064da4377 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -876,38 +876,23 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-set(CLANG_BOLT OFF CACHE STRING "Apply BOLT optimization to Clang. \
-  May be specified as Instrument or Perf or LBR to use a particular profiling \
-  mechanism.")
-string(TOUPPER "${CLANG_BOLT}" CLANG_BOLT)
-
-if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
+  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Pass extra flag in no-LBR mode
-  if (CLANG_BOLT STREQUAL "PERF")
-set(BOLT_NO_LBR "-nl")
-  endif()
-
-  if (CLANG_BOLT STREQUAL "INSTRUMENT")
-# Instrument clang with BOLT
-add_custom_target(clang-instrumented
-  DEPENDS ${CLANG_INSTRUMENTED}
-)
-add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-  DEPENDS clang llvm-bolt
-  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
--instrument --instrumentation-file-append-pid
---instrumentation-file=${BOLT_FDATA}
-  COMMENT "Instrumenting clang binary with BOLT"
-  VERBATIM
-)
-add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
-  else() # perf or LBR
-add_custom_target(clang-bolt-training-deps DEPENDS clang)
-  endif()
+  # Instrument clang with BOLT
+  add_custom_target(clang-instrumented
+DEPENDS ${CLANG_INSTRUMENTED}
+  )
+  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+DEPENDS clang llvm-bolt
+COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+  -instrument --instrumentation-file-append-pid
+  --instrumentation-file=${BOLT_FDATA}
+COMMENT "Instrumenting clang binary with BOLT"
+VERBATIM
+  )
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -921,7 +906,6 @@ if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
-  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM

diff  --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index eba2346b2f4ca12..0442f73e5426ac7 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
+set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")

diff  --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 601f40902fa34ea..c6d51863fb1b5c2 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,9 +62,7 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
-  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
-"Name of BOLT-instrumented Clang binary")
+if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -73,37 +71,16 @@ if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-bolt-training-deps clear-bolt-fdata clear-perf-data
+DEPENDS clang-instrumented clear-bolt-fdata
 )
 
   add_custom_targe

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2024-01-22 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Reverted. Strangely I couldn't repro from 
https://lab.llvm.org/buildbot/#/builders/225/builds/29950/ the failure locally. 
But will get to the bottom of it.

https://github.com/llvm/llvm-project/pull/69133
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 076240f - [CMake] Disable BOLT instrumentation of Clang on instrumented build

2022-10-19 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2022-10-19T18:24:05-07:00
New Revision: 076240fa062415b6470b79413559aff2bf5bf208

URL: 
https://github.com/llvm/llvm-project/commit/076240fa062415b6470b79413559aff2bf5bf208
DIFF: 
https://github.com/llvm/llvm-project/commit/076240fa062415b6470b79413559aff2bf5bf208.diff

LOG: [CMake] Disable BOLT instrumentation of Clang on instrumented build

This enables multi-stage PGO build optimized by BOLT using BOLT.cmake cache.

The issue is that `-DPGO_BUILD_CONFIGURATION` cache file is passed to both
stage2-instrumented and stage2-optimized builds (for them to be identical),
but in case of BOLT.cmake, it doesn't make sense to BOLT-instrument the
instrumented binary (it's not going to be optimized). Hence turn off
`CLANG_BOLT_INSTRUMENT` code if `LLVM_BUILD_INSTRUMENTED` is enabled.

The final workflow that enables multi-stage InstrPGO+ThinLTO+BOLT Clang build:
```
cmake /llvm -GNinja -DLLVM_ENABLE_LLD=ON \
  -DBOOTSTRAP_LLVM_ENABLE_LLD=ON -DBOOTSTRAP_BOOTSTRAP_LLVM_ENABLE_LLD=ON \
  -DPGO_INSTRUMENT_LTO=Thin -C llvm-project/clang/cmake/caches/BOLT-PGO.cmake
ninja stage2-clang++-bolt
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D136023

Added: 
clang/cmake/caches/BOLT-PGO.cmake

Modified: 
clang/CMakeLists.txt

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 87ce418424c96..202a5d2667943 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -868,7 +868,7 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT)
+if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
   set(CLANGXX_PATH ${CLANG_PATH}++)
   set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)

diff  --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
new file mode 100644
index 0..54827c124bcb1
--- /dev/null
+++ b/clang/cmake/caches/BOLT-PGO.cmake
@@ -0,0 +1,11 @@
+set(LLVM_ENABLE_PROJECTS "bolt;clang;lld" CACHE STRING "")
+
+set(CLANG_BOOTSTRAP_TARGETS
+  stage2-clang++-bolt
+  CACHE STRING "")
+set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS
+  clang++-bolt
+  CACHE STRING "")
+
+set(PGO_BUILD_CONFIGURATION ${CMAKE_CURRENT_LIST_DIR}/BOLT.cmake CACHE STRING 
"")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 3dab7fe - [CMake] Add clang-bolt target

2022-09-23 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2022-09-23T10:10:31+02:00
New Revision: 3dab7fede2019c399d793c43ca9ea5a4f2d5031f

URL: 
https://github.com/llvm/llvm-project/commit/3dab7fede2019c399d793c43ca9ea5a4f2d5031f
DIFF: 
https://github.com/llvm/llvm-project/commit/3dab7fede2019c399d793c43ca9ea5a4f2d5031f.diff

LOG: [CMake] Add clang-bolt target

This patch adds `CLANG_BOLT_INSTRUMENT` option that applies BOLT instrumentation
to Clang, performs a bootstrap build with the resulting Clang, merges resulting
fdata files into a single profile file, and uses it to perform BOLT optimization
on the original Clang binary.

The projects and targets used for bootstrap/profile collection are configurable 
via
`CLANG_BOLT_INSTRUMENT_PROJECTS` and `CLANG_BOLT_INSTRUMENT_TARGETS`.
The defaults are "llvm" and "count" respectively, which results in a profile 
with
~5.3B dynamically executed instructions.

The intended use of the functionality is through BOLT CMake cache file, similar
to PGO 2-stage build:
```
cmake /llvm -C /clang/cmake/caches/BOLT.cmake
ninja clang++-bolt # pulls clang-bolt
```

Stats with a recent checkout (clang-16), pre-built BOLT and Clang, 72vCPU/224G
| CMake configure with host Clang + BOLT.cmake | 1m6.592s
| Instrumenting Clang with BOLT | 2m50.508s
| CMake configure `llvm` with instrumented Clang | 5m46.364s (~5x slowdown)
| CMake build `not` with instrumented Clang |0m6.456s
| Merging fdata files | 0m9.439s
| Optimizing Clang with BOLT | 0m39.201s

Building Clang:
```cmake ../llvm-project/llvm -DCMAKE_C_COMPILER=... -DCMAKE_CXX_COMPILER=...
  -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang
  -DLLVM_TARGETS_TO_BUILD=Native -GNinja```

| | Release | BOLT-optimized
| cmake | 0m24.016s | 0m22.333s
| ninja clang | 5m55.692s | 4m35.122s

I know it's not rigorous, but shows a ballpark figure.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D132975

Added: 
clang/cmake/caches/BOLT.cmake

Modified: 
clang/CMakeLists.txt
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 02ce5de4652de..22b5118c83eda 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -443,7 +443,7 @@ CMAKE_DEPENDENT_OPTION(CLANG_PLUGIN_SUPPORT
   "HAVE_CLANG_PLUGIN_SUPPORT" OFF)
 
 # If libstdc++ is statically linked, clang-repl needs to statically link 
libstdc++
-# itself, which is not possible in many platforms because of current 
limitations in 
+# itself, which is not possible in many platforms because of current 
limitations in
 # JIT stack. (more platforms need to be supported by JITLink)
 if(NOT LLVM_STATIC_LINK_CXX_STDLIB)
   set(HAVE_CLANG_REPL_SUPPORT ON)
@@ -881,6 +881,118 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
+if (CLANG_BOLT_INSTRUMENT)
+  set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
+  set(CLANGXX_PATH ${CLANG_PATH}++)
+  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANGXX_INSTRUMENTED ${CLANGXX_PATH}-bolt.inst)
+  set(CLANG_OPTIMIZED ${CLANG_PATH}-bolt)
+  set(CLANGXX_OPTIMIZED ${CLANGXX_PATH}-bolt)
+
+  # Instrument clang with BOLT
+  add_custom_target(clang-instrumented
+DEPENDS ${CLANG_INSTRUMENTED}
+  )
+  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+DEPENDS clang llvm-bolt
+COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+  -instrument --instrumentation-file-append-pid
+  --instrumentation-file=${CMAKE_CURRENT_BINARY_DIR}/prof.fdata
+COMMENT "Instrumenting clang binary with BOLT"
+VERBATIM
+  )
+
+  # Make a symlink from clang-bolt.inst to clang++-bolt.inst
+  add_custom_target(clang++-instrumented
+DEPENDS ${CLANGXX_INSTRUMENTED}
+  )
+  add_custom_command(OUTPUT ${CLANGXX_INSTRUMENTED}
+DEPENDS clang-instrumented
+COMMAND ${CMAKE_COMMAND} -E create_symlink
+  ${CLANG_INSTRUMENTED}
+  ${CLANGXX_INSTRUMENTED}
+COMMENT "Creating symlink from BOLT instrumented clang to clang++"
+VERBATIM
+  )
+
+  # Build specified targets with instrumented Clang to collect the profile
+  set(STAMP_DIR ${CMAKE_CURRENT_BINARY_DIR}/bolt-instrumented-clang-stamps/)
+  set(BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/bolt-instrumented-clang-bins/)
+  set(build_configuration "$")
+  include(ExternalProject)
+  ExternalProject_Add(bolt-instrumentation-profile
+DEPENDS clang++-instrumented
+PREFIX bolt-instrumentation-profile
+SOURCE_DIR ${CMAKE_SOURCE_DIR}
+STAMP_DIR ${STAMP_DIR}
+BINARY_DIR ${BINARY_DIR}
+EXCLUDE_FROM_ALL 1
+CMAKE_ARGS
+${CLANG_BOLT_INSTRUMENT_EXTRA_CMAKE_FLAGS}
+# We shouldn't need to set this here, but INSTALL_DIR doesn't
+# seem to work, so instead I'm passing this through
+-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}
+-DCMAKE_C_COMPILER=${CLANG_INSTRUMENTED}
+-DCMAKE_CXX_COMPILER=${CLANGXX_INSTRUMENTED}
+

[clang] 1fbbf92 - [perf-training] Check extension in findFilesWithExtension

2023-01-10 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2023-01-10T11:46:51-08:00
New Revision: 1fbbf92e4fda3c7a3be1c02e1f7240135557846d

URL: 
https://github.com/llvm/llvm-project/commit/1fbbf92e4fda3c7a3be1c02e1f7240135557846d
DIFF: 
https://github.com/llvm/llvm-project/commit/1fbbf92e4fda3c7a3be1c02e1f7240135557846d.diff

LOG: [perf-training] Check extension in findFilesWithExtension

`findFilesWithExtension` helper checks for `endswith(extension)` instead of
exactly matching the file extension. This causes it to match unrelated files,
for example, `.profdata` files while matching `.fdata` files:

http://157.230.108.44:8011/#/builders/56/builds/247
```
Merging data from 
/worker/worker/bolt-x86_64-ubuntu-clang-bolt-gcc/build/tools/clang/prof.fdata.1124569.fdata...
Merging data from 
/worker/worker/bolt-x86_64-ubuntu-clang-bolt-gcc/build/tools/clang/test/Frontend/Output/optimization-remark-with-hotness-new-pm.c.tmp.profdata...
```

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D141342

Added: 


Modified: 
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/utils/perf-training/perf-helper.py 
b/clang/utils/perf-training/perf-helper.py
index c6a815e65473..d68ab3c72806 100644
--- a/clang/utils/perf-training/perf-helper.py
+++ b/clang/utils/perf-training/perf-helper.py
@@ -23,7 +23,7 @@ def findFilesWithExtension(path, extension):
   filenames = []
   for root, dirs, files in os.walk(path): 
 for filename in files:
-  if filename.endswith(extension):
+  if os.path.splitext(filename)[1] == extension:
 filenames.append(os.path.join(root, filename))
   return filenames
 



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 131b955 - [CMake] Include clang-bolt bootstrap target in BOLT-PGO.cmake

2023-01-27 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2023-01-27T10:46:34-08:00
New Revision: 131b955a4f7015c718c1780d0320353d488f85be

URL: 
https://github.com/llvm/llvm-project/commit/131b955a4f7015c718c1780d0320353d488f85be
DIFF: 
https://github.com/llvm/llvm-project/commit/131b955a4f7015c718c1780d0320353d488f85be.diff

LOG: [CMake] Include clang-bolt bootstrap target in BOLT-PGO.cmake

Preemptively include clang-bolt target in BOLT-PGO CMake cache file,
in preparation of https://reviews.llvm.org/D139454 to avoid breaking
bolt-x86_64-ubuntu-clang-bolt-lto-pgo buildbot.

Added: 


Modified: 
clang/cmake/caches/BOLT-PGO.cmake

Removed: 




diff  --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
index 54827c124bcb1..c329415465c94 100644
--- a/clang/cmake/caches/BOLT-PGO.cmake
+++ b/clang/cmake/caches/BOLT-PGO.cmake
@@ -1,9 +1,11 @@
 set(LLVM_ENABLE_PROJECTS "bolt;clang;lld" CACHE STRING "")
 
 set(CLANG_BOOTSTRAP_TARGETS
+  stage2-clang-bolt
   stage2-clang++-bolt
   CACHE STRING "")
 set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS
+  clang-bolt
   clang++-bolt
   CACHE STRING "")
 



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 5725c8d - [perf-training] Check extension in findFilesWithExtension

2023-01-17 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2023-01-17T21:54:00-08:00
New Revision: 5725c8ddd46becff95629515b1f0a26af047aa33

URL: 
https://github.com/llvm/llvm-project/commit/5725c8ddd46becff95629515b1f0a26af047aa33
DIFF: 
https://github.com/llvm/llvm-project/commit/5725c8ddd46becff95629515b1f0a26af047aa33.diff

LOG: [perf-training] Check extension in findFilesWithExtension

`findFilesWithExtension` helper checks for `endswith(extension)` instead of
exactly matching the file extension. This causes it to match unrelated files,
for example, `.profdata` files while matching `.fdata` files:

http://157.230.108.44:8011/#/builders/56/builds/247
```
Merging data from 
/worker/worker/bolt-x86_64-ubuntu-clang-bolt-gcc/build/tools/clang/prof.fdata.1124569.fdata...
Merging data from 
/worker/worker/bolt-x86_64-ubuntu-clang-bolt-gcc/build/tools/clang/test/Frontend/Output/optimization-remark-with-hotness-new-pm.c.tmp.profdata...
```

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D141342

Added: 


Modified: 
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/utils/perf-training/perf-helper.py 
b/clang/utils/perf-training/perf-helper.py
index c6a815e65473..abbdc77b2372 100644
--- a/clang/utils/perf-training/perf-helper.py
+++ b/clang/utils/perf-training/perf-helper.py
@@ -23,7 +23,7 @@ def findFilesWithExtension(path, extension):
   filenames = []
   for root, dirs, files in os.walk(path): 
 for filename in files:
-  if filename.endswith(extension):
+  if filename.endswith(f".{extension}"):
 filenames.append(os.path.join(root, filename))
   return filenames
 



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov created 
https://github.com/llvm/llvm-project/pull/69133

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617


>From 3987088f1b6f6b895f7563ad70c97074b0d4ded0 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT
 options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 +
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 ---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 65 +++
 6 files changed, 169 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${

[clang] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT options (PR #69133)

2023-10-15 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/69133

>From 96e644279ccec1970c42cca89c05aac186b872e6 Mon Sep 17 00:00:00 2001
From: Amir Aupov 
Date: Mon, 16 Oct 2023 01:08:28 +0200
Subject: [PATCH] [Clang][CMake] Support perf, LBR, and Instrument CLANG_BOLT
 options

Split up and refactor CLANG_BOLT_INSTRUMENT into support for
perf no-LBR and perf with LBR profiling modes.

Differential Revision: https://reviews.llvm.org/D143617
---
 clang/CMakeLists.txt  | 44 -
 clang/cmake/caches/BOLT.cmake |  2 +-
 clang/utils/perf-training/CMakeLists.txt  | 29 -
 clang/utils/perf-training/bolt.lit.cfg| 53 +---
 .../utils/perf-training/bolt.lit.site.cfg.in  |  2 +
 clang/utils/perf-training/perf-helper.py  | 63 +++
 6 files changed, 167 insertions(+), 26 deletions(-)

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 9b52c58be41e7f7..8f64d95cc394ffe 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -850,23 +850,38 @@ if (CLANG_ENABLE_BOOTSTRAP)
   endforeach()
 endif()
 
-if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "Apply BOLT optimization to Clang. \
+  May be specified as Instrument or Perf or LBR to use a particular profiling \
+  mechanism.")
+string(TOUPPER "${CLANG_BOLT}" uppercase_CLANG_BOLT)
+
+if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   set(CLANG_PATH ${LLVM_RUNTIME_OUTPUT_INTDIR}/clang)
-  set(CLANG_INSTRUMENTED ${CLANG_PATH}-bolt.inst)
+  set(CLANG_INSTRUMENTED 
${LLVM_RUNTIME_OUTPUT_INTDIR}/${CLANG_BOLT_INSTRUMENTED})
   set(BOLT_FDATA ${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/prof.fdata)
 
-  # Instrument clang with BOLT
-  add_custom_target(clang-instrumented
-DEPENDS ${CLANG_INSTRUMENTED}
-  )
-  add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
-DEPENDS clang llvm-bolt
-COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
-  -instrument --instrumentation-file-append-pid
-  --instrumentation-file=${BOLT_FDATA}
-COMMENT "Instrumenting clang binary with BOLT"
-VERBATIM
-  )
+  # Pass extra flag in no-LBR mode
+  if (uppercase_CLANG_BOLT STREQUAL "PERF")
+set(BOLT_NO_LBR "-nl")
+  endif()
+
+  if (uppercase_CLANG_BOLT STREQUAL "INSTRUMENT")
+# Instrument clang with BOLT
+add_custom_target(clang-instrumented
+  DEPENDS ${CLANG_INSTRUMENTED}
+)
+add_custom_command(OUTPUT ${CLANG_INSTRUMENTED}
+  DEPENDS clang llvm-bolt
+  COMMAND llvm-bolt ${CLANG_PATH} -o ${CLANG_INSTRUMENTED}
+-instrument --instrumentation-file-append-pid
+--instrumentation-file=${BOLT_FDATA}
+  COMMENT "Instrumenting clang binary with BOLT"
+  VERBATIM
+)
+add_custom_target(clang-bolt-training-deps DEPENDS clang-instrumented)
+  else() # perf or LBR
+add_custom_target(clang-bolt-training-deps DEPENDS clang)
+  endif()
 
   # Optimize original (pre-bolt) Clang using the collected profile
   set(CLANG_OPTIMIZED ${CMAKE_CURRENT_BINARY_DIR}/clang.bolt)
@@ -880,6 +895,7 @@ if (CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=hfsort+ -split-functions
   -split-all-cold -split-eh -dyno-stats -icf=1 -use-gnu-stack
+  ${BOLT_NO_LBR}
 COMMAND ${CMAKE_COMMAND} -E rename ${CLANG_OPTIMIZED} $
 COMMENT "Optimizing Clang with BOLT"
 VERBATIM
diff --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 0442f73e5426ac7..eba2346b2f4ca12 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -1,5 +1,5 @@
 set(CMAKE_BUILD_TYPE Release CACHE STRING "")
-set(CLANG_BOLT_INSTRUMENT ON CACHE BOOL "")
+set(CLANG_BOLT "INSTRUMENT" CACHE STRING "")
 set(CMAKE_EXE_LINKER_FLAGS "-Wl,--emit-relocs,-znow" CACHE STRING "")
 
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index c6d51863fb1b5c2..48fbee62a8636d1 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -62,7 +62,9 @@ if(APPLE AND DTRACE AND NOT LLVM_TOOL_LLVM_DRIVER_BUILD)
 DEPENDS generate-dtrace-logs)
 endif()
 
-if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
+if(CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
+  set(CLANG_BOLT_INSTRUMENTED "clang-bolt.inst" CACHE STRING
+"Name of BOLT-instrumented Clang binary")
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/bolt.lit.site.cfg.in
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/lit.site.cfg
@@ -71,16 +73,37 @@ if(CLANG_BOLT_INSTRUMENT AND NOT LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-bolt-fdata "Generating BOLT profile for Clang"
 ${CMAKE_CURRENT_BINARY_DIR}/bolt-fdata/
 EXCLUDE_FROM_CHECK_ALL
-DEPENDS clang-instrumented clear-bolt-fdata
+DEPENDS clang-bolt-training-deps clear-bolt-fd

[clang] 42129de - [Clang][CMake] Set up distribution target for Clang-BOLT

2023-02-07 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2023-02-07T15:06:15-08:00
New Revision: 42129deaa67b6c4b8fc82feb74b7e9fe25c99073

URL: 
https://github.com/llvm/llvm-project/commit/42129deaa67b6c4b8fc82feb74b7e9fe25c99073
DIFF: 
https://github.com/llvm/llvm-project/commit/42129deaa67b6c4b8fc82feb74b7e9fe25c99073.diff

LOG: [Clang][CMake] Set up distribution target for Clang-BOLT

Provide a way to install usable BOLT-optimized Clang
(clang + resource headers) using
`ninja clang-bolt install-distribution` with BOLT.cmake cache file
or `ninja stage2-clang-bolt stage2-install-distribution`
with BOLT-PGO.cmake cache file.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D140565

Added: 


Modified: 
clang/cmake/caches/BOLT-PGO.cmake
clang/cmake/caches/BOLT.cmake

Removed: 




diff  --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
index 616e632d1531b..1a04ca9a74e5e 100644
--- a/clang/cmake/caches/BOLT-PGO.cmake
+++ b/clang/cmake/caches/BOLT-PGO.cmake
@@ -2,10 +2,16 @@ set(LLVM_ENABLE_PROJECTS "bolt;clang;lld" CACHE STRING "")
 
 set(CLANG_BOOTSTRAP_TARGETS
   stage2-clang-bolt
+  stage2-distribution
+  stage2-install-distribution
   CACHE STRING "")
 set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS
   clang-bolt
+  distribution
+  install-distribution
   CACHE STRING "")
 
-set(PGO_BUILD_CONFIGURATION ${CMAKE_CURRENT_LIST_DIR}/BOLT.cmake CACHE STRING 
"")
+set(PGO_BUILD_CONFIGURATION
+  ${CMAKE_CURRENT_LIST_DIR}/BOLT.cmake
+  CACHE STRING "")
 include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)

diff  --git a/clang/cmake/caches/BOLT.cmake b/clang/cmake/caches/BOLT.cmake
index 65444c8044c3b..78ab3b64386a4 100644
--- a/clang/cmake/caches/BOLT.cmake
+++ b/clang/cmake/caches/BOLT.cmake
@@ -8,6 +8,13 @@ set(CLANG_BOLT_INSTRUMENT_EXTRA_CMAKE_FLAGS "" CACHE STRING "")
 set(LLVM_ENABLE_PROJECTS "bolt;clang" CACHE STRING "")
 set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "")
 
+# setup toolchain
+set(LLVM_INSTALL_TOOLCHAIN_ONLY ON CACHE BOOL "")
+set(LLVM_DISTRIBUTION_COMPONENTS
+  clang
+  clang-resource-headers
+  CACHE STRING "")
+
 # Disable function splitting enabled by default in GCC8+
 if("${CMAKE_CXX_COMPILER_ID}" MATCHES "GNU")
   set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fno-reorder-blocks-and-partition")



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 6a75aec - [Clang][perf-training] Remove trailing whitespace in perf-helper

2023-02-11 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2023-02-11T21:10:27-08:00
New Revision: 6a75aec090986d3c897af7cfed8836ff36b38181

URL: 
https://github.com/llvm/llvm-project/commit/6a75aec090986d3c897af7cfed8836ff36b38181
DIFF: 
https://github.com/llvm/llvm-project/commit/6a75aec090986d3c897af7cfed8836ff36b38181.diff

LOG: [Clang][perf-training] Remove trailing whitespace in perf-helper

Added: 


Modified: 
clang/utils/perf-training/perf-helper.py

Removed: 




diff  --git a/clang/utils/perf-training/perf-helper.py 
b/clang/utils/perf-training/perf-helper.py
index abbdc77b23724..f28eb0b731dd3 100644
--- a/clang/utils/perf-training/perf-helper.py
+++ b/clang/utils/perf-training/perf-helper.py
@@ -21,7 +21,7 @@
 
 def findFilesWithExtension(path, extension):
   filenames = []
-  for root, dirs, files in os.walk(path): 
+  for root, dirs, files in os.walk(path):
 for filename in files:
   if filename.endswith(f".{extension}"):
 filenames.append(os.path.join(root, filename))
@@ -102,7 +102,7 @@ def dtrace(args):
   dtrace_args.extend((
   'dtrace', '-xevaltime=exec',
   '-xbufsize=%dm' % (opts.buffer_size),
-  '-q', '-n', dtrace_script, 
+  '-q', '-n', dtrace_script,
   '-c', ' '.join(cmd)))
 
   if sys.platform == "darwin":
@@ -244,7 +244,7 @@ def get_symbols_with_prefix(symbol):
 # If we found too many possible symbols, ignore this as a prefix.
 if len(possible_symbols) > 100:
   print( "warning: ignoring symbol %r " % symbol +
-"(no match and too many possible suffixes)", file=sys.stderr) 
+"(no match and too many possible suffixes)", file=sys.stderr)
   continue
 
 # Report that we resolved a missing symbol.
@@ -284,7 +284,7 @@ def form_by_call_order_fair(symbol_lists):
   succs[a] = items = succs.get(a, [])
   if b not in items:
 items.append(b)
-  
+
   # Emit all the symbols, but make sure to always emit all successors from any
   # call list whenever we see a symbol.
   #
@@ -295,11 +295,11 @@ def form_by_call_order_fair(symbol_lists):
 for symbols in symbol_lists
 for node in symbols
 for s in ([node] + succs.get(node,[])))
- 
+
 def form_by_frequency(symbol_lists):
   # Form the order file by just putting the most commonly occurring symbols
   # first. This assumes the data files didn't use the oneshot dtrace method.
- 
+
   counts = {}
   for symbols in symbol_lists:
 for a in symbols:
@@ -308,14 +308,14 @@ def form_by_frequency(symbol_lists):
   by_count = list(counts.items())
   by_count.sort(key = lambda __n: -__n[1])
   return [s for s,n in by_count]
- 
+
 def form_by_random(symbol_lists):
   # Randomize the symbols.
   merged_symbols = uniq(s for symbols in symbol_lists
   for s in symbols)
   random.shuffle(merged_symbols)
   return merged_symbols
- 
+
 def form_by_alphabetical(symbol_lists):
   # Alphabetize the symbols.
   merged_symbols = list(set(s for symbols in symbol_lists for s in symbols))



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [compiler-rt] [flang] [llvm] [BOLT][BAT] Add support for three-way split functions (PR #93760)

2024-07-05 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/93760
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [compiler-rt] [flang] [llvm] [BOLT][BAT] Add support for three-way split functions (PR #93760)

2024-05-31 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/93760
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [compiler-rt] [flang] [llvm] [BOLT][BAT] Add support for three-way split functions (PR #93760)

2024-06-04 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/93760
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT][NFC] Track fragment relationships using EquivalenceClasses (PR #99979)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/99979
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT][NFC] Track fragment relationships using EquivalenceClasses (PR #99979)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/99979
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT] Support more than two jump table parents (PR #99988)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/99988
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT] Support more than two jump table parents (PR #99988)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/99988
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT] Add profile-use-pseudo-probes option (PR #100299)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited 
https://github.com/llvm/llvm-project/pull/100299
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [libcxx] [lld] [lldb] [llvm] [mlir] [BOLT] Add profile-use-pseudo-probes option (PR #100299)

2024-07-24 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed 
https://github.com/llvm/llvm-project/pull/100299
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Use disambiguated local names in BAT YAML (PR #91773)

2024-05-10 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/91773
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Use disambiguated local names in BAT YAML (PR #91773)

2024-05-10 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/91773
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Set entry counts in BAT YAML profile (PR #91775)

2024-05-10 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/91775
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Set entry counts in BAT YAML profile (PR #91775)

2024-05-10 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/91775
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [flang] [lld] [llvm] Use StringRef::operator== instead of StringRef::equals (NFC) (PR #91864)

2024-05-12 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov approved this pull request.

BOLT changes LGTM

https://github.com/llvm/llvm-project/pull/91864
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT][NFC] Rename DataAggregator::BranchInfo to TakenBranchInfo (PR #92017)

2024-05-16 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/92017
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT][NFC] Rename DataAggregator::BranchInfo to TakenBranchInfo (PR #92017)

2024-05-16 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/92017
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libcxx] [lldb] [llvm] [mlir] [BOLT][BAT] Add entries for deleted basic blocks (PR #91906)

2024-05-16 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/91906
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [compiler-rt] [flang] [libcxx] [libcxxabi] [lld] [lldb] [llvm] [mlir] [BOLT] Ignore special symbols as function aliases in updateELFSymbolTable (PR #92713)

2024-05-20 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/92713
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [compiler-rt] [flang] [libcxx] [libcxxabi] [lld] [lldb] [llvm] [mlir] [BOLT] Ignore special symbols as function aliases in updateELFSymbolTable (PR #92713)

2024-05-20 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/92713
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libcxx] [lldb] [llvm] [mlir] [BOLT][BAT] Add entries for deleted basic blocks (PR #91906)

2024-05-23 Thread Amir Ayupov via cfe-commits


aaupov wrote:

> Could you elaborate a bit better on why do we need a deleted block to be 
> present in the table? My memory fails me, aren't we using the translation 
> table just to map samples collected on the bolted binary? Where do the 
> deleted blocks become a problem?
> 
> Other than the motivation, the implementation itself looks good to me.

We need deleted blocks to correctly map optimized tail calls to containing 
basic blocks. Because we expand tail calls during CFG construction, and attach 
the profile to that CFG, we need those intermediate blocks to be present in the 
profile. bb

-with-two-tail-calls offers a good example:
after building CFG (details omitted):
```
.LBB00 (1 instructions, align : 1)
:   je  .Ltmp0 # Offset: 0

.LFT0 (2 instructions, align : 1)
0002:   ja  .Ltmp1 # Offset: 2
0004:   jmp .Ltmp2 # Offset: 4

.Ltmp0 (1 instructions, align : 1)
0006:   retq # Offset: 6

.Ltmp1 (1 instructions, align : 1)
0007:   jmp e # TAILCALL  # Offset: 7

.Ltmp2 (2 instructions, align : 1)
000c:   nop # Offset: 12 # Size: 1 # NOP: 1
000d:   jmp f # TAILCALL  # Offset: 13
```

after SCTC, we normally write out BAT from that state:
```
.LBB00 (1 instructions, align : 1)
:   je  .Ltmp0 # Offset: 0

.LFT0 (2 instructions, align : 1)
0002:   ja  e # TAILCALL  # Offset: 7 # CTCTakenCount: 4
0004:   jmp f # TAILCALL  # Offset: 13

.Ltmp0 (1 instructions, align : 1)
0009:   retq # Offset: 6
```

So in order to map this branch
```
0004:   jmp f # TAILCALL  # Offset: 13
```

to its block Ltmp2 we need to preserve the block entry in BAT, although it's 
removed from CFG.

https://github.com/llvm/llvm-project/pull/91906
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libcxx] [lldb] [llvm] [mlir] [BOLT][BAT] Add entries for deleted basic blocks (PR #91906)

2024-05-23 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/91906
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libcxx] [lldb] [llvm] [mlir] [BOLT][BAT] Add entries for deleted basic blocks (PR #91906)

2024-05-24 Thread Amir Ayupov via cfe-commits


aaupov wrote:

The entries for deleted basic blocks won't participate in 
`BAT::getFallthroughsInTrace` because trace boundary is looked up by output 
offsets: 
https://github.com/llvm/llvm-project/blob/098c6dfa8157681699a71fce9e3d94515e66311f/bolt/lib/Profile/BoltAddressTranslation.cpp#L529-L540

But deleted blocks will be used in `writeBATYAML` where we lookup containing 
blocks by input offsets: 
https://github.com/llvm/llvm-project/blob/098c6dfa8157681699a71fce9e3d94515e66311f/bolt/lib/Profile/DataAggregator.cpp#L2358-L2367

https://github.com/llvm/llvm-project/pull/91906
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Drop blocks without profile in BAT YAML (PR #107970)

2024-09-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited 
https://github.com/llvm/llvm-project/pull/107970
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Drop blocks without profile in BAT YAML (PR #107970)

2024-09-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited 
https://github.com/llvm/llvm-project/pull/107970
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lldb] [llvm] [BOLT] Drop blocks without profile in BAT YAML (PR #107970)

2024-09-11 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed 
https://github.com/llvm/llvm-project/pull/107970
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-10-28 Thread Amir Ayupov via cfe-commits


aaupov wrote:

Ping @wlei-llvm @WenleiHe 

https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [lld] [lldb] [llvm] [mlir] [BOLT] Match blocks with pseudo probes (PR #99891)

2024-11-08 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/99891
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [compiler-rt] [libc] [lld] [lldb] [llvm] [mlir] [BOLT] Match blocks with pseudo probes (PR #99891)

2024-11-12 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed https://github.com/llvm/llvm-project/pull/99891
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From efac76027bd9294d734c3eee77ad582ad0fa9ebe Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 26 +-
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 56 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 120 insertions(+), 13 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index c6496167d3828b..46e1412c0727aa 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -744,11 +744,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 00..34159068d5ea3f
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 49673790ff6e84..a3db3b7c1fcdd2 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -20,6 +20,7 @@ if(LLVM_BUILD_INSTRUMENTED)
 
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -29,6 +30,10 @@ if(LLVM_BUILD_INSTRUMENTED)
   if(NOT LLVM_PROFDATA)
 message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to 
llvm-profdata")
   else()
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PROFDATA_SAMPLE "--sample")
+endif()
 add_custom_command(
   OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata
   # generate-profraw is a custom_target which are always considered stale.
@@ -39,7 +44,7 @@ if(LLVM_BUILD_INSTRUMENTED)
   # Therefor we call the generate-profraw target manually as part of this 
custom
   # command, which will only run if clang or ${CLANG_PGO_TRAINING_DEPS} 
are updated.
   COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target 
generate-profraw
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/
+  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/ ${PROFDATA_SAMPLE}
   COMMENT "Merging profdata"
   DEPENDS clang ${CLANG_PGO_TRAINING_DEPS}
 )
@@ -49,6 +54,25 @@ if(LLVM_BUILD_INSTRUMENTED)
   USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS 
generate-profraw)
   ad

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-18 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From 2a5a3f2f55bd2b7479278fa72f6f05e54033532c Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 12 -
 clang/cmake/caches/CSSPGO.cmake|  3 ++
 clang/utils/perf-training/CMakeLists.txt   | 28 ++-
 clang/utils/perf-training/lit.cfg  |  6 +++
 clang/utils/perf-training/lit.site.cfg.in  |  1 +
 clang/utils/perf-training/perf-helper.py   | 56 ++
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 121 insertions(+), 14 deletions(-)
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index c6496167d3828b..46e1412c0727aa 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -744,11 +744,21 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+else()
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 00..34159068d5ea3f
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 49673790ff6e84..91883a21182340 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -20,6 +20,7 @@ if(LLVM_BUILD_INSTRUMENTED)
 
   add_custom_target(clear-profraw
 COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw
+COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
 COMMENT "Clearing old profraw data")
 
   if(NOT LLVM_PROFDATA)
@@ -39,10 +40,33 @@ if(LLVM_BUILD_INSTRUMENTED)
   # Therefor we call the generate-profraw target manually as part of this 
custom
   # command, which will only run if clang or ${CLANG_PGO_TRAINING_DEPS} 
are updated.
   COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target 
generate-profraw
-  COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} 
${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} 
${CMAKE_BINARY_DIR}/profiles/
-  COMMENT "Merging profdata"
   DEPENDS clang ${CLANG_PGO_TRAINING_DEPS}
 )
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  set(PROFDATA_SAMPLE "--sample")
+  if(NOT LLVM_PROFGEN)
+find_program(LLVM_PROFGEN llvm-profgen)
+  endif()
+
+  if(NOT LLVM_PROFGEN)
+message(STATUS "To enable converting CSSPGO samples LLVM_PROFGEN has 
to point to llvm-profgen")
+  endif()
+
+  # Convert perf profiles into profraw
+  add_custom_command(
+OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata
+COMMAND "${Python3_EXECUTABLE}" 
${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py perf2prof ${LLVM_PROFGEN} 
$ ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/
+COMMENT "Converting perf profiles into profraw"
+APPEND
+  )
+endif()
+add_custom_command(
+  OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/clang.p

[clang] [llvm] [CMake] Preserve clang-prebolt (PR #109351)

2024-09-23 Thread Amir Ayupov via cfe-commits


aaupov wrote:

> This commit is breaking LTO-PGO-BOLT build while optimizing clang with BOLT. 
> 
> 
> 
> ```
> 
> [3064/3065] Merging BOLT fdata
> 
> Using legacy profile format.
> 
> Profile from 2 files merged.
> 
> [3064/3065] Optimizing Clang with BOLT
> 
> BOLT-INFO: shared object or position-independent executable detected
> 
> BOLT-INFO: Target architecture: aarch64
> 
> BOLT-INFO: BOLT version: 51d1d03b17bb8bf66b6422c4482d8d615ab53409
> 
> BOLT-INFO: first alloc address is 0x0
> 
> BOLT-INFO: enabling relocation mode
> 
> BOLT-INFO: pre-processing profile using branch profile reader
> 
> BOLT-WARNING: 1 collisions detected while hashing binary objects. Use -v=1 to 
> see the list.
> 
> BOLT-INFO: number of removed linker-inserted veneers: 0
> 
> BOLT-INFO: 11906 out of 212101 functions in the binary (5.6%) have non-empty 
> execution profile
> 
> BOLT-INFO: 272 functions with profile could not be optimized
> 
> BOLT-INFO: profile for 1 objects was ignored
> 
> BOLT-INFO: removed 1 empty block
> 
> BOLT-INFO: basic block reordering modified layout of 4273 functions (35.89% 
> of profiled, 2.01% of total)
> 
> BOLT-INFO: UCE removed 1 blocks and 4 bytes of code
> 
> BOLT-INFO: 68 Functions were reordered by LoopInversionPass
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> :0: error: fixup value out of range
> 
> ```
> 
> 
> 
> More information on LTO-PGO-BOLT can be found on 
> https://llvm.org/docs/AdvancedBuilds.html

This change was tested on a buildbot: 
https://lab.llvm.org/buildbot/#/builders/113/builds/2934

Can you please try cleaning the build directory?

https://github.com/llvm/llvm-project/pull/109351
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [CMake] Preserve clang-prebolt (PR #109351)

2024-09-21 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov closed 
https://github.com/llvm/llvm-project/pull/109351
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [CMake] Preserve clang-prebolt (PR #109351)

2024-09-21 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited 
https://github.com/llvm/llvm-project/pull/109351
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [CMake] Preserve clang-prebolt (PR #109351)

2024-09-23 Thread Amir Ayupov via cfe-commits

aaupov wrote:

> @aaupov this happens on AArch64 target, and only with -split-strategy=cdsplit 
> enabled. Changing it to profile2, for example, fixes the issue

Thanks for clarification. I'll drop cdsplit as we never tested it on AArch64. 
ARM clang-bolt buildbot hasn't picked this change up yet so I didn't get the 
signal.

https://github.com/llvm/llvm-project/pull/109351
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] d61b259 - [Clang-BOLT] Drop CDSplit flag

2024-09-23 Thread Amir Ayupov via cfe-commits


Author: Amir Ayupov
Date: 2024-09-23T08:15:05-07:00
New Revision: d61b2590f8e360695a5298311855c8649337969f

URL: 
https://github.com/llvm/llvm-project/commit/d61b2590f8e360695a5298311855c8649337969f
DIFF: 
https://github.com/llvm/llvm-project/commit/d61b2590f8e360695a5298311855c8649337969f.diff

LOG: [Clang-BOLT] Drop CDSplit flag

Breaks AArch64 Clang-BOLT builds:
https://github.com/llvm/llvm-project/pull/109351#issuecomment-2368584365

Added: 


Modified: 
clang/CMakeLists.txt

Removed: 




diff  --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 94ce5968beb75e..27e8095534a65c 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -917,7 +917,7 @@ if (CLANG_BOLT AND NOT LLVM_BUILD_INSTRUMENTED)
   -data ${BOLT_FDATA}
   -reorder-blocks=ext-tsp -reorder-functions=cdsort -split-functions
   -split-all-cold -split-eh -dyno-stats -use-gnu-stack
-  -split-strategy=cdsplit -update-debug-sections
+  -update-debug-sections
   ${BOLT_NO_LBR}
 COMMENT "Optimizing Clang with BOLT"
 USES_TERMINAL



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From a3a8054675f295c15e7a7e45f7aebd2223072964 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 15 +-
 clang/cmake/caches/BOLT-CSSPGO.cmake   |  3 ++
 clang/cmake/caches/BOLT-PGO.cmake  |  3 +-
 clang/cmake/caches/CSSPGO.cmake|  2 +
 clang/utils/perf-training/CMakeLists.txt   | 36 +++--
 clang/utils/perf-training/perf-helper.py   | 60 +-
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 127 insertions(+), 21 deletions(-)
 create mode 100644 clang/cmake/caches/BOLT-CSSPGO.cmake
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 94ce5968beb75e..644db0f846000f 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -744,11 +744,22 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
-add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps generate-sprofdata)
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.sprofdata)
+else()
+  add_dependencies(clang-bootstrap-deps generate-profdata)
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/BOLT-CSSPGO.cmake 
b/clang/cmake/caches/BOLT-CSSPGO.cmake
new file mode 100644
index 00..b1c204ad57ac58
--- /dev/null
+++ b/clang/cmake/caches/BOLT-CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(BOLT_PGO_CMAKE_CACHE "CSSPGO" CACHE STRING "")
+set(BOOTSTRAP_CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/BOLT-PGO.cmake)
diff --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
index 1a04ca9a74e5e3..cc9410fd0e95c7 100644
--- a/clang/cmake/caches/BOLT-PGO.cmake
+++ b/clang/cmake/caches/BOLT-PGO.cmake
@@ -1,3 +1,4 @@
+set(BOLT_PGO_CMAKE_CACHE "PGO" CACHE STRING "")
 set(LLVM_ENABLE_PROJECTS "bolt;clang;lld" CACHE STRING "")
 
 set(CLANG_BOOTSTRAP_TARGETS
@@ -14,4 +15,4 @@ set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS
 set(PGO_BUILD_CONFIGURATION
   ${CMAKE_CURRENT_LIST_DIR}/BOLT.cmake
   CACHE STRING "")
-include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
+include(${CMAKE_CURRENT_LIST_DIR}/${BOLT_PGO_CMAKE_CACHE}.cmake)
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 00..59e08a64f8aad8
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,2 @@
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 49673790ff6e84..45a2b475dd1a70 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -6,6 +6,10 @@ set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" 
CACHE PATH
 set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source 
directory containing cmake project with source files to use for generating pgo 
data")
 set(CLANG_PGO_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to 
build the PGO training data.")
 
+add_custom_target(clear-perf-data
+  COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
+  COMMENT "Clearing old perf data")
+
 if(LLVM_BUILD_INSTRUMENTED)
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
@@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-profraw "Generating clang PGO data"
 ${

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov updated 
https://github.com/llvm/llvm-project/pull/79942

>From a3a8054675f295c15e7a7e45f7aebd2223072964 Mon Sep 17 00:00:00 2001
From: Amir Ayupov 
Date: Wed, 19 Jul 2023 20:30:29 -0700
Subject: [PATCH 1/2] [Clang][CMake] Add CSSPGO support to
 LLVM_BUILD_INSTRUMENTED

Build on Clang-BOLT infrastructure to collect sample profiles for CSSPGO.
Add clang/cmake/caches/CSSPGO.cmake to automate CSSPGO Clang build.

Differential Revision: https://reviews.llvm.org/D155419
---
 clang/CMakeLists.txt   | 15 +-
 clang/cmake/caches/BOLT-CSSPGO.cmake   |  3 ++
 clang/cmake/caches/BOLT-PGO.cmake  |  3 +-
 clang/cmake/caches/CSSPGO.cmake|  2 +
 clang/utils/perf-training/CMakeLists.txt   | 36 +++--
 clang/utils/perf-training/perf-helper.py   | 60 +-
 llvm/CMakeLists.txt|  3 ++
 llvm/cmake/modules/HandleLLVMOptions.cmake | 26 +-
 8 files changed, 127 insertions(+), 21 deletions(-)
 create mode 100644 clang/cmake/caches/BOLT-CSSPGO.cmake
 create mode 100644 clang/cmake/caches/CSSPGO.cmake

diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt
index 94ce5968beb75e..644db0f846000f 100644
--- a/clang/CMakeLists.txt
+++ b/clang/CMakeLists.txt
@@ -744,11 +744,22 @@ if (CLANG_ENABLE_BOOTSTRAP)
   if(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
 add_dependencies(clang-bootstrap-deps llvm-profdata)
 set(PGO_OPT -DLLVM_PROFDATA=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profdata)
+string(TOUPPER "${BOOTSTRAP_LLVM_BUILD_INSTRUMENTED}" 
BOOTSTRAP_LLVM_BUILD_INSTRUMENTED)
+if (BOOTSTRAP_LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps llvm-profgen)
+  list(APPEND PGO_OPT 
-DLLVM_PROFGEN=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-profgen)
+endif()
   endif()
 
   if(LLVM_BUILD_INSTRUMENTED)
-add_dependencies(clang-bootstrap-deps generate-profdata)
-set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+string(TOUPPER "${LLVM_BUILD_INSTRUMENTED}" LLVM_BUILD_INSTRUMENTED)
+if (LLVM_BUILD_INSTRUMENTED STREQUAL "CSSPGO")
+  add_dependencies(clang-bootstrap-deps generate-sprofdata)
+  set(PGO_OPT 
-DLLVM_SPROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.sprofdata)
+else()
+  add_dependencies(clang-bootstrap-deps generate-profdata)
+  set(PGO_OPT 
-DLLVM_PROFDATA_FILE=${CMAKE_CURRENT_BINARY_DIR}/utils/perf-training/clang.profdata)
+endif()
 # Use the current tools for LTO instead of the instrumented ones
 list(APPEND _BOOTSTRAP_DEFAULT_PASSTHROUGH
   CMAKE_CXX_COMPILER
diff --git a/clang/cmake/caches/BOLT-CSSPGO.cmake 
b/clang/cmake/caches/BOLT-CSSPGO.cmake
new file mode 100644
index 00..b1c204ad57ac58
--- /dev/null
+++ b/clang/cmake/caches/BOLT-CSSPGO.cmake
@@ -0,0 +1,3 @@
+set(BOLT_PGO_CMAKE_CACHE "CSSPGO" CACHE STRING "")
+set(BOOTSTRAP_CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/BOLT-PGO.cmake)
diff --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
index 1a04ca9a74e5e3..cc9410fd0e95c7 100644
--- a/clang/cmake/caches/BOLT-PGO.cmake
+++ b/clang/cmake/caches/BOLT-PGO.cmake
@@ -1,3 +1,4 @@
+set(BOLT_PGO_CMAKE_CACHE "PGO" CACHE STRING "")
 set(LLVM_ENABLE_PROJECTS "bolt;clang;lld" CACHE STRING "")
 
 set(CLANG_BOOTSTRAP_TARGETS
@@ -14,4 +15,4 @@ set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS
 set(PGO_BUILD_CONFIGURATION
   ${CMAKE_CURRENT_LIST_DIR}/BOLT.cmake
   CACHE STRING "")
-include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
+include(${CMAKE_CURRENT_LIST_DIR}/${BOLT_PGO_CMAKE_CACHE}.cmake)
diff --git a/clang/cmake/caches/CSSPGO.cmake b/clang/cmake/caches/CSSPGO.cmake
new file mode 100644
index 00..59e08a64f8aad8
--- /dev/null
+++ b/clang/cmake/caches/CSSPGO.cmake
@@ -0,0 +1,2 @@
+set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED "CSSPGO" CACHE STRING "")
+include(${CMAKE_CURRENT_LIST_DIR}/PGO.cmake)
diff --git a/clang/utils/perf-training/CMakeLists.txt 
b/clang/utils/perf-training/CMakeLists.txt
index 49673790ff6e84..45a2b475dd1a70 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -6,6 +6,10 @@ set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" 
CACHE PATH
 set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source 
directory containing cmake project with source files to use for generating pgo 
data")
 set(CLANG_PGO_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to 
build the PGO training data.")
 
+add_custom_target(clear-perf-data
+  COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py 
clean ${CMAKE_CURRENT_BINARY_DIR} perf.data
+  COMMENT "Clearing old perf data")
+
 if(LLVM_BUILD_INSTRUMENTED)
   configure_lit_site_cfg(
 ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
@@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED)
   add_lit_testsuite(generate-profraw "Generating clang PGO data"

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

2024-09-25 Thread Amir Ayupov via cfe-commits


https://github.com/aaupov edited https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

1 2 >

1 - 100 of 111 matches

Mail list logo