https://github.com/Keenuts created
https://github.com/llvm/llvm-project/pull/134844
When a callee is marked as `convergent`, some targets like HLSL/SPIR-V add a
convergent token to the call.
This is valid if both functions are marked as `convergent`.
ADCE/BDCE and other DCE passes were allowed to remove convergence intrinsics
when their token were unused.
This meant a leaf function could lose all its convergence intrinsics. This
would allow further optimization to remove the `convergent` attribute from the
callee.
Issue was the caller was not updated, and we now had a convergence token
attached to a call function calling a non-convergent function.
This can be solved in different ways:
- modify DCE passes to keep calls marked as convergent.
- add a 'hasSideEffect' to the intrinsics definition, preventing DCE to remove
the intrinsic.
- fix optimization passed removing the attribute by also removing the tokens
from the call.
I picked the second option: mark those as `hasSideEffects`, because convergence
intrinsics presence impact the divergence/reconvergence location of the IR,
hence their simple presence has an implicit meaning. This is particularly
important for SPIR-V as the control flow shape will depend on the presence of
those intrinsics.
An alternative I could agree to modifying each pass to keep the convergence
intrinsics. After all, they are important, but the widening the 'hadSideEffect'
meaning could be discussed.
I however would be against the 3rd solution: SPIR-V benefits greatly from the
convergence intrinsics presence to correctly structurize loops.
From b2dbe55df7cb7e9958f7e836c35136751736650a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Nathan=20Gau=C3=ABr?= <brio...@google.com>
Date: Mon, 7 Apr 2025 17:39:10 +0200
Subject: [PATCH] [IR] Mark convergence intrins as has-side-effect
When a callee is marked as `convergent`, some targets like HLSL/SPIR-V
add a convergent token to the call.
This is valid if both functions are marked as `convergent`.
ADCE/BDCE and other DCE passes were allowed to remove convergence
intrinsics when their token were unused.
This meant a leaf function could lose all its convergence intrinsics.
This would allow further optimization to remove the `convergent`
attribute from the callee.
Issue was the caller was not updated, and we now had a convergence token
attached to a call function calling a non-convergent function.
This can be solved in different ways:
- modify DCE passes to keep calls marked as convergent.
- add a 'hasSideEffect' to the intrinsics definition, preventing DCE to
remove the intrinsic.
- fix optimization passed removing the attribute by also removing the
tokens from the call.
I picked the second option: mark those as `hasSideEffects`, because
convergence intrinsics presence impact the divergence/reconvergence
location of the IR, hence their simple presence has an implicit meaning.
This is particularly important for SPIR-V as the control flow shape will
depend on the presence of those intrinsics.
An alternative I could agree to modifying each pass to keep the
convergence intrinsics. After all, they are important, but the widening
the 'hadSideEffect' meaning could be discussed.
I however would be against the 3rd solution: SPIR-V benefits greatly
from the convergence intrinsics presence to correctly structurize loops.
---
.../builtins/RWBuffer-constructor-opt.hlsl | 3 +-
clang/test/CodeGenHLSL/builtins/distance.hlsl | 8 ++++
clang/test/CodeGenHLSL/builtins/length.hlsl | 8 ++++
clang/test/CodeGenHLSL/builtins/reflect.hlsl | 8 ++++
.../test/CodeGenHLSL/builtins/smoothstep.hlsl | 8 ++++
clang/test/CodeGenSPIRV/Builtins/distance.c | 3 ++
clang/test/CodeGenSPIRV/Builtins/length.c | 3 ++
clang/test/CodeGenSPIRV/Builtins/reflect.c | 3 ++
clang/test/CodeGenSPIRV/Builtins/smoothstep.c | 4 ++
llvm/include/llvm/IR/Intrinsics.td | 9 ++--
llvm/test/Transforms/ADCE/convergence.ll | 46 ++++++++++++++++++
llvm/test/Transforms/BDCE/convergence.ll | 47 +++++++++++++++++++
llvm/test/Transforms/DCE/op_bundles.ll | 1 +
.../Transforms/FunctionAttrs/convergent.ll | 16 ++++++-
.../regression-convergence-tokens.ll | 8 ++--
.../LoopUnroll/convergent.controlled.ll | 8 ++++
16 files changed, 174 insertions(+), 9 deletions(-)
create mode 100644 llvm/test/Transforms/ADCE/convergence.ll
create mode 100644 llvm/test/Transforms/BDCE/convergence.ll
diff --git a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor-opt.hlsl
b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor-opt.hlsl
index 56c523f6bc8cf..25062b8537aca 100644
--- a/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor-opt.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/RWBuffer-constructor-opt.hlsl
@@ -1,5 +1,5 @@
// RUN: %clang_cc1 -triple dxil-pc-shadermodel6.3-library -x hlsl -emit-llvm
-O3 -o - %s | FileCheck %s
-// RUN: %clang_cc1 -triple spirv-vulkan-compute -x hlsl -emit-llvm -O3 -o - %s
| FileCheck %s
+// RUN: %clang_cc1 -triple spirv-vulkan-compute -x hlsl -emit-llvm -O3 -o - %s
| FileCheck %s --check-prefixes=CHECK,SPIRV
// All referenced to an unused resource should be removed by optimizations.
RWBuffer<float> Buf : register(u5, space3);
@@ -10,6 +10,7 @@ void main() {
// CHECK-NOT: resource.handlefrombinding
// CHECK: define void @main()
// CHECK-NEXT: entry:
+// SPIRV-NEXT: %0 = tail call token @llvm.experimental.convergence.entry()
// CHECK-NEXT: ret void
// CHECK-NOT: resource.handlefrombinding
}
diff --git a/clang/test/CodeGenHLSL/builtins/distance.hlsl
b/clang/test/CodeGenHLSL/builtins/distance.hlsl
index e830903261c8c..1d8f986bd12eb 100644
--- a/clang/test/CodeGenHLSL/builtins/distance.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/distance.hlsl
@@ -16,6 +16,7 @@
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z18test_distance_halfDhDh(
// SPVCHECK-SAME: half noundef nofpclass(nan inf) [[X:%.*]], half noundef
nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn half
[[X]], [[Y]]
// SPVCHECK-NEXT: [[ELT_ABS_I:%.*]] = tail call reassoc nnan ninf nsz arcp
afn noundef half @llvm.fabs.f16(half [[SUB_I]])
// SPVCHECK-NEXT: ret half [[ELT_ABS_I]]
@@ -33,6 +34,7 @@ half test_distance_half(half X, half Y) { return distance(X,
Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z19test_distance_half2Dv2_DhS_(
// SPVCHECK-SAME: <2 x half> noundef nofpclass(nan inf) [[X:%.*]], <2 x half>
noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <2 x
half> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v2f16(<2 x half> [[SUB_I]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
@@ -50,6 +52,7 @@ half test_distance_half2(half2 X, half2 Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z19test_distance_half3Dv3_DhS_(
// SPVCHECK-SAME: <3 x half> noundef nofpclass(nan inf) [[X:%.*]], <3 x half>
noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <3 x
half> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v3f16(<3 x half> [[SUB_I]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
@@ -67,6 +70,7 @@ half test_distance_half3(half3 X, half3 Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z19test_distance_half4Dv4_DhS_(
// SPVCHECK-SAME: <4 x half> noundef nofpclass(nan inf) [[X:%.*]], <4 x half>
noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <4 x
half> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v4f16(<4 x half> [[SUB_I]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
@@ -83,6 +87,7 @@ half test_distance_half4(half4 X, half4 Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z19test_distance_floatff(
// SPVCHECK-SAME: float noundef nofpclass(nan inf) [[X:%.*]], float noundef
nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn float
[[X]], [[Y]]
// SPVCHECK-NEXT: [[ELT_ABS_I:%.*]] = tail call reassoc nnan ninf nsz arcp
afn noundef float @llvm.fabs.f32(float [[SUB_I]])
// SPVCHECK-NEXT: ret float [[ELT_ABS_I]]
@@ -100,6 +105,7 @@ float test_distance_float(float X, float Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z20test_distance_float2Dv2_fS_(
// SPVCHECK-SAME: <2 x float> noundef nofpclass(nan inf) [[X:%.*]], <2 x
float> noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <2 x
float> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v2f32(<2 x float> [[SUB_I]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
@@ -117,6 +123,7 @@ float test_distance_float2(float2 X, float2 Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z20test_distance_float3Dv3_fS_(
// SPVCHECK-SAME: <3 x float> noundef nofpclass(nan inf) [[X:%.*]], <3 x
float> noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <3 x
float> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v3f32(<3 x float> [[SUB_I]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
@@ -134,6 +141,7 @@ float test_distance_float3(float3 X, float3 Y) { return
distance(X, Y); }
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z20test_distance_float4Dv4_fS_(
// SPVCHECK-SAME: <4 x float> noundef nofpclass(nan inf) [[X:%.*]], <4 x
float> noundef nofpclass(nan inf) [[Y:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SUB_I:%.*]] = fsub reassoc nnan ninf nsz arcp afn <4 x
float> [[X]], [[Y]]
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v4f32(<4 x float> [[SUB_I]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
diff --git a/clang/test/CodeGenHLSL/builtins/length.hlsl
b/clang/test/CodeGenHLSL/builtins/length.hlsl
index 2d4bbd995298f..9597d33f70d62 100644
--- a/clang/test/CodeGenHLSL/builtins/length.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/length.hlsl
@@ -20,6 +20,7 @@
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z16test_length_halfDh(
// SPVCHECK-SAME: half noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0:[0-9]+]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[ELT_ABS_I:%.*]] = tail call reassoc nnan ninf nsz arcp
afn noundef half @llvm.fabs.f16(half [[P0]])
// SPVCHECK-NEXT: ret half [[ELT_ABS_I]]
//
@@ -42,6 +43,7 @@ half test_length_half(half p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z17test_length_half2Dv2_Dh(
// SPVCHECK-SAME: <2 x half> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v2f16(<2 x half> [[P0]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
//
@@ -61,6 +63,7 @@ half test_length_half2(half2 p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z17test_length_half3Dv3_Dh(
// SPVCHECK-SAME: <3 x half> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v3f16(<3 x half> [[P0]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
//
@@ -80,6 +83,7 @@ half test_length_half3(half3 p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z17test_length_half4Dv4_Dh(
// SPVCHECK-SAME: <4 x half> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef half @llvm.spv.length.v4f16(<4 x half> [[P0]])
// SPVCHECK-NEXT: ret half [[SPV_LENGTH_I]]
//
@@ -98,6 +102,7 @@ half test_length_half4(half4 p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z17test_length_floatf(
// SPVCHECK-SAME: float noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[ELT_ABS_I:%.*]] = tail call reassoc nnan ninf nsz arcp
afn noundef float @llvm.fabs.f32(float [[P0]])
// SPVCHECK-NEXT: ret float [[ELT_ABS_I]]
//
@@ -117,6 +122,7 @@ float test_length_float(float p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z18test_length_float2Dv2_f(
// SPVCHECK-SAME: <2 x float> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v2f32(<2 x float> [[P0]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
//
@@ -136,6 +142,7 @@ float test_length_float2(float2 p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z18test_length_float3Dv3_f(
// SPVCHECK-SAME: <3 x float> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v3f32(<3 x float> [[P0]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
//
@@ -155,6 +162,7 @@ float test_length_float3(float3 p0)
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z18test_length_float4Dv4_f(
// SPVCHECK-SAME: <4 x float> noundef nofpclass(nan inf) [[P0:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_LENGTH_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef float @llvm.spv.length.v4f32(<4 x float> [[P0]])
// SPVCHECK-NEXT: ret float [[SPV_LENGTH_I]]
//
diff --git a/clang/test/CodeGenHLSL/builtins/reflect.hlsl
b/clang/test/CodeGenHLSL/builtins/reflect.hlsl
index 35ee059697c4b..3f1f653e0f0f9 100644
--- a/clang/test/CodeGenHLSL/builtins/reflect.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/reflect.hlsl
@@ -18,6 +18,7 @@
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z17test_reflect_halfDhDh(
// SPVCHECK-SAME: half noundef nofpclass(nan inf) [[I:%.*]], half noundef
nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[MUL_I:%.*]] = fmul reassoc nnan ninf nsz arcp afn half
[[I]], 0xH4000
// SPVCHECK-NEXT: [[TMP0:%.*]] = fmul reassoc nnan ninf nsz arcp afn half
[[N]], [[N]]
// SPVCHECK-NEXT: [[MUL2_I:%.*]] = fmul reassoc nnan ninf nsz arcp afn half
[[TMP0]], [[MUL_I]]
@@ -42,6 +43,7 @@ half test_reflect_half(half I, half N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <2 x half>
@_Z18test_reflect_half2Dv2_DhS_(
// SPVCHECK-SAME: <2 x half> noundef nofpclass(nan inf) [[I:%.*]], <2 x half>
noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <2 x half> @llvm.spv.reflect.v2f16(<2 x half> [[I]], <2 x
half> [[N]])
// SPVCHECK-NEXT: ret <2 x half> [[SPV_REFLECT_I]]
//
@@ -63,6 +65,7 @@ half2 test_reflect_half2(half2 I, half2 N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <3 x half>
@_Z18test_reflect_half3Dv3_DhS_(
// SPVCHECK-SAME: <3 x half> noundef nofpclass(nan inf) [[I:%.*]], <3 x half>
noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <3 x half> @llvm.spv.reflect.v3f16(<3 x half> [[I]], <3 x
half> [[N]])
// SPVCHECK-NEXT: ret <3 x half> [[SPV_REFLECT_I]]
//
@@ -84,6 +87,7 @@ half3 test_reflect_half3(half3 I, half3 N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <4 x half>
@_Z18test_reflect_half4Dv4_DhS_(
// SPVCHECK-SAME: <4 x half> noundef nofpclass(nan inf) [[I:%.*]], <4 x half>
noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <4 x half> @llvm.spv.reflect.v4f16(<4 x half> [[I]], <4 x
half> [[N]])
// SPVCHECK-NEXT: ret <4 x half> [[SPV_REFLECT_I]]
//
@@ -103,6 +107,7 @@ half4 test_reflect_half4(half4 I, half4 N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z18test_reflect_floatff(
// SPVCHECK-SAME: float noundef nofpclass(nan inf) [[I:%.*]], float noundef
nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[MUL_I:%.*]] = fmul reassoc nnan ninf nsz arcp afn float
[[I]], 2.000000e+00
// SPVCHECK-NEXT: [[TMP0:%.*]] = fmul reassoc nnan ninf nsz arcp afn float
[[N]], [[N]]
// SPVCHECK-NEXT: [[MUL2_I:%.*]] = fmul reassoc nnan ninf nsz arcp afn
float [[TMP0]], [[MUL_I]]
@@ -127,6 +132,7 @@ float test_reflect_float(float I, float N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <2 x float>
@_Z19test_reflect_float2Dv2_fS_(
// SPVCHECK-SAME: <2 x float> noundef nofpclass(nan inf) [[I:%.*]], <2 x
float> noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <2 x float> @llvm.spv.reflect.v2f32(<2 x float> [[I]], <2 x
float> [[N]])
// SPVCHECK-NEXT: ret <2 x float> [[SPV_REFLECT_I]]
//
@@ -148,6 +154,7 @@ float2 test_reflect_float2(float2 I, float2 N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <3 x float>
@_Z19test_reflect_float3Dv3_fS_(
// SPVCHECK-SAME: <3 x float> noundef nofpclass(nan inf) [[I:%.*]], <3 x
float> noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <3 x float> @llvm.spv.reflect.v3f32(<3 x float> [[I]], <3 x
float> [[N]])
// SPVCHECK-NEXT: ret <3 x float> [[SPV_REFLECT_I]]
//
@@ -169,6 +176,7 @@ float3 test_reflect_float3(float3 I, float3 N) {
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <4 x float>
@_Z19test_reflect_float4Dv4_fS_(
// SPVCHECK-SAME: <4 x float> noundef nofpclass(nan inf) [[I:%.*]], <4 x
float> noundef nofpclass(nan inf) [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_REFLECT_I:%.*]] = tail call reassoc nnan ninf nsz
arcp afn noundef <4 x float> @llvm.spv.reflect.v4f32(<4 x float> [[I]], <4 x
float> [[N]])
// SPVCHECK-NEXT: ret <4 x float> [[SPV_REFLECT_I]]
//
diff --git a/clang/test/CodeGenHLSL/builtins/smoothstep.hlsl
b/clang/test/CodeGenHLSL/builtins/smoothstep.hlsl
index f2328c7330e6c..de95c11a138e7 100644
--- a/clang/test/CodeGenHLSL/builtins/smoothstep.hlsl
+++ b/clang/test/CodeGenHLSL/builtins/smoothstep.hlsl
@@ -22,6 +22,7 @@
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) half
@_Z20test_smoothstep_halfDhDhDh(
// SPVCHECK-SAME: half noundef nofpclass(nan inf) [[MIN:%.*]], half noundef
nofpclass(nan inf) [[MAX:%.*]], half noundef nofpclass(nan inf) [[X:%.*]])
local_unnamed_addr #[[ATTR0:[0-9]+]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef half @llvm.spv.smoothstep.f16(half [[MIN]], half [[MAX]],
half [[X]])
// SPVCHECK-NEXT: ret half [[SPV_SMOOTHSTEP_I]]
//
@@ -43,6 +44,7 @@ half test_smoothstep_half(half Min, half Max, half X) {
return smoothstep(Min, M
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <2 x half>
@_Z21test_smoothstep_half2Dv2_DhS_S_(
// SPVCHECK-SAME: <2 x half> noundef nofpclass(nan inf) [[MIN:%.*]], <2 x
half> noundef nofpclass(nan inf) [[MAX:%.*]], <2 x half> noundef nofpclass(nan
inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <2 x half> @llvm.spv.smoothstep.v2f16(<2 x half> [[MIN]],
<2 x half> [[MAX]], <2 x half> [[X]])
// SPVCHECK-NEXT: ret <2 x half> [[SPV_SMOOTHSTEP_I]]
//
@@ -64,6 +66,7 @@ half2 test_smoothstep_half2(half2 Min, half2 Max, half2 X) {
return smoothstep(M
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <3 x half>
@_Z21test_smoothstep_half3Dv3_DhS_S_(
// SPVCHECK-SAME: <3 x half> noundef nofpclass(nan inf) [[MIN:%.*]], <3 x
half> noundef nofpclass(nan inf) [[MAX:%.*]], <3 x half> noundef nofpclass(nan
inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <3 x half> @llvm.spv.smoothstep.v3f16(<3 x half> [[MIN]],
<3 x half> [[MAX]], <3 x half> [[X]])
// SPVCHECK-NEXT: ret <3 x half> [[SPV_SMOOTHSTEP_I]]
//
@@ -85,6 +88,7 @@ half3 test_smoothstep_half3(half3 Min, half3 Max, half3 X) {
return smoothstep(M
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <4 x half>
@_Z21test_smoothstep_half4Dv4_DhS_S_(
// SPVCHECK-SAME: <4 x half> noundef nofpclass(nan inf) [[MIN:%.*]], <4 x
half> noundef nofpclass(nan inf) [[MAX:%.*]], <4 x half> noundef nofpclass(nan
inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <4 x half> @llvm.spv.smoothstep.v4f16(<4 x half> [[MIN]],
<4 x half> [[MAX]], <4 x half> [[X]])
// SPVCHECK-NEXT: ret <4 x half> [[SPV_SMOOTHSTEP_I]]
//
@@ -106,6 +110,7 @@ half4 test_smoothstep_half4(half4 Min, half4 Max, half4 X)
{ return smoothstep(M
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) float
@_Z21test_smoothstep_floatfff(
// SPVCHECK-SAME: float noundef nofpclass(nan inf) [[MIN:%.*]], float noundef
nofpclass(nan inf) [[MAX:%.*]], float noundef nofpclass(nan inf) [[X:%.*]])
local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef float @llvm.spv.smoothstep.f32(float [[MIN]], float
[[MAX]], float [[X]])
// SPVCHECK-NEXT: ret float [[SPV_SMOOTHSTEP_I]]
//
@@ -127,6 +132,7 @@ float test_smoothstep_float(float Min, float Max, float X)
{ return smoothstep(M
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <2 x float>
@_Z22test_smoothstep_float2Dv2_fS_S_(
// SPVCHECK-SAME: <2 x float> noundef nofpclass(nan inf) [[MIN:%.*]], <2 x
float> noundef nofpclass(nan inf) [[MAX:%.*]], <2 x float> noundef
nofpclass(nan inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <2 x float> @llvm.spv.smoothstep.v2f32(<2 x float>
[[MIN]], <2 x float> [[MAX]], <2 x float> [[X]])
// SPVCHECK-NEXT: ret <2 x float> [[SPV_SMOOTHSTEP_I]]
//
@@ -148,6 +154,7 @@ float2 test_smoothstep_float2(float2 Min, float2 Max,
float2 X) { return smooths
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <3 x float>
@_Z22test_smoothstep_float3Dv3_fS_S_(
// SPVCHECK-SAME: <3 x float> noundef nofpclass(nan inf) [[MIN:%.*]], <3 x
float> noundef nofpclass(nan inf) [[MAX:%.*]], <3 x float> noundef
nofpclass(nan inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <3 x float> @llvm.spv.smoothstep.v3f32(<3 x float>
[[MIN]], <3 x float> [[MAX]], <3 x float> [[X]])
// SPVCHECK-NEXT: ret <3 x float> [[SPV_SMOOTHSTEP_I]]
//
@@ -169,6 +176,7 @@ float3 test_smoothstep_float3(float3 Min, float3 Max,
float3 X) { return smooths
// SPVCHECK-LABEL: define spir_func noundef nofpclass(nan inf) <4 x float>
@_Z22test_smoothstep_float4Dv4_fS_S_(
// SPVCHECK-SAME: <4 x float> noundef nofpclass(nan inf) [[MIN:%.*]], <4 x
float> noundef nofpclass(nan inf) [[MAX:%.*]], <4 x float> noundef
nofpclass(nan inf) [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// SPVCHECK-NEXT: [[ENTRY:.*:]]
+// SPVCHECK-NEXT: {{.*}} = tail call token
@llvm.experimental.convergence.entry()
// SPVCHECK-NEXT: [[SPV_SMOOTHSTEP_I:%.*]] = tail call reassoc nnan ninf
nsz arcp afn noundef <4 x float> @llvm.spv.smoothstep.v4f32(<4 x float>
[[MIN]], <4 x float> [[MAX]], <4 x float> [[X]])
// SPVCHECK-NEXT: ret <4 x float> [[SPV_SMOOTHSTEP_I]]
//
diff --git a/clang/test/CodeGenSPIRV/Builtins/distance.c
b/clang/test/CodeGenSPIRV/Builtins/distance.c
index 76c684b932c1d..13dfade1a189a 100644
--- a/clang/test/CodeGenSPIRV/Builtins/distance.c
+++ b/clang/test/CodeGenSPIRV/Builtins/distance.c
@@ -9,6 +9,7 @@ typedef float float4 __attribute__((ext_vector_type(4)));
// CHECK-LABEL: define spir_func float @test_distance_float2(
// CHECK-SAME: <2 x float> noundef [[X:%.*]], <2 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0:[0-9]+]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_DISTANCE:%.*]] = tail call float
@llvm.spv.distance.v2f32(<2 x float> [[X]], <2 x float> [[Y]])
// CHECK-NEXT: ret float [[SPV_DISTANCE]]
//
@@ -17,6 +18,7 @@ float test_distance_float2(float2 X, float2 Y) { return
__builtin_spirv_distance
// CHECK-LABEL: define spir_func float @test_distance_float3(
// CHECK-SAME: <3 x float> noundef [[X:%.*]], <3 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_DISTANCE:%.*]] = tail call float
@llvm.spv.distance.v3f32(<3 x float> [[X]], <3 x float> [[Y]])
// CHECK-NEXT: ret float [[SPV_DISTANCE]]
//
@@ -25,6 +27,7 @@ float test_distance_float3(float3 X, float3 Y) { return
__builtin_spirv_distance
// CHECK-LABEL: define spir_func float @test_distance_float4(
// CHECK-SAME: <4 x float> noundef [[X:%.*]], <4 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_DISTANCE:%.*]] = tail call float
@llvm.spv.distance.v4f32(<4 x float> [[X]], <4 x float> [[Y]])
// CHECK-NEXT: ret float [[SPV_DISTANCE]]
//
diff --git a/clang/test/CodeGenSPIRV/Builtins/length.c
b/clang/test/CodeGenSPIRV/Builtins/length.c
index 59e7c298dd816..3c6e4592e02f8 100644
--- a/clang/test/CodeGenSPIRV/Builtins/length.c
+++ b/clang/test/CodeGenSPIRV/Builtins/length.c
@@ -9,6 +9,7 @@ typedef float float4 __attribute__((ext_vector_type(4)));
// CHECK-LABEL: define spir_func float @test_length_float2(
// CHECK-SAME: <2 x float> noundef [[X:%.*]]) local_unnamed_addr
#[[ATTR0:[0-9]+]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_LENGTH:%.*]] = tail call float
@llvm.spv.length.v2f32(<2 x float> [[X]])
// CHECK-NEXT: ret float [[SPV_LENGTH]]
//
@@ -17,6 +18,7 @@ float test_length_float2(float2 X) { return
__builtin_spirv_length(X); }
// CHECK-LABEL: define spir_func float @test_length_float3(
// CHECK-SAME: <3 x float> noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_LENGTH:%.*]] = tail call float
@llvm.spv.length.v3f32(<3 x float> [[X]])
// CHECK-NEXT: ret float [[SPV_LENGTH]]
//
@@ -25,6 +27,7 @@ float test_length_float3(float3 X) { return
__builtin_spirv_length(X); }
// CHECK-LABEL: define spir_func float @test_length_float4(
// CHECK-SAME: <4 x float> noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_LENGTH:%.*]] = tail call float
@llvm.spv.length.v4f32(<4 x float> [[X]])
// CHECK-NEXT: ret float [[SPV_LENGTH]]
//
diff --git a/clang/test/CodeGenSPIRV/Builtins/reflect.c
b/clang/test/CodeGenSPIRV/Builtins/reflect.c
index f51ac27a07457..e1e784fb8b961 100644
--- a/clang/test/CodeGenSPIRV/Builtins/reflect.c
+++ b/clang/test/CodeGenSPIRV/Builtins/reflect.c
@@ -9,6 +9,7 @@ typedef float float4 __attribute__((ext_vector_type(4)));
// CHECK-LABEL: define spir_func <2 x float> @test_reflect_float2(
// CHECK-SAME: <2 x float> noundef [[X:%.*]], <2 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0:[0-9]+]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_REFLECT:%.*]] = tail call <2 x float>
@llvm.spv.reflect.v2f32(<2 x float> [[X]], <2 x float> [[Y]])
// CHECK-NEXT: ret <2 x float> [[SPV_REFLECT]]
//
@@ -17,6 +18,7 @@ float2 test_reflect_float2(float2 X, float2 Y) { return
__builtin_spirv_reflect(
// CHECK-LABEL: define spir_func <3 x float> @test_reflect_float3(
// CHECK-SAME: <3 x float> noundef [[X:%.*]], <3 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_REFLECT:%.*]] = tail call <3 x float>
@llvm.spv.reflect.v3f32(<3 x float> [[X]], <3 x float> [[Y]])
// CHECK-NEXT: ret <3 x float> [[SPV_REFLECT]]
//
@@ -25,6 +27,7 @@ float3 test_reflect_float3(float3 X, float3 Y) { return
__builtin_spirv_reflect(
// CHECK-LABEL: define spir_func <4 x float> @test_reflect_float4(
// CHECK-SAME: <4 x float> noundef [[X:%.*]], <4 x float> noundef [[Y:%.*]])
local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_REFLECT:%.*]] = tail call <4 x float>
@llvm.spv.reflect.v4f32(<4 x float> [[X]], <4 x float> [[Y]])
// CHECK-NEXT: ret <4 x float> [[SPV_REFLECT]]
//
diff --git a/clang/test/CodeGenSPIRV/Builtins/smoothstep.c
b/clang/test/CodeGenSPIRV/Builtins/smoothstep.c
index 714db4d9f728c..1c7a37a89d059 100644
--- a/clang/test/CodeGenSPIRV/Builtins/smoothstep.c
+++ b/clang/test/CodeGenSPIRV/Builtins/smoothstep.c
@@ -9,6 +9,7 @@ typedef float float4 __attribute__((ext_vector_type(4)));
// CHECK-LABEL: define spir_func float @test_smoothstep_float(
// CHECK-SAME: float noundef [[MIN:%.*]], float noundef [[MAX:%.*]], float
noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_SMOOTHSTEP:%.*]] = tail call float
@llvm.spv.smoothstep.f32(float [[MIN]], float [[MAX]], float [[X]])
// CHECK-NEXT: ret float [[SPV_SMOOTHSTEP]]
//
@@ -17,6 +18,7 @@ float test_smoothstep_float(float Min, float Max, float X) {
return __builtin_sp
// CHECK-LABEL: define spir_func <2 x float> @test_smoothstep_float2(
// CHECK-SAME: <2 x float> noundef [[MIN:%.*]], <2 x float> noundef
[[MAX:%.*]], <2 x float> noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_SMOOTHSTEP:%.*]] = tail call <2 x float>
@llvm.spv.smoothstep.v2f32(<2 x float> [[MIN]], <2 x float> [[MAX]], <2 x
float> [[X]])
// CHECK-NEXT: ret <2 x float> [[SPV_SMOOTHSTEP]]
//
@@ -25,6 +27,7 @@ float2 test_smoothstep_float2(float2 Min, float2 Max, float2
X) { return __built
// CHECK-LABEL: define spir_func <3 x float> @test_smoothstep_float3(
// CHECK-SAME: <3 x float> noundef [[MIN:%.*]], <3 x float> noundef
[[MAX:%.*]], <3 x float> noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_SMOOTHSTEP:%.*]] = tail call <3 x float>
@llvm.spv.smoothstep.v3f32(<3 x float> [[MIN]], <3 x float> [[MAX]], <3 x
float> [[X]])
// CHECK-NEXT: ret <3 x float> [[SPV_SMOOTHSTEP]]
//
@@ -33,6 +36,7 @@ float3 test_smoothstep_float3(float3 Min, float3 Max, float3
X) { return __built
// CHECK-LABEL: define spir_func <4 x float> @test_smoothstep_float4(
// CHECK-SAME: <4 x float> noundef [[MIN:%.*]], <4 x float> noundef
[[MAX:%.*]], <4 x float> noundef [[X:%.*]]) local_unnamed_addr #[[ATTR0]] {
// CHECK-NEXT: [[ENTRY:.*:]]
+// CHECK-NEXT: [[TMP0:%.*]] = tail call token
@llvm.experimental.convergence.entry()
// CHECK-NEXT: [[SPV_SMOOTHSTEP:%.*]] = tail call <4 x float>
@llvm.spv.smoothstep.v4f32(<4 x float> [[MIN]], <4 x float> [[MAX]], <4 x
float> [[X]])
// CHECK-NEXT: ret <4 x float> [[SPV_SMOOTHSTEP]]
//
diff --git a/llvm/include/llvm/IR/Intrinsics.td
b/llvm/include/llvm/IR/Intrinsics.td
index d10b07ccd91c2..ea08877466f73 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -2858,11 +2858,14 @@ def int_ptrauth_sign_generic :
//===------- Convergence Intrinsics
---------------------------------------===//
def int_experimental_convergence_entry
- : DefaultAttrsIntrinsic<[llvm_token_ty], [], [IntrNoMem, IntrConvergent]>;
+ : DefaultAttrsIntrinsic<[llvm_token_ty], [],
+ [IntrNoMem, IntrConvergent, IntrHasSideEffects]>;
def int_experimental_convergence_anchor
- : DefaultAttrsIntrinsic<[llvm_token_ty], [], [IntrNoMem, IntrConvergent]>;
+ : DefaultAttrsIntrinsic<[llvm_token_ty], [],
+ [IntrNoMem, IntrConvergent, IntrHasSideEffects]>;
def int_experimental_convergence_loop
- : DefaultAttrsIntrinsic<[llvm_token_ty], [], [IntrNoMem, IntrConvergent]>;
+ : DefaultAttrsIntrinsic<[llvm_token_ty], [],
+ [IntrNoMem, IntrConvergent, IntrHasSideEffects]>;
//===----------------------------------------------------------------------===//
// Target-specific intrinsics
diff --git a/llvm/test/Transforms/ADCE/convergence.ll
b/llvm/test/Transforms/ADCE/convergence.ll
new file mode 100644
index 0000000000000..b1d3922b2237a
--- /dev/null
+++ b/llvm/test/Transforms/ADCE/convergence.ll
@@ -0,0 +1,46 @@
+; RUN: opt %s -passes=adce -S | FileCheck %s
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define i32 @foo(i32 %a) #0 {
+define i32 @foo(i32 %a) #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.entry()
+ %0 = call token @llvm.experimental.convergence.entry()
+ ret i32 %a
+}
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define void @bar() #0 {
+define void @bar() #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.anchor()
+ %0 = call token @llvm.experimental.convergence.anchor()
+ ret void
+}
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define void @baz() #0 {
+define void @baz() #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.entry()
+ %0 = call token @llvm.experimental.convergence.entry()
+ br label %header
+
+header:
+; CHECK: %1 = call token @llvm.experimental.convergence.loop() [
"convergencectrl"(token %0) ]
+ %1 = call token @llvm.experimental.convergence.loop() [
"convergencectrl"(token %0) ]
+ br i1 true, label %body, label %exit
+
+body:
+ br label %header
+
+exit:
+ ret void
+}
+
+declare token @llvm.experimental.convergence.entry() #1
+declare token @llvm.experimental.convergence.anchor() #1
+declare token @llvm.experimental.convergence.loop() #1
+
+attributes #0 = { convergent }
+attributes #1 = { convergent nocallback nofree nosync nounwind willreturn
memory(none) }
diff --git a/llvm/test/Transforms/BDCE/convergence.ll
b/llvm/test/Transforms/BDCE/convergence.ll
new file mode 100644
index 0000000000000..c5a84a30980e7
--- /dev/null
+++ b/llvm/test/Transforms/BDCE/convergence.ll
@@ -0,0 +1,47 @@
+; RUN: opt %s -passes=bdce -S | FileCheck %s
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define i32 @foo(i32 %a) #0 {
+define i32 @foo(i32 %a) #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.entry()
+ %0 = call token @llvm.experimental.convergence.entry()
+ ret i32 %a
+}
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define void @bar() #0 {
+define void @bar() #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.anchor()
+ %0 = call token @llvm.experimental.convergence.anchor()
+ ret void
+}
+
+; CHECK: Function Attrs: convergent
+; CHECK-NEXT: define void @baz() #0 {
+define void @baz() #0 {
+entry:
+; CHECK: %0 = call token @llvm.experimental.convergence.entry()
+ %0 = call token @llvm.experimental.convergence.entry()
+ br label %header
+
+header:
+; CHECK: %1 = call token @llvm.experimental.convergence.loop() [
"convergencectrl"(token %0) ]
+ %1 = call token @llvm.experimental.convergence.loop() [
"convergencectrl"(token %0) ]
+ br i1 true, label %body, label %exit
+
+body:
+ br label %header
+
+exit:
+ ret void
+}
+
+declare token @llvm.experimental.convergence.entry() #1
+declare token @llvm.experimental.convergence.anchor() #1
+declare token @llvm.experimental.convergence.loop() #1
+
+attributes #0 = { convergent }
+attributes #1 = { convergent nocallback nofree nosync nounwind willreturn
memory(none) }
+
diff --git a/llvm/test/Transforms/DCE/op_bundles.ll
b/llvm/test/Transforms/DCE/op_bundles.ll
index 0d3b4db8265e8..6726e1f1a0f9d 100644
--- a/llvm/test/Transforms/DCE/op_bundles.ll
+++ b/llvm/test/Transforms/DCE/op_bundles.ll
@@ -4,6 +4,7 @@
define void @dead_readfirstlane_convergencetoken(<2 x i32> %src) convergent {
; CHECK-LABEL: define void @dead_readfirstlane_convergencetoken(
; CHECK-SAME: <2 x i32> [[SRC:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT: [[T:%.*]] = tail call token
@llvm.experimental.convergence.entry()
; CHECK-NEXT: ret void
;
%t = tail call token @llvm.experimental.convergence.entry()
diff --git a/llvm/test/Transforms/FunctionAttrs/convergent.ll
b/llvm/test/Transforms/FunctionAttrs/convergent.ll
index fe8029d39d924..e46981c06affe 100644
--- a/llvm/test/Transforms/FunctionAttrs/convergent.ll
+++ b/llvm/test/Transforms/FunctionAttrs/convergent.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
UTC_ARGS: --function-signature --check-attributes
-; RUN: opt -passes=function-attrs -S < %s | FileCheck %s
+; RUN: opt -passes=function-attrs -S %s | FileCheck %s
define i32 @nonleaf() convergent {
; CHECK: Function Attrs: mustprogress nofree norecurse nosync nounwind
willreturn memory(none)
@@ -129,3 +129,17 @@ define i32 @noopt_friend() convergent {
%a = call i32 @noopt()
ret i32 0
}
+
+define i32 @leaf_convergent() convergent {
+; CHECK: Function Attrs: convergent mustprogress nofree norecurse nosync
nounwind willreturn memory(none)
+; CHECK-LABEL: define {{[^@]+}}@leaf_convergent
+; CHECK-SAME: () #[[ATTR7:[0-9]+]] {
+; CHECK-NEXT: %1 = call token @llvm.experimental.convergence.entry()
+; CHECK-NEXT: ret i32 0
+ %1 = call token @llvm.experimental.convergence.entry()
+ ret i32 0
+}
+
+declare token @llvm.experimental.convergence.entry() #1
+
+attributes #1 = { convergent nocallback nofree nosync nounwind willreturn
memory(none) }
diff --git a/llvm/test/Transforms/IRNormalizer/regression-convergence-tokens.ll
b/llvm/test/Transforms/IRNormalizer/regression-convergence-tokens.ll
index 88eff971b9576..559dca38cbb0f 100644
--- a/llvm/test/Transforms/IRNormalizer/regression-convergence-tokens.ll
+++ b/llvm/test/Transforms/IRNormalizer/regression-convergence-tokens.ll
@@ -6,10 +6,10 @@ define i32 @nested(i32 %src) #0 {
; CHECK-LABEL: define i32 @nested(
; CHECK-SAME: i32 [[A0:%.*]]) #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: [[BB15160:.*:]]
-; CHECK-NEXT: [[T1:%.*]] = call token @llvm.experimental.convergence.entry()
-; CHECK-NEXT: %"vl77672llvm.experimental.convergence.anchor()" = call token
@llvm.experimental.convergence.anchor()
-; CHECK-NEXT: %"op68297(vl77672)" = call i32
@llvm.amdgcn.readfirstlane.i32(i32 [[A0]]) [ "convergencectrl"(token
%"vl77672llvm.experimental.convergence.anchor()") ]
-; CHECK-NEXT: ret i32 %"op68297(vl77672)"
+; CHECK-NEXT: %"op91455llvm.experimental.convergence.entry()" = call token
@llvm.experimental.convergence.entry()
+; CHECK-NEXT: %"vl15160llvm.experimental.convergence.anchor()" = call token
@llvm.experimental.convergence.anchor()
+; CHECK-NEXT: %"op68297(vl15160)" = call i32
@llvm.amdgcn.readfirstlane.i32(i32 [[A0]]) [ "convergencectrl"(token
%"vl15160llvm.experimental.convergence.anchor()") ]
+; CHECK-NEXT: ret i32 %"op68297(vl15160)"
;
%t1 = call token @llvm.experimental.convergence.entry()
%t2 = call token @llvm.experimental.convergence.anchor()
diff --git a/llvm/test/Transforms/LoopUnroll/convergent.controlled.ll
b/llvm/test/Transforms/LoopUnroll/convergent.controlled.ll
index 7fd4eb18f16eb..89a0867c92a53 100644
--- a/llvm/test/Transforms/LoopUnroll/convergent.controlled.ll
+++ b/llvm/test/Transforms/LoopUnroll/convergent.controlled.ll
@@ -431,6 +431,7 @@ define i32 @extended_inner_loop_2(i32 %n, i1 %cond) {
; CHECK-NEXT: entry:
; CHECK-NEXT: br label [[L3:%.*]]
; CHECK: l3:
+; CHECK-NEXT: [[TOK_LOOP:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2:
; CHECK-NEXT: [[TOK_L2:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -438,6 +439,7 @@ define i32 @extended_inner_loop_2(i32 %n, i1 %cond) {
; CHECK-NEXT: br i1 [[COND:%.*]], label [[L2]], label [[LATCH:%.*]],
!llvm.loop [[LOOP4]]
; CHECK: latch:
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2]]) ]
+; CHECK-NEXT: [[TOK_LOOP_1:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_1:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.1:
; CHECK-NEXT: [[TOK_L2_1:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -445,6 +447,7 @@ define i32 @extended_inner_loop_2(i32 %n, i1 %cond) {
; CHECK-NEXT: br i1 [[COND]], label [[L2_1]], label [[LATCH_1:%.*]],
!llvm.loop [[LOOP4]]
; CHECK: latch.1:
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2_1]]) ]
+; CHECK-NEXT: [[TOK_LOOP_2:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_2:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.2:
; CHECK-NEXT: [[TOK_L2_2:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -452,6 +455,7 @@ define i32 @extended_inner_loop_2(i32 %n, i1 %cond) {
; CHECK-NEXT: br i1 [[COND]], label [[L2_2]], label [[LATCH_2:%.*]],
!llvm.loop [[LOOP4]]
; CHECK: latch.2:
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2_2]]) ]
+; CHECK-NEXT: [[TOK_LOOP_3:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_3:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.3:
; CHECK-NEXT: [[TOK_L2_3:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -491,6 +495,7 @@ define i32 @unroll_nest(i32 %n, i1 %cond) {
; CHECK-NEXT: entry:
; CHECK-NEXT: br label [[L3:%.*]]
; CHECK: l3:
+; CHECK-NEXT: [[TOK_LOOP:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2:
; CHECK-NEXT: [[TOK_L2:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -501,6 +506,7 @@ define i32 @unroll_nest(i32 %n, i1 %cond) {
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2_1]]) ]
; CHECK-NEXT: br i1 [[COND]], label [[L2]], label [[LATCH]], !llvm.loop
[[LOOP9:![0-9]+]]
; CHECK: latch:
+; CHECK-NEXT: [[TOK_LOOP_1:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_12:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.12:
; CHECK-NEXT: [[TOK_L2_11:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -511,6 +517,7 @@ define i32 @unroll_nest(i32 %n, i1 %cond) {
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2_1_1]]) ]
; CHECK-NEXT: br i1 [[COND]], label [[L2_12]], label [[LATCH_1]],
!llvm.loop [[LOOP9]]
; CHECK: latch.1:
+; CHECK-NEXT: [[TOK_LOOP_2:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_2:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.2:
; CHECK-NEXT: [[TOK_L2_2:%.*]] = call token
@llvm.experimental.convergence.anchor()
@@ -521,6 +528,7 @@ define i32 @unroll_nest(i32 %n, i1 %cond) {
; CHECK-NEXT: call void @f() [ "convergencectrl"(token [[TOK_L2_1_2]]) ]
; CHECK-NEXT: br i1 [[COND]], label [[L2_2]], label [[LATCH_2]], !llvm.loop
[[LOOP9]]
; CHECK: latch.2:
+; CHECK-NEXT: [[TOK_LOOP_3:%.*]] = call token
@llvm.experimental.convergence.anchor()
; CHECK-NEXT: br label [[L2_3:%.*]], !llvm.loop [[LOOP4]]
; CHECK: l2.3:
; CHECK-NEXT: [[TOK_L2_3:%.*]] = call token
@llvm.experimental.convergence.anchor()
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
