[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard and reorganize the test dialect (PR #89424)

2024-04-19 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89424
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard and reorganize the test dialect (PR #89424)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 


@@ -228,6 +332,7 @@ void TestDialect::initialize() {
   >();
   registerOpsSyntax();
   addOperations();
+  registerTestDialectOperations(this);

Mogball wrote:

As you noted, this isn't needed. I think this got lost in the merge resolution. 
I'll delete it.

https://github.com/llvm/llvm-project/pull/89424
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball created 
https://github.com/llvm/llvm-project/pull/89628

[mlir][test] Shard the Test Dialect (NFC)

This PR uses the new op sharding mechanism in tablegen to shard the test
dialect's op definitions. This breaks the definition of ops into
multiple source files, speeding up compile time of the test dialect
dramatically. This improves developer cycle times when iterating on the
test dialect.

>From 498ea497b99e1328fc16cbc41ebf37575c1c4a59 Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 16:39:58 +
Subject: [PATCH] [mlir][test] Shard the Test Dialect (NFC)

This PR uses the new op sharding mechanism in tablegen to shard the test
dialect's op definitions. This breaks the definition of ops into
multiple source files, speeding up compile time of the test dialect
dramatically. This improves developer cycle times when iterating on the
test dialect.
---
 mlir/test/lib/Dialect/Test/CMakeLists.txt |  6 +++--
 mlir/test/lib/Dialect/Test/TestDialect.cpp|  5 +---
 mlir/test/lib/Dialect/Test/TestOps.cpp|  1 -
 .../mlir/test/BUILD.bazel | 25 +++
 4 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/mlir/test/lib/Dialect/Test/CMakeLists.txt 
b/mlir/test/lib/Dialect/Test/CMakeLists.txt
index f63e4d330e6ac1..fab89378093326 100644
--- a/mlir/test/lib/Dialect/Test/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/Test/CMakeLists.txt
@@ -31,8 +31,6 @@ mlir_tablegen(TestOpEnums.cpp.inc -gen-enum-defs)
 add_public_tablegen_target(MLIRTestEnumDefIncGen)
 
 set(LLVM_TARGET_DEFINITIONS TestOps.td)
-mlir_tablegen(TestOps.h.inc -gen-op-decls)
-mlir_tablegen(TestOps.cpp.inc -gen-op-defs)
 mlir_tablegen(TestOpsDialect.h.inc -gen-dialect-decls -dialect=test)
 mlir_tablegen(TestOpsDialect.cpp.inc -gen-dialect-defs -dialect=test)
 mlir_tablegen(TestPatterns.inc -gen-rewriters)
@@ -43,6 +41,8 @@ mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls)
 mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs)
 add_public_tablegen_target(MLIRTestOpsSyntaxIncGen)
 
+add_sharded_ops(TestOps 20)
+
 # Exclude tests from libMLIR.so
 add_mlir_library(MLIRTestDialect
   TestAttributes.cpp
@@ -56,6 +56,7 @@ add_mlir_library(MLIRTestDialect
   TestTypes.cpp
   TestOpsSyntax.cpp
   TestDialectInterfaces.cpp
+  ${SHARDED_SRCS}
 
   EXCLUDE_FROM_LIBMLIR
 
@@ -66,6 +67,7 @@ add_mlir_library(MLIRTestDialect
   MLIRTestTypeDefIncGen
   MLIRTestOpsIncGen
   MLIRTestOpsSyntaxIncGen
+  MLIRTestOpsShardGen
 
   LINK_LIBS PUBLIC
   MLIRControlFlowInterfaces
diff --git a/mlir/test/lib/Dialect/Test/TestDialect.cpp 
b/mlir/test/lib/Dialect/Test/TestDialect.cpp
index 77fd7e61bd3a06..bfb9592e638288 100644
--- a/mlir/test/lib/Dialect/Test/TestDialect.cpp
+++ b/mlir/test/lib/Dialect/Test/TestDialect.cpp
@@ -326,12 +326,9 @@ struct TestOpEffectInterfaceFallback
 void TestDialect::initialize() {
   registerAttributes();
   registerTypes();
-  addOperations<
-#define GET_OP_LIST
-#include "TestOps.cpp.inc"
-  >();
   registerOpsSyntax();
   addOperations();
+  registerTestDialectOperations(this);
   registerDynamicOp(getDynamicGenericOp(this));
   registerDynamicOp(getDynamicOneOperandTwoResultsOp(this));
   registerDynamicOp(getDynamicCustomParserPrinterOp(this));
diff --git a/mlir/test/lib/Dialect/Test/TestOps.cpp 
b/mlir/test/lib/Dialect/Test/TestOps.cpp
index ce7e476be74e65..47d5b1b19121ef 100644
--- a/mlir/test/lib/Dialect/Test/TestOps.cpp
+++ b/mlir/test/lib/Dialect/Test/TestOps.cpp
@@ -14,5 +14,4 @@
 using namespace mlir;
 using namespace test;
 
-#define GET_OP_CLASSES
 #include "TestOps.cpp.inc"
diff --git a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel 
b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
index dc5f4047c286db..b98f7eb5613af4 100644
--- a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
+++ b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
@@ -4,7 +4,7 @@
 
 load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
 load("//llvm:lit_test.bzl", "package_path")
-load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library")
+load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library", 
"gentbl_sharded_ops", "td_library")
 
 package(
 default_visibility = ["//visibility:public"],
@@ -151,14 +151,6 @@ gentbl_cc_library(
 name = "TestOpsIncGen",
 strip_include_prefix = "lib/Dialect/Test",
 tbl_outs = [
-(
-["-gen-op-decls"],
-"lib/Dialect/Test/TestOps.h.inc",
-),
-(
-["-gen-op-defs"],
-"lib/Dialect/Test/TestOps.cpp.inc",
-),
 (
 [
 "-gen-dialect-decls",
@@ -370,12 +362,25 @@ cc_library(
 ],
 )
 
+gentbl_sharded_ops(
+name = "TestDialectOpSrcs",
+hdr_out = "lib/Dialect/Test/TestOps.h.inc",
+shard_count = 20,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "lib/Dialect/Test/TestOps.cpp",
+src_out = "lib/Dialect/Test/TestOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball updated 
https://github.com/llvm/llvm-project/pull/89628

>From 478b4e8e9523ab80e65f115274b2f2ae4423faf4 Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 16:39:58 +
Subject: [PATCH] [mlir][test] Shard the Test Dialect (NFC)

This PR uses the new op sharding mechanism in tablegen to shard the test
dialect's op definitions. This breaks the definition of ops into
multiple source files, speeding up compile time of the test dialect
dramatically. This improves developer cycle times when iterating on the
test dialect.

stack-info: PR: https://github.com/llvm/llvm-project/pull/89628, branch: 
users/Mogball/stack/1
---
 mlir/test/lib/Dialect/Test/CMakeLists.txt |  6 +++--
 mlir/test/lib/Dialect/Test/TestDialect.cpp|  5 +---
 mlir/test/lib/Dialect/Test/TestOps.cpp|  1 -
 .../mlir/test/BUILD.bazel | 25 +++
 4 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/mlir/test/lib/Dialect/Test/CMakeLists.txt 
b/mlir/test/lib/Dialect/Test/CMakeLists.txt
index f63e4d330e6ac1..fab89378093326 100644
--- a/mlir/test/lib/Dialect/Test/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/Test/CMakeLists.txt
@@ -31,8 +31,6 @@ mlir_tablegen(TestOpEnums.cpp.inc -gen-enum-defs)
 add_public_tablegen_target(MLIRTestEnumDefIncGen)
 
 set(LLVM_TARGET_DEFINITIONS TestOps.td)
-mlir_tablegen(TestOps.h.inc -gen-op-decls)
-mlir_tablegen(TestOps.cpp.inc -gen-op-defs)
 mlir_tablegen(TestOpsDialect.h.inc -gen-dialect-decls -dialect=test)
 mlir_tablegen(TestOpsDialect.cpp.inc -gen-dialect-defs -dialect=test)
 mlir_tablegen(TestPatterns.inc -gen-rewriters)
@@ -43,6 +41,8 @@ mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls)
 mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs)
 add_public_tablegen_target(MLIRTestOpsSyntaxIncGen)
 
+add_sharded_ops(TestOps 20)
+
 # Exclude tests from libMLIR.so
 add_mlir_library(MLIRTestDialect
   TestAttributes.cpp
@@ -56,6 +56,7 @@ add_mlir_library(MLIRTestDialect
   TestTypes.cpp
   TestOpsSyntax.cpp
   TestDialectInterfaces.cpp
+  ${SHARDED_SRCS}
 
   EXCLUDE_FROM_LIBMLIR
 
@@ -66,6 +67,7 @@ add_mlir_library(MLIRTestDialect
   MLIRTestTypeDefIncGen
   MLIRTestOpsIncGen
   MLIRTestOpsSyntaxIncGen
+  MLIRTestOpsShardGen
 
   LINK_LIBS PUBLIC
   MLIRControlFlowInterfaces
diff --git a/mlir/test/lib/Dialect/Test/TestDialect.cpp 
b/mlir/test/lib/Dialect/Test/TestDialect.cpp
index 77fd7e61bd3a06..bfb9592e638288 100644
--- a/mlir/test/lib/Dialect/Test/TestDialect.cpp
+++ b/mlir/test/lib/Dialect/Test/TestDialect.cpp
@@ -326,12 +326,9 @@ struct TestOpEffectInterfaceFallback
 void TestDialect::initialize() {
   registerAttributes();
   registerTypes();
-  addOperations<
-#define GET_OP_LIST
-#include "TestOps.cpp.inc"
-  >();
   registerOpsSyntax();
   addOperations();
+  registerTestDialectOperations(this);
   registerDynamicOp(getDynamicGenericOp(this));
   registerDynamicOp(getDynamicOneOperandTwoResultsOp(this));
   registerDynamicOp(getDynamicCustomParserPrinterOp(this));
diff --git a/mlir/test/lib/Dialect/Test/TestOps.cpp 
b/mlir/test/lib/Dialect/Test/TestOps.cpp
index ce7e476be74e65..47d5b1b19121ef 100644
--- a/mlir/test/lib/Dialect/Test/TestOps.cpp
+++ b/mlir/test/lib/Dialect/Test/TestOps.cpp
@@ -14,5 +14,4 @@
 using namespace mlir;
 using namespace test;
 
-#define GET_OP_CLASSES
 #include "TestOps.cpp.inc"
diff --git a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel 
b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
index dc5f4047c286db..b98f7eb5613af4 100644
--- a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
+++ b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
@@ -4,7 +4,7 @@
 
 load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
 load("//llvm:lit_test.bzl", "package_path")
-load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library")
+load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library", 
"gentbl_sharded_ops", "td_library")
 
 package(
 default_visibility = ["//visibility:public"],
@@ -151,14 +151,6 @@ gentbl_cc_library(
 name = "TestOpsIncGen",
 strip_include_prefix = "lib/Dialect/Test",
 tbl_outs = [
-(
-["-gen-op-decls"],
-"lib/Dialect/Test/TestOps.h.inc",
-),
-(
-["-gen-op-defs"],
-"lib/Dialect/Test/TestOps.cpp.inc",
-),
 (
 [
 "-gen-dialect-decls",
@@ -370,12 +362,25 @@ cc_library(
 ],
 )
 
+gentbl_sharded_ops(
+name = "TestDialectOpSrcs",
+hdr_out = "lib/Dialect/Test/TestOps.h.inc",
+shard_count = 20,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "lib/Dialect/Test/TestOps.cpp",
+src_out = "lib/Dialect/Test/TestOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+td_file = "lib/Dialect/Test/TestOps.td",
+test = True,
+deps = [":TestOpTdFiles"],
+)
+
 cc_library(
 name = "TestDialect",
 srcs = glob(
 ["lib/Dialect/Test/*.cpp"],
 exclude = ["lib/Dialect/Test/TestTo

[llvm-branch-commits] [mlir] [mlir][test] Reorganize the test dialect (PR #89424)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89424
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [mlir] [mlir][test] Reorganize the test dialect (PR #89424)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89424
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball updated 
https://github.com/llvm/llvm-project/pull/89628

>From 478b4e8e9523ab80e65f115274b2f2ae4423faf4 Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 16:39:58 +
Subject: [PATCH] [mlir][test] Shard the Test Dialect (NFC)

This PR uses the new op sharding mechanism in tablegen to shard the test
dialect's op definitions. This breaks the definition of ops into
multiple source files, speeding up compile time of the test dialect
dramatically. This improves developer cycle times when iterating on the
test dialect.

stack-info: PR: https://github.com/llvm/llvm-project/pull/89628, branch: 
users/Mogball/stack/1
---
 mlir/test/lib/Dialect/Test/CMakeLists.txt |  6 +++--
 mlir/test/lib/Dialect/Test/TestDialect.cpp|  5 +---
 mlir/test/lib/Dialect/Test/TestOps.cpp|  1 -
 .../mlir/test/BUILD.bazel | 25 +++
 4 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/mlir/test/lib/Dialect/Test/CMakeLists.txt 
b/mlir/test/lib/Dialect/Test/CMakeLists.txt
index f63e4d330e6ac1..fab89378093326 100644
--- a/mlir/test/lib/Dialect/Test/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/Test/CMakeLists.txt
@@ -31,8 +31,6 @@ mlir_tablegen(TestOpEnums.cpp.inc -gen-enum-defs)
 add_public_tablegen_target(MLIRTestEnumDefIncGen)
 
 set(LLVM_TARGET_DEFINITIONS TestOps.td)
-mlir_tablegen(TestOps.h.inc -gen-op-decls)
-mlir_tablegen(TestOps.cpp.inc -gen-op-defs)
 mlir_tablegen(TestOpsDialect.h.inc -gen-dialect-decls -dialect=test)
 mlir_tablegen(TestOpsDialect.cpp.inc -gen-dialect-defs -dialect=test)
 mlir_tablegen(TestPatterns.inc -gen-rewriters)
@@ -43,6 +41,8 @@ mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls)
 mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs)
 add_public_tablegen_target(MLIRTestOpsSyntaxIncGen)
 
+add_sharded_ops(TestOps 20)
+
 # Exclude tests from libMLIR.so
 add_mlir_library(MLIRTestDialect
   TestAttributes.cpp
@@ -56,6 +56,7 @@ add_mlir_library(MLIRTestDialect
   TestTypes.cpp
   TestOpsSyntax.cpp
   TestDialectInterfaces.cpp
+  ${SHARDED_SRCS}
 
   EXCLUDE_FROM_LIBMLIR
 
@@ -66,6 +67,7 @@ add_mlir_library(MLIRTestDialect
   MLIRTestTypeDefIncGen
   MLIRTestOpsIncGen
   MLIRTestOpsSyntaxIncGen
+  MLIRTestOpsShardGen
 
   LINK_LIBS PUBLIC
   MLIRControlFlowInterfaces
diff --git a/mlir/test/lib/Dialect/Test/TestDialect.cpp 
b/mlir/test/lib/Dialect/Test/TestDialect.cpp
index 77fd7e61bd3a06..bfb9592e638288 100644
--- a/mlir/test/lib/Dialect/Test/TestDialect.cpp
+++ b/mlir/test/lib/Dialect/Test/TestDialect.cpp
@@ -326,12 +326,9 @@ struct TestOpEffectInterfaceFallback
 void TestDialect::initialize() {
   registerAttributes();
   registerTypes();
-  addOperations<
-#define GET_OP_LIST
-#include "TestOps.cpp.inc"
-  >();
   registerOpsSyntax();
   addOperations();
+  registerTestDialectOperations(this);
   registerDynamicOp(getDynamicGenericOp(this));
   registerDynamicOp(getDynamicOneOperandTwoResultsOp(this));
   registerDynamicOp(getDynamicCustomParserPrinterOp(this));
diff --git a/mlir/test/lib/Dialect/Test/TestOps.cpp 
b/mlir/test/lib/Dialect/Test/TestOps.cpp
index ce7e476be74e65..47d5b1b19121ef 100644
--- a/mlir/test/lib/Dialect/Test/TestOps.cpp
+++ b/mlir/test/lib/Dialect/Test/TestOps.cpp
@@ -14,5 +14,4 @@
 using namespace mlir;
 using namespace test;
 
-#define GET_OP_CLASSES
 #include "TestOps.cpp.inc"
diff --git a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel 
b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
index dc5f4047c286db..b98f7eb5613af4 100644
--- a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
+++ b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
@@ -4,7 +4,7 @@
 
 load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
 load("//llvm:lit_test.bzl", "package_path")
-load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library")
+load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library", 
"gentbl_sharded_ops", "td_library")
 
 package(
 default_visibility = ["//visibility:public"],
@@ -151,14 +151,6 @@ gentbl_cc_library(
 name = "TestOpsIncGen",
 strip_include_prefix = "lib/Dialect/Test",
 tbl_outs = [
-(
-["-gen-op-decls"],
-"lib/Dialect/Test/TestOps.h.inc",
-),
-(
-["-gen-op-defs"],
-"lib/Dialect/Test/TestOps.cpp.inc",
-),
 (
 [
 "-gen-dialect-decls",
@@ -370,12 +362,25 @@ cc_library(
 ],
 )
 
+gentbl_sharded_ops(
+name = "TestDialectOpSrcs",
+hdr_out = "lib/Dialect/Test/TestOps.h.inc",
+shard_count = 20,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "lib/Dialect/Test/TestOps.cpp",
+src_out = "lib/Dialect/Test/TestOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+td_file = "lib/Dialect/Test/TestOps.td",
+test = True,
+deps = [":TestOpTdFiles"],
+)
+
 cc_library(
 name = "TestDialect",
 srcs = glob(
 ["lib/Dialect/Test/*.cpp"],
 exclude = ["lib/Dialect/Test/TestTo

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball updated 
https://github.com/llvm/llvm-project/pull/89628

>From 478b4e8e9523ab80e65f115274b2f2ae4423faf4 Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 16:39:58 +
Subject: [PATCH] [mlir][test] Shard the Test Dialect (NFC)

This PR uses the new op sharding mechanism in tablegen to shard the test
dialect's op definitions. This breaks the definition of ops into
multiple source files, speeding up compile time of the test dialect
dramatically. This improves developer cycle times when iterating on the
test dialect.

stack-info: PR: https://github.com/llvm/llvm-project/pull/89628, branch: 
users/Mogball/stack/1
---
 mlir/test/lib/Dialect/Test/CMakeLists.txt |  6 +++--
 mlir/test/lib/Dialect/Test/TestDialect.cpp|  5 +---
 mlir/test/lib/Dialect/Test/TestOps.cpp|  1 -
 .../mlir/test/BUILD.bazel | 25 +++
 4 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/mlir/test/lib/Dialect/Test/CMakeLists.txt 
b/mlir/test/lib/Dialect/Test/CMakeLists.txt
index f63e4d330e6ac1..fab89378093326 100644
--- a/mlir/test/lib/Dialect/Test/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/Test/CMakeLists.txt
@@ -31,8 +31,6 @@ mlir_tablegen(TestOpEnums.cpp.inc -gen-enum-defs)
 add_public_tablegen_target(MLIRTestEnumDefIncGen)
 
 set(LLVM_TARGET_DEFINITIONS TestOps.td)
-mlir_tablegen(TestOps.h.inc -gen-op-decls)
-mlir_tablegen(TestOps.cpp.inc -gen-op-defs)
 mlir_tablegen(TestOpsDialect.h.inc -gen-dialect-decls -dialect=test)
 mlir_tablegen(TestOpsDialect.cpp.inc -gen-dialect-defs -dialect=test)
 mlir_tablegen(TestPatterns.inc -gen-rewriters)
@@ -43,6 +41,8 @@ mlir_tablegen(TestOpsSyntax.h.inc -gen-op-decls)
 mlir_tablegen(TestOpsSyntax.cpp.inc -gen-op-defs)
 add_public_tablegen_target(MLIRTestOpsSyntaxIncGen)
 
+add_sharded_ops(TestOps 20)
+
 # Exclude tests from libMLIR.so
 add_mlir_library(MLIRTestDialect
   TestAttributes.cpp
@@ -56,6 +56,7 @@ add_mlir_library(MLIRTestDialect
   TestTypes.cpp
   TestOpsSyntax.cpp
   TestDialectInterfaces.cpp
+  ${SHARDED_SRCS}
 
   EXCLUDE_FROM_LIBMLIR
 
@@ -66,6 +67,7 @@ add_mlir_library(MLIRTestDialect
   MLIRTestTypeDefIncGen
   MLIRTestOpsIncGen
   MLIRTestOpsSyntaxIncGen
+  MLIRTestOpsShardGen
 
   LINK_LIBS PUBLIC
   MLIRControlFlowInterfaces
diff --git a/mlir/test/lib/Dialect/Test/TestDialect.cpp 
b/mlir/test/lib/Dialect/Test/TestDialect.cpp
index 77fd7e61bd3a06..bfb9592e638288 100644
--- a/mlir/test/lib/Dialect/Test/TestDialect.cpp
+++ b/mlir/test/lib/Dialect/Test/TestDialect.cpp
@@ -326,12 +326,9 @@ struct TestOpEffectInterfaceFallback
 void TestDialect::initialize() {
   registerAttributes();
   registerTypes();
-  addOperations<
-#define GET_OP_LIST
-#include "TestOps.cpp.inc"
-  >();
   registerOpsSyntax();
   addOperations();
+  registerTestDialectOperations(this);
   registerDynamicOp(getDynamicGenericOp(this));
   registerDynamicOp(getDynamicOneOperandTwoResultsOp(this));
   registerDynamicOp(getDynamicCustomParserPrinterOp(this));
diff --git a/mlir/test/lib/Dialect/Test/TestOps.cpp 
b/mlir/test/lib/Dialect/Test/TestOps.cpp
index ce7e476be74e65..47d5b1b19121ef 100644
--- a/mlir/test/lib/Dialect/Test/TestOps.cpp
+++ b/mlir/test/lib/Dialect/Test/TestOps.cpp
@@ -14,5 +14,4 @@
 using namespace mlir;
 using namespace test;
 
-#define GET_OP_CLASSES
 #include "TestOps.cpp.inc"
diff --git a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel 
b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
index dc5f4047c286db..b98f7eb5613af4 100644
--- a/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
+++ b/utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel
@@ -4,7 +4,7 @@
 
 load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
 load("//llvm:lit_test.bzl", "package_path")
-load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library")
+load("//mlir:tblgen.bzl", "gentbl_cc_library", "td_library", 
"gentbl_sharded_ops", "td_library")
 
 package(
 default_visibility = ["//visibility:public"],
@@ -151,14 +151,6 @@ gentbl_cc_library(
 name = "TestOpsIncGen",
 strip_include_prefix = "lib/Dialect/Test",
 tbl_outs = [
-(
-["-gen-op-decls"],
-"lib/Dialect/Test/TestOps.h.inc",
-),
-(
-["-gen-op-defs"],
-"lib/Dialect/Test/TestOps.cpp.inc",
-),
 (
 [
 "-gen-dialect-decls",
@@ -370,12 +362,25 @@ cc_library(
 ],
 )
 
+gentbl_sharded_ops(
+name = "TestDialectOpSrcs",
+hdr_out = "lib/Dialect/Test/TestOps.h.inc",
+shard_count = 20,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "lib/Dialect/Test/TestOps.cpp",
+src_out = "lib/Dialect/Test/TestOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+td_file = "lib/Dialect/Test/TestOps.td",
+test = True,
+deps = [":TestOpTdFiles"],
+)
+
 cc_library(
 name = "TestDialect",
 srcs = glob(
 ["lib/Dialect/Test/*.cpp"],
 exclude = ["lib/Dialect/Test/TestTo

[llvm-branch-commits] [mlir] [mlir][test] Reorganize the test dialect (PR #89424)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

Mogball wrote:

> LG, but please make sure to remove all spurious headers from the commit 
> description (seems like it should start at "This PR massively ...")

Yep. The tool removes it before landing the commit

https://github.com/llvm/llvm-project/pull/89424
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [mlir] [mlir][ods] Add documentation on how to use sharded op definitions (NFC) (PR #89664)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball created 
https://github.com/llvm/llvm-project/pull/89664

[mlir][ods] Add documentation on how to use sharded op definitions (NFC)

This adds explanations and instructions on how to set up a dialect for
sharded op definitions to the MLIR documentation.

>From 8991d4f35210cdf5f08a57552058b5606c2f1e57 Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 20:40:10 +
Subject: [PATCH] [mlir][ods] Add documentation on how to use sharded op
 definitions (NFC)

This adds explanations and instructions on how to set up a dialect for
sharded op definitions to the MLIR documentation.
---
 mlir/docs/DefiningDialects/Operations.md | 94 
 1 file changed, 94 insertions(+)

diff --git a/mlir/docs/DefiningDialects/Operations.md 
b/mlir/docs/DefiningDialects/Operations.md
index 729393d5362673..79a0cc55f13840 100644
--- a/mlir/docs/DefiningDialects/Operations.md
+++ b/mlir/docs/DefiningDialects/Operations.md
@@ -1114,6 +1114,100 @@ void process(AddOp op, ArrayRef newOperands) {
 }
 ```
 
+ Sharded Operation Definitions
+
+Large dialects with many operations may struggle with C++ compile time of
+generated op definitions, due to large compilation units. `mlir-tblgen`
+provides the ability to shard op definitions by splitting them up evenly
+by passing `-op-shard-count` to `-gen-op-defs` and `-gen-op-decls`. The tool
+will generate a single include file for the definitions broken up by
+`GET_OP_DEFS_${N}` where `${N}` is the shard number. A shard can be compiled in
+a single compilation unit by adding a file like this to your dialect library:
+
+```c++
+#include "mlir/IR/Operation.h"
+// Add any other required includes.
+
+// Utilities shared by generated op definitions: custom directive parsers,
+// printers, etc.
+#include "OpUtils.h"
+
+#define GET_OP_DEFS_0
+#include "MyDialectOps.cpp.inc"
+```
+
+Note: this requires restructing shared utility functions within the dialect
+library so they can be shared by multiple compilation units. I.e. instead of
+defining `static` methods in the same source file, you should declare them in a
+shared header and define them in their own source file.
+
+The op registration hooks are also sharded, because the template instantiation
+can take a very long time to compile. Operations should be registered in your
+dialect like:
+
+```c++
+void MyDialect::initialize() {
+  registerMyDialectOperations(this);
+}
+```
+
+CMake and Bazel functions are included to make sharding dialects easier.
+Assuming you have organized your operation utility functions into their own
+header, define a file that looks like the one above, but without the `#define`:
+
+```c++
+// MyDialectOps.cpp
+#include "mlir/IR/Operation.h"
+
+#include "OpUtils.h"
+
+#include "MyDialectOps.cpp.inc"
+```
+
+In CMake, remove the manual `mlir_tablegen` invocations and replace them with:
+
+```cmake
+set(LLVM_TARGET_DEFINITIONS MyDialectOps.td)
+add_sharded_ops(MyDialectOps 8) # shard the op definitions by 8
+
+add_mlir_library(MyDialect
+  MyDialect.cpp
+  MyDialectOpDefs.cpp
+  ${SHARDED_SRCS}
+
+  DEPENDS
+  MLIRTestOpsShardGen
+)
+```
+
+This will automatically duplicate the `MyDialectOps.cpp` source file and add 
the
+`#define` up the number of shards indicated.
+
+It is recommended that any out-of-line op member functions (like verifiers) be
+defined in a separate source file. In this example, it is called
+`MyDialectOpDefs.cpp`.
+
+In Bazel, remove the `-gen-op-defs` and `-gen-op-decls` invocations, and add
+
+```bazel
+gentbl_sharded_ops(
+name = "MyDialectOpSrcs",
+hdr_out = "MyDialectOps.h.inc",
+shard_count = 8,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "MyDialectOps.cpp",
+src_out = "MyDialectOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+td_file = "MyDialectOps.td",
+deps = [":MyDialectOpsTdFiles"],
+)
+
+cc_library(
+name = "MyDialect",
+srcs = glob(["MyDialect/*.cpp"]) + [":MyDialectOpSrcs"]
+)
+```
+
 ## Constraints
 
 Constraint is a core concept in table-driven operation definition: operation

___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [mlir] [mlir][ods] Add documentation on how to use sharded op definitions (NFC) (PR #89664)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball updated 
https://github.com/llvm/llvm-project/pull/89664

>From f8c449c81fb62dc063eaf9be3dc0cc81407afd1d Mon Sep 17 00:00:00 2001
From: Mogball 
Date: Mon, 22 Apr 2024 20:40:10 +
Subject: [PATCH] [mlir][ods] Add documentation on how to use sharded op
 definitions (NFC)

This adds explanations and instructions on how to set up a dialect for
sharded op definitions to the MLIR documentation.

stack-info: PR: https://github.com/llvm/llvm-project/pull/89664, branch: 
users/Mogball/stack/3
---
 mlir/docs/DefiningDialects/Operations.md | 94 
 1 file changed, 94 insertions(+)

diff --git a/mlir/docs/DefiningDialects/Operations.md 
b/mlir/docs/DefiningDialects/Operations.md
index 729393d5362673..79a0cc55f13840 100644
--- a/mlir/docs/DefiningDialects/Operations.md
+++ b/mlir/docs/DefiningDialects/Operations.md
@@ -1114,6 +1114,100 @@ void process(AddOp op, ArrayRef newOperands) {
 }
 ```
 
+ Sharded Operation Definitions
+
+Large dialects with many operations may struggle with C++ compile time of
+generated op definitions, due to large compilation units. `mlir-tblgen`
+provides the ability to shard op definitions by splitting them up evenly
+by passing `-op-shard-count` to `-gen-op-defs` and `-gen-op-decls`. The tool
+will generate a single include file for the definitions broken up by
+`GET_OP_DEFS_${N}` where `${N}` is the shard number. A shard can be compiled in
+a single compilation unit by adding a file like this to your dialect library:
+
+```c++
+#include "mlir/IR/Operation.h"
+// Add any other required includes.
+
+// Utilities shared by generated op definitions: custom directive parsers,
+// printers, etc.
+#include "OpUtils.h"
+
+#define GET_OP_DEFS_0
+#include "MyDialectOps.cpp.inc"
+```
+
+Note: this requires restructing shared utility functions within the dialect
+library so they can be shared by multiple compilation units. I.e. instead of
+defining `static` methods in the same source file, you should declare them in a
+shared header and define them in their own source file.
+
+The op registration hooks are also sharded, because the template instantiation
+can take a very long time to compile. Operations should be registered in your
+dialect like:
+
+```c++
+void MyDialect::initialize() {
+  registerMyDialectOperations(this);
+}
+```
+
+CMake and Bazel functions are included to make sharding dialects easier.
+Assuming you have organized your operation utility functions into their own
+header, define a file that looks like the one above, but without the `#define`:
+
+```c++
+// MyDialectOps.cpp
+#include "mlir/IR/Operation.h"
+
+#include "OpUtils.h"
+
+#include "MyDialectOps.cpp.inc"
+```
+
+In CMake, remove the manual `mlir_tablegen` invocations and replace them with:
+
+```cmake
+set(LLVM_TARGET_DEFINITIONS MyDialectOps.td)
+add_sharded_ops(MyDialectOps 8) # shard the op definitions by 8
+
+add_mlir_library(MyDialect
+  MyDialect.cpp
+  MyDialectOpDefs.cpp
+  ${SHARDED_SRCS}
+
+  DEPENDS
+  MLIRTestOpsShardGen
+)
+```
+
+This will automatically duplicate the `MyDialectOps.cpp` source file and add 
the
+`#define` up the number of shards indicated.
+
+It is recommended that any out-of-line op member functions (like verifiers) be
+defined in a separate source file. In this example, it is called
+`MyDialectOpDefs.cpp`.
+
+In Bazel, remove the `-gen-op-defs` and `-gen-op-decls` invocations, and add
+
+```bazel
+gentbl_sharded_ops(
+name = "MyDialectOpSrcs",
+hdr_out = "MyDialectOps.h.inc",
+shard_count = 8,
+sharder = "//mlir:mlir-src-sharder",
+src_file = "MyDialectOps.cpp",
+src_out = "MyDialectOps.cpp.inc",
+tblgen = "//mlir:mlir-tblgen",
+td_file = "MyDialectOps.td",
+deps = [":MyDialectOpsTdFiles"],
+)
+
+cc_library(
+name = "MyDialect",
+srcs = glob(["MyDialect/*.cpp"]) + [":MyDialectOpSrcs"]
+)
+```
+
 ## Constraints
 
 Constraint is a core concept in table-driven operation definition: operation

___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89628
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][ods] Allow sharding of op definitions (PR #89423)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89423
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][ods] Allow sharding of op definitions (PR #89423)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

Mogball wrote:

> Before I forget: we should add documentation for this, including the how to 
> structure the dialect to support it.

Added in https://github.com/llvm/llvm-project/pull/89664

https://github.com/llvm/llvm-project/pull/89423
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [llvm] [mlir] [mlir][test] Shard the Test Dialect (NFC) (PR #89628)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89628
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [mlir] [mlir][ods] Add documentation on how to use sharded op definitions (NFC) (PR #89664)

2024-04-22 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball edited 
https://github.com/llvm/llvm-project/pull/89664
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits

[llvm-branch-commits] [mlir] [mlir][Properties] Shorten "Property" to "Prop" in most places (PR #120368)

2024-12-18 Thread Jeff Niu via llvm-branch-commits 

https://github.com/Mogball approved this pull request.

Makes sense

https://github.com/llvm/llvm-project/pull/120368
___
llvm-branch-commits mailing list
llvm-branch-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-branch-commits