[
https://issues.apache.org/jira/browse/CASSANDRA-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952749#comment-17952749
]
Andy Tolbert edited comment on CASSANDRA-20328 at 5/20/25 12:54 AM:
--------------------------------------------------------------------
So I was looking things over and I realize now why I probably didn't migrate
these dtests:
# There was already precedence for doing this; fqltool
({{{}FqlReplayDDLExclusionTest{}}}, {{{}FqlTombstoneHandlingTest{}}}) has a
separate source path (tools/fqltool), but its dtests exist in
test/distributed/org/apache/cassandra/distributed/test/fql that use code from
fqltool. We also have tests that invoke cassandra stress, such as
{{IPMembershipTest}} and {{CassandrastressTest}} that are also in test/unit.
Although in these two former cases, they address the class names as direct
strings (e.g. \{{
ToolRunner.invokeClass("org.apache.cassandra.fqltool.FullQueryLogTool")}}),
while the BulkLoader tests reference the class directly (e.g.
{{{}ToolRunner.invokeClass(BulkLoader.class){}}}). In either case, you need the
classes on your class path to successfully run the tests, so the dependency
still exists regardless, you just won't get an error in your IDE.
# It would require separate ant tasks and test pipelines for dtests, which
might be more overhead than we want, but I'll think hard about this as I think
it's probably the right thing to do.
That said, it looks like I completely missed migrating {{BulkLoaderTest}} to
tools/sstableloader tests, i'll take care of it and also continue to vet what
else I may have missed.
I think ideally we want tools to be isolated from the main source and test. Let
me see what I can do about this in CASSANDRA-20329. I will also look into
updating {{cassandra.classpath}} to exclude tools libs, as we don't want to
pull those in (thanks for pointing that out).
[~benedict], since you mentioned you are not using {{{}generate-idea-files{}}},
I'm curious to know what else might not be working? I expect by moving
sstableloader code to the tools directory, it's possible that those new
source/test directories didn't get properly detected. Generally would be
interesting to know what worked / didn't work.
As far as IDE issues go, please let me know if you see any other problems. My
expectation is that {{ant generate-idea-files}} will properly configure an IDE,
but there may be more knock on effects for people pulling down latest trunk
that I didn't anticipate. I'd be curious to know if there are any other
problems so I can fix them up right away. I did a bunch of testing in IntelliJ,
Eclipse (for which I found some existing problems which i've attempted to
address in CASSANDRA-20497) and Netbeans.
was (Author: andrew.tolbert):
So I was looking things over and I realize now why I probably didn't migrate
these dtests:
# There was already precedence for doing this; fqltool
({{{}FqlReplayDDLExclusionTest{}}}, {{{}FqlTombstoneHandlingTest{}}}) has a
separate source path (tools/fqltool), but its dtests exist in
test/distributed/org/apache/cassandra/distributed/test/fql that use code from
fqltool. We also have tests that invoke cassandra stress, such as
{{IPMembershipTest}} and {{CassandrastressTest}} that are also in test/unit.
Although in these two former cases, they address the class names as direct
strings (e.g. \{{
ToolRunner.invokeClass("org.apache.cassandra.fqltool.FullQueryLogTool")}}),
while the BulkLoader tests reference the class directly (e.g.
{{{}ToolRunner.invokeClass(BulkLoader.class){}}}). In either case, you need the
classes on your class path to successfully run the tests, so the dependency
still exists regardless, you just won't get a compiler error.
# It would require separate ant tasks and test pipelines for dtests, which
might be more overhead than we want, but I'll think hard about this as I think
it's probably the right thing to do.
That said, it looks like I completely missed migrating {{BulkLoaderTest}} to
tools/sstableloader tests, i'll take care of it and also continue to vet what
else I may have missed.
I think ideally we want tools to be isolated from the main source and test. Let
me see what I can do about this in CASSANDRA-20329. I will also look into
updating {{cassandra.classpath}} to exclude tools libs, as we don't want to
pull those in (thanks for pointing that out).
[~benedict], since you mentioned you are not using {{{}generate-idea-files{}}},
I'm curious to know what else might not be working? I expect by moving
sstableloader code to the tools directory, it's possible that those new
source/test directories didn't get properly detected. Generally would be
interesting to know what worked / didn't work.
As far as IDE issues go, please let me know if you see any other problems. My
expectation is that {{ant generate-idea-files}} will properly configure an IDE,
but there may be more knock on effects for people pulling down latest trunk
that I didn't anticipate. I'd be curious to know if there are any other
problems so I can fix them up right away. I did a bunch of testing in IntelliJ,
Eclipse (for which I found some existing problems which i've attempted to
address in CASSANDRA-20497) and Netbeans.
> Migrate sstableloader code to its own tools directory
> -----------------------------------------------------
>
> Key: CASSANDRA-20328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20328
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Tool/sstable
> Reporter: Andy Tolbert
> Assignee: Andy Tolbert
> Priority: Normal
> Labels: pull-request-available
> Fix For: 5.x
>
> Attachments: CASSANDRA-0413-20328_ci_summary.html,
> CASSANDRA-20328-511-0_ci_summary.html, CASSANDRA-20328-515-0_ci_summary.html,
> CASSANDRA-20328-test_results.html,
> ci_summary_tolbertam_CASSANDRA-20328_238.html,
> results_details_tolbertam_CASSANDRA-20328_238.tar.xz
>
> Time Spent: 15h 10m
> Remaining Estimate: 0h
>
> Akin to fqltool and stress, we should move sstableloader to its own tools
> directory. This is needed to decouple the java driver as a dependency from
> server code ([CASSANDRA-20326]).
> For backwards compatibility reasons, we should retain {{bin/sstableloader}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
