sreejasahithi commented on code in PR #8409:
URL: https://github.com/apache/ozone/pull/8409#discussion_r2077611483
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/logs/container/utils/SQLDBConstants.java:
##########
@@ -68,6 +68,10 @@ public final class SQLDBConstants {
"WHERE d.container_id = ? ORDER BY d.datanode_id ASC, d.timestamp ASC;";
public static final String CREATE_DCL_CONTAINER_STATE_TIME_INDEX = "CREATE
INDEX IF NOT EXISTS " +
"idx_dcl_container_state_time ON DatanodeContainerLogTable(container_id,
container_state, timestamp);";
+ public static final String CREATE_CONTAINER_ID_INDEX = "CREATE INDEX IF NOT
EXISTS idx_containerlog_container_id " +
+ "ON ContainerLogTable(container_id);";
+ public static final String SELECT_DISTINCT_CONTAINER_IDS_QUERY =
+ "SELECT DISTINCT container_id FROM ContainerLogTable";
Review Comment:
Thanks for the suggestion! but this query will not work for all cases to
identify containers with duplicate OPEN state because it filters only on the
latest state being OPEN, while the current query works because it considers all
historical states and doesn't limit based on the current/latest one.
for example:
```
Timestamp | Container ID | Datanode ID | Container State |
BCSID | Message | Index Value
-------------------------------------------------------------------------------------------------------------------------------
2024-06-18 13:23:27,652 | 2002 | 100 | OPEN | 0
| No error | 0
2024-12-10 13:53:40,574 | 2002 | 100 | CLOSING |
8705627 | No error | 0
2024-12-10 13:54:42,593 | 2002 | 100 | QUASI_CLOSED |
8705627 | Ratis group removed | 0
2024-12-10 13:54:52,236 | 2002 | 100 | CLOSED |
8705627 | No error | 0
2024-12-11 02:35:43,315 | 2002 | 100 | DELETED |
8705627 | Empty container deleted | 0
2024-06-18 13:23:27,651 | 2002 | 200 | OPEN | 0
| No error | 0
2024-12-10 13:48:37,699 | 2002 | 200 | CLOSING |
8705627 | No error | 0
2024-12-10 13:48:37,703 | 2002 | 200 | CLOSED |
8705627 | No error | 0
2024-12-11 02:36:01,778 | 2002 | 200 | DELETED |
8705627 | Empty container deleted | 0
2024-07-22 11:21:28,184 | 2002 | 300 | OPEN | 0
| No error | 0
2024-08-29 14:11:30,262 | 2002 | 300 | CLOSING |
77 | No error | 0
2024-08-29 14:11:30,267 | 2002 | 300 | CLOSED |
77 | No error | 0
2024-07-22 11:21:28,175 | 2002 | 250 | OPEN | 0
| No error | 0
2024-08-29 14:11:17,284 | 2002 | 250 | CLOSING |
77 | No error | 0
2024-08-29 14:11:17,603 | 2002 | 250 | CLOSED |
77 | No error | 0
2024-06-18 13:23:27,655 | 2002 | 350 | OPEN | 0
| No error | 0
2024-12-10 13:47:37,394 | 2002 | 350 | CLOSING |
8705627 | No error | 0
2024-12-10 13:48:37,912 | 2002 | 350 | CLOSED |
8705627 | No error | 0
2024-12-11 02:35:51,425 | 2002 | 350 | DELETED |
8705627 | Empty container deleted | 0
```
##########
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/logs/container/utils/ContainerDatanodeDatabase.java:
##########
@@ -541,5 +541,44 @@ private List<DatanodeContainerInfo>
getContainerLogData(Long containerID, Connec
return logEntries;
}
+
+ private void createIdxContainerlogContainerId(Connection conn) throws
SQLException {
+ String sql = SQLDBConstants.CREATE_CONTAINER_ID_INDEX;
+ try (Statement stmt = conn.createStatement()) {
+ stmt.execute(sql);
+ }
+ }
+
+ public void findDuplicateOpenContainer() throws SQLException {
+ String sql = SQLDBConstants.SELECT_DISTINCT_CONTAINER_IDS_QUERY;
+
+ try (Connection connection = getConnection()) {
+
+ createIdxContainerlogContainerId(connection);
+
+ try (PreparedStatement statement = connection.prepareStatement(sql);
+ ResultSet resultSet = statement.executeQuery()) {
+ int count = 0;
+
+ while (resultSet.next()) {
+ Long containerID = resultSet.getLong("container_id");
+ List<DatanodeContainerInfo> logEntries =
getContainerLogData(containerID, connection);
+
logEntries.sort(Comparator.comparing(DatanodeContainerInfo::getTimestamp));
+ boolean hasIssue = checkForMultipleOpenStates(logEntries);
+ if (hasIssue) {
+ count++;
+ out.println("Container ID: " + containerID);
Review Comment:
Thanks for the suggestion! The current logic to check duplicate OPEN states
is already implemented in `checkForMultipleOpenStates()`, which is also used by
another command (ozone debug log container info). In order to get the count as
well when we do the check, we would need to implement another method, but the
logic would be mostly redundant to the existing one.
So if users want more details (like how duplicate OPEN), they can use `ozone
debug log container --db=<path to db> info <container id>`. That gives the
complete state transition history of a particular container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
