This is an automated email from the ASF dual-hosted git repository.
liyang pushed a commit to branch doc5.0
in repository https://gitbox.apache.org/repos/asf/kylin.git
The following commit(s) were added to refs/heads/doc5.0 by this push:
new 32d6c97e5a Add metadata migration, Remove maintenance mode
32d6c97e5a is described below
commit 32d6c97e5aa5f2d669d58734fcc7ac82d128c0d5
Author: Pengfei Zhan <[email protected]>
AuthorDate: Fri Sep 27 19:55:27 2024 +0800
Add metadata migration, Remove maintenance mode
---
.../operations/access-control/user_management.md | 4 +-
...nt_dependency_check.md => environment_check.md} | 8 +--
...ata_backup_restore.md => metadata_operation.md} | 30 ++++++--
.../cli_tool/metadata_tool/intro.md | 18 -----
...den_modify_metadata_during_maintenance_mode.png | Bin 72307 -> 0 bytes
..._metadata_promoting_during_maintenance_mode.png | Bin 57544 -> 0 bytes
.../system-operation/images/maintenance_mode.png | Bin 91393 -> 0 bytes
.../system_recover_from_maintenance_mode.png | Bin 111166 -> 0 bytes
.../operations/system-operation/junk_file_clean.md | 1 -
.../system-operation/maintenance_mode.md | 76 ---------------------
website/docs/quickstart/intro.md | 2 +-
website/sidebars.ts | 20 +-----
12 files changed, 34 insertions(+), 125 deletions(-)
diff --git a/website/docs/operations/access-control/user_management.md
b/website/docs/operations/access-control/user_management.md
index e494bbd46d..3f30c7eb23 100644
--- a/website/docs/operations/access-control/user_management.md
+++ b/website/docs/operations/access-control/user_management.md
@@ -67,12 +67,10 @@ $KYLIN_HOME/bin/admin-tool.sh admin-password-reset
When the parameter `kylin.metadata.random-admin-password.enabled=false`, it
will not regenerate a random password but the fixed password `KYLIN`. If the
parameter `kylin.metadata.random-admin-password.enabled` is set from `false` to
`true` , it will regenerate a random password and display it on the console
after all the Kylin nodes restarted.
-**Caution** When run this command, Kylin will enter maintenance mode. If the
command is interrupted by force, you may need to exit maintain mode manually.
Refer to [Maintenance Mode](../system-operation/maintenance_mode.md).
-
### <span id="pwd">Reset password for Non-admin</span>
-Click `username` --> `Setup` on the top right corner of the navigation bar.
In the pop-up window, user need to provide the old password and repeat the new
password twice to reset password.
+Click `username` --> `Setup` in the top right corner of the navigation bar.
In the pop-up window, user need to provide the old password and repeat the new
password twice to reset password.
### <span id="group">Assign a User to a Group</span>
diff --git
a/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
b/website/docs/operations/system-operation/cli_tool/environment_check.md
similarity index 95%
rename from
website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
rename to website/docs/operations/system-operation/cli_tool/environment_check.md
index 5d204c979e..26cdd386d1 100644
---
a/website/docs/operations/system-operation/cli_tool/environment_dependency_check.md
+++ b/website/docs/operations/system-operation/cli_tool/environment_check.md
@@ -1,14 +1,14 @@
---
-title: Environment Dependency Check
+title: Environment Check
language: en
-sidebar_label: Environment Dependency Check
-pagination_label: Environment Dependency Check
+sidebar_label: Environment Check
+pagination_label: Environment Check
toc_min_heading_level: 2
toc_max_heading_level: 6
pagination_prev: null
pagination_next: null
keywords:
- - environment dependency check
+ - environment check
draft: false
last_update:
date: 08/16/2022
diff --git
a/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
b/website/docs/operations/system-operation/cli_tool/metadata_operation.md
similarity index 73%
rename from
website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
rename to
website/docs/operations/system-operation/cli_tool/metadata_operation.md
index 50a40642b4..c896202847 100644
---
a/website/docs/operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore.md
+++ b/website/docs/operations/system-operation/cli_tool/metadata_operation.md
@@ -1,14 +1,14 @@
---
-title: System Metadata Backup and Restore
+title: Metadata Operation
language: en
-sidebar_label: System Metadata Backup and Restore
-pagination_label: System Metadata Backup and Restore
+sidebar_label: Metadata Operation
+pagination_label: Metadata Operation
toc_min_heading_level: 2
toc_max_heading_level: 6
pagination_prev: null
pagination_next: null
keywords:
- - system metadata backup and restore
+ - metadata operation
draft: false
last_update:
date: 08/16/2022
@@ -85,4 +85,24 @@ Metadata recovery is required in Kylin with the **command
line**.
- `METADATA_BACKUP_PATH` - required, represents the metadata path that are
going to be recovered, the default value is `${KYLIN_HOME}/meta_backups/`
- `--after-truncate` - optional, if this parameter is added, the project
metadata will be completely restored, otherwise only the deleted and modified
metadata will be restored, and the new metadata will still be retained.
-**Caution** When run this command, Kylin will enter maintenance mode. If the
command is interrupted by force, you may need to exit maintain mode manually.
Refer [Maintenance Mode](../../maintenance_mode.md).
+
+## Metadata Migration {#metadata_migration}
+
+Since Kylin 5.0-alpha and Kylin 5.0.0 underwent a metadata refactoring, you
will need to use this tool to perform a metadata migration on versions prior to
5.0.0. The steps for migration are as follows:
+
++ Backup the metadata
+ ```shell
+ $KYLIN_HOME/bin/metastore.sh backup METADATA_BACKUP_PATH
+ ```
++ Perform metadata conversion
+ ```shell
+ $KYLIN_HOME/bin/kylin.sh
org.apache.kylin.common.persistence.metadata.MigrateKEMetadataTool {inputPath}
{outputPath}
+ ```
++ Restore the metadata
+ ```shell
+ $KYLIN_HOME/bin/metastore.sh restore METADATA_RESTORE_PATH
+ ```
+:::tip Tips
++ Configure the new KE and ensure it starts normally. Ensure that at least the
following parameters are configured: `metadata_url`, `zookeeper`, other
required parameters. If the metadata were stored in MySQL, provide the
MySQL-related JAR package.
++ During metadata migration and import, a large number of intermediate values
may be stored in memory. If you encounter OutOfMemory (OOM) issues, adjust the
memory parameters and try again.
+:::
diff --git
a/website/docs/operations/system-operation/cli_tool/metadata_tool/intro.md
b/website/docs/operations/system-operation/cli_tool/metadata_tool/intro.md
deleted file mode 100644
index 6eb17487ff..0000000000
--- a/website/docs/operations/system-operation/cli_tool/metadata_tool/intro.md
+++ /dev/null
@@ -1,18 +0,0 @@
----
-title: Metadata Tool
-language: en
-sidebar_label: Metadata Tool
-pagination_label: Metadata Tool
-toc_min_heading_level: 2
-toc_max_heading_level: 6
-pagination_prev: null
-pagination_next: null
-keywords:
- - metadata tool
-draft: false
-last_update:
- date: 08/16/2022
----
-
-This chapter introduces the metadata tool.
-
diff --git
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
deleted file mode 100644
index 373277d910..0000000000
Binary files
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_during_maintenance_mode.png
and /dev/null differ
diff --git
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
b/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
deleted file mode 100644
index 5adfae28a0..0000000000
Binary files
a/website/docs/operations/system-operation/images/forbidden_modify_metadata_promoting_during_maintenance_mode.png
and /dev/null differ
diff --git
a/website/docs/operations/system-operation/images/maintenance_mode.png
b/website/docs/operations/system-operation/images/maintenance_mode.png
deleted file mode 100644
index eb94f1d29f..0000000000
Binary files
a/website/docs/operations/system-operation/images/maintenance_mode.png and
/dev/null differ
diff --git
a/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
b/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
deleted file mode 100644
index 0007e33fef..0000000000
Binary files
a/website/docs/operations/system-operation/images/system_recover_from_maintenance_mode.png
and /dev/null differ
diff --git a/website/docs/operations/system-operation/junk_file_clean.md
b/website/docs/operations/system-operation/junk_file_clean.md
index cc517119ba..b131297180 100644
--- a/website/docs/operations/system-operation/junk_file_clean.md
+++ b/website/docs/operations/system-operation/junk_file_clean.md
@@ -77,7 +77,6 @@ This command supports standard short and long parameters. The
parameter descript
- `-r number`: The number of requests per second when accessing cloud
environment object storage. `-r 10` means 10 requests per second. You can use
this parameter to limit the frequency of requests for object storage in the
cloud environment by the junk file cleaning tool to avoid errors due to
exceeding the request frequency limit.
- `-t number`: The number of request retries when accessing the cloud
environment object storage fails. `-t 3` means to retry 3 times.
-**Note**: Whether this command executes metadata junk file cleaning through -m
or data junk file cleaning through -c, Kylin will enter maintenance mode. If
junk file cleaning is forcibly interrupted, you need to manually exit
maintenance mode. Refer to [ Maintenance Mode](maintenance_mode.md).
In addition, from Kylin 5, the new command line tool `FastRoutineTool`
````sh
diff --git a/website/docs/operations/system-operation/maintenance_mode.md
b/website/docs/operations/system-operation/maintenance_mode.md
deleted file mode 100644
index 3c4b3aa323..0000000000
--- a/website/docs/operations/system-operation/maintenance_mode.md
+++ /dev/null
@@ -1,76 +0,0 @@
----
-title: Maintenance Mode
-language: en
-sidebar_label: Maintenance Mode
-pagination_label: Maintenance Mode
-toc_min_heading_level: 2
-toc_max_heading_level: 6
-pagination_prev: null
-pagination_next: null
-keywords:
- - maintenance mode
-draft: false
-last_update:
- date: 08/16/2022
----
-
-## Basic Concepts
-
-When in maintenance mode, Kylin nodes will not be able to modify metadata, and
only read operations such as queries will not be affected
-
-## Entering maintenance mode
- Run the following command to enter maintenance mode:
- ```sh
- ${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.MaintainModeTool -on
-reason ${reason}
- ```
-
- **Note**:
-
-- You need to use the argument -reason to explain why you turn on maintenance
mode。
-
- - When it is in maintenance mode, it is not allowed to enter maintenance
mode again. You need to manually exit maintenance mode first
-
-- `${reason}` refers to the reason for entering maintenance mode. There are no
fixed characters. You can enter any word, such as `upgrade`
-
- When in maintenance mode, you will get following prompt:
-
- 
-
- metadata modification operations, such as create project, load/unload
tables, create/update models, etc.,
- can't be executed.
-
- If you do the above, the following errors will occur:
-
- 
-
- 
-
-## Exiting Maintenance mode
- Run the following command to exit maintenance mode:
- ```sh
- ${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.MaintainModeTool -off
- ```
- **Note**：While system is already in maintenance mode, the system will prompt
you after executing command. In speciality, you can use `-f` or `--force`
parameter to force to turn off maintenance mode,
- to release all occupation of project.
-
- After the command was executed, you may not modify metadata immediately. It
should be waiting Kylin nodes to
- get the resources. You can refer following properties:
- ```properties
- kylin.server.leader-race.heart-beat-interval=30
- ```
- Its unit is seconds. You should wait up to 30 seconds.
-
- During the maintenance, if you try to modify metadata, you will get
following prompt:
-
- 
-
-
-## Recommendations for the use of maintenance mode
-
-Maintenance mode is provided to prevent metadata write conflicts in the system.
-In principle, users are not recommended to turn on or turn off this function
manually.
-When running the following commands, the system will automatically turn on the
maintenance mode, and turn off the maintenance mode after the operation ends.
-
- - [reset admin password](../access-control/user_management.md)
- - [junk file clean](junk_file_clean.md)
- - [metadata
restore](cli_tool/metadata_tool/metadata_backup_restore.md#metadata-restore----metadata_restore)
diff --git a/website/docs/quickstart/intro.md b/website/docs/quickstart/intro.md
index daf793e107..bb7b1791e8 100644
--- a/website/docs/quickstart/intro.md
+++ b/website/docs/quickstart/intro.md
@@ -163,7 +163,7 @@ In the `conf` directory under the root directory of the
installation package, yo
2. Start Kylin with the startup script.
- Run the following command to start Kylin. When it is first started, the
system will run a series of scripts to check whether the system environment has
met the requirements. For details, please refer to the [Environment Dependency
Check](../operations/system-operation/cli_tool/environment_dependency_check.md)
chapter.
+ Run the following command to start Kylin. When it is first started, the
system will run a series of scripts to check whether the system environment has
met the requirements. For details, please refer to the [Environment Dependency
Check](../operations/system-operation/cli_tool/environment_check) chapter.
```shell
${KYLIN_HOME}/bin/kylin.sh start
diff --git a/website/sidebars.ts b/website/sidebars.ts
index a995353f8d..90fed31d07 100644
--- a/website/sidebars.ts
+++ b/website/sidebars.ts
@@ -613,25 +613,15 @@ const sidebars = {
items: [
{
type: 'doc',
- id:
'operations/system-operation/cli_tool/environment_dependency_check',
+ id:
'operations/system-operation/cli_tool/environment_check',
},
{
type: 'doc',
id:
'operations/system-operation/cli_tool/diagnosis'
},
{
- type: 'category',
- label: 'Metadata Tool',
- link: {
- type: 'doc',
- id:
'operations/system-operation/cli_tool/metadata_tool/intro',
- },
- items: [
- {
- type: 'doc',
- id:
'operations/system-operation/cli_tool/metadata_tool/metadata_backup_restore'
- },
- ],
+ type: 'doc',
+ id:
'operations/system-operation/cli_tool/metadata_operation'
},
{
type: 'doc',
@@ -639,10 +629,6 @@ const sidebars = {
},
],
},
- {
- type: 'doc',
- id: 'operations/system-operation/maintenance_mode'
- },
{
type: 'doc',
id: 'operations/system-operation/guardian',
