This is an automated email from the ASF dual-hosted git repository.
songjian pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel-web.git
The following commit(s) were added to refs/heads/main by this push:
new 61bfaca1 [Feature] Add README and fix some bugs (#46)
61bfaca1 is described below
commit 61bfaca1d7db3f238722d1c7521472ab305d159d
Author: Tyrantlucifer <tyrantluci...@gmail.com>
AuthorDate: Sun Mar 26 15:27:17 2023 +0800
[Feature] Add README and fix some bugs (#46)
* [Improve][seatunnel-server] Update api style follow front-end &
product-design
* [Improve][seatunnel-server] Update api style follow front-end &
product-design
* [Improve][seatunnel-server] Bug fix
* [Improve][seatunnel-server] Update api style follow front-end &
product-design (#12)
* [Improve][seatunnel-server] Update api style follow front-end &
product-design
* [Improve][seatunnel-server] Update api style follow front-end &
product-design
* [Improve][seatunnel-server] Bug fix
---------
Co-authored-by: dijie <di...@clickzetta.com>
* [Improve][seatunnel-server] Bugfix
* [Improve][seatunnel-server] add build script for build code & docker image
* [Improve][seatunnel-server] bugfix
* [Hotfix] Add license headers
* [Hotfix] Fix 404 dead link
* [Hotfix] Fix package maven plugin
* [Hotfix] Fix dependency license
---------
Co-authored-by: dijie <di...@clickzetta.com>
Co-authored-by: dijie <nj18652727...@gmail.com>
---
.github/workflows/backend.yml | 2 +-
.gitignore | 1 +
README.md | 126 +++++++++++++++++++++
build.sh | 53 +++++++++
docker/backend.dockerfile | 30 +++++
docs/images/datapipeline.png | Bin 0 -> 97439 bytes
docs/images/job.png | Bin 0 -> 72583 bytes
docs/images/user.png | Bin 0 -> 77240 bytes
pom.xml | 6 +-
seatunnel-server/seatunnel-app/pom.xml | 82 ++++++++++++--
.../src/main/assembly/seatunnel-web.xml | 58 ++++++++++
.../src/main/bin/seatunnel-backend-daemon.sh | 81 +++++++++++++
.../seatunnel-scheduler-dolphinscheduler/pom.xml | 4 +
seatunnel-server/seatunnel-server-common/pom.xml | 4 +
seatunnel-server/seatunnel-spi/pom.xml | 4 +
task.png | Bin 0 -> 110176 bytes
tools/dependencies/known-dependencies.txt | 2 -
17 files changed, 438 insertions(+), 15 deletions(-)
diff --git a/.github/workflows/backend.yml b/.github/workflows/backend.yml
index 6553aaba..efc139b1 100644
--- a/.github/workflows/backend.yml
+++ b/.github/workflows/backend.yml
@@ -79,7 +79,7 @@ jobs:
[[ ${{ needs.dead-link.result }} == 'success' ]] || exit 1;
build:
- if: github.repository == 'apache/incubator-seatunnel'
+ if: github.repository == 'apache/incubator-seatunnel-web'
name: Build
needs: [ sanity-check ]
strategy:
diff --git a/.gitignore b/.gitignore
index 757280ec..1197e9bd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -47,3 +47,4 @@ Test.scala
test.conf
spark-warehouse
*.flattened-pom.xml
+/seatunnel-ui/package-lock.json
diff --git a/README.md b/README.md
new file mode 100644
index 00000000..30444830
--- /dev/null
+++ b/README.md
@@ -0,0 +1,126 @@
+# Apache SeaTunnel (Incubating)
+
+<img src="https://seatunnel.apache.org/image/logo.png"; alt="seatunnel logo"
height="200px" align="right" />
+
+[](https://github.com/apache/incubator-seatunnel/actions/workflows/backend.yml)
+[](https://join.slack.com/t/apacheseatunnel/shared_invite/zt-123jmewxe-RjB_DW3M3gV~xL91pZ0oVQ)
+[](https://twitter.com/ASFSeaTunnel)
+
+---
+[](README.md)
+
+SeaTunnel was formerly named Waterdrop , and renamed SeaTunnel since October
12, 2021.
+
+---
+
+So, What we are?
+
+An open-source web console to manage your seatunnel-script, and would push
them to any scheduling-system easily.
+Click it if your want to know more about our design.
👉🏻[Design](https://github.com/apache/incubator-seatunnel/issues/1947)
+
+
+## How to start
+
+First, we need clone this project from Github.
+
+```shell
+git clone https://github.com/apache/incubator-seatunnel-web.git
+```
+
+Then, setup up configuration of db and more.
+```shell
+vim seatunnel-server/seatunnel-app/src/main/resources/application.yml
+```
+
+Notice:
+At present, we only support the following scheduler systems: dolphinscheduler,
more scheduler systems will be supported in the future;
+And for easier use, we plan to build our own scheduling system in Seatunnel.
+
+Here is a sample parameter configuration for Seatunnel integration
dolphinscheduler:
+```yaml
+ds:
+ script:
+ # The path where the script is stored
+ dir: /dj
+ project:
+ # The default project name of dolphinscheduler
+ default: test_dj
+ tenant:
+ # Which tenant been used to submit script
+ default: default
+ api:
+ # The dolphinscheduler user token
+ token: 12345678
+ # The dolphinscheduler api prefix address
+ prefix: http://127.0.0.1:12345/dolphinscheduler
+```
+
+Now comes the crucial part, this is about your account security, please modify
the Jwt secret key and algorithm.
+
+```yaml
+jwt:
+ expireTime: 86400
+ secretKey: https://github.com/apache/incubator-seatunnel
+ algorithm: HS256
+```
+
+
+Next, execute sql to create table .(Your must create database first by
yourself)
+```shell
+# Replace `username` & `dbName` with the real username and database name.
+# We will provided script in future.
+mysql -u username -p [dbName] < bin/seatunnl.sql
+```
+
+Now, you've done all the preparatory work, launch our app.
+
+### Launch it in IntelliJ IDEA
+
+Starting the back end of St in idea is really simple, just run the main method
of `SeatunnelApplication.java` in the `seatunnel-app` module.
+And the log will tell u anything you need to know.
+
+### Start it in the command line
+
+```shell
+# start backend
+
+# for build code
+sh build.sh code
+
+# for build image
+sh build.sh image
+
+# and then start docker container
+docker run apache/seatunnel-web
+
+```
+
+### start frontend
+You can use a Web server such as Apache HTTP Server or Nginx to start
front-end applications. Deploy the built front-end code to the root directory
of the Web server, start the Web server, and enter the URL of the Web server in
a browser to access the application.
+
+If you want start in dev mode:
+```shell
+cd seatunnel-ui
+npm install
+npm run dev
+```
+
+### How to use it
+
+After all the pre-work is done, we can open the following URL:
127.0.0.1:7890(please replace it according to your configuration) to use it.
+
+Now ,let me show you how to use it.
+
+#### User manage
+
+
+
+#### Datapipeline manage
+
+
+
+#### Job manage
+
+
+#### Task manage
+
\ No newline at end of file
diff --git a/build.sh b/build.sh
new file mode 100644
index 00000000..d48afbf4
--- /dev/null
+++ b/build.sh
@@ -0,0 +1,53 @@
+#!/bin/sh
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -
+WORKDIR=$(
+ cd "$(dirname "$0")" || exit
+ pwd
+)
+
+DOCKER_VERSION=1.0.0-snapshot
+
+# build code
+code() {
+ /bin/sh $WORKDIR/mvnw clean package -DskipTests
+ # mv release zip
+ mv $WORKDIR/seatunnel-server/seatunnel-app/target/seatunnel-web.zip $WORKDIR/
+}
+
+# build image
+image() {
+ docker buildx build --load --no-cache -t
apache/seatunnel-web:$DOCKER_VERSION -t apache/seatunnel-web:latest -f
$WORKDIR/docker/backend.dockerfile .
+}
+
+# main
+case "$1" in
+"code")
+ code
+ ;;
+"image")
+ image
+ ;;
+*)
+ echo "Usage: seatunnel-daemon.sh {start|stop|status}"
+ exit 1
+ ;;
+esac
+set +
diff --git a/docker/backend.dockerfile b/docker/backend.dockerfile
new file mode 100644
index 00000000..c4e5b31a
--- /dev/null
+++ b/docker/backend.dockerfile
@@ -0,0 +1,30 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+FROM openjdk:8-jre-alpine
+
+ENV DOCKER true
+ENV TZ Asia/Shanghai
+ENV SEATUNNEL_WEB_HOME /opt/app/seatunnel-web
+
+WORKDIR $SEATUNNEL_WEB_HOME
+
+ADD ../seatunnel-server/seatunnel-app/target/seatunnel-web/
$SEATUNNEL_WEB_HOME/
+
+EXPOSE 8080
+
+CMD [ "/bin/sh", "/opt/app/seatunnel-web/bin/seatunnel-backend-daemon.sh",
"start" ]
\ No newline at end of file
diff --git a/docs/images/datapipeline.png b/docs/images/datapipeline.png
new file mode 100644
index 00000000..052b8b9b
Binary files /dev/null and b/docs/images/datapipeline.png differ
diff --git a/docs/images/job.png b/docs/images/job.png
new file mode 100644
index 00000000..bc857f2b
Binary files /dev/null and b/docs/images/job.png differ
diff --git a/docs/images/user.png b/docs/images/user.png
new file mode 100644
index 00000000..fd694e41
Binary files /dev/null and b/docs/images/user.png differ
diff --git a/pom.xml b/pom.xml
index 8f380eb4..e5195ce0 100644
--- a/pom.xml
+++ b/pom.xml
@@ -94,6 +94,7 @@
<maven-pmd-plugin.version>3.8</maven-pmd-plugin.version>
<maven-license-maven-plugin>1.20</maven-license-maven-plugin>
<maven-scm-provider-jgit.version>1.9.5</maven-scm-provider-jgit.version>
+ <exec-maven-plugin.version>3.1.0</exec-maven-plugin.version>
<checkstyle.fails.on.error>true</checkstyle.fails.on.error>
<lombok.version>1.18.0</lombok.version>
<skipUT>false</skipUT>
@@ -202,11 +203,6 @@
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>${slf4j.version}</version>
- </dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
diff --git a/seatunnel-server/seatunnel-app/pom.xml
b/seatunnel-server/seatunnel-app/pom.xml
index 8b1dddf1..aef91185 100644
--- a/seatunnel-server/seatunnel-app/pom.xml
+++ b/seatunnel-server/seatunnel-app/pom.xml
@@ -25,6 +25,11 @@
<artifactId>seatunnel-app</artifactId>
+ <properties>
+ <java.version>1.8</java.version>
+ <frontend.project.name>seatunnel-ui</frontend.project.name>
+ </properties>
+
<dependencies>
<dependency>
<groupId>org.apache.seatunnel</groupId>
@@ -136,12 +141,24 @@
<groupId>org.apache.seatunnel</groupId>
<artifactId>seatunnel-spi</artifactId>
<version>${project.version}</version>
+ <exclusions>
+ <exclusion>
+ <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.slf4j</groupId>
+ </exclusion>
+ </exclusions>
</dependency>
<dependency>
<groupId>org.apache.seatunnel</groupId>
<artifactId>seatunnel-scheduler-dolphinscheduler</artifactId>
<version>${project.version}</version>
+ <exclusions>
+ <exclusion>
+ <artifactId>slf4j-log4j12</artifactId>
+ <groupId>org.slf4j</groupId>
+ </exclusion>
+ </exclusions>
</dependency>
<dependency>
@@ -180,21 +197,72 @@
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-compiler-plugin</artifactId>
+ <artifactId>maven-jar-plugin</artifactId>
<configuration>
- <source>1.8</source>
- <target>1.8</target>
+ <excludes>
+ <exclude>*.yml</exclude>
+ <exclude>*.yaml</exclude>
+ <exclude>*.xml</exclude>
+ </excludes>
</configuration>
</plugin>
+
+ <plugin>
+ <groupId>com.github.eirslett</groupId>
+ <artifactId>frontend-maven-plugin</artifactId>
+ <version>1.11.3</version>
+ <configuration>
+
<workingDirectory>${project.basedir}/../../${frontend.project.name}</workingDirectory>
+ </configuration>
+ <executions>
+ <execution>
+ <id>install node and npm</id>
+ <goals>
+ <goal>install-node-and-npm</goal>
+ </goals>
+ <configuration>
+ <nodeVersion>v16.16.0</nodeVersion>
+ <npmVersion>8.19.2</npmVersion>
+ </configuration>
+ </execution>
+ <execution>
+ <id>install</id>
+ <goals>
+ <goal>npm</goal>
+ </goals>
+ <phase>generate-resources</phase>
+ <configuration>
+ <arguments>install --ignore-scripts</arguments>
+ </configuration>
+ </execution>
+ <execution>
+ <id>build</id>
+ <goals>
+ <goal>npm</goal>
+ </goals>
+ <configuration>
+ <arguments>run build:prod</arguments>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
<plugin>
- <groupId>org.springframework.boot</groupId>
- <artifactId>spring-boot-maven-plugin</artifactId>
- <version>${spring-boot.version}</version>
+ <artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
+ <id>seatunnel-web</id>
<goals>
- <goal>repackage</goal>
+ <goal>single</goal>
</goals>
+ <phase>package</phase>
+ <configuration>
+ <finalName>seatunnel-web</finalName>
+ <descriptors>
+
<descriptor>src/main/assembly/seatunnel-web.xml</descriptor>
+ </descriptors>
+ <appendAssemblyId>false</appendAssemblyId>
+ </configuration>
</execution>
</executions>
</plugin>
diff --git a/seatunnel-server/seatunnel-app/src/main/assembly/seatunnel-web.xml
b/seatunnel-server/seatunnel-app/src/main/assembly/seatunnel-web.xml
new file mode 100644
index 00000000..0d1f4935
--- /dev/null
+++ b/seatunnel-server/seatunnel-app/src/main/assembly/seatunnel-web.xml
@@ -0,0 +1,58 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+
+<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0
http://maven.apache.org/xsd/assembly-2.1.0.xsd";>
+ <id>seatunnel-web</id>
+ <formats>
+ <format>dir</format>
+ <format>zip</format>
+ </formats>
+ <includeBaseDirectory>false</includeBaseDirectory>
+ <baseDirectory>seatunnel-web</baseDirectory>
+ <fileSets>
+ <fileSet>
+ <directory>${basedir}/src/main/resources</directory>
+ <includes>
+ <include>*.yml</include>
+ <include>*.yaml</include>
+ <include>*.xml</include>
+ </includes>
+ <outputDirectory>conf</outputDirectory>
+ </fileSet>
+ <fileSet>
+ <directory>${basedir}/src/main/bin</directory>
+ <includes>
+ <include>*.sh</include>
+ </includes>
+ <outputDirectory>bin</outputDirectory>
+ </fileSet>
+ <fileSet>
+ <directory>${basedir}/../../seatunnel-ui/dist</directory>
+ <includes>
+ <include>*</include>
+ </includes>
+ <outputDirectory>dist</outputDirectory>
+ </fileSet>
+ </fileSets>
+ <dependencySets>
+ <dependencySet>
+ <outputDirectory>libs</outputDirectory>
+ </dependencySet>
+ </dependencySets>
+</assembly>
diff --git
a/seatunnel-server/seatunnel-app/src/main/bin/seatunnel-backend-daemon.sh
b/seatunnel-server/seatunnel-app/src/main/bin/seatunnel-backend-daemon.sh
new file mode 100644
index 00000000..285f39cd
--- /dev/null
+++ b/seatunnel-server/seatunnel-app/src/main/bin/seatunnel-backend-daemon.sh
@@ -0,0 +1,81 @@
+#!/bin/sh
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -
+# usage: seatunnel-backend-daemon.sh <start|stop|status>
+
+WORKDIR=$(cd "$(dirname "$0")" || exit; pwd)
+
+# start
+start() {
+ echo "starting seatunnel..."
+
+ JAVA_OPTS="${JAVA_OPTS} -server -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails
-Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"
+ SPRING_OPTS="${SPRING_OPTS} -Dspring.config.name=application.yml
-Dspring.config.location=classpath:application.yml"
+
+ # check env JAVA_HOME
+ if [ -z "$JAVA_HOME" ]; then
+ echo "JAVA_HOME is not set"
+ exit 1
+ fi
+
+ echo "$WORKDIR"
+ $JAVA_HOME/bin/java $JAVA_OPTS \
+ -cp "$WORKDIR/../conf":"$WORKDIR/../libs/*" \
+ $SPRING_OPTS \
+ org.apache.seatunnel.app.SeatunnelApplication
+ echo "seatunnel started"
+}
+# stop
+stop() {
+ echo "stopping seatunnel..."
+ pid=$(jcmd | grep -i 'seatunnel-app-.*jar' | grep -v grep | awk '{print $1}')
+ if [ -n "$pid" ]; then
+ kill -15 $pid
+ echo "seatunnel stopped"
+ else
+ echo "seatunnel is not running"
+ fi
+}
+
+#status
+status() {
+ pid=$(jcmd | grep -i 'seatunnel-app-.*jar' | grep -v grep | awk '{print $1}')
+ if [ -n "$pid" ]; then
+ echo "seatunnel is running"
+ else
+ echo "seatunnel is not running"
+ fi
+}
+
+# main
+case "$1" in
+"start")
+ start
+ ;;
+"stop")
+ stop
+ ;;
+"status")
+ status
+ ;;
+*)
+ echo "Usage: seatunnel-daemon.sh {start|stop|status}"
+ exit 1
+esac
\ No newline at end of file
diff --git
a/seatunnel-server/seatunnel-scheduler/seatunnel-scheduler-dolphinscheduler/pom.xml
b/seatunnel-server/seatunnel-scheduler/seatunnel-scheduler-dolphinscheduler/pom.xml
index 5bf89de0..099115ad 100644
---
a/seatunnel-server/seatunnel-scheduler/seatunnel-scheduler-dolphinscheduler/pom.xml
+++
b/seatunnel-server/seatunnel-scheduler/seatunnel-scheduler-dolphinscheduler/pom.xml
@@ -25,6 +25,10 @@
<artifactId>seatunnel-scheduler-dolphinscheduler</artifactId>
+ <properties>
+ <java.version>1.8</java.version>
+ </properties>
+
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
diff --git a/seatunnel-server/seatunnel-server-common/pom.xml
b/seatunnel-server/seatunnel-server-common/pom.xml
index e170fdfb..6eb22aac 100644
--- a/seatunnel-server/seatunnel-server-common/pom.xml
+++ b/seatunnel-server/seatunnel-server-common/pom.xml
@@ -25,6 +25,10 @@
<artifactId>seatunnel-server-common</artifactId>
+ <properties>
+ <java.version>1.8</java.version>
+ </properties>
+
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
diff --git a/seatunnel-server/seatunnel-spi/pom.xml
b/seatunnel-server/seatunnel-spi/pom.xml
index 318c16c6..614c9025 100644
--- a/seatunnel-server/seatunnel-spi/pom.xml
+++ b/seatunnel-server/seatunnel-spi/pom.xml
@@ -25,6 +25,10 @@
<artifactId>seatunnel-spi</artifactId>
+ <properties>
+ <java.version>1.8</java.version>
+ </properties>
+
<dependencies>
<dependency>
<groupId>org.apache.seatunnel</groupId>
diff --git a/task.png b/task.png
new file mode 100644
index 00000000..8169be52
Binary files /dev/null and b/task.png differ
diff --git a/tools/dependencies/known-dependencies.txt
b/tools/dependencies/known-dependencies.txt
index 53d49110..055e9389 100644
--- a/tools/dependencies/known-dependencies.txt
+++ b/tools/dependencies/known-dependencies.txt
@@ -38,7 +38,6 @@ jjwt-impl-0.10.7.jar
jjwt-jackson-0.10.7.jar
jsoup-1.14.3.jar
jul-to-slf4j-1.7.36.jar
-log4j-1.2.17.jar
logback-classic-1.2.11.jar
logback-core-1.2.11.jar
mapstruct-1.0.0.Final.jar
@@ -55,7 +54,6 @@ seatunnel-common-2.1.3.jar
seatunnel-config-base-2.1.1.jar
seatunnel-config-shade-2.1.1.jar
slf4j-api-1.7.25.jar
-slf4j-log4j12-1.7.25.jar
snakeyaml-1.29.jar
spring-aop-5.3.20.jar
spring-beans-5.3.20.jar
![https://seatunnel.apache.org/image/logo.png"](https://seatunnel.apache.org/image/logo.png"){kind=link}
