This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/asf-site by this push: new 4689281 Automated deployment: Wed Jul 7 07:20:38 UTC 2021 42d721ddaf2cc975148878e0ab44cab649a0afec 4689281 is described below commit 46892812043e806ac40e557df58883f3b695dec2 Author: gosonzhang <gosonzh...@users.noreply.github.com> AuthorDate: Wed Jul 7 07:20:39 2021 +0000 Automated deployment: Wed Jul 7 07:20:38 UTC 2021 42d721ddaf2cc975148878e0ab44cab649a0afec --- docs/en-us/modules/agent/img/architecture.png | Bin 0 -> 43613 bytes docs/en-us/modules/agent/quick_start.md | 18 ++++-- docs/en-us/modules/dataproxy/quick_start.md | 10 +--- docs/zh-cn/modules/agent/quick_start.md | 76 +++++++++++++------------- docs/zh-cn/modules/dataproxy/quick_start.md | 11 +--- en-us/docs/modules/agent/img/architecture.png | Bin 0 -> 43613 bytes en-us/docs/modules/agent/quick_start.html | 13 +++-- en-us/docs/modules/agent/quick_start.json | 2 +- en-us/docs/modules/agent/quick_start.md | 18 ++++-- en-us/docs/modules/dataproxy/quick_start.html | 9 +-- en-us/docs/modules/dataproxy/quick_start.json | 2 +- en-us/docs/modules/dataproxy/quick_start.md | 10 +--- zh-cn/docs/modules/agent/quick_start.html | 41 +++++++------- zh-cn/docs/modules/agent/quick_start.json | 2 +- zh-cn/docs/modules/agent/quick_start.md | 76 +++++++++++++------------- zh-cn/docs/modules/dataproxy/quick_start.html | 9 +-- zh-cn/docs/modules/dataproxy/quick_start.json | 2 +- zh-cn/docs/modules/dataproxy/quick_start.md | 11 +--- 18 files changed, 151 insertions(+), 159 deletions(-) diff --git a/docs/en-us/modules/agent/img/architecture.png b/docs/en-us/modules/agent/img/architecture.png new file mode 100644 index 0000000..1138fe1 Binary files /dev/null and b/docs/en-us/modules/agent/img/architecture.png differ diff --git a/docs/en-us/modules/agent/quick_start.md b/docs/en-us/modules/agent/quick_start.md index cdbdb9b..985c249 100644 --- a/docs/en-us/modules/agent/quick_start.md +++ b/docs/en-us/modules/agent/quick_start.md @@ -1,19 +1,25 @@ ## 1、install + env requirements: + java JDK 1.8 + Maven 3.6 + + + mvn clean package -DskipTests - The tgz installation package can be found in the target under the project + The tgz installation package can be found in the target under the agent-release ## 2、Configuration -###2.1 Agent configuration - The agent supports two modes of operation: local operation and online operation + +###2.1 Agent configuration + Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as follows: -agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) +agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (the class name for fetch tasks, default ManagerFetcher) agent.local.ip=Write local ip agent.manager.vip.http.host=manager host agent.manager.vip.http.port=manager port @@ -31,8 +37,8 @@ Create a new directory jobs in conf, vim job1.json: put - { - +{ + "job": { "dir": { diff --git a/docs/en-us/modules/dataproxy/quick_start.md b/docs/en-us/modules/dataproxy/quick_start.md index 3abdc59..f98748f 100644 --- a/docs/en-us/modules/dataproxy/quick_start.md +++ b/docs/en-us/modules/dataproxy/quick_start.md @@ -14,8 +14,8 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ### config tube vim conf/flume.conf - In vim command line mode, execute the following replacement commands, where xxx is the tube address - :%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port + In vim command line mode, execute the following replacement commands, where ip:port is the tube address + :%s/TUBE_LIST/ip:port,ip:port ### Environmental preparation cd bin @@ -24,7 +24,7 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ### config manager vim conf/commmon.properties - 写入以下配置: + write: cluster_id=1 manager_hosts=xx.xx.xx.xx:port @@ -34,7 +34,3 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ## 4、check telnet 127.0.0.1 46801 -## 5、test - use proxySdk to test - - diff --git a/docs/zh-cn/modules/agent/quick_start.md b/docs/zh-cn/modules/agent/quick_start.md index 419f0bd..c387c0f 100644 --- a/docs/zh-cn/modules/agent/quick_start.md +++ b/docs/zh-cn/modules/agent/quick_start.md @@ -1,19 +1,23 @@ - ##一、安装 + 环境要求: + java JDK 1.8 + Maven 3.6 + + + inlong目录下执行命令 mvn clean package -DskipTests - 在项目下的target里面可以找到tgz安装包 - + 在项目下的agent-release/target里面可以找到tgz安装包 -## 二、配置 -###2.1 Agent 相关设置 +## 二、配置 agent 支持两种运行模式:本地运行以及线上运行 +###2.1 Agent 线上运行相关设置 + 线上运行需要从inlong-manager拉取配置,配置conf/agent.properties如下: agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) agent.local.ip=写入本机ip @@ -27,7 +31,7 @@ agent.manager.vip.http.port=manager port {"cluster_id":45,"isInterVisit":1,"size":1,"address": [{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0} -###2.3 本地 job配置 +###2.3 agent本地运行job配置 如果不使用线上模式,可以使用本地文件新建读取任务 在conf中新建目录jobs, vim job1.json: @@ -89,41 +93,37 @@ vim job1.json: # http default port agent.http.port=可用端口 - curl --location --request POST 'http://localhost:8129/config/job' \ - --header 'Content-Type: application/json' \ - --data '{ - "job": { - "dir": { - "path": "", - "pattern": "/data/inlong-agent/test.log" - }, - "trigger": "org.apache.inlong.agent.plugin.trigger.DirectoryTrigger", - "id": 1, - "thread": { - "running": { - "core": "4" - }, - "onejob": true - }, - "name": "fileAgentTest", - "source": "org.apache.inlong.agent.plugin.sources.TextFileSource", - "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink", - "channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel" - }, - "bus": { - "bid": "thirtybid10", - "tdmanager": { - "port": "8099", - "host": "http://tl-tdbank-tdmanger.tencent-distribute.com" - } - }, - "op": "add" - }' +curl --location --request POST 'http://localhost:8129/config/job' \ +--header 'Content-Type: application/json' \ +--data '{ +"job": { +"dir": { +"path": "", +"pattern": "/data/inlong-agent/test.log" +}, +"trigger": "org.apache.inlong.agent.plugin.trigger.DirectoryTrigger", +"id": 1, +"thread": { +"running": { +"core": "4" +} +}, +"name": "fileAgentTest", +"source": "org.apache.inlong.agent.plugin.sources.TextFileSource", +"sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink", +"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel" +}, +"proxy": { +"bid": "thirtybid10" +}, +"op": "add" +}' 其中参数分别为: - op: + pattern: 代表读取/data/inlong-agent/test.log文件 + proxy.bid: 代表发送到proxy使用的bid ##三、运行 解压后如下命令运行 - sh agent.sh start + sh agent.sh start \ No newline at end of file diff --git a/docs/zh-cn/modules/dataproxy/quick_start.md b/docs/zh-cn/modules/dataproxy/quick_start.md index dfc65d4..337ab40 100644 --- a/docs/zh-cn/modules/dataproxy/quick_start.md +++ b/docs/zh-cn/modules/dataproxy/quick_start.md @@ -1,6 +1,4 @@ - - -##一、安装 +## 一、安装 mvn clean package -DskipTests @@ -16,8 +14,8 @@ ### 配置tube地址 vim conf/flume.conf - vim命令行模式,执行以下替换命令,其中xxx为tube地址 - :%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port + vim命令行模式,执行以下替换命令,其中ip:port为tube master地址,可配置多个 + :%s/TUBE_LIST/ip1:port,ip2:port ### 环境准备 cd bin @@ -36,7 +34,4 @@ ## 四、检查启动情况 telnet 127.0.0.1 46801 -## 五、测试 - 使用proxySdk发送数据测试 - diff --git a/en-us/docs/modules/agent/img/architecture.png b/en-us/docs/modules/agent/img/architecture.png new file mode 100644 index 0000000..1138fe1 Binary files /dev/null and b/en-us/docs/modules/agent/img/architecture.png differ diff --git a/en-us/docs/modules/agent/quick_start.html b/en-us/docs/modules/agent/quick_start.html index 026adb0..0afaac4 100644 --- a/en-us/docs/modules/agent/quick_start.html +++ b/en-us/docs/modules/agent/quick_start.html @@ -13,15 +13,20 @@ </head> <body> <div id="root"><div class="documentation-page" data-reactroot=""><header class="header-container header-container-normal"><div class="header-body"><a href="/en-us/index.html"><a href="//www.apache.org"><img class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div class="logo-split"></div><img class="logo tube" style="width:120px;top:12px;position:absolute" src="/img/Tube logo.svg"/></a><div class="search search-normal"><span class="icon-search"></span></div><span class= [...] -<pre><code>mvn clean package -DskipTests +<pre><code>env requirements: +java JDK 1.8 +Maven 3.6 + -The tgz installation package can be found in the target under the project + +mvn clean package -DskipTests + +The tgz installation package can be found in the target under the agent-release </code></pre> <h2>2、Configuration</h2> -<p>###2.1 Agent configuration</p> <p>The agent supports two modes of operation: local operation and online operation</p> +<p>###2.1 Agent configuration</p> <p>Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as follows: -agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) +agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (the class name for fetch tasks, default ManagerFetcher) agent.local.ip=Write local ip agent.manager.vip.http.host=manager host agent.manager.vip.http.port=manager port</p> diff --git a/en-us/docs/modules/agent/quick_start.json b/en-us/docs/modules/agent/quick_start.json index 1a6f7d2..fa24221 100644 --- a/en-us/docs/modules/agent/quick_start.json +++ b/en-us/docs/modules/agent/quick_start.json @@ -1,6 +1,6 @@ { "filename": "quick_start.md", - "__html": "<h2>1、install</h2>\n<pre><code>mvn clean package -DskipTests\n\nThe tgz installation package can be found in the target under the project\n</code></pre>\n<h2>2、Configuration</h2>\n<p>###2.1 Agent configuration</p>\n<p>The agent supports two modes of operation: local operation and online operation</p>\n<p>Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as follows:\nagent.fetcher.classname=org.apache.inlong.agent [...] + "__html": "<h2>1、install</h2>\n<pre><code>env requirements:\njava JDK 1.8\nMaven 3.6 +\n\n\nmvn clean package -DskipTests\n\nThe tgz installation package can be found in the target under the agent-release\n</code></pre>\n<h2>2、Configuration</h2>\n<p>The agent supports two modes of operation: local operation and online operation</p>\n<p>###2.1 Agent configuration</p>\n<p>Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as f [...] "link": "/en-us/docs/modules/agent/quick_start.html", "meta": {} } \ No newline at end of file diff --git a/en-us/docs/modules/agent/quick_start.md b/en-us/docs/modules/agent/quick_start.md index cdbdb9b..985c249 100644 --- a/en-us/docs/modules/agent/quick_start.md +++ b/en-us/docs/modules/agent/quick_start.md @@ -1,19 +1,25 @@ ## 1、install + env requirements: + java JDK 1.8 + Maven 3.6 + + + mvn clean package -DskipTests - The tgz installation package can be found in the target under the project + The tgz installation package can be found in the target under the agent-release ## 2、Configuration -###2.1 Agent configuration - The agent supports two modes of operation: local operation and online operation + +###2.1 Agent configuration + Online operation needs to pull the configuration from inlong-manager, the configuration conf/agent.properties is as follows: -agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) +agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (the class name for fetch tasks, default ManagerFetcher) agent.local.ip=Write local ip agent.manager.vip.http.host=manager host agent.manager.vip.http.port=manager port @@ -31,8 +37,8 @@ Create a new directory jobs in conf, vim job1.json: put - { - +{ + "job": { "dir": { diff --git a/en-us/docs/modules/dataproxy/quick_start.html b/en-us/docs/modules/dataproxy/quick_start.html index fd76d9e..bcf196b 100644 --- a/en-us/docs/modules/dataproxy/quick_start.html +++ b/en-us/docs/modules/dataproxy/quick_start.html @@ -24,8 +24,8 @@ tar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz </code></pre> <h3>config tube</h3> <pre><code>vim conf/flume.conf -In vim command line mode, execute the following replacement commands, where xxx is the tube address -:%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port +In vim command line mode, execute the following replacement commands, where ip:port is the tube address +:%s/TUBE_LIST/ip:port,ip:port </code></pre> <h3>Environmental preparation</h3> <pre><code>cd bin @@ -34,7 +34,7 @@ cd .. </code></pre> <h3>config manager</h3> <pre><code>vim conf/commmon.properties -写入以下配置: +write: cluster_id=1 manager_hosts=xx.xx.xx.xx:port </code></pre> @@ -44,9 +44,6 @@ manager_hosts=xx.xx.xx.xx:port <h2>4、check</h2> <pre><code>telnet 127.0.0.1 46801 </code></pre> -<h2>5、test</h2> -<pre><code>use proxySdk to test -</code></pre> </div></section><footer class="footer-container"><div class="footer-body"><img src="/img/incubator-logo.svg"/><div class="cols-container"><div class="col col-24"><p>Apache InLong (incubating) is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with [...] <script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script> <script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script> diff --git a/en-us/docs/modules/dataproxy/quick_start.json b/en-us/docs/modules/dataproxy/quick_start.json index 8bc0b34..4a0c33d 100644 --- a/en-us/docs/modules/dataproxy/quick_start.json +++ b/en-us/docs/modules/dataproxy/quick_start.json @@ -1,6 +1,6 @@ { "filename": "quick_start.md", - "__html": "<h2>1、install</h2>\n<pre><code>mvn clean package -DskipTests\n</code></pre>\n<p>The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under the inlong-dataproxy/dataproxy-dist/target project</p>\n<pre><code>un tar it\ntar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>2、configuration</h2>\n<pre><code>cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>config tube</h3>\n<pre> [...] + "__html": "<h2>1、install</h2>\n<pre><code>mvn clean package -DskipTests\n</code></pre>\n<p>The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under the inlong-dataproxy/dataproxy-dist/target project</p>\n<pre><code>un tar it\ntar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>2、configuration</h2>\n<pre><code>cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>config tube</h3>\n<pre> [...] "link": "/en-us/docs/modules/dataproxy/quick_start.html", "meta": {} } \ No newline at end of file diff --git a/en-us/docs/modules/dataproxy/quick_start.md b/en-us/docs/modules/dataproxy/quick_start.md index 3abdc59..f98748f 100644 --- a/en-us/docs/modules/dataproxy/quick_start.md +++ b/en-us/docs/modules/dataproxy/quick_start.md @@ -14,8 +14,8 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ### config tube vim conf/flume.conf - In vim command line mode, execute the following replacement commands, where xxx is the tube address - :%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port + In vim command line mode, execute the following replacement commands, where ip:port is the tube address + :%s/TUBE_LIST/ip:port,ip:port ### Environmental preparation cd bin @@ -24,7 +24,7 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ### config manager vim conf/commmon.properties - 写入以下配置: + write: cluster_id=1 manager_hosts=xx.xx.xx.xx:port @@ -34,7 +34,3 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t ## 4、check telnet 127.0.0.1 46801 -## 5、test - use proxySdk to test - - diff --git a/zh-cn/docs/modules/agent/quick_start.html b/zh-cn/docs/modules/agent/quick_start.html index f6aed32..06a10a6 100644 --- a/zh-cn/docs/modules/agent/quick_start.html +++ b/zh-cn/docs/modules/agent/quick_start.html @@ -13,13 +13,18 @@ </head> <body> <div id="root"><div class="documentation-page" data-reactroot=""><header class="header-container header-container-normal"><div class="header-body"><a href="/zh-cn/index.html"><a href="//www.apache.org"><img class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div class="logo-split"></div><img class="logo tube" style="width:120px;top:12px;position:absolute" src="/img/Tube logo.svg"/></a><div class="search search-normal"><span class="icon-search"></span></div><span class= [...] -<pre><code>mvn clean package -DskipTests +<pre><code>环境要求: +java JDK 1.8 +Maven 3.6 + -在项目下的target里面可以找到tgz安装包 +inlong目录下执行命令 +mvn clean package -DskipTests + +在项目下的agent-release/target里面可以找到tgz安装包 </code></pre> <h2>二、配置</h2> -<p>###2.1 Agent 相关设置</p> <p>agent 支持两种运行模式:本地运行以及线上运行</p> +<p>###2.1 Agent 线上运行相关设置</p> <p>线上运行需要从inlong-manager拉取配置,配置conf/agent.properties如下: agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) agent.local.ip=写入本机ip @@ -30,7 +35,7 @@ agent.manager.vip.http.port=manager port</p> <p>内部写入:</p> <pre><code>{"cluster_id":45,"isInterVisit":1,"size":1,"address": [{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0} </code></pre> -<p>###2.3 本地 job配置 +<p>###2.3 agent本地运行job配置 如果不使用线上模式,可以使用本地文件新建读取任务 在conf中新建目录jobs, vim job1.json:</p> @@ -84,9 +89,9 @@ proxy.bid: 写入proxy时使用的bid类型 agent.http.enable=true # http default port agent.http.port=可用端口 - -curl --location --request POST 'http://localhost:8129/config/job' \ ---header 'Content-Type: application/json' \ +</code></pre> +<p>curl --location --request POST '<a href="http://localhost:8129/config/job">http://localhost:8129/config/job</a>' <br> +--header 'Content-Type: application/json' <br> --data '{ "job": { "dir": { @@ -98,31 +103,25 @@ curl --location --request POST 'http://localhost:8129/config/job' \ "thread": { "running": { "core": "4" -}, -"onejob": true +} }, "name": "fileAgentTest", "source": "org.apache.inlong.agent.plugin.sources.TextFileSource", "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink", "channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel" }, -"bus": { -"bid": "thirtybid10", -"tdmanager": { -"port": "8099", -"host": "http://tl-tdbank-tdmanger.tencent-distribute.com" -} +"proxy": { +"bid": "thirtybid10" }, "op": "add" -}' - -其中参数分别为: -op: +}'</p> +<pre><code>其中参数分别为: +pattern: 代表读取/data/inlong-agent/test.log文件 +proxy.bid: 代表发送到proxy使用的bid </code></pre> <p>##三、运行 解压后如下命令运行</p> -<pre><code>sh agent.sh start -</code></pre> +<pre><code>sh agent.sh start</code></pre> </div></section><footer class="footer-container"><div class="footer-body"><img src="/img/incubator-logo.svg"/><div class="cols-container"><div class="col col-24"><p>Apache InLong (incubating) is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with [...] <script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script> <script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script> diff --git a/zh-cn/docs/modules/agent/quick_start.json b/zh-cn/docs/modules/agent/quick_start.json index 8e5852e..4048cdd 100644 --- a/zh-cn/docs/modules/agent/quick_start.json +++ b/zh-cn/docs/modules/agent/quick_start.json @@ -1,6 +1,6 @@ { "filename": "quick_start.md", - "__html": "<p>##一、安装</p>\n<pre><code>mvn clean package -DskipTests\n\n在项目下的target里面可以找到tgz安装包\n</code></pre>\n<h2>二、配置</h2>\n<p>###2.1 Agent 相关设置</p>\n<p>agent 支持两种运行模式:本地运行以及线上运行</p>\n<p>线上运行需要从inlong-manager拉取配置,配置conf/agent.properties如下:\nagent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher)\nagent.local.ip=写入本机ip\nagent.manager.vip.http.host=manager host\nagent.manager.vip.http.port=manager port</p>\n<p>###2.2 Proxy 相关设置\n在agent [...] + "__html": "<p>##一、安装</p>\n<pre><code>环境要求:\njava JDK 1.8\nMaven 3.6 +\n\ninlong目录下执行命令\nmvn clean package -DskipTests\n\n在项目下的agent-release/target里面可以找到tgz安装包\n</code></pre>\n<h2>二、配置</h2>\n<p>agent 支持两种运行模式:本地运行以及线上运行</p>\n<p>###2.1 Agent 线上运行相关设置</p>\n<p>线上运行需要从inlong-manager拉取配置,配置conf/agent.properties如下:\nagent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher)\nagent.local.ip=写入本机ip\nagent.manager.vip.http.host=manager host\nagent [...] "link": "/zh-cn/docs/modules/agent/quick_start.html", "meta": {} } \ No newline at end of file diff --git a/zh-cn/docs/modules/agent/quick_start.md b/zh-cn/docs/modules/agent/quick_start.md index 419f0bd..c387c0f 100644 --- a/zh-cn/docs/modules/agent/quick_start.md +++ b/zh-cn/docs/modules/agent/quick_start.md @@ -1,19 +1,23 @@ - ##一、安装 + 环境要求: + java JDK 1.8 + Maven 3.6 + + + inlong目录下执行命令 mvn clean package -DskipTests - 在项目下的target里面可以找到tgz安装包 - + 在项目下的agent-release/target里面可以找到tgz安装包 -## 二、配置 -###2.1 Agent 相关设置 +## 二、配置 agent 支持两种运行模式:本地运行以及线上运行 +###2.1 Agent 线上运行相关设置 + 线上运行需要从inlong-manager拉取配置,配置conf/agent.properties如下: agent.fetcher.classname=org.apache.inlong.agent.plugin.fetcher.ManagerFetcher (设置任务获取的类名,默认为ManagerFetcher) agent.local.ip=写入本机ip @@ -27,7 +31,7 @@ agent.manager.vip.http.port=manager port {"cluster_id":45,"isInterVisit":1,"size":1,"address": [{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0} -###2.3 本地 job配置 +###2.3 agent本地运行job配置 如果不使用线上模式,可以使用本地文件新建读取任务 在conf中新建目录jobs, vim job1.json: @@ -89,41 +93,37 @@ vim job1.json: # http default port agent.http.port=可用端口 - curl --location --request POST 'http://localhost:8129/config/job' \ - --header 'Content-Type: application/json' \ - --data '{ - "job": { - "dir": { - "path": "", - "pattern": "/data/inlong-agent/test.log" - }, - "trigger": "org.apache.inlong.agent.plugin.trigger.DirectoryTrigger", - "id": 1, - "thread": { - "running": { - "core": "4" - }, - "onejob": true - }, - "name": "fileAgentTest", - "source": "org.apache.inlong.agent.plugin.sources.TextFileSource", - "sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink", - "channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel" - }, - "bus": { - "bid": "thirtybid10", - "tdmanager": { - "port": "8099", - "host": "http://tl-tdbank-tdmanger.tencent-distribute.com" - } - }, - "op": "add" - }' +curl --location --request POST 'http://localhost:8129/config/job' \ +--header 'Content-Type: application/json' \ +--data '{ +"job": { +"dir": { +"path": "", +"pattern": "/data/inlong-agent/test.log" +}, +"trigger": "org.apache.inlong.agent.plugin.trigger.DirectoryTrigger", +"id": 1, +"thread": { +"running": { +"core": "4" +} +}, +"name": "fileAgentTest", +"source": "org.apache.inlong.agent.plugin.sources.TextFileSource", +"sink": "org.apache.inlong.agent.plugin.sinks.TdBusSink", +"channel": "org.apache.inlong.agent.plugin.channel.MemoryChannel" +}, +"proxy": { +"bid": "thirtybid10" +}, +"op": "add" +}' 其中参数分别为: - op: + pattern: 代表读取/data/inlong-agent/test.log文件 + proxy.bid: 代表发送到proxy使用的bid ##三、运行 解压后如下命令运行 - sh agent.sh start + sh agent.sh start \ No newline at end of file diff --git a/zh-cn/docs/modules/dataproxy/quick_start.html b/zh-cn/docs/modules/dataproxy/quick_start.html index 2095bd0..1646c62 100644 --- a/zh-cn/docs/modules/dataproxy/quick_start.html +++ b/zh-cn/docs/modules/dataproxy/quick_start.html @@ -12,7 +12,7 @@ <link rel="stylesheet" href="/build/documentation.css" /> </head> <body> - <div id="root"><div class="documentation-page" data-reactroot=""><header class="header-container header-container-normal"><div class="header-body"><a href="/zh-cn/index.html"><a href="//www.apache.org"><img class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div class="logo-split"></div><img class="logo tube" style="width:120px;top:12px;position:absolute" src="/img/Tube logo.svg"/></a><div class="search search-normal"><span class="icon-search"></span></div><span class= [...] + <div id="root"><div class="documentation-page" data-reactroot=""><header class="header-container header-container-normal"><div class="header-body"><a href="/zh-cn/index.html"><a href="//www.apache.org"><img class="logo apache" style="width:120px" src="/img/asf_logo.svg"/></a><div class="logo-split"></div><img class="logo tube" style="width:120px;top:12px;position:absolute" src="/img/Tube logo.svg"/></a><div class="search search-normal"><span class="icon-search"></span></div><span class= [...] <pre><code>mvn clean package -DskipTests </code></pre> <p>在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p> @@ -24,8 +24,8 @@ tar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz </code></pre> <h3>配置tube地址</h3> <pre><code>vim conf/flume.conf -vim命令行模式,执行以下替换命令,其中xxx为tube地址 -:%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port +vim命令行模式,执行以下替换命令,其中ip:port为tube master地址,可配置多个 +:%s/TUBE_LIST/ip1:port,ip2:port </code></pre> <h3>环境准备</h3> <pre><code>cd bin @@ -44,9 +44,6 @@ manager_hosts=xx.xx.xx.xx:port <h2>四、检查启动情况</h2> <pre><code>telnet 127.0.0.1 46801 </code></pre> -<h2>五、测试</h2> -<pre><code>使用proxySdk发送数据测试 -</code></pre> </div></section><footer class="footer-container"><div class="footer-body"><img src="/img/incubator-logo.svg"/><div class="cols-container"><div class="col col-24"><p>Apache InLong (incubating) is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with [...] <script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script> <script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script> diff --git a/zh-cn/docs/modules/dataproxy/quick_start.json b/zh-cn/docs/modules/dataproxy/quick_start.json index 48e5127..f3991c3 100644 --- a/zh-cn/docs/modules/dataproxy/quick_start.json +++ b/zh-cn/docs/modules/dataproxy/quick_start.json @@ -1,6 +1,6 @@ { "filename": "quick_start.md", - "__html": "<p>##一、安装</p>\n<pre><code>mvn clean package -DskipTests\n</code></pre>\n<p>在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>解压\ntar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>二、配置</h2>\n<pre><code>cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>配置tube地址</h3>\n<pre><code>vim conf/flume.conf\nvim命令行模式,执行以下替换命令,其中xxx为tube地址\n:%s/TUBE_LIST/ [...] + "__html": "<h2>一、安装</h2>\n<pre><code>mvn clean package -DskipTests\n</code></pre>\n<p>在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz</p>\n<pre><code>解压\ntar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz\n</code></pre>\n<h2>二、配置</h2>\n<pre><code>cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin\n</code></pre>\n<h3>配置tube地址</h3>\n<pre><code>vim conf/flume.conf\nvim命令行模式,执行以下替换命令,其中ip:port为tube master地址,可配置多 [...] "link": "/zh-cn/docs/modules/dataproxy/quick_start.html", "meta": {} } \ No newline at end of file diff --git a/zh-cn/docs/modules/dataproxy/quick_start.md b/zh-cn/docs/modules/dataproxy/quick_start.md index dfc65d4..337ab40 100644 --- a/zh-cn/docs/modules/dataproxy/quick_start.md +++ b/zh-cn/docs/modules/dataproxy/quick_start.md @@ -1,6 +1,4 @@ - - -##一、安装 +## 一、安装 mvn clean package -DskipTests @@ -16,8 +14,8 @@ ### 配置tube地址 vim conf/flume.conf - vim命令行模式,执行以下替换命令,其中xxx为tube地址 - :%s/TUBE_LIST/xx.xx.xx.xx:port,xx.xx.xx.xx:port + vim命令行模式,执行以下替换命令,其中ip:port为tube master地址,可配置多个 + :%s/TUBE_LIST/ip1:port,ip2:port ### 环境准备 cd bin @@ -36,7 +34,4 @@ ## 四、检查启动情况 telnet 127.0.0.1 46801 -## 五、测试 - 使用proxySdk发送数据测试 -