Re: [PR] [INLONG-989][Doc] Add HTTP report usage demo in Quick Start [inlong-website]
dockerzhang merged PR #1001: URL: https://github.com/apache/inlong-website/pull/1001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Improve][Doc] Add HTTP report usage demo in Quick Start [inlong-website]
dockerzhang closed issue #989: [Improve][Doc] Add HTTP report usage demo in Quick Start URL: https://github.com/apache/inlong-website/issues/989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-11233][SDK] Transform SQL supports mid function [inlong]
Zkplo opened a new pull request, #11234: URL: https://github.com/apache/inlong/pull/11234 Fixes #11233 ### Motivation Add a mid function name to the SubstringFunction class. And add test case code. ### Modifications ### Verifying this change *(Please pick either of the following options)* - [ ] This change is a trivial rework/code cleanup without any test coverage. - [x] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong-website) branch master updated: [INLONG-1002][Doc] Add document for Transform SDK UDF extension (#1007)
This is an automated email from the ASF dual-hosted git repository.
aloyszhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 3a92365bbc9 [INLONG-1002][Doc] Add document for Transform SDK UDF
extension (#1007)
3a92365bbc9 is described below
commit 3a92365bbc983aa07871d49f5775ffcf4ce417b3
Author: ChunLiang Lu
AuthorDate: Sat Sep 28 21:04:29 2024 +0800
[INLONG-1002][Doc] Add document for Transform SDK UDF extension (#1007)
---
docs/development/extension_sort/transform_udf.md | 285 +
.../development/extension_sort/transform_udf.md| 283
2 files changed, 568 insertions(+)
diff --git a/docs/development/extension_sort/transform_udf.md
b/docs/development/extension_sort/transform_udf.md
new file mode 100644
index 000..75ec3afc66d
--- /dev/null
+++ b/docs/development/extension_sort/transform_udf.md
@@ -0,0 +1,285 @@
+---
+title: Transform UDF extension
+sidebar_position: 3
+---
+
+# Basic Concepts
+The following are some basic concepts that need to be understood during the
development process:
+- Transform SQL functions, including arithmetic functions (such as abs,
power), time functions (such as localtime, date_format), string functions (such
as locate, translate), etc. Functions generally have one or more parameters,
and their function is to perform some transformation operation on the input
data, and then output the transformed result.
+- Transform SQL parser, there are mainly two types of parsers, one is the
parser class for type, which is used to convert the original data into the
corresponding type object, such as DateParser can convert the input data into a
Date object in Java, which is convenient for further conversion operations; The
other is the parser class for calculation expressions, which is used to perform
certain calculation operations on the converted original data and output the
calculation result (simila [...]
+- Transform SQL operators, mainly some logical operators, such as (and, or,
not), etc., to implement some logical judgment operations, and the output
result is a Boolean value.
+
+# Function Development
+This section introduces how to expand a new function.
+
+## Create Function Class File
+The function implementation class is stored in this
[directory](https://github.com/apache/inlong/tree/master/inlong-sdk/transform-sdk/src/main/java/org/apache/inlong/sdk/transform/process/function).
After determining the function you want to expand, create a new class in this
directory, and the class name consists of function name + Function, such as
AbsFunction.
+
+## Basic Code Framework Construction
+
+After creating the class, build the basic framework of the code, taking
AbsFunction as an example:
+```java
+/**
+ * AbsFunction
+ * description: abs(numeric)--returns the absolute value of numeric
+ */
+@TransformFunction(names = {"abs"})
+public class AbsFunction implements ValueParser {
+
+@Override
+public Object parse(SourceData sourceData, int rowIndex, Context context) {
+
+}
+}
+```
+
+Add corresponding class comments and @TransformFunction annotation for the
function. The function needs to implement the ValueParser interface and
override the parse method in the interface.
+
+## Add Constructor and ValueParser Object
+
+Add a parameterized constructor and related ValueParser member variables to
the function. In the constructor, parse the function expression and initialize
the parameter parser object. Taking AbsFunction as an example:
+
+```java
+private ValueParser numberParser;
+
+public AbsFunction(Function expr) {
+ numberParser =
OperatorTools.buildParser(expr.getParameters().getExpressions().get(0));
+}
+```
+
+The number of ValueParser objects is the same as the number of function
parameters.
+
+## Function Implement
+
+Override the parse method, parse the parameters and implement the function
logic, and calculate the function return value. Taking AbsFunction as an
example:
+
+```java
+@Override
+public Object parse(SourceData sourceData, int rowIndex, Context context) {
+ Object numberObj = numberParser.parse(sourceData, rowIndex, context);
+ BigDecimal numberValue = OperatorTools.parseBigDecimal(numberObj);
+ return numberValue.abs();
+}
+```
+
+## Add Unit Test Code
+
+Each function needs to pass unit tests to verify whether the function logic is
correct. The unit test class is located in this directory. All unit test
functions for each function are placed in the same unit test class, and the
unit test class is named in the format of Test + function name + Function,
taking testAbsFunction() as an example:
+
+```java
+@Test
+public void testAbsFunction() throws Exception {
+ String transformSql = "select abs(numeric1) from source";
+ TransformConfig config = new TransformConfig(transformSql);
+
Re: [I] [Improve][Doc] Add document for Transform SDK UDF extension [inlong-website]
aloyszhang closed issue #1002: [Improve][Doc] Add document for Transform SDK UDF extension URL: https://github.com/apache/inlong-website/issues/1002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-1002][Doc] Add document for Transform SDK UDF extension [inlong-website]
aloyszhang merged PR #1007: URL: https://github.com/apache/inlong-website/pull/1007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong-website) branch master updated: [INLONG-989][Doc] Add HTTP report usage demo in Quick Start (#1001)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 45312c8e847 [INLONG-989][Doc] Add HTTP report usage demo in Quick
Start (#1001)
45312c8e847 is described below
commit 45312c8e84755823b83440e791551dc8987a856b
Author: Goson Zhang <4675...@qq.com>
AuthorDate: Sat Sep 28 20:18:37 2024 +0800
[INLONG-989][Doc] Add HTTP report usage demo in Quick Start (#1001)
---
docs/quick_start/data_http_report/_category_.json | 4 ++
docs/quick_start/data_http_report/http_report.md | 55 +
.../data_http_report/img/http_data_preview_en.png | Bin 0 -> 63494 bytes
.../data_http_report/img/http_data_view_en.png | Bin 0 -> 20893 bytes
.../data_http_report/img/http_dataproxy_en.png | Bin 0 -> 26970 bytes
.../data_http_report/img/http_group_stream_en.png | Bin 0 -> 48136 bytes
.../data_http_report/img/http_stream_define_en.png | Bin 0 -> 41117 bytes
.../quick_start/data_http_report/http_report.md| 55 +
.../data_http_report/img/http_data_preview_cn.png | Bin 0 -> 63609 bytes
.../data_http_report/img/http_data_view_cn.png | Bin 0 -> 21355 bytes
.../data_http_report/img/http_dataproxy_cn.png | Bin 0 -> 29109 bytes
.../data_http_report/img/http_group_stream_cn.png | Bin 0 -> 49126 bytes
.../data_http_report/img/http_stream_define_cn.png | Bin 0 -> 39714 bytes
13 files changed, 114 insertions(+)
diff --git a/docs/quick_start/data_http_report/_category_.json
b/docs/quick_start/data_http_report/_category_.json
new file mode 100644
index 000..1539f4f8940
--- /dev/null
+++ b/docs/quick_start/data_http_report/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "HTTP Report",
+ "position": 4
+}
\ No newline at end of file
diff --git a/docs/quick_start/data_http_report/http_report.md
b/docs/quick_start/data_http_report/http_report.md
new file mode 100644
index 000..957258c5324
--- /dev/null
+++ b/docs/quick_start/data_http_report/http_report.md
@@ -0,0 +1,55 @@
+---
+title: HTTP Report Example
+sidebar_position: 1
+---
+
+In the following content, we will use a complete example to introduce how to
use HTTP to report data, quickly verify whether the applied {groupId, streamId}
is effective, and whether the data is accepted by InLong DataProxy and
correctly written to the MQ cluster.
+
+## Prepare resources
+### Apply for InLong group and stream
+We need to apply for {groupId, streamId} in InLong Manager first. As shown in
the following figure, we have applied for {test_http, test_stream} information
and the administrator has approved it:
+
+
+In the application report stream, we defined that the data of this report
stream is reported in CSV format. The data content consists of three fields
(ID, Name, Desc) separated by vertical bars ("|"):
+
+
+### Find the IP and port of the DataProxy node that supports HTTP access
+InLong supports direct data reporting via HTTP. In this reporting example, we
directly select a DataProxy that supports HTTP reporting from the resource
details page of the InLong group to report the message. In the demonstration
environment, the HTTP receiving port opened by DataProxy is 47805, as shown
below:
+
+
+At this point, we have obtained the InLong group and stream information
required for data reporting, as well as the DataProxy node IP and port
information to be reported by HTTP reporting. Next, we can report data through
HTTP to verify whether the requested InLong group and stream, pipeline are
available.
+
+## Report data via HTTP
+According to the HTTP reporting protocol requirements of InLong, we use curl
tool to construct an HTTP instruction as shown below for execution. In the body
part, we construct a record containing three field values according to the
format definition of test_stream. {dataproxy_ip:dataproxy_httpport} is the
DataProxy IP and port for receiving the reported message. You can replace it
with the corresponding information in your environment:
+
+```bash
+curl -X POST -d
'groupId=test_http&streamId=test_stream&dt=data_time&body=1|name_1|desc_record_one&cnt=1'
http://{dataproxy_ip:dataproxy_httpport}/dataproxy/message
+```
+- Parameter Description：
+
+| parameter | meaning | Remark |
+|---|---|-|
+| groupId | Data stream group id | |
+| streamId | Data stream ID| |
+| body | Data content to be pushed | |
+| dt| Data time to be pushed|timestamp in millisecond
|
+| cnt | The count of data pieces to be pushed
Re: [I] [Improve][Doc] Add docs of TubeMQ Load Node [inlong-website]
aloyszhang closed issue #999: [Improve][Doc] Add docs of TubeMQ Load Node URL: https://github.com/apache/inlong-website/issues/999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-999][Doc] Add docs of TubeMQ Load Node [inlong-website]
aloyszhang merged PR #1008: URL: https://github.com/apache/inlong-website/pull/1008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong-website) branch master updated (3a92365bbc9 -> e922fa2d2e5)
This is an automated email from the ASF dual-hosted git repository. aloyszhang pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/inlong-website.git from 3a92365bbc9 [INLONG-1002][Doc] Add document for Transform SDK UDF extension (#1007) add e922fa2d2e5 [INLONG-999][Doc] Add docs of TubeMQ Load Node (#1008) No new revisions were added by this update. Summary of changes: docs/data_node/extract_node/tube.md| 83 +++--- docs/data_node/load_node/tube.md | 82 + .../current/data_node/extract_node/tube.md | 77 +--- .../current/data_node/load_node/tube.md| 80 + 4 files changed, 272 insertions(+), 50 deletions(-) create mode 100644 docs/data_node/load_node/tube.md create mode 100644 i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/tube.md
[I] [Improve][Doc] Add Transform component introduction. [inlong-website]
luchunliang opened a new issue, #1011: URL: https://github.com/apache/inlong-website/issues/1011 ### Description Add Transform component introduction. ### Are you willing to submit PR? - [X] Yes, I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Bump axios, @docusaurus/core, @docusaurus/plugin-content-docs and @docusaurus/preset-classic [inlong-website]
dependabot[bot] commented on PR #974: URL: https://github.com/apache/inlong-website/pull/974#issuecomment-2380587952 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Bump dompurify and swagger-ui-react [inlong-website]
dependabot[bot] commented on PR #976: URL: https://github.com/apache/inlong-website/pull/976#issuecomment-2380588011 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong-website) branch dependabot/npm_and_yarn/multi-23dea901ca deleted (was 064767894b7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/multi-23dea901ca in repository https://gitbox.apache.org/repos/asf/inlong-website.git was 064767894b7 Bump axios, @docusaurus/core, @docusaurus/plugin-content-docs and @docusaurus/preset-classic The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(inlong-website) branch dependabot/npm_and_yarn/multi-8a753ca33f deleted (was 1a82c5a15de)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/multi-8a753ca33f in repository https://gitbox.apache.org/repos/asf/inlong-website.git was 1a82c5a15de Bump dompurify and swagger-ui-react The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[PR] [INLONG-11216][SDK] Transform support STR_TO_MAP() function [inlong]
emptyOVO opened a new pull request, #11232: URL: https://github.com/apache/inlong/pull/11232 Fixes #11216 ### Motivation ### Modifications add StrToMapFucntion.class and provide unit tests ### Verifying this change *(Please pick either of the following options)* - [ ] This change is a trivial rework/code cleanup without any test coverage. - [x] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Bump axios, @docusaurus/core, @docusaurus/plugin-content-docs and @docusaurus/preset-classic [inlong-website]
dockerzhang closed pull request #974: Bump axios, @docusaurus/core, @docusaurus/plugin-content-docs and @docusaurus/preset-classic URL: https://github.com/apache/inlong-website/pull/974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Bump dompurify and swagger-ui-react [inlong-website]
dockerzhang closed pull request #976: Bump dompurify and swagger-ui-react URL: https://github.com/apache/inlong-website/pull/976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [INLONG-11209][SDK] Transform TRIM() function add usage of BTRIM() [inlong]
emptyOVO opened a new pull request, #11231: URL: https://github.com/apache/inlong/pull/11231 Fixes #11209 ### Motivation add usage of `btrim` , provide unit tests ### Modifications ### Verifying this change *(Please pick either of the following options)* - [ ] This change is a trivial rework/code cleanup without any test coverage. - [x] This change is already covered by existing tests, such as: *(please describe tests)* - [ ] This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a follow-up issue for adding the documentation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Improve][Doc] Add InLong sort format usage and extend doc [inlong-website]
baomingyu opened a new issue, #1012: URL: https://github.com/apache/inlong-website/issues/1012 ### Description _No response_ ### Are you willing to submit PR? - [X] Yes, I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [InLong-1012][Doc] Add InLong sort format usage and extend doc [inlong-website]
aloyszhang commented on code in PR #1013: URL: https://github.com/apache/inlong-website/pull/1013#discussion_r1779886340 ## i18n/zh-CN/docusaurus-plugin-content-docs/current/development/extension_sort/inlong_sort_data_organization_and_binary_protocol.md: ## @@ -0,0 +1,61 @@ +--- +title: InLong 分拣数据组织及协议解析 +sidebar_position: 5 +--- +## 总览 + +本文面向 InLong-Sort-Formats 数据分拣开发人员, 尝试尽可能全面地阐述开发一个数据格式的数据解析过程。 +InLong-Sort-Formats 模块支持两大类的数据格式解析，分别基于 Flink Row 和 Flink RowData 类型实现，这两类实现仅仅是，使用的 Flink API 不同，本文基于 Flink RowData 方式的实现进行描述。 +目前，InLong-Sort 支持如下几种格式(通过 InLongMsg 格式封装的 6 种，原始的数据格式 3 种): +- InLongMsg binlog +- InLongMSg CSV +- InLongMsg KV +- InLongMsg Tlog-CSV +- InLongMsg Tlog-KV +- InLongMsg PB +- CSV +- KV +- JSON + +## 开发之前 + +- InLongMsg 格式介绍参照 [InLongMsg](img/inlong_msg.md)； Review Comment: broken link -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [InLong-1012][Doc] Add InLong sort format usage and extend doc [inlong-website]
aloyszhang commented on code in PR #1013: URL: https://github.com/apache/inlong-website/pull/1013#discussion_r1779889199 ## docs/development/extension_sort/inlong_sort_data_organization_and_binary_protocol.md: ## @@ -0,0 +1,71 @@ +--- +title: InLong sort format extend +sidebar_position: 5 +--- +## Overview + +This article is aimed at InLong-Sort-Formats data format parsing developers and aims to comprehensively explain the process of developing data parsing for a data format. + +The InLong-Sort-Formats module supports two major types of data format parsing, implemented based on the Flink Row and Flink RowData types. These two implementations differ only in the Flink API used. This article will describe the implementation based on the Flink RowData. + +Currently, InLong-Sort supports the following formats, including 6 formats encapsulated in the InLongMsg format and 3 original data formats: +- InLongMsg binlog +- InLongMSg CSV +- InLongMsg KV +- InLongMsg Tlog-CSV +- InLongMsg Tlog-KV +- InLongMsg PB +- CSV +- KV +- JSON +By implementing the data parsing process for these formats, developers can effectively handle and process data in the InLong-Sort module. + +## Before Development + +- InLongMsg refer to [InLongMsg](img/inlong_msg.md); Review Comment: broken link -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [INLONG-1009][Doc] Add InLongMsg format definition and usage doc [inlong-website]
aloyszhang merged PR #1010: URL: https://github.com/apache/inlong-website/pull/1010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(inlong-website) branch master updated: [INLONG-1009][Doc] Add InLongMsg format definition and usage doc (#1010)
This is an automated email from the ASF dual-hosted git repository.
aloyszhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new a7b95c23071 [INLONG-1009][Doc] Add InLongMsg format definition and
usage doc (#1010)
a7b95c23071 is described below
commit a7b95c23071d0835717f94884421c0fb49d3af10
Author: Goson Zhang <4675...@qq.com>
AuthorDate: Sun Sep 29 09:55:34 2024 +0800
[INLONG-1009][Doc] Add InLongMsg format definition and usage doc (#1010)
---
docs/data_node/load_node/auto_consumption.md | 2 +-
.../binary_protocol/img/inlongmsg_frame.png| Bin 0 -> 28605 bytes
.../binary_protocol/img/inlongmsg_v1.png | Bin 0 -> 27298 bytes
.../binary_protocol/img/inlongmsg_v2.png | Bin 0 -> 28570 bytes
.../binary_protocol/img/inlongmsg_v3.png | Bin 0 -> 29195 bytes
.../binary_protocol/img/inlongmsg_v4.png | Bin 0 -> 24826 bytes
.../binary_protocol/img/inlongmsg_v4_bodydata.png | Bin 0 -> 5355 bytes
docs/development/binary_protocol/inlong_msg.md | 191
docs/development/inlong_msg.md | 50 --
.../data_node/load_node/auto_consumption.md| 2 +-
.../binary_protocol/img/inlongmsg_frame.png| Bin 0 -> 28605 bytes
.../binary_protocol/img/inlongmsg_v1.png | Bin 0 -> 27298 bytes
.../binary_protocol/img/inlongmsg_v2.png | Bin 0 -> 28570 bytes
.../binary_protocol/img/inlongmsg_v3.png | Bin 0 -> 29195 bytes
.../binary_protocol/img/inlongmsg_v4.png | Bin 0 -> 24826 bytes
.../binary_protocol/img/inlongmsg_v4_bodydata.png | Bin 0 -> 5355 bytes
.../development/binary_protocol/inlong_msg.md | 195 +
.../current/development/inlong_msg.md | 50 --
18 files changed, 388 insertions(+), 102 deletions(-)
diff --git a/docs/data_node/load_node/auto_consumption.md
b/docs/data_node/load_node/auto_consumption.md
index 847a2df1c14..972d8e71054 100644
--- a/docs/data_node/load_node/auto_consumption.md
+++ b/docs/data_node/load_node/auto_consumption.md
@@ -6,4 +6,4 @@ sidebar_position: 2
## Overview
**Auto Consumption** meanings receive data from Message Queue Services (TubeMQ
or Pulsar) directly, you can consume the message from MQ
by [Pulsar SDK
Client](https://pulsar.apache.org/docs/en/2.8.3/client-libraries/) or [TubeMQ
SDK Client](modules/tubemq/clients_java.md),
-after that, you have to [Parse the InLongMsg](development/inlong_msg.md) to
get raw data for forward processing.
\ No newline at end of file
+after that, you have to [Parse the
InLongMsg](development/binary_protocol/inlong_msg.md) to get raw data for
forward processing.
\ No newline at end of file
diff --git a/docs/development/binary_protocol/img/inlongmsg_frame.png
b/docs/development/binary_protocol/img/inlongmsg_frame.png
new file mode 100644
index 000..0c57142eedb
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_frame.png differ
diff --git a/docs/development/binary_protocol/img/inlongmsg_v1.png
b/docs/development/binary_protocol/img/inlongmsg_v1.png
new file mode 100644
index 000..fdb1a1c1932
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_v1.png differ
diff --git a/docs/development/binary_protocol/img/inlongmsg_v2.png
b/docs/development/binary_protocol/img/inlongmsg_v2.png
new file mode 100644
index 000..694025c796f
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_v2.png differ
diff --git a/docs/development/binary_protocol/img/inlongmsg_v3.png
b/docs/development/binary_protocol/img/inlongmsg_v3.png
new file mode 100644
index 000..11207719f50
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_v3.png differ
diff --git a/docs/development/binary_protocol/img/inlongmsg_v4.png
b/docs/development/binary_protocol/img/inlongmsg_v4.png
new file mode 100644
index 000..48e8284f9f2
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_v4.png differ
diff --git a/docs/development/binary_protocol/img/inlongmsg_v4_bodydata.png
b/docs/development/binary_protocol/img/inlongmsg_v4_bodydata.png
new file mode 100644
index 000..2a10737e27f
Binary files /dev/null and
b/docs/development/binary_protocol/img/inlongmsg_v4_bodydata.png differ
diff --git a/docs/development/binary_protocol/inlong_msg.md
b/docs/development/binary_protocol/inlong_msg.md
new file mode 100644
index 000..d1f157b8bc5
--- /dev/null
+++ b/docs/development/binary_protocol/inlong_msg.md
@@ -0,0 +1,191 @@
+---
+title: InLongMsg format definition and usage
+sidebar_position: 1
+---
+
+import {siteVariables} from '../../version';
+
+## Overview
+
+Users report data to the InLong system through SDK, HTTP, Agent and other data
reporting methods. InLong's DataProxy component packages t
Re: [I] [Improve][Doc] Add InLongMsg format definition and usage doc [inlong-website]
aloyszhang closed issue #1009: [Improve][Doc] Add InLongMsg format definition and usage doc URL: https://github.com/apache/inlong-website/issues/1009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [InLong-1012][Doc] Add InLong sort format usage and extend doc [inlong-website]
baomingyu opened a new pull request, #1013: URL: https://github.com/apache/inlong-website/pull/1013 Fixes #1012 Add InLong sort format usage and extend doc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@inlong.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
