szaszm commented on code in PR #1966: URL: https://github.com/apache/nifi-minifi-cpp/pull/1966#discussion_r2298386958
########## libminifi/test/libtest/integration/HTTPHandlers.cpp: ########## @@ -280,7 +301,7 @@ void HeartbeatHandler::verifyJsonHasAgentManifest(const rapidjson::Document& roo REQUIRE(manifestHash.length() == 128); // throws if not a valid hexadecimal hash - const auto hashVec = utils::string::from_hex(manifestHash); + const auto hashVec = minifi::utils::string::from_hex(manifestHash); Review Comment: why? When we're in the minifi namespace, we don't need to mention it when using things from within the namespace. ########## SITE_TO_SITE.md: ########## @@ -0,0 +1,133 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Table of Contents +- [Site-to-Site Overview](#site-to-site-overview) +- [Site-to-Site Configuration](#site-to-site-configuration) + - [Site-to-Site Configuration on NiFi side](#site-to-site-configuration-on-nifi-side) + - [Site-to-Site Configuration on MiNiFi C++ side](#site-to-site-configuration-on-minifi-c-side) +- [Additional examples](#additional-examples) + +## Site-to-Site Overview + +Site-to-Site protocol allows data to be transferred between MiNiFi C++ and NiFi instances. MiNiFi C++ can send or receive data from NiFi using remote process groups. This is useful for scenarios where you want to send data from MiNiFi C++ to NiFi or vice versa. Site-to-Site protocol support raw TCP and HTTP protocols. + +At the moment site-to-site protocol is only supported between MiNiFi C++ and NiFi instances, it cannot be used to transfer data between multiple MiNiFi C++ instances. It is recommended to use processors like InvokeHTTP and ListenHTTP to transfer data between MiNiFi C++ instances. Review Comment: Is there a technical reason for this limitation? ########## SITE_TO_SITE.md: ########## @@ -0,0 +1,133 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Table of Contents +- [Site-to-Site Overview](#site-to-site-overview) +- [Site-to-Site Configuration](#site-to-site-configuration) + - [Site-to-Site Configuration on NiFi side](#site-to-site-configuration-on-nifi-side) + - [Site-to-Site Configuration on MiNiFi C++ side](#site-to-site-configuration-on-minifi-c-side) +- [Additional examples](#additional-examples) + +## Site-to-Site Overview + +Site-to-Site protocol allows data to be transferred between MiNiFi C++ and NiFi instances. MiNiFi C++ can send or receive data from NiFi using remote process groups. This is useful for scenarios where you want to send data from MiNiFi C++ to NiFi or vice versa. Site-to-Site protocol support raw TCP and HTTP protocols. + +At the moment site-to-site protocol is only supported between MiNiFi C++ and NiFi instances, it cannot be used to transfer data between multiple MiNiFi C++ instances. It is recommended to use processors like InvokeHTTP and ListenHTTP to transfer data between MiNiFi C++ instances. + +## Site-to-Site Configuration + +### Site-to-Site Configuration on NiFi side + +On NiFi side, site-to-site protocol is configured by creating input and output ports. The input port is used to receive data from MiNiFi C++ and the output port is used to send data to MiNiFi C++. The input and output ports can be created in the NiFi UI by dragging and dropping the input and output port icons onto the canvas. + +To use the input or output port of the NiFi flow in the MiNiFi C++ flow, the instance id of the port should be used. The instance id can be found in the NiFi UI by clicking on the input or output port and looking at the operation panel. It can be copied from that panel, or from the port "instanceIdentifier" field from configuration json file in the NiFi conf directory. + +### Site-to-Site Configuration on MiNiFi C++ side + +Site-to-Site protocol is configured on the MiNiFi C++ side by using remote process groups in the configuration. The remote process group represents the NiFi endpoint and uses the instance ids of the ports created on the NiFi side. The remote process group can be configured to use either raw TCP or HTTP protocol. + +Here is a yaml example of how to configure site-to-site protocol in MiNiFi C++ where the MiNiFi C++ instance is sending data to NiFi using raw socket protocol: + +```yaml +MiNiFi Config Version: 3 +Flow Controller: + name: Simple GenerateFlowFile to RPG +Processors: + - id: b0c04f28-0158-1000-0000-000000000000 + name: GenerateFlowFile + class: org.apache.nifi.processors.standard.GenerateFlowFile + scheduling strategy: TIMER_DRIVEN + scheduling period: 5 sec + auto-terminated relationships list: [] + Properties: + Data Format: Text + Unique FlowFiles: false + Custom Text: Custom text +Connections: + - id: b0c0c3cc-0158-1000-0000-000000000000 + name: GenerateFlowFile/succes/nifi-inputport + source id: b0c04f28-0158-1000-0000-000000000000 + destination id: de7cc09a-0196-1000-2c63-ee6b4319ffb6 + source relationship name: success +Remote Process Groups: + - id: b0c09ff0-0158-1000-0000-000000000000 + name: "RPG" + url: http://localhost:8080/nifi + timeout: 20 sec + yield period: 10 sec + transport protocol: RAW + Input Ports: + - id: de7cc09a-0196-1000-2c63-ee6b4319ffb6 # this is the instance id of the input port created in NiFi + name: nifi-inputport + max concurrent tasks: 1 + use compression: false # currently not supported and ignored in MiNiFi C++ + batch size: + size: 10 MB + count: 10 + duration: 30 sec Review Comment: What does this do? I see it showed up in many places, but doesn't seem to be documented. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
