The only items that we set are: Remote Process Groups: URLs ... Transport Protocol: HTTP Communication Timeout: 60 sec Yield Duration: 2 sec
Input Ports Concurrent Tasks: 7 Batch Settings Count: 2000 Size: 1000MB Duration: not set On Tue, Oct 2, 2018 at 3:47 PM Bryan Bende <[email protected]> wrote: > It looks like there may have been images attached that didn't make it > through. > > I'm not sure why S2S would be slow, it is made to handle very high > throughput. > > What I meant by #4 was... did you perform any specific configuration of the > RPG on cluster 1 or the input port on cluster 2, such as changing batch > settings or modifying concurrent tasks, or did you use all default values? > > > On Tue, Oct 2, 2018 at 2:01 PM roozbeh karimi <[email protected]> > wrote: > > > Hello Bryan > > 1) The communication is site to site between two separate > > instances/clusters of NiFi. > > 2) We have tried S2S in both raw and http modes, and neither made a > > difference. > > 3) The data flows but very slow, and after a while the back pressure > > reaches the threshold and we start losing data. > > 4) We have set up the RPG input port. > > 5) If we replace RPG with a posthttp processor on cluster 1 and a > listener > > processor on cluster 2 (where rpg nodes were), we don't see any back > > pressure buildup on cluster 1 and the data flows between the two > instances > > very fast. > > > > > > [image: image.png] > > [image: image.png] > > > > On Tue, Oct 2, 2018 at 11:55 AM Bryan Bende <[email protected]> wrote: > > > >> Hello, > >> > >> Can you describe the setup a little bit more? > >> > >> Is this site-to-site between two separate NiFi instances, or is it > >> within the same cluster for purposes of redistribution? > >> > >> If it's separate instances then what type of instances, clustered or > >> standalone? > >> > >> Have you made any changes to the configuration of the RPG, the port > >> settings within the RPG, or the Input Port on the remote side? > >> > >> Is some data being transferred through the RPG and it is just slow, or > >> is no data making it through? > >> > >> -Bryan > >> > >> On Mon, Oct 1, 2018 at 11:53 PM roozbeh karimi <[email protected]> > >> wrote: > >> > > >> > Hello All, > >> > > >> > Are there any best practices and guides for setting up RPG? With my > set > >> up, > >> > I am seeing a large volume of back pressure is being built up and > hardly > >> > any data is being transmitted from the source NiFi node to the nodes > in > >> > RPG. If I replace RPG with PostHttp on the source node and a listener > >> > processor on the destination node, no back pressure is built and the > >> data > >> > is transmitted smoothly between the source and destination nodes. > >> > > >> > I'd appreciate any feedback on to go about tuning my RPG set up. > >> > > >> > Thanks > >> > RK > >> > > >
