Thank you, Val. Ilya, please let me know if the PR looks ok.
On Thu, 29 Apr 2021, 00:19 Valentin Kulichenko, < valentin.kuliche...@gmail.com> wrote: > I'm OK with the design. > > Ilya, please feel free to merge if the implementation and tests look good > to you. > > -Val > > On Wed, Apr 28, 2021 at 1:07 AM Atri Sharma <a...@apache.org> wrote: > > > Hi Ilya and Val, > > > > Thank you for taking a look and providing insights. I have updated the > > PR and raised another iteration. > > > > Val, I have moved the configuration to TcpDiscoverySpi. > > > > Please see and let me know your thoughts and comments. > > > > Regards, > > > > Atri > > > > On Wed, Apr 28, 2021 at 2:11 AM Valentin Kulichenko > > <valentin.kuliche...@gmail.com> wrote: > > > > > > Hi Atri, > > > > > > I've noticed that you added the property to the IgniteConfiguration, > but > > > it's applied only within the discovery. I feel like something is wrong > > here. > > > > > > If this feature only relates to the discovery, then we should have the > > > configuration property on the TcpDiscoverySpi instead. Otherwise, the > > > filtering should be applied to all network components (although I'm not > > > necessarily sure what that would imply). > > > > > > What do you think? > > > > > > -Val > > > > > > On Tue, Apr 27, 2021 at 2:00 AM Atri Sharma <a...@apache.org> wrote: > > > > > > > Hi Val and Ilya, > > > > > > > > Thank you for taking the time to pursue this issue. > > > > > > > > I have raised a new PR for the discussed approach. Please see and let > > > > me know what you think: > > > > > > > > https://github.com/apache/ignite/pull/9048 > > > > > > > > Regards, > > > > > > > > Atri > > > > > > > > On Thu, Apr 22, 2021 at 3:34 PM Ilya Kasnacheev > > > > <ilya.kasnach...@gmail.com> wrote: > > > > > > > > > > Hello! > > > > > > > > > > I'm still not fully convinced, but Val's approach sounds rational > to > > me. > > > > > > > > > > Regards, > > > > > -- > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > чт, 22 апр. 2021 г. в 12:45, Atri Sharma <a...@apache.org>: > > > > > > > > > > > Hello! > > > > > > > > > > > > I actually saw the shared container scenario being tried by > > somebody > > > > > > who wanted an external script to monitor all IPs being used by > his > > > > > > clusters and hence thought of this idea. Another thing that came > in > > > > > > was the Firewall blocking a few IP addresses, hence the idea. > > > > > > > > > > > > I feel that the footprint of this change is small, and can be > > useful > > > > > > for esoteric use cases too without really interfering in any > > existing > > > > > > code path. Val's suggestion seems the right way to go since it > > gives > > > > > > the functionality without much change. > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > On Thu, Apr 22, 2021 at 2:47 PM Ilya Kasnacheev > > > > > > <ilya.kasnach...@gmail.com> wrote: > > > > > > > > > > > > > > Hello! > > > > > > > > > > > > > > AFAIK, a S3 container, Azure blob container, etc, is a > relatively > > > > > > > lightweight entity, similar to a table in an SQL database. Why > > would > > > > > > > different clusters need to share the same discovery storage > > > > container? > > > > > > > When I tested Azure IP finder, it created several blob > containers > > > > for me > > > > > > on > > > > > > > demand, based on the parameter passed to IP finder. If I wanted > > to > > > > have > > > > > > > more than one cluster it should have been seamless already. > > > > > > > > > > > > > > I can theoretically see how address filtering may be useful to > > remove > > > > > > > public / private addresses or Docker gateway address, but it is > > > > usually > > > > > > > handled by setting localHost setting, although requiring tuning > > it > > > > for > > > > > > each > > > > > > > instance individually. Overall benefit seems to small. > > > > > > > > > > > > > > This is why I am asking, do you have any specific scenario in > > mind > > > > where > > > > > > > this feature is an enabler? How did you arrive at the > conclusion > > to > > > > go > > > > > > > forward with it? > > > > > > > > > > > > > > Regards, > > > > > > > -- > > > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > > > > > > чт, 22 апр. 2021 г. в 07:51, Atri Sharma <a...@apache.org>: > > > > > > > > > > > > > > > Hi Val, > > > > > > > > > > > > > > > > Consider a scenario where multiple Ignite clusters are > running > > and > > > > for > > > > > > > > operational ease (and also compliance, in some cases, e.g. to > > make > > > > > > > > auditing easier), people can configure cloud based IP finders > > to > > > > share > > > > > > > > the same container (blob container in Azure, S3 container in > > AWS > > > > etc). > > > > > > > > > > > > > > > > In such a case, IPs for all clusters will be in the same > > container. > > > > > > > > IPFinders of both the clusters will read the entire list. In > > this > > > > > > > > case, address filtering will help ignore the irrelevant IP > > > > addresses. > > > > > > > > > > > > > > > > Thank you for pointing me to the alternate direction. Let me > > > > research > > > > > > > > that and revert. > > > > > > > > > > > > > > > > Atri > > > > > > > > > > > > > > > > On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko > > > > > > > > <valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Hi Atri, > > > > > > > > > > > > > > > > > > Can you describe the scenario in a little more detail? What > > > > exactly > > > > > > do > > > > > > > > you > > > > > > > > > mean by a container shared by multiple clusters? What are > the > > > > > > > > consequences > > > > > > > > > of this? How does the proposed solution solve the problem? > > > > > > > > > > > > > > > > > > Also, I would suggest revisiting the design - I'm not sure > > such > > > > > > filtering > > > > > > > > > should be done on the IP finder level. Why not do this on > > the SPI > > > > > > level > > > > > > > > > instead? I would simply add something like "addressFilter" > > to the > > > > > > > > > TcpDiscoverySpi. The filter can be a generic > > IgnitePredicate, so > > > > you > > > > > > will > > > > > > > > > be able to provide any implementations, including regex or > > > > anything > > > > > > else. > > > > > > > > > > > > > > > > > > -Val > > > > > > > > > > > > > > > > > > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma < > a...@apache.org > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > When a container is shared by multiple clusters, then > this > > can > > > > be > > > > > > > > useful > > > > > > > > > > for filtering IPs. > > > > > > > > > > > > > > > > > > > > Also, things like VPC based barriers can be circumvented > > using > > > > this > > > > > > > > > > technique. > > > > > > > > > > > > > > > > > > > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, < > > > > > > ilya.kasnach...@gmail.com > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hello! > > > > > > > > > > > > > > > > > > > > > > What are the expected use cases for this feature? Can > you > > > > please > > > > > > > > > > elaborate? > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > -- > > > > > > > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma < > > a...@apache.org>: > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > > > > > > > I have opened the following JIRA for the said topic: > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14606 > > > > > > > > > > > > > > > > > > > > > > > > The concept is to filter IPs based on a pattern or a > > > > blocklist > > > > > > in > > > > > > > > > > > > IPFinders while consuming IPs. This is more pertinent > > for > > > > cloud > > > > > > > > based > > > > > > > > > > > > IPFinders since they can have shared containers. > > > > > > > > > > > > > > > > > > > > > > > > For the moment, I have implemented regex based > > filtering: > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14607 > > > > > > > > > > > > > > > > > > > > > > > > for Azure Blob Storage IP Finder. Over time, we can > > extend > > > > the > > > > > > > > same to > > > > > > > > > > > > other IP finders. > > > > > > > > > > > > > > > > > > > > > > > > Please see the PR: > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/ignite/pull/9024 > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > > > > > Atri > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > > > > > > > Atri > > > > > > > > > > > > Apache Concerted > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > > > > > > > > > > > > > Atri > > > > > > > > Apache Concerted > > > > > > > > > > > > > > > > > > > > -- > > > > > > Regards, > > > > > > > > > > > > Atri > > > > > > Apache Concerted > > > > > > > > > > > > > > -- > > > > Regards, > > > > > > > > Atri > > > > Apache Concerted > > > > > > > > -- > > Regards, > > > > Atri > > Apache Concerted > > >
