Hi Atri, I've noticed that you added the property to the IgniteConfiguration, but it's applied only within the discovery. I feel like something is wrong here.
If this feature only relates to the discovery, then we should have the configuration property on the TcpDiscoverySpi instead. Otherwise, the filtering should be applied to all network components (although I'm not necessarily sure what that would imply). What do you think? -Val On Tue, Apr 27, 2021 at 2:00 AM Atri Sharma <a...@apache.org> wrote: > Hi Val and Ilya, > > Thank you for taking the time to pursue this issue. > > I have raised a new PR for the discussed approach. Please see and let > me know what you think: > > https://github.com/apache/ignite/pull/9048 > > Regards, > > Atri > > On Thu, Apr 22, 2021 at 3:34 PM Ilya Kasnacheev > <ilya.kasnach...@gmail.com> wrote: > > > > Hello! > > > > I'm still not fully convinced, but Val's approach sounds rational to me. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > чт, 22 апр. 2021 г. в 12:45, Atri Sharma <a...@apache.org>: > > > > > Hello! > > > > > > I actually saw the shared container scenario being tried by somebody > > > who wanted an external script to monitor all IPs being used by his > > > clusters and hence thought of this idea. Another thing that came in > > > was the Firewall blocking a few IP addresses, hence the idea. > > > > > > I feel that the footprint of this change is small, and can be useful > > > for esoteric use cases too without really interfering in any existing > > > code path. Val's suggestion seems the right way to go since it gives > > > the functionality without much change. > > > > > > Thoughts? > > > > > > On Thu, Apr 22, 2021 at 2:47 PM Ilya Kasnacheev > > > <ilya.kasnach...@gmail.com> wrote: > > > > > > > > Hello! > > > > > > > > AFAIK, a S3 container, Azure blob container, etc, is a relatively > > > > lightweight entity, similar to a table in an SQL database. Why would > > > > different clusters need to share the same discovery storage > container? > > > > When I tested Azure IP finder, it created several blob containers > for me > > > on > > > > demand, based on the parameter passed to IP finder. If I wanted to > have > > > > more than one cluster it should have been seamless already. > > > > > > > > I can theoretically see how address filtering may be useful to remove > > > > public / private addresses or Docker gateway address, but it is > usually > > > > handled by setting localHost setting, although requiring tuning it > for > > > each > > > > instance individually. Overall benefit seems to small. > > > > > > > > This is why I am asking, do you have any specific scenario in mind > where > > > > this feature is an enabler? How did you arrive at the conclusion to > go > > > > forward with it? > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > чт, 22 апр. 2021 г. в 07:51, Atri Sharma <a...@apache.org>: > > > > > > > > > Hi Val, > > > > > > > > > > Consider a scenario where multiple Ignite clusters are running and > for > > > > > operational ease (and also compliance, in some cases, e.g. to make > > > > > auditing easier), people can configure cloud based IP finders to > share > > > > > the same container (blob container in Azure, S3 container in AWS > etc). > > > > > > > > > > In such a case, IPs for all clusters will be in the same container. > > > > > IPFinders of both the clusters will read the entire list. In this > > > > > case, address filtering will help ignore the irrelevant IP > addresses. > > > > > > > > > > Thank you for pointing me to the alternate direction. Let me > research > > > > > that and revert. > > > > > > > > > > Atri > > > > > > > > > > On Wed, Apr 21, 2021 at 10:46 PM Valentin Kulichenko > > > > > <valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > Hi Atri, > > > > > > > > > > > > Can you describe the scenario in a little more detail? What > exactly > > > do > > > > > you > > > > > > mean by a container shared by multiple clusters? What are the > > > > > consequences > > > > > > of this? How does the proposed solution solve the problem? > > > > > > > > > > > > Also, I would suggest revisiting the design - I'm not sure such > > > filtering > > > > > > should be done on the IP finder level. Why not do this on the SPI > > > level > > > > > > instead? I would simply add something like "addressFilter" to the > > > > > > TcpDiscoverySpi. The filter can be a generic IgnitePredicate, so > you > > > will > > > > > > be able to provide any implementations, including regex or > anything > > > else. > > > > > > > > > > > > -Val > > > > > > > > > > > > On Wed, Apr 21, 2021 at 9:04 AM Atri Sharma <a...@apache.org> > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > When a container is shared by multiple clusters, then this can > be > > > > > useful > > > > > > > for filtering IPs. > > > > > > > > > > > > > > Also, things like VPC based barriers can be circumvented using > this > > > > > > > technique. > > > > > > > > > > > > > > On Wed, 21 Apr 2021, 15:49 Ilya Kasnacheev, < > > > ilya.kasnach...@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hello! > > > > > > > > > > > > > > > > What are the expected use cases for this feature? Can you > please > > > > > > > elaborate? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > -- > > > > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > > > > > > > > > ср, 21 апр. 2021 г. в 08:23, Atri Sharma <a...@apache.org>: > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > > > I have opened the following JIRA for the said topic: > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14606 > > > > > > > > > > > > > > > > > > The concept is to filter IPs based on a pattern or a > blocklist > > > in > > > > > > > > > IPFinders while consuming IPs. This is more pertinent for > cloud > > > > > based > > > > > > > > > IPFinders since they can have shared containers. > > > > > > > > > > > > > > > > > > For the moment, I have implemented regex based filtering: > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14607 > > > > > > > > > > > > > > > > > > for Azure Blob Storage IP Finder. Over time, we can extend > the > > > > > same to > > > > > > > > > other IP finders. > > > > > > > > > > > > > > > > > > Please see the PR: > > > > > > > > > > > > > > > > > > https://github.com/apache/ignite/pull/9024 > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > Atri > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > Atri > > > > > > > > > Apache Concerted > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > > > > > > Atri > > > > > Apache Concerted > > > > > > > > > > > -- > > > Regards, > > > > > > Atri > > > Apache Concerted > > > > > -- > Regards, > > Atri > Apache Concerted >
