Max, Denis, Partition Awareness [1] is our main reason to have a specialized k8s discovery mechanism in Thin Clients.
k8s ingress mechanism is fine for a single client connection, but in Partition Aware mode thin clients need to connect to every server node, and track nodes as they enter or leave the topology. Denis is right, current tickets [2] [3] are about Thin Clients being deployed in the same k8s cluster as server nodes. Using k8s APIs we can get every pod address and establish connections. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients [2] https://issues.apache.org/jira/browse/IGNITE-13011 <https://issues.apache.org/jira/browse/IGNITE-13204> [3] https://issues.apache.org/jira/browse/IGNITE-13204 On Thu, Aug 13, 2020 at 10:48 AM Denis Magda <dma...@apache.org> wrote: > We're discussing the case when the thin client and server nodes are a part > of a single Kubernetes cluster. > > Ignite thick client uses KubernetesIPFinder > <https://apacheignite.readme.io/docs/kubernetes-ip-finder> to > auto-discover > other Ignite pods. The IP finder gets IP addresses of other Ignite pods via > a special Service instance, and then the thick client uses those addresses > to communicate to the rest of the cluster bypassing the Service. I > believe the ultimate goal of the JIRA task is to support a similar > capability for the thin client. > > However, you solution might be the way to go. Still, I wonder how the thin > client will handle cases when a server pod, the client was connected to, > goes down, and the Service connects the client to another pod (considering > the session affinity option). It might break some internal state of the > connection on the client-side and lead to unpredictable behavior. Also, if > the partition-awareness feature is enabled, then the thin client will start > bypassing the Service for most of the key-value requests connecting to the > nodes directly using their IP addresses. > > - > Denis > > > On Wed, Aug 12, 2020 at 11:07 PM Max Timonin <timonin.ma...@gmail.com> > wrote: > > > I'm not sure what you mean by inside/outside of kubernetes. Service name > is > > visible within k8s environment. I've described a case with connection > from > > another pod that is part of k8s cluster. > > > > To provide connection outside of Kubernetes one should configure Ingress > > [1]. It will have a fixed address assigned by a cluster administrator. > Any > > request outside of k8s to the Ingress will be proxied inside k8s env to > the > > Service and then to a pod automatically without need to provide any IP > > addresses in a client configuration. > > > > [1] https://kubernetes.io/docs/concepts/services-networking/ingress/ > > > > On Thu, Aug 13, 2020 at 1:39 AM Denis Magda <dma...@apache.org> wrote: > > > > > Max, > > > > > > That improvement automates the cluster discovery if both the cluster > and > > > thin clients are deployed *inside* of Kubernetes. That's my reading of > > the > > > ticket's description. While you're referring to the case when the > client > > is > > > deployed outside of a K8S environment. > > > > > > @Vladimir Pligin <vpli...@gridgain.com>, could you please join the > > thread. > > > I've seen you joined the discussion in JIRA. > > > > > > - > > > Denis > > > > > > > > > On Wed, Aug 12, 2020 at 12:06 PM Max Timonin <timonin.ma...@gmail.com> > > > wrote: > > > > > > > Hi, Igniters > > > > > > > > I'm now discovering the issue > > > > https://issues.apache.org/jira/browse/IGNITE-13204 and wondering > what > > > is a > > > > case that requires a client to be able to discover a cluster? > > > > > > > > I believe that discovery is a goal of kubernetes itself. We could > > assign > > > a > > > > DNS name for a Service [1] and then any request to the Service will > be > > > > forwarded to any of Ignite pods behind it. So there is no need to > > extract > > > > pods IP to provide the connection. For example, I've configured a > k8s > > > > service with name "ignite" and port mapping 10800:9888. Then just > > connect > > > > to it with sqlline by a string "ignite:9888". > > > > > > > > I think that it's OK that clients should be configured with those > > > settings > > > > as they aren't changed during cluster live. > > > > > > > > Also k8s provides SessionAffinity [2] to guarantee forwarding traffic > > > from > > > > one client to the same pod for every request. So there is no need to > > > > provide IP for a specific pod if a state is required. > > > > > > > > Maybe I'm missing something about this feature? Could somebody > provide > > > more > > > > details for this task? > > > > > > > > Below is the example config and connection string: > > > > > > > > apiVersion: v1 > > > > kind: Service > > > > metadata: > > > > name: ignite > > > > spec: > > > > ports: > > > > - port: 9888 > > > > targetPort: 10800 > > > > selector: > > > > app: ignite > > > > > > > > ./bin/sqlline.sh --verbose=true -u jdbc:ignite:thin://ignite:9888 > > > > 0: jdbc:ignite:thin://ignite:9888> > > > > > > > > [1] https://kubernetes.io/docs/concepts/services-networking/service/ > > > > [2] > > > > > > > > > > > > > > https://kubernetes.io/docs/concepts/services-networking/service/#proxy-mode-userspace > > > > > > > > > >
