Igor, I am sorry it took me a while to fully understand your reasoning.
“Update user software first, then update the server” approach still looks somewhat weird to me (I think of Let's Encrypt client as an example of “normal” approach in Python world), but since this approach is vivid, I just have to take it into account, so I must agree with you. I just want to reiterate on one downside of such multi-protocol client, that was not yet addressed (not in Jira tasks or in docs, at least). Imagine a coder wrote a program with the latest client, using a feature available only in latest binary protocol. When the coder tests his program against the latest Ignite cluster, the program works perfectly. But then the end user runs the program against the previous version of the server, which client is still backwards-compatible with, the program runs, but at some point it tries to use the latest feature of the binary protocol and fails with some cryptic message. The end user is clueless, so as the coder. To avoid such a case, we must include an explicit parameter in our client's initialization method, that would set the desired protocol version(s) the user application is designed to work with. This parameter should be explicit, i.e. not have a default value, since it just will be useless the other way. And yes, this parameter renders all the software built with previous client versions incompatible with the new client. I think this problem concerns not only the Python client, but all the thin clients. What do you think? On Wed, 2019-02-13 at 13:45 +0300, Igor Sapego wrote: > The approach you suggest looks to me pretty much the same as > installing a new version of client software in C++ or Java. The issue > here that we break existing installed software and require for user > to update software in order to have ability to connect to a server. > Just imagine that application which made with thin client is not used > by a developer that knows how to use pip and all the stuff, but > someone with another background. Imagine, you have thousands of such > users. And now imagine, you want to update your servers. > > Best Regards, > Igor > > > On Tue, Feb 12, 2019 at 8:51 PM Dmitry Melnichuk < > dmitry.melnic...@nobitlost.com> wrote: > > > Igor, > > > > Thank you for your explanation. I think the matter begins to clear > > up > > for me now. > > > > The backward compatibility issue you described can not be applied > > to > > Python, because Python applications, unlike Java ones, do not have > > to > > be built. They rely on package manager (pip, conda, et c.) to run > > anywhere, including production. > > > > At the stage of deployment, the package manager collects > > dependencies > > using a specially crafted response file, often called > > `requirements.txt`. > > > > For example, to ensure that their application will work with the > > current _and_ future minor versions of pyignite, the user may > > include a > > line in their `requirements.txt` file: > > > > pyignite < x > > > > where x is a next major version number. In compliance with semantic > > versioning, the line is basically says: “Use the latest available > > version, that is earlier than x”. > > > > When upgrading Ignite server, system administrator or devops > > engineer > > must also update or recreate the app's environment, or update OS- > > level > > packages, or redeploy the app using Docker − the exact procedure > > may > > vary, but in any case it should be completely standard − to deliver > > the > > latest suitable dependencies. > > > > And then the same app connects to a latest Ignite server. > > > > Here is more about how pip understands versions: > > > > https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers > > > > What we really need to do for this to work seamlessly, is to > > establish > > the clear relation between products' versions. Regretfully, I have > > not > > done this before; just did not expect for this issue to come up. I > > think it would be best for pyignite major and minor to be set > > according > > to the Ignite binary protocol versions, i.e. pyignite 1.2.z handles > > Ignite binary protocol v1.2, and so on. But that is another matter. > > > > On Tue, 2019-02-12 at 13:39 +0300, Igor Sapego wrote: > > > Hi, > > > > > > Well, this approach was used for quite some time in ODBC and > > > JDBC, > > > so thin client has just inherited it, as we have the same port > > > and > > > the > > > same handshake message header. Maybe that's why you could not > > > find > > > any mention of versioning in thin client context. > > > > > > 3. Let's start from this point as this is the root of the issue. > > > Preconditions > > > are simple - the backward compatibility. We should guarantee that > > > user > > > can use any version of thin client with any version of server > > > while > > > they > > > share the same major version. This can be important, if someone > > > have > > > user applications built with thin client and they don't want to > > > break > > > their > > > user applications, when they just want to update servers. What is > > > appropriate here is to give user some kind of warning that they > > > use > > > outdated client or server. > > > > > > Can you explain how your approach is going to work in the example > > > above? I'm not quite sure I understand you correctly. > > > > > > 1. The API for this can be as simple as a set of allowed version > > > in > > > a configuration for both server and client. Maybe we should > > > discuss > > > such > > > kind of API as it can be useful for example in terms of security. > > > > > > 2. Complexity is the price developers have to pay for better user > > > experience. > > > Nothing new here. Software is not designed for developers, it's > > > designed > > > for a user. > > > > > > Best Regards, > > > Igor > > > > > > > > > On Tue, Feb 12, 2019 at 11:56 AM Dmitry Melnichuk < > > > dmitry.melnic...@nobitlost.com> wrote: > > > > > > > Hello fellow igniters! > > > > > > > > I recently started reconnaissance before the > > > > implementation of IEP-23 [1] in Python thin client. I think it > > > > is > > > > an > > > > interesting and much promising feature. As I understand, the > > > > changes > > > > associated with this feature will result in a new version of > > > > the > > > > binary > > > > protocol. > > > > > > > > What bothers me is not the changes required by the > > > > implementation itself, but the upcoming changes in the way that > > > > the > > > > fallback to the previous versions of the protocol is going to > > > > be > > > > handled. > > > > > > > > I have not found a open discussion on this topic neither on > > > > this mailing list, nor in Jira. If such a discussion took > > > > place, > > > > please > > > > inform me with a link. Based on what I have heard so far, I > > > > made > > > > the > > > > following conclusions: > > > > > > > > 1) the client is now expected to handle multiple > > > > binary protocol versions; > > > > 2) since the protocol is implemented the way > > > > the client first reports its version to the server on > > > > handshake, > > > > and > > > > not the other way, the client now must try to connect to the > > > > server > > > > using its preferred version of the protocol, and in case of an > > > > error, > > > > reconnect with another version. > > > > > > > > These changes are raised some concerns > > > > for me, for example: > > > > > > > > 1) the version negotiation mechanism implemented on > > > > client can not be made transparent to the end user. We must > > > > give > > > > the > > > > user an option, so that they could choose from the entire set > > > > of > > > > versions supported by the client, a subset of the versions > > > > their > > > > application is designed to work with. That will complicate the > > > > user > > > > API; > > > > > > > > 2) the complexity of the client will also be increased, > > > > especially > > > > considering unit testing. Certain tests will require a certain > > > > version > > > > of Ignite server, so they can no longer be implemented > > > > environment- > > > > agnostically, leading to a weird mix on unit and integration > > > > levels; > > > > > > > > 3) > > > > I do not see, or not aware of, the preconditions to such > > > > changes. I > > > > think the number of potential applications that can benefit > > > > from > > > > multi- > > > > protocol client is quite low. Usually all the hassle of version > > > > matching lie on the deployment level, above the app level. > > > > Please > > > > prove > > > > me wrong on this subject. > > > > > > > > Instead of supporting multiple binary > > > > protocols inside one client package, I would strongly prefer to > > > > support > > > > each version of the protocol separately, in the corresponding > > > > branch of > > > > the package repository. It would still be possible for the end > > > > user > > > > to > > > > use multiple protocols in one app, but with certain code-level > > > > and > > > > deployment-level efforts combined. > > > > > > > > Again, I most probably lack awareness > > > > on the subject, and therefore my conclusions may be premature, > > > > but > > > > anyway, let us discuss. I will appreciate any bit of knowledge > > > > from > > > > the > > > > community. > > > > > > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP- > > > > 23:+Best+Effort+Affinity+for+thin+clients > > > > > > > >
