On Thu, Feb 1, 2018 at 12:03 PM, Ivan Kelly <iv...@apache.org> wrote:
> Ok, I think we're on the same page as to what is needed. The tool > needs to be in place before any version is EOL. > > I do think we should retain some BC tests with the older versions, of the > form: > - init cluster with 4.1.0 - run 4.7.0 upgrade tool > - start latest (4.8.0-SNAPSHOT) bookie and client > that is fine. > > Another thing to consider is that ledger metadata upgrade will involve > downtime for the client. This is fine, they can have a maintenance > window, but we should be clear in the documentation that this is > needed. > > -Ivan > > On Thu, Feb 1, 2018 at 5:58 PM, Sijie Guo <guosi...@gmail.com> wrote: > > I think you misunderstood what I was saying. > > > > > > On Thu, Feb 1, 2018 at 4:53 AM, Ivan Kelly <iv...@apache.org> wrote: > > > >> >> What do we mean by stopping support for BC? > >> > > >> > It means we will NOT consider BC problems with those old releases. > >> > >> If EOL means ignoring all BC considerations, then I'm -1 on these > >> proposals. But I don't think we need to go to those extremes. > >> I think we need to specify what BC guarantees we give on any version, > >> and then EOL policies will follow from that. The 3 big BC > >> considerations, as I see it, are: > >> 1) Data on the bookie > >> 2) Client wire protocol > >> 3) Data shared between bookies and clients (i.e. zookeeper metadata) > >> > >> For 1 we can have quite a short timeframe for EOL old software. > >> Bookies can be upgraded individually, and this is controlled by the > >> administrator. > >> For 2 we need a bit longer, because within an organisation there may > >> be many different versions of the client knocking around and it's hard > >> to get everyone to upgrade. > >> For 3 we can pretty much never have a hard break. If a cluster was > >> initialized with an old version, they must be able to able to upgrade > >> that cluster to the latest version of the software. Otherwise they > >> will stick with the old version forever. > >> > > > > when I am saying stop all BC consideration, I mean: > > > > 1) all the data and metadata will be upgraded to the minimal version we > > claim to support. > > 2) all the client would have to upgrade to the minimal version we claim > to > > support for wire protocol. > > > > Let's take 4.7.0 as the example, when I am saying enforce EOL at 4.7.0. > It > > means 4.7.0 will still compatible with 4.1.0. but at 4.7.0 we are > thinking > > about dropping 4.1.x, 4.2.x and 4.3.x. > > > > so at 4.7.0: > > > > - we will have to develop a tool to upgrade metadata and data to the > > version that will be recognized by 4.4.x and onwards releases. > > > > so at 4.8.0: > > > > - we will not consider clients older than 4.4.x. all the legal code can > be > > potentially removed. > > > > That means if a cluster (either metadata/data produced by clients older > > than 4.4.x or still have old clients) want to upgrade to 4.8.0, it has to > > first upgrade the cluster (both metadata and data) to 4.7.0 following the > > instructions provided at 4.7.0 > > and upgrade all the clients to minimal version that 4.8.0 will support. > > After all is done first, the cluster can be upgraded to 4.8.0. > > > > > > Hope this example make things clear. > > > > Enforcing EOL means dropping the BC considerations between the oldest > > version and newest version and limit the BC considerations within a > limited > > number of versions which is more controllable, maintenanable. Using the > > example above, > > it means if you want to upgrade from 4.7.x to 4.8.x, please use the tools > > provided in 4.7.x to ensure metadata/data are upgraded to latest version > > and no more clients older than 4.4.0 talk to the cluster. > > > > > >> > >> > for example, the password and digest type was introduced at a very old > >> > release. > >> > There is code assuming ledgers might not have password or digest. > >> However, > >> > the releases onwards would already have those fields. > >> > If we stop supporting those old version, we can remove that special BC > >> > code, which will make code clearer to maintain. > >> > >> This code is in the client. I'm fine with stopping support for all > >> clients older than a certain era. However, yahoo is still on 4.3, so > >> we should at least support back that far until they've been migrated > >> forward. > >> > > > > the yahoo branch is between 4.3 and 4.4. it is a special case that we > need > > to take care of. we are not dropping it out of consideration radar. > > > > > >> > >> >> The BC > >> >> tests should help in this regard, but we need keep testing scenarios > >> >> where the original cluster was brought up in the oldest version > >> >> (4.1.0). > >> > > >> > If we stop supporting 4.1.0, we don't need to test those scenarios. We > >> > started with from the lowest version we are claiming to support. > >> > >> Are we supporting the software or data that was created with that > >> software? That's two different things. Sure, we're not going to fix > >> bugs on pre-4.4.0 releases, but there are pre-4.4.0 clusters running > >> in the wild (yahoo being an example), and they have to be usable with > >> the latest software, otherwise they will never upgrade. I had this in > >> my previous job where they made a big breaking BC change in the > >> metadata between two versions. While it did allow us to delete a lot > >> of code, we ended up supporting two codebases forever, as customers > >> were unable to upgrade to the latest version. > >> > > > > Please check my example above to see if that makes sense to you. > > > > > >> > >> > That is the whole point of having EOL. > >> It's not clear what the point of having EOL is, since we don't have a > >> clearly defined EOL policy nor BC policy. > >> > > > > Please check my example above to see if that makes sense to you. > > > > > >> > >> -Ivan > >> >
