On 4/4/19 at 11:22 PM, john.matts...@ericsson.com (John
Mattsson) wrote:
I think "this requires both endpoints to adopt the change
simultaneously" is a problem as it makes it impossible to
introduce this mechanism in many existing deployments. I would
expect a solution that can be introduced gradually (some nodes
at a time) in existing deployments supporting RFC 8446 and/or
RFC 5246.
Having seen a successful deployment and migration to a new
incompatible version of a protocol, I think we too rapidly
dismiss the possibility. In security protocols, we gain a lot of
simplicity, and therefore security, by not having a
compatibility mode. I wrote a short paper about this deployment
for an amateur radio publication which I have included below.
Cheers - Bill
-----------------------------------------------------
Flag Day with FT8
Bill Frantz, AE6JV
West Valley Amateur Radio Association
Northern California DX Club
Northern California Contest Club
Abstract
Recently a community of over 20,000 users did a conversion
between release 1 and release 2 of a amateur radio digital
communications protocol called FT8. The old version of the
protocol was not compatible with the new version, resulting in a
"flag day" event for users. However, the conversion was rapid
and proceeded smoothly. Lessons from this conversion should be
useful for other software projects.
--------------------------------------------------
Introduction
The new FT8 digital transmission mode has taken the amateur
radio world by storm. Adoption has been overwhelmingly rapid,
going from nothing at its first release in July 2017 to the most
popular mode for many uses in December 2018.
FT8 is the invention of Steven Franke, K9AN, and Joe Taylor,
K1JT. (Joe won the Nobel Prize in Physics laureate for his
discovery with Russell Hulse of a "new type of pulsar, a
discovery that has opened up new possibilities for the study of
gravitation."[1]) It is a "weak signal" mode using 50 Hz
bandwidth for each signal, and can decode stations that are 24
dB below the voice signal bandwidth noise level.
It is an evolving protocol and has recently been through a major
change, from 75 bit messages to 77 bit messages. When you make a
major change in a computer protocol, and indeed, FT8 is a
computer protocol, there are two ways to introduce the change.
(1) You can code the new protocol to inter-operate with the old
protocol, or (2) you can have a "Flag Day" where everyone
changes to the new protocol at the same time. (A "flag day" is
when everyone has to adopt the new protocol at about the same
time or lose connection with those who have adopted it.)
Historically, computer engineers have avoided flag days like the
plague. Getting everyone to change at the same time is almost
impossible, and it is easier to design inter-operation. The FT8
implementors made the other choice; to have a flag day. It is
interesting examine why they made that choice and how they
succeeded with that choice.
Bill Somerville, G4WJS reported on January 16, 2019 [2]
"The situation with the 75-bit to 77-bit update of the FT8
protocol has
several constraints that are uncommon elsewhere and particularly
uncommon when they are all combined, including:
"1) It is a communications protocol where every user wishes
to be able
to communicate with any other, 2) the old version will not understand
the new protocol, at least without an upgrade, 3) WSJT-X is FOSS
[Freee Open Source Software] with no charges for use or
upgrade, 4)
the core WSJT development team is tiny and has no wish to support
multiple GA [General Availability] versions.
"Given these constraints and a desire not to consume extra valuable
digital modes bandwidth, any opportunity for old protocol
users who do
not wish to upgrade should be avoided. Having a new version
that still
talks the old protocol just exacerbates the potential schism
of users
since it allows those reluctant to upgrade to continue using
the old
protocol, at least until the number of indecipherable signals which
are QRM [interference] to their decoder overwhelms them."
Joe Taylor reported on January 17, 2019 [2]:
"1. We published a schedule, three months in advance,
specifying target
dates for intermediate candidate releases for beta testing. We
publicized this schedule as widely as possible, and we stuck
to it.
"2. The first three candidate releases included "bi-lingual" capability
for the old and new protocols. This helped to encourage beta testing
and feedback from interested users, and helped to show users that
upgrading was a simple drop-in replacement.
"3. Candidate releases 4 and 5 were "new protocol only".
They allowed
us to eliminate large amounts of obsolete code and fix most
of the
bugs introduced by these wholesale changes."
Operations are usually a lot more pleasant when everyone is
running the same version of a protocol. As Bill Somerville notes
above, keeping the community of users together has a significant
long-term value.
Inter-operation logic tends to be buggy as noted by Joe Taylor
above, and those problems "just go away" with a flag day. A flag
day also allows a development team to concentrate on more
valuable tasks.
Also, with only 77 bits in a message, spending a bit or several
on a message format ID is very expensive. Considering that it
takes 15 seconds to send each of these messages, and another 15
seconds to receive the reply, every bit is precious.
Another consideration is that the old and new formats can
coexist operating on different frequencies. They just can't talk
to each other. People who haven't converted can still
communicate with others who haven't converted, somewhat easing
the need for everyone to change "at once". However seeing a lot
of strong signals that can't be decoded is a significant
incentive to find out what the problem is and fix it.
--------------------------------------------------
Changeover Time Line
So what was the result when the team announced they would have
Release 2.0 available on December 10, 2018 and expected everyone
to convert by January 1, 2019? In a word, success! From the
wsjt-x developer email list:
Joe Tailor reported [3]:
"Monitoring 20 meters just 3 hours after WSJT-X 2.0 had been released,
I found that nearly half of the active stations were already
using it.
I made 18 QSOs [contacts] in about half an hour.
"According to PSK Reporter* statistics, at least 1620
callsigns have
been making QSOs with the new version. This is now ~7.5
hours after
the v2.0 GA release."
Dwight NS9I reported [4]: 70 q's [contacts] and all is well ...
amazed at the activity, including DX [foreign stations] on all
the bands.
On December 12, Tom OH6VDA reported [5]: And right now
aproximately 8000 V2.0 users of a total of 21 000 WSJT-X users,
according to Pskreporter.info#. :)
On December 12, Richard KF5OIM reported [6]: wjstx 2.0.0 has
landed in the Fedora testing repositories for Fedora 28, 29,
Rawhide and Fedora EPEL 7 (CentOS,SL,etc)
On December 12, Ted K9IMM reported [7]:
I have a multi-band FT8 skimmer. It skims 7 bands from 160M-15M
(excluding 60M).
I converted the skimmer to V2.0.0 yesterday about this time.
Ordinarily, my V1.9.1 spot rate was about 10,000-12,000 spots
per day.
In the last 24 hours my V2.0.0 spot rate is 7700/day.
So...Either there are a huge amount of new FT8 users running V2.0.0,
or users are converting to the new version at a very satisfyingly
rapid rate.
Pretty amazing for a release 2 days old.
On December 14, Joe Taylor reported [8]:
This morning I ran an instance of WSJT-X v1.9.1 in parallel with
WSJT-X 2.0. An informal count of decodes in the old and new protocols
shows far more activity using the new 77-bit messages. Here
are the
rough statistics, given as the ratio of new- to old-style decodes:
For FT8:
-----------
17m: 10:1
20m: 3:1
30m: 6:1
40m 20:1
On December, 14, Charlie WD5BJT reported [9]: I mostly operate
on the 6 meter band using FT-8. Of the many operators on the
band last night, I decoded only two users of the old protocol.
On January 2, 2019, a late adopter reported [10]: My version of
WSJT-X is 1.8.0 I don't decode anything, WF [Waterfall, a way
of seeing all the signals on a band] clearly has big signals on
it that come and go every 15 sec. Do I need to upgrade?
--------------------------------------------------
Conclusions
This change was a very successful flag day. There is almost no
one using the old version after the end of the cut-over period.
There are almost no complaints about the need to change.
Things that made this success possible:
Most of the usage of FT8 is concentrated in a small
percentage of the users. This small percentage of users were
early adopters. As a result, nearly half to the signals on one
band were in the new format only 3 hours after the release of
the new version.
There were a number of features in the new version of the
software that would attract users to convert. These included
better classification of a received station's location (for
people interested in long distance communication) and support
for more kinds of on-the-air ham radio activities.
Conversion was a simple drop-in replacement without a lot of
effort required from the user.
File data formats did not change except for the details of
some debugging information. Information about contacts made with
the old version were available to the new version.
A use could easily switch back and forth between the old and
new version, easing concerns of early adopters.
Software developers with similar advantages should give more
consideration to having a flag day instead of including
inter-operation code in their applications. A flag day might be
particularly attractive if there is a strong desire to keep from
splitting the user community, or if the development team is small.
--------------------------------------------------
* PSK Reporter is a world-wide amateur radio service which
listens for digital signals and reports which ones it hears via
the Internet. See: <https://pskreporter.info/> and <https://pskreporter.info/cgi-bin/pskstats.pl>
# PSK Reporter can take up to a week to notice that a station is
using the new version.
--------------------------------------------------
Acknowledgements
Thanks to Joe Taylor K1JT and Bill Somerville G4WJS for their
comments. Their insite into the development process was very
valuable in developing this paper.
Thanks to Wolfgang Meister OE1MWW for information about PSK
Reporter and other software supporting FT8.
Thanks to Gary Hinson ZL2iFB's "FT8 Operating Guide"
<http://www.g4ifb.com/FT8_Hinson_tips_for_HF_DXers.pdf> for some
of the early history of FT8
--------------------------------------------------
References
[1] <https://en.wikipedia.org/wiki/Joseph_Hooton_Taylor_Jr.>
[2] <https://sourceforge.net/p/wsjt/mailman/message/36519613/>
[3] <https://sourceforge.net/p/wsjt/mailman/message/36490442/>
[4]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/35619a06-08ca-a82e-2b96-5784c5aa9285%40bayland.net/#msg36490399>
[5]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/1419952134.2738348.1544608313582%40mail.yahoo.com/#msg36491719>
[6]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/CAN3TeO2DJFcNO_XeK9jZSZtJJDNk0u_E3VkHqc2byg58PddjKA%40mail.gmail.com/#msg36492022>
[7]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/003901d49258%24139b4ff0%243ad1efd0%24%40offex.com/#msg36492293>
[8]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/49fe71f4-bf4c-4c9c-33ee-7d73c6620046%40princeton.edu/#msg36494001>
[9]
<https://sourceforge.net/p/wsjt/mailman/wsjt-devel/thread/2015959274.4026.1544810692848%40wamui-dingo.atl.sa.earthlink.net/#msg36494040>
[10] Post to the Northern California Contest Club reflector on
January 2, 2019 at 1533 PDT.
-----------------------------------------------------
---------------------------------------------------------------------------
Bill Frantz |"After all, if the conventional wisdom was
working, the
408-356-8506 | rate of systems being compromised would be
going down,
www.pwpconsult.com | wouldn't it?" -- Marcus Ranum
_______________________________________________
TLS mailing list
TLS@ietf.org
https://www.ietf.org/mailman/listinfo/tls
