<https://labs.ripe.net/author/emileaben/the-resilience-of-the-internet-in-ukraine/>
At the end of the second week of the Russian invasion of Ukraine, the
devastating effects of this war are very visible in mainstream media. Over two
million people have fled the violence so far, and immense damage is being done
to cities, towns and villages across the country.
In spite of all that is happening, the Ukrainian Internet appears to be holding
up. And while it is terrible to see the resiliency of a country's Internet
tested in this way, it is good to see that it is still functioning. There has
been some speculation as to why this is. For example, it may be that the
Ukrainian Internet is deliberately not being targeted by the invaders. Or it
may be that it is structured in such a way that makes it hard to attack.
In this article, we take a closer look at certain factors that may be
contributing to the the resiliency of the Ukrainian Internet.
Lack of market concentration at end-user networks
One interesting dimension of the Ukrainian Internet is the lack of market
concentration in the end-user market. If we calculate the Herfindahl Hirschman
index (HHI), an economic indicator for market concentration, we find that
Ukraine is one of the least concentrated markets worldwide. Note that we
calculate the HHI based on APNIC data for which networks serve users in a
particular country. From this data we take all networks with over 1% users for
the particular country, and from that we calculate HHI. Because we convert from
percentages to fractions (divide by 100) we get an HHI between 0 (not
concentrated) and 1 (monopoly):
Top 10 least concentrated markets for end-user per network (ASN) Country
HHI
1 Brazil 0.018
2 Russia 0.047
3 United States 0.05
4 Ukraine 0.052
5 Lebanon 0.067
6 Singapore 0.069
7 Albania 0.072
8 Guadelope 0.081
9 South Africa 0.083
10 Japan 0.087
Why is this important? There are no dominant players in the market, so if an
individual network goes down, this has a relatively small effect on the whole
network.
It is also relevant to note that, for the most part, the networks with
significant amounts of users are Ukrainian companies. For the Russian regime to
shut down Ukrainian networks, they would have to force these Ukrainian
companies to do so. This is different from situations where a government shuts
down the Internet in the country it governs, as we saw in Egypt in the Arab
Spring. In that case, the Egyptian government ordered Egyptian networks to shut
down the Internet and the networks complied.
We also look at how these networks are interconnected. For the networks where
we have RIPE Atlas probes, we can measure the path between these networks. This
is depicted in Figure 1:
Figure 1: How Ukrainian end-user networks interconnect, as seen from RIPE Atlas
The outer ring of this visualisation represents end-user networks. For this
visualisation, the only networks that are shown (as green arcs) in the circle
are those that serve over 1% of the population of the country. In the case of
Ukraine, you see a large 'gap' in the left side of circle (roughly 55%). This
is due to the fact that more than half of the networks serving end-users in
Ukraine serve less than 1% of the population.
For the networks that we have data for, you can see that there are many options
between the networks, we either see them directly interconnecting, or
interconnection is mediated by one of the many Internet Exchange Points (IXPs)
in Ukraine (depicted by orange circles in Figure 1). Note that we don't
directly observe resiliency, because the paths that we measure are the active
paths, and due to how Internet routing works, we can't see backup paths until
they are activated.
Resiliency in IXPs
Zooming in on the IXPs, we see 19 IXPs in Ukraine listed in PeeringDB:
Figure 2: IXPs in Ukraine (source PeeringDB)
If we measure the path between all networks that we have RIPE Atlas probes in,
we see 13 IXPs in these paths. This indicates there are a lot of IXPs active,
which facilitates resilient interconnection. We visualise the paths between
probes in Figure 3. Each cell here represents a path between RIPE Atlas probes
in Ukraine, and you can see that the majority of these paths are mediated by
IXPs (the total of coloured cells). It's also striking how many different IXPs
are used, indicating that there is not a single dominant IXP.
Probes are ordered north (top left of Figure 3) to south (bottom right).
Figure 3: IXPs between RIPE Atlas probes in Ukraine
Market concentration at transit networks
If we move our focus to transit networks, we see similar patterns. For the
influence (directly and indirectly) of networks over particular countries, we
look at the AS Hegemony scores in the Internet Health Report. This data is
derived from RIPE RIS and Routeviews, and these hegemony scores indicate how
influential one network is over a part of the Internet. For instance how
influential a particular ISP is over the population of users in a particular
country. This is what that looks like for Ukraine:
Figure 4: Top 10 of dependencies for Ukranian end-users (3rd column)
We can also calculate this score for the influence of a particular ISP over the
networks in a particular country (the difference is that not all networks have
the same number of users, so in the first case the influence is weighted over
users, the second it's weighted over networks).
Figure 5: Top 10 of dependencies for Ukrainian networks (ASNs, last column)
If there are networks that Ukrainian networks depend on, these seem to be the
large international players, many of them so called "tier1" networks, and note
that Russian transit networks are absent.
Mobile vs Fixed Internet
The data we collect and see is mostly about the fixed Internet. Insights into
what happens with mobile Internet are a lot harder to obtain from the data we
collect in RIPE RIS and RIPE Atlas. The best public data we found on this is in
Cloudflare Radar. This data shows how, during the beginning of the war, the
traffic they received shifted towards more mobile use. This also explains the
slow but steady decline we see indicators of the fixed Internet (for instance
the number of connected RIPE Atlas probes). With 2 million people fleeing in a
country of 44 million people, it is expected that the fixed Internet is less
used, due to people shutting down their home and/or office Internet routers.
Figure 6: Cloudflare Radar data suggests shift away from fixed towards mobile
Internet
Figure 7: RIPE Atlas connected probes in Ukraine shows a slow but steady
decline since the invasion started
Other similar indicators of user activity see the same trend; e.g. Google
transparency report, Gatech IODA.
The Physical Layer
In the end, the Internet infrastructure in Ukraine is a man-made structure, at
the very core consisting of routers, a transport medium (fibre mostly) and
electricity to power it all. Loss of power or extensive damage to the
underlying fibres would mean no more physical connectivity; i.e. the Internet
stops working.
Cables
There is a really good explanation of the role physical fibres play in keeping
the Internet in Ukraine connected from Telegeography . What we see there also
suggests resilience on the physical layer. In short, there are diverse fibre
paths from multiple organisations providing services over these fibres.
That said, physical infrastructure is being damaged in specific parts of the
country. Even as that is happening, though, engineers on the ground are doing
incredible work to carry out repairs under profoundly difficult circumstances
(more on this below).
Power
Internet infrastructure needs electricity to function. With power going out in
Mariupol, we are hearing reports of the devastating effects this is having on
people in the city, which of course go well beyond loss of Internet
connectivity alone.
Focusing on the Internet here, though, we do indeed see that with this loss of
power, Internet connectivity appears to have been badly affected in the city.
An example of this is the local provider CDS, who operate out of Mariupol. This
ISP, which represents 0.3% of all users in Ukraine, went dark on the night of
the 28 February (see Figure 8). Again, despite the resilience of the Ukrainian
Internet, we are seeing damage as a result of the war.
Figure 8: Gatech IODA showing signals for CDS (AS43554)
Humans
There are other indications of the determination of network operators in
Ukraine to keep the Internet running. We see outages, but also quick recovery
of networks, in what can only be obscenely difficult circumstances. We reported
on the sabotage of Triolan earlier. Another example of this is Ukrtelekom,
which serves around 5% of the Ukrainian population. They had an outage on the
morning of 8 March, but this only lasted for 3 hours thanks to the efforts of
network operators.
Figure 9: Gatech IODA showing signals for Ukrtelekom (AS6849)
According to reports that we have received from the ground, network operators
from different providers are teaming up to help each other - for example by
repairing fibres - so as to make sure that the people relying on the Internet
are able to keep on communicating with one another.
An engineer in Kyiv working on repairing a fibre (shared on LinkedIn)
(Image source)
Conclusion
We hope to have provided some insights as to why the Ukrainian Internet is
remaining resilient under these dire circumstances. Due to low market
concentration at multiple levels and the relatively high number of interconnect
facilities, there are no obvious chokepoints, or individual networks whose loss
would have a crippling effect on the Internet in Ukraine. Qrator resilience
ratings suggest the same.
But there is a break-point for all infrastructure. For instance, if power goes
out, or if there are large scale physical disruptions, these likely will have
significant effects on the Internet in Ukraine. As we receive more information
about power shortages, this can have an effect on the Internet in Ukraine as a
whole.
For more background on the Ukrainian Internet we refer to Alex Semenyaka's
presentation at the Kyiv Internet days in 2019.
Again, our thoughts are with the Ukrainian people, and we continue to hope for
a peaceful resolution of this conflict soon. We encourage anyone who knows of
specific efforts to help Ukrainian network operators to leave further
information on those efforts in the comments below.
_______________________________________________
nexa mailing list
nexa@server-nexa.polito.it
https://server-nexa.polito.it/cgi-bin/mailman/listinfo/nexa
