Hi, A few months ago qemu.org hit network bandwidth limits and incurred costs for exceeding them. Since then we have implemented several changes to make continuous integration systems more bandwidth-efficient and reduce the biggest sources of traffic to qemu.org.
During the Mar-Apr billing cycle qemu.org still exceeded its network bandwidth limit but only by a small amount. Bandwidth consumption needs to be under ~6-7 TB/month. Below are the details of how we're doing. Thank you to Paolo Bonzini, Thomas Huth, Philippe Mathieu-Daudé, Daniel Berrangé, and everyone who helped with bandwidth reduction. The main change was a move to GitLab.com, which now serves the main QEMU git repository URLs. We also updated documentation and links to encourage people to use these new URLs. qemu.org bandwidth usage has been as follows: - Jan: 12.56 TB - Feb: 10.55 TB - Mar: 10.28 TB - Apr: 7.62 TB In May qemu.org has averaged 232.25 GB/day so far putting it on track for 7 TB total this month. The top 3 web traffic users are Google Cloud and Amazon Web Services IP addresses. This suggests that some continuous integration systems are still accessing qemu.org git repositories. It is unlikely that these are crawlers because User-Agent web stats show that crawlers only consume a few GB whereas the top three hosts consume 10s or 100s of GB each. Roughly 75% of traffic is git (https), 25% is tarball downloads, and the rest is wiki/web/miscellaneous traffic. Fun fact: qemu-4.2.0.tar.xz is the most popular download! I will send another update in 2 months so we can see where bandwidth usage finally settled. At that point we can decide whether more steps are necessary. Thanks, Stefan
