On 03/08/2018 11:33 AM, Sid Hayn wrote: > https://github.com/EttusResearch/uhddev/blob/master/images/manifest.txt > > This address is invalid, assuming you meant this one? > > https://github.com/EttusResearch/uhd/blob/master/images/manifest.txt
Yes! Thanks for pointing that out. -- M > > Thanks, > Zero > > On Wed, Mar 7, 2018 at 8:12 PM, Martin Braun <martin.br...@ettus.com > <mailto:martin.br...@ettus.com>> wrote: > > Hi all, > > we've had a bunch of great discussions on the packaging of FPGA images > (binaries) and on packaging UHD in general. Since there were many > different discussions, I'd like to summarize them all in one big email. > Thanks to Rick, Maitland, and Phil for providing feedback. > > If you just use UHD via git, none of this affects you. > > ## Release tags > > Going forward, we'll change the format of the release tags. Before, it > was release_003_011_000_000 and so on. Now, it'll be v3.11.0.0, which is > a much more standard format. Personally, I'm happy to make this change > -- we were hesitant to change it for consistency's sake, but it seems no > one actually cares about the old format anyway. If you're pulling from > github, this means you get nice, reproducible URLs like this: > https://github.com/EttusResearch/uhd/releases/tag/v3.11.0.0 > <https://github.com/EttusResearch/uhd/releases/tag/v3.11.0.0> > > ...and the tarball for the sources: > https://github.com/EttusResearch/uhd/archive/v3.11.0.0.tar.gz > <https://github.com/EttusResearch/uhd/archive/v3.11.0.0.tar.gz> > > ...and so on. I'd be interested to see who cares about the old tag > format. For now, we'll double-tag (and we might re-tag old versions, > too, if there's interest) to keep things easy. > > ## Getting FPGA images for releases > > We have two paths here. First, we have the uhd_images_downloader format > (I'll talk more about that below). You can grab images from the > images/manifest.txt file, but I understand that's cumbersome. > > So, on top of the uhd_images_downloader format, we're considering > bringing back an archive per release, but we're still figuring out the > details of this. > > We have for the sake of feedback uploaded an archive here: > https://github.com/EttusResearch/uhd/releases/tag/v3.11.0.0 > <https://github.com/EttusResearch/uhd/releases/tag/v3.11.0.0> > > I'd love to know if this format is suitable for people, and if things > are missing. If this is all everyone needs, we'll find a way to automate > this and make it error-proof. > > However, this means that you will be downloading *all* default images > (currently, that's over 200 MB after unpacking), some of which rarely or > never change (e.g., the USRP1 image will always be the same). > If you want to tailor the images you download (e.g., you might only care > about the embedded images), pulling the individual packages might be a > better option. The problem is, we don't have an easy way of identifying > which ones there are without requiring to run a Python script at package > build time. > Something like this works, if you have access to the UHD source code > (bash code): > > SRC_URI=http://files.ettus.com/binaries/cache/`cat > <http://files.ettus.com/binaries/cache/`cat> images/manifest.txt | > grep -v '#' | awk -F' ' '{print $3}' | grep b200mini` > > ...and it would limit the images to the b200mini images, but it's not > exactly handy. If people have suggestions, I'd be curious to hear them. > > ## FPGA Versions > > There was a question why we don't version our FPGA images (e.g., why > does the X300 image not have version 1.2.3, and we say this version > works with UHD 3.11.0.0). There are several problems with this: First, > we have over 40 FPGA images (not all are packaged in the default image). > Versioning them all correctly is already a hard logistical issue. > Second, how do we create meaningful version numbers? The FPGA images > consist of many subcomponents, all of which would need to go into an > image version. The X300 and X310 images, for example, should always have > similar versions when they have similar behaviour. But what if we fix > something only for X300? Then, both the X310 and X300 images have 5 > variants (XG, HG, XA, HA, and AA). What if we fix a 10GigE issue? How do > we keep the 10 version numbers for the {X300, X310} X {HG, XG, HA, AA, > XA} in sync? Then of course, we have components on all X-Series that we > also use on the E310. So that makes the version-number-space even more > higher-dimensional. And so on. > Finally, we consider the FPGA source code part of UHD. That means, for > version 3.11.0.0 of UHD, there is exactly 1 commit on the FPGA > repository that matches the source code. The images built off of this > commit are the ones that matter, and so they already have a version (the > UHD version, in this case, 3.11.0.0). > For any given UHD commit, the correct set of FPGA images is the one > tracked in images/manifest.txt. > > ## The new images downloader > > If you're used to using uhd_images_downloader, nothing changes for you. > Here's the contents of an older email explaining the motivation and > changes in greater detail. > Note that we'll start using this tool to distribute even more things > going forward. We have a bunch of things where building binaries is > getting more and more complicated for end users, and we'll try and make > it easy to update devices. > > > ``` > Subject: [UHD] Changes to uhd_images_downloader > Date: Tue, 23 Jan 2018 15:33:14 -0800 > From: Martin Braun <martin.br...@ettus.com > <mailto:martin.br...@ettus.com>> > > For the next UHD version, we will be updating uhd_images_downloader. The > problem with the old uhd_images_downloader is that it refers to one big > zip file for all the images. As we add more and more devices, we not > only have to update the zip file more often, it also becomes bigger and > bigger. If you're running uhd_images_downloader on an E310, it's most > likely that you only want the E310 images, and not the megabytes of > other FPGA images. > > The new tool fixes all those issues by breaking up the zip file into > more, smaller zip files. On top of that, the new tool has a bunch of > other improvements: > > - It maintains an inventory of downloaded files, and skips downloading > them again if they didn't change. This means if we roll out a fix for > the X300 only, and you run uhd_images_downloader without any arguments, > it won't try and update anything else. Should you delete the inventory, > it'll simply get everything, like it did before. > - You can now filter by type. Use the -t switch to match against a > regular expression, e.g. '-t b200mini' will only download a b200mini > file. > - We'll be adding more image types soon that are not FPGA images, such > as filesystem updates. > - 'uhd_images_downloader -l' will list all the targets you can download. > > In general, this should improve things and make other things easier. > However, since we had to basically rewrite the tool, which had otherwise > matured, it's likely that the new tool has shortcomings. We'd like to > hear from you regarding those. > > There are a few problems we are already tracking: > - We currently have no archives other than zip files. We're intending to > add .tar.xz in the future, and you will be able to use > uhd_images_downloader to fetch those (assuming your system can unzip xz > archives) > - The tool output changed, and we're still fine tuning the output. > > > If you're a package maintainer, you probably hard-coded the URL for the > FPGA images in your package creation script. > We track the URLs of the individual archives, along with their SHA256 > hash, in a manifest that is checked into git: > https://github.com/EttusResearch/uhddev/blob/master/images/manifest.txt > <https://github.com/EttusResearch/uhddev/blob/master/images/manifest.txt> > > The base URL for all those is http://files.ettus.com/binaries/cache > <http://files.ettus.com/binaries/cache>, so > > http://files.ettus.com/binaries/cache/usrp1/fpga-6bea23d/usrp1_usrp1_fpga_default.zip > > <http://files.ettus.com/binaries/cache/usrp1/fpga-6bea23d/usrp1_usrp1_fpga_default.zip> > will fetch the USRP1 FPGA images. > > We're happy for any feedback -- ideally before we release the next > version. > > Cheers, > Martin > ``` > > > > Hopefully this is helpful. Again, I'd like to thank everyone for > providing feedback. > > Cheers, > Martin > > > > _______________________________________________ > Discuss-gnuradio mailing list > Discuss-gnuradio@gnu.org <mailto:Discuss-gnuradio@gnu.org> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio > <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio> > > _______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
