On Sun, Aug 14, 2011 at 10:46 AM, Paul Wise <p...@debian.org> wrote: > [...]
To possibly clarify, the current development process is: 1) Artists (by which I mean audio too) create art in various programs and save in various formats, with proprietary source formats (.max (3ds Max), .psd (Photoshop), .psp (Paint Shop Pro), whatever presumably expensive systems musicians use, etc) and open source (.blend (Blender), .wav (for sound effects), etc), or sometimes only save in 'flattened' formats like .png (particularly if they're modifying someone else's art and don't have whatever program the original source format was for). 2) Sometimes they put those original source formats on some FTP site or some separate SVN repository, but not in a reliable or organised way, and sometimes they lose them entirely. 3) They export to standard formats (.dae for 3D models and animations, .png for textures, .ogg for audio; some old content uses .dds for textures and the game engine's custom binary model/animation formats (.pmd/.psa)) and put those exported files in the main SVN repository. 4) Some files in SVN are generated by running some tools (also in SVN) on other files in SVN, then manually committing the output when necessary. (In particular, constructing the font textures - I don't remember any others.) 5) The game executable can be run in a mode that collects all the data files from an SVN checkout, does some processing and compression and conversion into more efficient file formats (.dae -> custom binary formats (.pmd/.psa), .xml -> custom binary format (.xmb), .png (or any other texture) -> .dds), and sticks them all into a single .zip file. (If the game is run without first processing the data files and packing them into a zip, e.g. if you run it directly from an SVN checkout, it will do the same processing work at runtime when it loads each individual file, so it works transparently (except for delays of tens of seconds while it converts the files) - the zip is merely an optimistion.) 6) When creating 0ad-*-unix-data.tar.xz files for releases, we run the executable to pack all of binaries/data/mods/public/ (from SVN) into public.zip and only include public.zip in the release tarball. 'Building' the package from that tarball just involves copying files. It sounds like you're asking for: 1b) Artists create art only in DFSG-free applications and save in the original lossless editable formats and put those files in the main SVN repository. 2b) There is a build script that can automatically convert all those files into formats the engine can load (which probably involves invoking Blender and Gimp and Audacity etc from the command line to export from their formats). 3b) (Maybe we have some system that uploads the converted output into SVN so developers can develop without the huge pain and slowness of having to run all those conversions themselves.) 4b) We release a tarball with those original source files and the build script, then the distro package scripts run the build script (eventually producing a public.zip the game can load, like in the current system). I think the fundamental problem is that 1b is never going to happen - we'd have to throw out all our art and half our artists and start again with open source tools, since very little is currently done in that way. Using the new process only for a portion of the art doesn't seem useful - either it's necessary for all art, or else it's not necessary and there's no point spending time setting up a new pipeline for only some of it. (I'm not claiming the current process is ideal - I believe we ought to be much more organised about storing original source files for art, and ought to converge on fewer applications for creating it, since that would make development smoother, but that'll be a gradual change if it happens at all, and doesn't seem relevant from a DFSG-compliance perspective.) Am I understanding this correctly? Is the problem fatal? It would certainly be technically possible to change the process to get rid of step 4 and to make step 6 build public.zip as part of the distro package build script, using a new release tarball that has the files directly from SVN (as opposed to the current one that only contains the pregenerated public.zip), though it would introduce a (slightly circular) build-time dependency on the game executable package (which contains the code to generate the .zip) and I don't know if the change would really solve any problems. > There is a ZIP file in there too. It seems a bit rediculous to put a > compressed file inside the already compressed tarball and even more > rediculous to put such a file in a VCS (didn't look if it was there > too). Assuming you mean public.zip, it's an uncompressed zip file (to avoid interfering with tarball compression). It's not in the VCS - it's generated by step 6 above. > Also the ./public/art/LICENSE.txt file indicates that all the > materials from CGTextures are missing their source form. For most of the textures we can't usefully distribute the original source version since it's either lost or in a format like .psd; for the ones using CGTextures materials we couldn't distribute them in any case since CGTextures doesn't allow distribution of their work in unmodified form (but we got explicit permission to distribute the derived bitmap textures as CC-BY-SA). > I find it completely bizarre that they are using an Excel spreadsheet > to store techtree info. Does the game really require a Excel file > reading code?? That file is just documentation. Not sure what it's doing in SVN there - should delete it some time (probably when implementing techtrees and verifying that it's obsolete). > There are two RAR files. Those aren't used and should be deleted some time (after verifying they don't contain anything useful). > Those xmb files look like binary versions of the corresponding xml > files. Yep, via step 6. > There are some weird binary formats. One of them contained a string: > "God Knows", perhaps that should be replaced with "God Knows WTF these > files are and how to modify them" ;) I think those would be the .pmd/.psa files, which are generated either by step 6 or step 3 (exported directly to the custom binary formats). > I wonder about the DDS files. They can be opened in GIMP with gimp-dds > but their names seem to indicate that there are TGA files that are the > source form of them. Indeed, they're from step 6 (conversions of the texture files in SVN that are .tga or .png or .dds). > IMO the DDS files should be created at > runtime/build time from their source TGA files, or the game should > simply switch to using the TGA files. We need mipmapping, which is only really supported by DDS. Also we want S3TC compression (saves >75% video memory) which needs DDS. Creating the compressed DDS at runtime can take a couple of seconds per texture - we do that when running the game from an SVN checkout, but that's not good for normal players. So we do do it at build time (but in the build step before creating the tarball). > There are a bunch of pre-encoded Ogg files. I don't know what is the > source but Ogg definitely is not it. I think most audio files we only have in .ogg format, but we're gradually replacing them and trying to be more organised with storing .wav versions of the replacements. But storing .wav in the main SVN repository sounds non-ideal (the music alone will add a few hundred megabytes to the download), so we're currently sticking them in a separate parallel SVN repository and ought to find a better way to handle them eventually. -- Philip Taylor exc...@gmail.com -- To UNSUBSCRIBE, email to debian-wnpp-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cafakgupx3ob3br6798wqqukoagdczuoqucequvb50yiopq+...@mail.gmail.com
