Hi Qian, Good work, many thanks. Comments inline.
On Fri, Aug 7, 2015 at 2:30 AM, Qian Hong <fract...@gmail.com> wrote: > Dear MSYS2 developers and users, > > > I've recently do some experiment with MSYS2 on Appveyor CI. > I created a pull request for review: > https://github.com/Alexpux/MSYS2-packages/pull/319 > > It's an early draft, so I'd like to explain several details and ask > for feedback here. > > *** > > Summary: > > Currently, the Appveyor CI scripts in PR 319 works as below: > > - When a new commit or new pull request touch one subdirectory of > MSYS2-packages, the buildbot will compile the PKGBUILD in that > directory. > > For example, if msys2-runtime/PKGBUILD is touched, the buildbot will > cd to msys2-runtime first, and then run `makepkg -s -f --noconfirm > --skippgpcheck` > > If more than one subdirectory is touched in one commit, only the first > subdirectory will be build. Appveyor provides 40min of build time by > default, it is impossible to build too many packages. > > - When the appveyor.yml file itself is touched, or any other top level > file is touched, the buildbot will build ./appveyor/PKGBUILD, this is > some kind of "self test", to make sure the appveyor.yml script and > those wrapper scripts are not broke. > > Related source code for automatically decide which subdirectory to > build is in appveyor/build.sh > > https://github.com/fracting/MSYS2-packages/blob/2cb3291d6ae0e2db0070729bbdee30287ccb9c79/appveyor/build.sh#L16 > > *** > > Internal details: > > 1. The script firstly use choco to download some helper tools like > axel and 7zip, then download and extract msys2 tar ball, install > base-devel / msys2-devel / git and initialize git. > > Related source code is in appveyor/msys2-prepare.cmd > > https://github.com/fracting/MSYS2-packages/blob/2cb3291d6ae0e2db0070729bbdee30287ccb9c79/appveyor/msys2-prepare.cmd#L2 > > To simplify the process, one idea is to contribute a msys2 package to > the chocolatey repo, so we can simply install msys2 in one command > line. It seems no one done yet, if no one want to do that I'll take a > look but not soon. See the comment from "Kitty" on http://www.hanselman.com/blog/AptGetForWindowsOneGetAndChocolateyOnWindows10.aspx This is the danger of binary repackaging. We shouldn't promote malware or systems that deliver it. I would rather wait until AppVeyor adds MSYS2 support than add MSYS2 to chocolatey. > > 2. After installation and initialization, `appveyor\msys2-wrapper.cmd > appveyor\build.sh` is called, which is the complex part. > > It is complex mostly because we have to work around > http://help.appveyor.com/discussions/problems/912-problem-building-mono-with-cygwin-inputoutput-redirection > > quote: > ``` > I keep on getting "bad file descriptor" errors during > configuration/build and it looks like it is related to Cygwin trying > to use exec to redirect I/O. I have tested this independently on an > Azure Windows Server 2012 VM so it is looking to me as though it may > be related to Appveyor configuration? > ``` > It appears that the stdin stream file descriptor is not (available) as > needed by autotools/configure. > ``` > > To work around the above issue, I created appveyor\msys2-wrapper.cmd. > appveyor\msys2-wrapper.cmd starts a separate CMD window, which has > normal stdin stream; then it call msys2 /usr/bin/script in the new CMD > window, which call `appveyor/build.sh` in msys2 bash, and collect all > output to a typescript file; finally the wrapper dump the typescript > file to the original CMD console in Appveyor. > > During the above process, exit status is handled carefully. > > Source code is in appveyor/msys2-wrapper.cmd > https://github.com/fracting/MSYS2-packages/blob/2cb3291d6ae0e2db0070729bbdee30287ccb9c79/appveyor/msys2-wrapper.cmd > > 3. While the wrapper scripts are complex, the appveyor.yml script > itself is pretty simple, just 10 lines: > > https://github.com/fracting/MSYS2-packages/blob/2cb3291d6ae0e2db0070729bbdee30287ccb9c79/appveyor.yml > > My idea is packaging the wrapper scripts and hosting them in some > website, then this solution can be easily shared to more users. Normal > user should only need a few lines like below in their appveyor.yml: > > install: > - command_to_download_and_install_msys2 > - command_to_download_and_install_wrapper_scripts > > build_script: > - msys2-wrapper.cmd path/to/build.sh > # or - msys2-wrapper.cmd -c "makepkg -f -s --skippgpcheck --noconfirm" > > > *** > > Test result: > > According to my limited tests so far, the Appveyor scripts works as > expected, reports PASS when build passes, reports FAIL when build > fails: > > make: compiles fine but tests failed > https://ci.appveyor.com/project/fracting/msys2-packages/build/1.0.48 > > python2: compiles fine but tests failed > https://ci.appveyor.com/project/fracting/msys2-packages/build/1.0.47 > > zlib: compiles fine and tests passed > https://ci.appveyor.com/project/fracting/msys2-packages/build/1.0.46 > > git: failed to compile test suite > https://ci.appveyor.com/project/fracting/msys2-packages/build/1.0.45 > > *** > > Pull request - > > It seems Appveyor has a plan to support MSYS2 officially, that would > be great for us. Before official supported, I think my solution can be > treat as a temporary workaround. Personally I need an Appveyor build > slave to compile and record as many as packages in MSYS2-packages, so > I can compare the build & test result with my Travis CI Wine test > result: > https://github.com/fracting/wine-fracting/wiki/MSYS2-on-Wine#build-status > > I admit my solution is not simple, but I'm glad to subscribe the build > result and respond to any failure caused by the build scripts in > Appveyor. Once Appveyor officially supports MSYS2, I'm glad to work on > migrating my old Appveyor scripts to a new solution officially > recommended by Appveyor. > > Is there any chance this solution could be accept to Alexpux's tree, > at least for some experiment? If it doesn't work we can revert > anytime. Sure! Make a pull request to MSYS2-packages, perhaps in a folder called continuous-integration/AppVeyor > > If this could be considered, I'll start to invest build 64bit msys2 > packages and 32/64 bit MinGW packages in Appveyor, those would be > pretty easy base on the current solution. > > Is there any further feature you like? Is there anything you don't > like? I'm open to any suggestions, thank you! Some suggestions: 1. Ideally we need separate reports from each of the 4 different variants, {x86_64, i686} * {MINGW-packages, MSYS2-packages}. Of course for MSYS2 packages you'd need to run a i686 MSYS2 and a x86_64 MSYS2. For the MINGW-packages, you'd need to run the build from x86_64 MSYS2 twice, once as "MINGW_INSTALLS=mingw32 makepkg-mingw ...", then as "MINGW_INSTALLS=mingw64 makepkg-mingw ..." 2. If we can identify the following issues distinctly that would be awesome: 2.1. Unclean patches (grep the log for things such as "patch unexpectedly ends in middle of line", "Hunk #1 succeeded at 2560 with fuzz 1.") 2.2. Patch failures. 2.3. Build failures separate to check failures. For 2.3, you can do a build with --nocheck, but I'm not sure if it's possible to then run just check without rebuilding! Hopefully there's at least a hacky way to do this. 3. Saving the logs somewhere would also be useful for users who encounter build failures could compare theirs logs against. I don't know if this is possible. ATM I'm quite busy so these are just some quick notes, -- Best regards, Ray. > > -- > Regards, > Qian Hong > > - > http://www.winehq.org > > ------------------------------------------------------------------------------ > _______________________________________________ > Msys2-users mailing list > Msys2-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/msys2-users ------------------------------------------------------------------------------ _______________________________________________ Msys2-users mailing list Msys2-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/msys2-users
