On Sun, Dec 20, 2015 at 9:15 PM, <[email protected]> wrote: > > Hello, > > We're using msys2 (from 2015.05.12) as part of a front-end for a > build system. > > With the original msys, we experienced crashes in the automatic > conversion of the command line from Windows to POSIX-style > arguments. Using MSYS2_ARG_CONV_EXCL="*" with msys2 to disable this > automatic conversion has allowed us to make steady progress for > seven months, but recently the command tail passed to the build > system has changed, and we're again seeing this crash issue. >
Do you mean that you are seeing arguments being converted from POSIX form to Windows form? I implemented the ="*" thing and never used it, so it's quite likely that some changes outside it have caused it to no longer work, however MSYS2_ARG_CONV_EXCL itself is used extensively. I'm keen for us not to crash here so if you have a simple testcase I'd appreciate it. To be honest, if the ground has shifted around arg_heuristic_with_exclusions so that things that once held true no long do, it should be easy to track down and fix. > The crash manifests itself as 'Bad Address' when attempting to launch > a program from the build process. This 'Bad Address' is a SEGV. > This crash is somewhat, but not entirely, related to the command > line that is used. The crash happens while processing a bash > command executed from a Makefile. > > Examination of arg_heuristic_with_exclusions() and its use in > child_info_spawn::worker() yields strong suspicions -- for example, > modifying the data returned from getenv(). > > Modifying the sources that we are currently using would be ideal, > but they are controlled by another group. I needed to install the > source locally to investigate and test. However, using the newest > code that's committed yields a different problem: > > Several programs are reporting that they are not receiving their > command line arguments. > > For example, 'mkdir' reports 'missing operand'. However, as you > might expect, manually executing the Makefile command that failed > always succeeds. Given that code works in the code from 2015.05.12, > and that it requires the full infrastructure of the build process to > fail, I suspect other memory corruption issues. > > Interestingly, by pre-populating the build output directories using > the msys2 tools from 2015.05.12, I can reproduce the bad address > failure with the newest code that I just installed. I have made > modifications to the newest code and I have been able to remove theB > 'Bad Address' problem (I think), but I'm now stuck with the > situation where programs that are run do not seem to be passed the > command line. There's a CYGWIN env var that we renamed to MSYS and it has an option (wincmdln) for whether to fill in the Windows process command lines or not, and by default this is set to false, so I'm not sure if you are talking about that here or not. i.e. by default, Cygwin and MSYS2 processes don't show any command lines in the Windows Task Manager. https://cygwin.com/cygwin-ug-net/using-cygwinenv.html. I guess you probably don't mean the Windows process' view of the command line though. > > I'd like to get this resolved, but being new to looking at this > code, I have a few issues. I'm hoping you folks can point me to > documentation, or better workflows: > > o How do I efficiently build? Get a 20+ core workstation, with as much memory as possible and run everything from an ImDisk Virtual Disk. Ok, other ideas .. for many packages you can get away with using the fact that package () calls make install, and "make install" depends on "make all" and "pacman -R" (repackage) avoids re-extracting, rebuilding etc etc .. but, unfortunately for msys2-runtime, this isn't possible (it's probably not impossible either, it's just not setup that way for some reason). It seems at the bottom of the msys2-runtime/PKGBUILD I added some instructions for myself for just this kind of situation, for i686, modify for x86_64 and fix the paths. With no msys-2.0.dll-linked-to programs or shells running: open cmd.exe set "PATH=C:\\msys32\\usr\\bin;%PATH%" E: pushd E:\m2\repo-MSYS2\msys2-runtime\src\build-i686-pc-msys\i686-pc-msys\winsup\cygwin C:/msys32/usr/bin/bash -c "LANG=C && make" copy /y new-msys-2.0.dll C:\msys32\usr\bin\msys-2.0.dll popd C: C:/msys32/usr/bin/strace ls / > C:/strace.txt 2>&1 > > Right now, I'm just modifying msys2-runtime. I can modify the > source code, and run 'makepkg -sLf', but this fetches the current > sources from git, overwriting my changes. > > For now, I put my changes back while all the ./configure work is > being done, but this is really cumbersome and error prone. (It's > worked so far, but I don't trust it.) > > What's the best way to build without having my changes overwritten > with each build? > To make sure you aren't losing your changes when you're hacking on it and building a package the normal way, since this package is based on a git repository it's quite easy to get into the flow of using git to generate patches and adding them to the PKGBUILD and iterating on that. Here's how I do it: .. assuming you're running an msys2 shell, in the MSYS2-packages/msys2-runtime folder: pushd src/msys2-runtime nano winsup/cygwin/path.cc <make some changes and exit> # If this is a brand new patch then: git commit -a -m "path.cc: Attempted fix for segfault" git format-patch -6 # <5+1> # where 5 is the amount of patches there were before this new one <now add the new patch to the PKGBUILD in two places, source array and in the git -am part of perpare() > # Else if this is modification to a patch you're iterating on then: git add path.cc git commit --amend # EndIf git format-patch -6 mv *.patch ../.. popd updpkgsums git add *.patch PKGBUILD > o What is the preferred mechanism for contributing patches? > Fork https://github.com/Alexpux/MSYS2-packages or https://github.com/Alexpux/MINGW-packages and make pull requests. > o Where is the source to 'mkdir'? > It is part of msys2/coreutils. To track down the MSYS2 package that any MSYS2 file came from you can use pkgfile: $ pacman -S pkgfile $ pkgfile --update $ pkgfile $(which mkdir.exe) # using ther correct extension is important! msys/coreutils .. then inspecting MSYS2-packages/PKGBUILD yeilds: pkgname=coreutils pkgver=8.24 source=(ftp://ftp.gnu.org/gnu/${pkgname}/${pkgname}-${pkgver}.tar.xz{,.sig} > I'd like to modify it to output the full command like that is > actually passed. > > o How can I see debug_printf output? > > Is there a better way to see debug_printf() than running strace? > No. I guess you read https://github.com/Alexpux/Cygwin/blob/msys2-master/winsup/cygwin/how-to-debug-cygwin.txt ? > This is important, because the tools are being run from inside a > larger piece of code and needing to run everything under strace > will be very hard. > You could hack it up to behave differently, to write out to a huge area of memory instead or something like that? Throw enough memory at the problem! > o How can I get & modify the sources from 2015.05.12? It's going to be a little bit fiddly to be honest. You're going to have to clone msys2-runtime (ok no problem) and also: https://github.com/Alexpux/Cygwin.git (msys2-master branch) then run "git log ." in the msys2-runtime folder and redirect the output to a file. This folder has changed about 20 times since then. At each of those date points, you need to figure out where the HEAD was of msys2-master of Cygwin.git with a bit of git-fu (it may not have changed very often), then you need to do a bisect by hand probably: git checkout sha1_from_bisected-MSYS2-packages -- msys2-runtime/ .. then change the PKGBUILD from source=('msys2-runtime'::'git+https://github.com/Alexpux/Cygwin.git#branch=msys2-master' .. to source=('msys2-runtime'::'git+https://github.com/Alexpux/Cygwin.git#branch=sha1_you_figured_out_where_the_HEAD_was-of_msys2-master-at-that-bisected-date' .. but really, if you have a good hunch about some suspect looking code in arg_heuristic_with_exclusions(), child_info_spawn::worker() and getenv() then that sounds like a much saner approach. I'll try to take a look at it some time tomorrow if I get a chance to see what's going on. Anyway, I hope this has been useful to some extent. Cheers, Ray. > > At least that version had only one known issue -- the 'Bad > Address' and not two (losing arguments from invoked programs). If > I can modify that code, I have better confidence that the memory > corruption issue we see is resolved, and won't have to immediately > figure out why other issues are cropping up. > > o How can I easily see the logs of what's changed in the past 7 > months? > > thanks, > thutt > -- > > ------------------------------------------------------------------------------ > _______________________________________________ > Msys2-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/msys2-users ------------------------------------------------------------------------------ _______________________________________________ Msys2-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/msys2-users
