On Mon, Dec 21, 2015 at 2:17 AM, <[email protected]> wrote:
> Ray Donnelly writes:
> > On Sun, Dec 20, 2015 at 9:15 PM, <[email protected]> wrote:
> > >
> > > Hello,
> > >
> > > We're using msys2 (from 2015.05.12) as part of a front-end for a
> > > build system.
> > >
> > > With the original msys, we experienced crashes in the automatic
> > > conversion of the command line from Windows to POSIX-style
> > > arguments. Using MSYS2_ARG_CONV_EXCL="*" with msys2 to disable this
> > > automatic conversion has allowed us to make steady progress for
> > > seven months, but recently the command tail passed to the build
> > > system has changed, and we're again seeing this crash issue.
> > >
> >
> > Do you mean that you are seeing arguments being converted from POSIX
> > form to Windows form? I implemented the ="*" thing and never used it,
> > so it's quite likely that some changes outside it have caused it to no
> > longer work, however MSYS2_ARG_CONV_EXCL itself is used extensively.
> > I'm keen for us not to crash here so if you have a simple testcase I'd
> > appreciate it. To be honest, if the ground has shifted around
> > arg_heuristic_with_exclusions so that things that once held true no
> > long do, it should be easy to track down and fix.
>
> I do not have a simple test case that I can release; we have not
> pinned down exactly what the crash is, but have some high level
> commands that do reproduce faithfully. Narrowing it down further
> hits a meta stable state -- further reductions work sometimes.
>
> We know exactly which command is failing, but have not been able to
> reproduce the crash by _only_ executing the command that ultimately
> crashes. This might not be corruption in Bash, but may be the version
> of Gnu Make? We are still investigating.
>
> In short, we have a tool that generates a sophisticated build process
> that uses Gnu Make, after the gnerated Makefile performs a
> significant amount of bookkeeping work, it attempts to start a
> subordinate build process. The subordinate process is written in
> Python, and the command line that's executed from within Make takes
> the form:
>
> program --option-0 value --option-1 value -- command-tail
>
> The command tail is many thousands of bytes long, contains things
> that look just like environment variable settings:
>
> V0="value" V1="value" V2="value value"
>
> Some of the values are pathnames with drive letters, but no '\', only
> '/'.
>
> >
>
> <snip>
>
> >
> > There's a CYGWIN env var that we renamed to MSYS and it has an
> > option (wincmdln) for whether to fill in the Windows process
> > command lines or not, and by default this is set to false, so I'm
> > not sure if you are talking about that here or not. i.e. by
> > default, Cygwin and MSYS2 processes don't show any command lines in
> > the Windows Task Manager.
> > https://cygwin.com/cygwin-ug-net/using-cygwinenv.html. I guess you
> > probably don't mean the Windows process' view of the command line
> > though.
>
> No, not talking about anything with Windows tools. Here's an
> example, for clarity; the Makefile executes:
>
> mkdir --parents <path>
>
> When run from the command line, that works reliably.
>
> When run from the infrastructure described above, using the
> 2015.05.12 version, it works reliably.
>
> When run from the infrastructure described above, with the msys64
> that I just installed, I see:
>
> mkdir: missing operand
>
> This is reproduced on the command line with invocations like:
>
> mkdir --parents
>
> or
>
> mkdir
>
> I conclude that 'something' is not executing mkdir properly. The
> Makefile is using the bash shell to execute the commands, so the
> natural assumption is that something in bash (or futher down in the
> posix/windows layer) is having issues. This behavior is with and
> without my changes.
>
> > >
> > > I'd like to get this resolved, but being new to looking at this
> > > code, I have a few issues. I'm hoping you folks can point me to
> > > documentation, or better workflows:
> > >
> > > o How do I efficiently build?
> >
> > Get a 20+ core workstation, with as much memory as possible and run
> > everything from an ImDisk Virtual Disk.
>
> Speed's not the issue; the issue is efficiency in my work.... you
> address some a bit below.
>
> >
> > Ok, other ideas .. for many packages you can get away with using the
> > fact that package () calls make install, and "make install" depends on
> > "make all" and "pacman -R" (repackage) avoids re-extracting,
> > rebuilding etc etc
> >
> > .. but, unfortunately for msys2-runtime, this isn't possible (it's
> > probably not impossible either, it's just not setup that way for some
> > reason).
>
> Drat.
Don't be put off, there's no reason that maybe an hour or two's
investigation and hacking in the msys2-runtime PKGBUILD wouldn't lead
to a solution modelled along the lines of:
build() {
..
make
}
package() {
..
make install
}
.. and then makepkg -RL would be useful for this package, and we'd
gladly accept such a patch if it didn't break anything too.
>
> > It seems at the bottom of the msys2-runtime/PKGBUILD I added some
> > instructions for myself for just this kind of situation, for i686,
> > modify for x86_64 and fix the paths. With no msys-2.0.dll-linked-to
> > programs or shells running:
>
> Below, what is your E: drive?
E: is just where I do my MSYS2 hacking on my laptop as it has a tiny
C: SSD drive. (i.e. E:\m2\repo-MSYS2 is perhaps your equivalent of
C:\msys64\home\thutt\MSYS2-packages)
>
> > open cmd.exe
> > set "PATH=C:\\msys32\\usr\\bin;%PATH%"
> > E:
> > pushd
> E:\m2\repo-MSYS2\msys2-runtime\src\build-i686-pc-msys\i686-pc-msys\winsup\cygwin
> > C:/msys32/usr/bin/bash -c "LANG=C && make"
> > copy /y new-msys-2.0.dll C:\msys32\usr\bin\msys-2.0.dll
> > popd
> > C:
> > C:/msys32/usr/bin/strace ls / > C:/strace.txt 2>&1
>
> Is the strace for getting new debug messages?
> Is there any way to get the debug messages without strace?
> > >
> > > Right now, I'm just modifying msys2-runtime. I can modify the
> > > source code, and run 'makepkg -sLf', but this fetches the current
> > > sources from git, overwriting my changes.
> > >
> > > For now, I put my changes back while all the ./configure work is
> > > being done, but this is really cumbersome and error prone. (It's
> > > worked so far, but I don't trust it.)
> > >
> > > What's the best way to build without having my changes overwritten
> > > with each build?
> > >
> >
> > To make sure you aren't losing your changes when you're hacking on it
> > and building a package the normal way, since this package is based on
> > a git repository it's quite easy to get into the flow of using git to
> > generate patches and adding them to the PKGBUILD and iterating on
> > that. Here's how I do it:
> > .. assuming you're running an msys2 shell, in the
> > MSYS2-packages/msys2-runtime folder:
> > pushd src/msys2-runtime
> > nano winsup/cygwin/path.cc
> > <make some changes and exit>
> > # If this is a brand new patch then:
> > git commit -a -m "path.cc: Attempted fix for segfault"
> > git format-patch -6 # <5+1> # where 5 is the amount of patches there
> > were before this new one
> > <now add the new patch to the PKGBUILD in two places, source array
> > and in the git -am part of perpare() >
> > # Else if this is modification to a patch you're iterating on then:
> > git add path.cc
> > git commit --amend
> > # EndIf
> > git format-patch -6
> > mv *.patch ../..
> > popd
> > updpkgsums
> > git add *.patch PKGBUILD
>
> This is the kind of information that I was hoping would appear.
> Thanks. (Saw your other message, too).
>
> >
> > > o What is the preferred mechanism for contributing patches?
> > >
> >
> > Fork https://github.com/Alexpux/MSYS2-packages or
> > https://github.com/Alexpux/MINGW-packages and make pull requests.
>
> Ok. Th
>
> >
> > > o Where is the source to 'mkdir'?
> > >
> >
>
> <snip>
>
> > It's going to be a little bit fiddly to be honest. You're going to
> > have to clone msys2-runtime (ok no problem) and also:
> > https://github.com/Alexpux/Cygwin.git (msys2-master branch)
> > then run "git log ." in the msys2-runtime folder and redirect the
> > output to a file. This folder has changed about 20 times since then.
> > At each of those date points, you need
> > to figure out where the HEAD was of msys2-master of Cygwin.git with a
> > bit of git-fu (it may not have changed very often), then you need to
> > do a bisect by hand probably:
>
> That's what I feared.
>
> > Anyway, I hope this has been useful to some extent.
>
> Yes, very much so. It gives me a lot of information to make faster
> progress.
>
> thutt
> --
------------------------------------------------------------------------------
_______________________________________________
Msys2-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/msys2-users
