Re: [PATCH] Implement sched_[gs]etaffinity()
On Apr 10 21:06, Mark Geisert wrote:
> ---
> newlib/libc/include/sched.h | 4 +++
> winsup/cygwin/sched.cc | 68 +
> 2 files changed, 72 insertions(+)
>
> diff --git a/newlib/libc/include/sched.h b/newlib/libc/include/sched.h
> index 1016235bb..e3a5b97e5 100644
> --- a/newlib/libc/include/sched.h
> +++ b/newlib/libc/include/sched.h
> @@ -92,6 +92,10 @@ int sched_yield( void );
>
> #if __GNU_VISIBLE
> int sched_getcpu(void);
> +
> +typedef uint64_t cpu_set_t; /* ...until cpuset(7) exists */
> +int sched_getaffinity(pid_t, size_t, cpu_set_t *);
> +int sched_setaffinity(pid_t, size_t, const cpu_set_t *);
> #endif
>
> #ifdef __cplusplus
> diff --git a/winsup/cygwin/sched.cc b/winsup/cygwin/sched.cc
> index 10168e641..496e08857 100644
> --- a/winsup/cygwin/sched.cc
> +++ b/winsup/cygwin/sched.cc
> @@ -424,4 +424,72 @@ sched_getcpu ()
>return pnum.Group * __get_cpus_per_group () + pnum.Number;
> }
>
> +int
> +sched_getaffinity (pid_t pid, size_t cpusetsize, cpu_set_t *mask)
> +{
> + int status = 0;
> + HANDLE process = pid ? OpenProcess(PROCESS_QUERY_INFORMATION, FALSE, pid)
> + : GetCurrentProcess ();
This needs to grab the pinfo(pid) aund use p->dwProcessId, as you
noted on cygwin-developers already.
Two more notes:
- You could use GetCurrentProcess() in case of pid == myself->pid, too.
- PROCESS_QUERY_LIMITED_INFORMATION should be sufficent per MSDN.
PROCESS_QUERY_INFORMATION was required pre-Vista only.
Thanks,
Corinna
--
Corinna Vinschen
Cygwin Maintainer
signature.asc
Description: PGP signature
[PATCH] Cygwin: fork: remember child as late as possible
Otherwise, when the child does fail to reload dlls and terminates, we
produce a SIGCHILD signal, even if we did not succeed in starting up the
child process at all. Also, we would need to reap that child somewhere.
---
winsup/cygwin/fork.cc | 71 ---
1 file changed, 40 insertions(+), 31 deletions(-)
diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 74ee9acf4..a5b45f851 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -186,14 +186,14 @@ frok::child (volatile char * volatile here)
cygheap->fdtab.fixup_after_fork (hParent);
- /* If we haven't dynamically loaded any dlls, just signal the parent.
- Otherwise, tell the parent that we've loaded all the dlls
- and wait for the parent to fill in the loaded dlls' data/bss. */
- if (!load_dlls)
-sync_with_parent ("performed fork fixup", false);
- else
+ /* If we have dynamically loaded some dlls, we need anoter stop to
+ wait for the parent to fill in the loaded dll's data/bss. */
+ if (load_dlls)
sync_with_parent ("loaded dlls", true);
+ /* Signal the parent. */
+ sync_with_parent ("performed fork fixup", false);
+
init_console_handler (myself->ctty > 0);
ForceCloseHandle1 (fork_info->forker_finished, forker_finished);
@@ -420,20 +420,6 @@ frok::parent (volatile char * volatile stack_here)
child.hProcess = hchild;
ch.postfork (child);
- /* Hopefully, this will succeed. The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards. This requires more bookkeeping than I like, though,
- so we'll just do it the easy way. So, terminate any child process if
- we can't actually record the pid in the internal table. */
- if (!child.remember (false))
-{
- this_errno = EAGAIN;
-#ifdef DEBUGGING0
- error ("child remember failed");
-#endif
- goto cleanup;
-}
-
/* CHILD IS STOPPED */
debug_printf ("child is alive (but stopped)");
@@ -483,20 +469,20 @@ frok::parent (volatile char * volatile stack_here)
}
}
- /* Start thread, and then wait for it to reload dlls. */
- resume_child (forker_finished);
- if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
-{
- this_errno = EAGAIN;
- error ("died waiting for dll loading");
- goto cleanup;
-}
-
/* If DLLs were loaded in the parent, then the child has reloaded all
of them and is now waiting to have all of the individual data and
bss sections filled in. */
if (load_dlls)
{
+ /* Start the child up, and then wait for it to reload dlls. */
+ resume_child (forker_finished);
+ if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
+ {
+ this_errno = EAGAIN;
+ error ("died waiting for dll loading");
+ goto cleanup;
+ }
+
/* CHILD IS STOPPED */
/* write memory of reloaded dlls */
for (dll *d = dlls.istart (DLL_LOAD); d; d = dlls.inext ())
@@ -514,8 +500,31 @@ frok::parent (volatile char * volatile stack_here)
goto cleanup;
}
}
- /* Start the child up again. */
- resume_child (forker_finished);
+}
+
+ /* Hopefully, this will succeed. The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards. This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way. So, terminate any child process if
+ we can't actually record the pid in the internal table.
+ Note: child.remember () needs the subsequent WFMO in ch.sync (),
+ to perform the asynchronous start of the controlling threads. */
+ if (!child.remember (false))
+{
+ this_errno = EAGAIN;
+#ifdef DEBUGGING0
+ error ("child remember failed");
+#endif
+ goto cleanup;
+}
+
+ /* Start the child up, finally. */
+ resume_child (forker_finished);
+ if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
+{
+ this_errno = EAGAIN;
+ error ("died waiting for dll loading");
+ goto cleanup;
}
ForceCloseHandle (forker_finished);
--
2.19.2
[PATCH] Cygwin: use win pid+threadid for forkables dirname
Rather than newest last write time of all dlls loaded, use the forking
process' windows pid and windows thread id as directory name to create
the forkable hardlinks into. While this may create hardlinks more
often, it does avoid conflicts between dlls not having the newest last
write time.
---
winsup/cygwin/forkable.cc | 26 +++---
1 file changed, 7 insertions(+), 19 deletions(-)
diff --git a/winsup/cygwin/forkable.cc b/winsup/cygwin/forkable.cc
index d1b0f5723..4580610b1 100644
--- a/winsup/cygwin/forkable.cc
+++ b/winsup/cygwin/forkable.cc
@@ -340,30 +340,18 @@ exename (PWCHAR buf, ssize_t bufsize)
return format_IndexNumber (buf, bufsize, &d->fii.IndexNumber);
}
-/* Into buf if not NULL, write the newest dll's LastWriteTime.
+/* Into buf if not NULL, write the current Windows Thread Identifier.
Return the number of characters (that would be) written. */
static int
-lwtimename (PWCHAR buf, ssize_t bufsize)
+winthrname (PWCHAR buf, ssize_t bufsize)
{
if (!buf)
-return sizeof (LARGE_INTEGER) * 2;
- if (bufsize >= 0 && bufsize <= (int)sizeof (LARGE_INTEGER) * 2)
+return sizeof (DWORD) * 4;
+ if (bufsize >= 0 && bufsize <= (int)sizeof (DWORD) * 4)
return 0;
- LARGE_INTEGER newest = { 0 };
- /* Need by-handle-file-information for _all_ loaded dlls,
- as most recent ctime forms the hardlinks directory. */
- dll *d = &dlls.start;
- while ((d = d->next))
-{
- /* LastWriteTime more properly tells the last file-content modification
-time, because a newly created hardlink may have a different
-CreationTime compared to the original file. */
- if (d->fbi.LastWriteTime.QuadPart > newest.QuadPart)
- newest = d->fbi.LastWriteTime;
-}
-
- return __small_swprintf (buf, L"%016X", newest);
+ return __small_swprintf (buf, L"%08X%08X",
+ GetCurrentProcessId(), GetCurrentThreadId());
}
struct namepart {
@@ -382,7 +370,7 @@ forkable_nameparts[] = {
{ L"", sidname, true, true, },
{ L"", exename, false, false, },
{ MUTEXSEP,NULL, false, false, },
- { L"",lwtimename, true, true, },
+ { L"", winthrname, true, true, },
{ NULL, NULL },
};
--
2.19.2
[rebase PATCH] Introduce --no-rebase flag
The --no-rebase flag is to update the database for new files, without
performing a rebase. The file names provided should have been rebased
using the --oblivious flag just before.
---
rebase.c | 48 +++-
1 file changed, 35 insertions(+), 13 deletions(-)
diff --git a/rebase.c b/rebase.c
index 56537d6..6347c6c 100644
--- a/rebase.c
+++ b/rebase.c
@@ -69,9 +69,10 @@ WORD machine = IMAGE_FILE_MACHINE_I386;
ULONG64 image_base = 0;
ULONG64 low_addr;
BOOL down_flag = FALSE;
-BOOL image_info_flag = FALSE;
+BOOL image_info_flag = FALSE; /* implies --no-rebase, but without --database */
BOOL image_storage_flag = FALSE;
BOOL image_oblivious_flag = FALSE;
+BOOL perform_rebase_flag = TRUE;
BOOL force_rebase_flag = FALSE;
ULONG offset = 0;
int args_index = 0;
@@ -533,9 +534,9 @@ load_image_info ()
{
img_info_list[i].name = NULL;
/* Ensure that existing database entries are not touched when
-* --oblivious is active, even if they are out-of sync with
-* reality. */
- if (image_oblivious_flag)
+* --oblivious or --no-rebase is active, even if they are
+* out-of sync with reality. */
+ if (image_oblivious_flag || !perform_rebase_flag)
img_info_list[i].flag.cannot_rebase = 2;
}
/* Eventually read the strings. */
@@ -584,8 +585,8 @@ load_image_info ()
static BOOL
set_cannot_rebase (img_info_t *img)
{
- /* While --oblivious is active, cannot_rebase is set to 2 on loading
- * the database entries */
+ /* While --oblivious or --no-rebase is active, cannot_rebase
+ * is set to 2 on loading the database entries */
if (img->flag.cannot_rebase <= 1 )
{
int fd = open (img->name, O_WRONLY);
@@ -878,7 +879,7 @@ collect_image_info (const char *pathname)
/* Skip if not rebaseable, but only if we're collecting for rebasing,
not if we're collecting for printing only. */
- if (!image_info_flag && !is_rebaseable (pathname))
+ if (perform_rebase_flag && !is_rebaseable (pathname))
{
if (!quiet)
fprintf (stderr, "%s: skipped because not rebaseable\n", pathname);
@@ -928,8 +929,16 @@ collect_image_info (const char *pathname)
}
img_info_list[img_info_size].slot_size
= roundup2 (img_info_list[img_info_size].size, ALLOCATION_SLOT);
- img_info_list[img_info_size].flag.needs_rebasing = 1;
- img_info_list[img_info_size].flag.cannot_rebase = 0;
+ if (perform_rebase_flag)
+{
+ img_info_list[img_info_size].flag.needs_rebasing = 1;
+ img_info_list[img_info_size].flag.cannot_rebase = 0;
+}
+ else
+{
+ img_info_list[img_info_size].flag.needs_rebasing = 0;
+ img_info_list[img_info_size].flag.cannot_rebase = 2;
+}
/* This back and forth from POSIX to Win32 is a way to get a full path
more thoroughly. For instance, the difference between /bin and
/usr/bin will be eliminated. */
@@ -970,7 +979,9 @@ collect_image_info (const char *pathname)
}
#endif
if (verbose)
-fprintf (stderr, "rebasing %s because filename given on command line\n",
img_info_list[img_info_size].name);
+fprintf (stderr, "%s %s because filename given on command line\n",
+perform_rebase_flag ? "rebasing" : "considering",
+img_info_list[img_info_size].name);
++img_info_size;
return TRUE;
}
@@ -1173,6 +1184,7 @@ static struct option long_options[] = {
{"offset", required_argument, NULL, 'o'},
{"oblivious",no_argument, NULL, 'O'},
{"quiet",no_argument, NULL, 'q'},
+ {"no-rebase",no_argument, NULL, 'R'},
{"database", no_argument, NULL, 's'},
{"touch",no_argument, NULL, 't'},
{"filelist", required_argument, NULL, 'T'},
@@ -1182,7 +1194,7 @@ static struct option long_options[] = {
{NULL, no_argument, NULL, 0 }
};
-static const char *short_options = "48b:dhino:OqstT:vV";
+static const char *short_options = "48b:dhino:OqRstT:vV";
void
parse_args (int argc, char *argv[])
@@ -1212,6 +1224,7 @@ parse_args (int argc, char *argv[])
break;
case 'i':
image_info_flag = TRUE;
+ perform_rebase_flag = FALSE;
break;
case 'o':
offset = string_to_ulonglong (optarg);
@@ -1220,6 +1233,10 @@ parse_args (int argc, char *argv[])
case 'q':
quiet = TRUE;
break;
+ case 'R':
+ perform_rebase_flag = FALSE;
+ image_storage_flag = TRUE;
+ break;
case 'O':
image_oblivious_flag = TRUE;
/* -O implies -s, which in turn implies -d, so intentionally
@@ -1264,8 +1281,8 @@ parse_args (int argc, char *argv[])
}
}
- if ((image_base == 0 && !image_info_flag && !image_storage_flag)
- || (image_base && image_info_flag))
+ if ((image_base == 0 && perform_rebase_flag && !image_storage_flag)
+ || (force_rebase_flag && !perform_rebase_flag))
{
usage ();
Re: [PATCH RFC] fork: reduce chances for "address space is already occupied" errors
Hi Michael, On Apr 3 14:22, Corinna Vinschen wrote: > On Apr 3 11:18, Michael Haubenwallner wrote: > > On 4/1/19 5:56 PM, Corinna Vinschen wrote: > > > On Apr 1 16:56, Corinna Vinschen wrote: > > >> On Apr 1 16:28, Michael Haubenwallner wrote: > > >>> On 3/28/19 9:30 PM, Corinna Vinschen wrote: > > can you please collect the base addresses of all DLLs generated during > > the build, plus their size and make a sorted list? It would be > > interesting to know if the hash algorithm in ld is actually as bad > > as I conjecture. > > >>> > > >>> Please find attached the output of rebase -i for the dlls after > > >>> bootstrap > > >>> on Cygwin 3.0.4, each built with ld from binutils-2.31.1. > > > > > > Oh, wait. That's not what I was looking for. The addresses are ok, but > > > the paths *must* be the ones at the time the DLLs have been created, > > > because that's what ld uses when creating the image base addresses. The > > > addresses combined with the installation paths don't make sense anymore. > > > > So I have intercepted the ld.exe to show 'rebase -i' on any just created > > dll, > > tell about the exact -o argument to ld, and the current directory. > > > > This is with binutils-2.31.1 > > > > Anything else needed? > > No, that should be sufficient, thanks for collecting this! Nick Clifton, one of the binutils maintainers, made the following suggestion in PM: Allow the ld flag --enable-auto-image-base to take a filename as argument. The idea: The file is used by ld to generate the start address for the next built DLL. Mechanism: 1.1. If ld links a DLL and if the file given to --enable-auto-image-base doesn't exist, ld will give the DLL the start address of the auto image base range. 1.2: Next time, if ld links a DLL and if the file given to --enable-auto-image-base exists, it will use the address in that file as the start address for th just built DLL. 2. It will store that address, plus the size of the DLL, rounded up to 64K, in that file. 3. If the auto image base range is at an end, ld will wrap back to the start address of the auto image base range. TBD: A way to enable this feature without having to change all packages' build systems. That way you could build hundreds of DLLs in a project and use them immediately without having to rebase. This is just in a discussion state, nothing has happend yet, but what do you think in general? Corinna -- Corinna Vinschen Cygwin Maintainer signature.asc Description: PGP signature
Re: [PATCH] Cygwin: fork: remember child as late as possible
On Apr 12 15:31, Michael Haubenwallner wrote: > Otherwise, when the child does fail to reload dlls and terminates, we > produce a SIGCHILD signal, even if we did not succeed in starting up the > child process at all. Also, we would need to reap that child somewhere. I'm not that happy with the patch. The patch is doing two things, one is to remember child as late as possible, the other is to change the child startup in case the parent loaded dlls. Either write a more descriptive commit message or split this into two patches. I'd prefer two patches. It may help debugging. Thanks, Corinna -- Corinna Vinschen Cygwin Maintainer signature.asc Description: PGP signature
Re: [PATCH] Cygwin: use win pid+threadid for forkables dirname
On Apr 12 15:32, Michael Haubenwallner wrote: > Rather than newest last write time of all dlls loaded, use the forking > process' windows pid and windows thread id as directory name to create > the forkable hardlinks into. While this may create hardlinks more > often, it does avoid conflicts between dlls not having the newest last > write time. > --- > winsup/cygwin/forkable.cc | 26 +++--- > 1 file changed, 7 insertions(+), 19 deletions(-) Pushed. Thanks, Corinna -- Corinna Vinschen Cygwin Maintainer signature.asc Description: PGP signature
Re: [rebase PATCH] Introduce --no-rebase flag
On Apr 12 15:52, Michael Haubenwallner wrote: > The --no-rebase flag is to update the database for new files, without Wouldn't something like --merge-files be more descriptive? Corinna -- Corinna Vinschen Cygwin Maintainer signature.asc Description: PGP signature
