Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

Hi,
I was wondering about the possibility of FreeBSD to provide an official
supported graphical environment.


UNIX (so FreeBSD) never had "standard graphical environment" or "graphical 
environment" at all.


Xorg is standard in FreeBSD and most unices for graphics hardware support.

There are thousands and more programs available that uses Xorg, including 
tens of "graphical environments", and everyone is free to select any of 
them if needed. Or none of them if not needed.


Is it that bad to allow people control their computer and willingly choose 
what to use?





___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

This idea would precisely serve the purpose of removing this need and
eliminate redundancy of toolkits, when it comes to essential utilities
that FreeBSD would want to provide, like devices automounting,
partitioning (taking advantage of the system features) and so on... but
it's just an idea, of course.


is anything of this really FreeBSD specific?
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

regard GUI as a third-party bonus.


This is according to *your* use cases though. There are many of us who
do not put X - or any graphical environment - on our FreeBSD servers.


and there are many of us that do not put any "graphical environment" while 
using Xorg, making actually productive one.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

want to develop GUI applications on FreeBSD, supporting features as
panel integration, reliable messageboxes and other trivial things, on
other operating systems, that are apparently unavailable on UNIX without
pulling in significant portions of lots of environments.


this make sense. but it is not FreeBSD specific at all.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

   Replying more to the Wayland comments, yes..
FreeBSD/NetBSD/OpenBSD need to implement the Wayland `protocol`
because xorg-server development is slowly being killed over time, but


that's the main reason i already frozen package tree, so i will be able to 
use Xorg in 5 years or more.


Wayland is the step that destroyed main adventage of X11 protocol - 
network transparency. And yes i use it heavily.


Actually i don't see any real future for wayland and linux. Linux is 
already pushed out by *BSD on the professional side, and by 
Windows, Mac OS X,"smart"phones, android, etc... on avarage user size, 
with real "linux users" being less than 0.1% of all.


Android is linux based but mostly kernel side.

Of course you may see much higher stats as a lot of people have linux 
installed altogether with windows or Mac OS X, they normally use the 
latter, but want to be proud of being linux user.


Linux now is mostly hype. Still it's very loud about it but that's only 
media reality.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

From a programmer's point of view, GUI is a protocol, a graphical

language. It's true. But users don't care. Users don't care how their
graphical commands are being implemented.


Such users don't use FreeBSD, or at least doesn't have admin rights.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

actually come out of it). I then faced the problem that there are lots
of GUI toolkits, lots of scenarios to take into account, lots of desktop


You cannot change it. There are lots of GUI toolking and none are really 
consistent. None are part of FreeBSD and none will.


If you want to write GUI programs like this you just have to select single 
one.


Or better - spend you time on more productive things that making another 
graphic toy. Even drinking beer is more productive.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

To be succinct: this is not OSX/Windows. True Unix and Unix clones can
be decoupled from a desktop environment enough that forcing everyone
to have one choice for desktop user experience doesn't make sense, and
the fact that there isn't a common GUI development toolkit (GTK, QT,
etc) encourages fragmentation of effort further (I think it's called
the Bazaar model of development :P).


That's all true. But do anyone understand why there is still so much 
pressure for every open source OS and specifically *BSD on "default 
desktop environment" or similar ideas?


Such a pressure exist for 20 years at least, and - between other results - 
started demise of linux as trusty high performance system.


Why people still not learned from faults and keep harming good open source 
projects that way?

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

make a GUI application for FreeBSD? You are asking yourself what desktop
environment will work for sure on FreeBSD? There you have it, Blah DE
works just well and is perfectly documented."


use any X toolking you want (well almost, i recommend avoiding Qt) and use 
it properly without assuming any "desktop environment" running and your 
program will work just fine.


What's a problem?
No standard for "toolbar widgets"? Really it's completely unimportant 
thing for anyone treating computer as something more than a toy.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

sound programming

[Wojciech Puchar]

where can i find up to day beginner howto of programming sound (C 
language) in FreeBSD?


particularly how to record/process/mix/play sound realtime - i mean having 
low delays.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: do we always have acpi_cpu for a cpu?

[Andriy Gapon]

[ping]

on 11/09/2012 09:32 Andriy Gapon said the following:
> 
> I think that we always expect to have a one-to-one correspondence between
> acpi_cpu devices and actual (APIC) CPUs.  acpi_pcpu_get_id() seems to even
> assert that, if I am reading the code correctly.
> 
> The following patch adds the assert to acpi_cpu_idle as well and also removes
> what I believe to be an obsolete comment about HTT CPUs.
> 
> acpi_cpu: expect every cpu to have a corresponding acpi_cpu object
> 
> ... via Processor object in ASL namespace.
> 
> diff --git a/sys/dev/acpica/acpi_cpu.c b/sys/dev/acpica/acpi_cpu.c
> index 15201f9..203ed02 100644
> --- a/sys/dev/acpica/acpi_cpu.c
> +++ b/sys/dev/acpica/acpi_cpu.c
> @@ -925,23 +925,15 @@ acpi_cpu_idle()
>  uint32_t start_time, end_time;
>  int  bm_active, cx_next_idx, i;
> 
> +sc = cpu_softc[PCPU_GET(cpuid)];
> +KASSERT(sc != NULL, ("acpi_cpu_idle: CPU without ACPI CPU"));
> +
>  /* If disabled, return immediately. */
>  if (cpu_disable_idle) {
>   ACPI_ENABLE_IRQS();
>   return;
>  }
> 
> -/*
> - * Look up our CPU id to get our softc.  If it's NULL, we'll use C1
> - * since there is no ACPI processor object for this CPU.  This occurs
> - * for logical CPUs in the HTT case.
> - */
> -sc = cpu_softc[PCPU_GET(cpuid)];
> -if (sc == NULL) {
> - acpi_cpu_c1();
> - return;
> -}
> -
>  /* Find the lowest state that has small enough latency. */
>  cx_next_idx = 0;
>  if (cpu_disable_deep_sleep)
> 


-- 
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Chris Rees]

Can you perhaps read the whole thread and organise your thoughts into just
one email?

Chris
On 18 Sep 2012 09:09, "Wojciech Puchar" 
wrote:

> To be succinct: this is not OSX/Windows. True Unix and Unix clones can
>> be decoupled from a desktop environment enough that forcing everyone
>> to have one choice for desktop user experience doesn't make sense, and
>> the fact that there isn't a common GUI development toolkit (GTK, QT,
>> etc) encourages fragmentation of effort further (I think it's called
>> the Bazaar model of development :P).
>>
>
> That's all true. But do anyone understand why there is still so much
> pressure for every open source OS and specifically *BSD on "default desktop
> environment" or similar ideas?
>
> Such a pressure exist for 20 years at least, and - between other results -
> started demise of linux as trusty high performance system.
>
> Why people still not learned from faults and keep harming good open source
> projects that way?
> __**_
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-**hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@**
> freebsd.org "
>
>
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

I spent years using Linux before I truly appreciated the key difference between a "desktop 
environment" and a "graphical environment". Probably because everyone had to have a 
desktop environment.

I define graphical environment as simply X11 and a window manager.


good that you as first one defined things. I am too a user of "graphical 
environment" as per your definition, using Xorg and fvwm2 with heavily 
modified config.



That's all you need to run Firefox, Gimp, etc. Because x11 is the underlying 
base, any toolkit (gtk, qt, whatever) will work just fine. A developer can pick 
the toolkit they're most comfortable with and it will work on anyone's system.


There are few exceptions for badly written programs, mostly QT based that 
gets eg. terrible performance or leave lots of dangling processes after 
you exit. But these are exceptions not rule.



IMHO, a graphical environment is useful for running applications like Firefox 
and Gimp.
I never run either of these on a server so I would never want to install 
even a graphical environment on my servers.


Well i actually installed it on a few. If i have connected monitor and i 
am present there and just want to view a webpage or do something requiring

X11 - why not?



I have no use at all for desktop environments. They're often bloated, buggy, 
and provide no real value to me.


Even if there would be "desktop enviroment" that starts in less than a 
second, reacts on everything below single video frame time and used close 
to zero CPU time i would not use one, as there is no benefit.


IMHO the only "benefit" is wasted lots of screen space for taskbars, 
menubars, windows frames and titles etc...



___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Anton Shterenlikht]

Date: Tue, 18 Sep 2012 09:54:41 +0200 (CEST)
From: Wojciech Puchar 

Actually i don't see any real future for wayland and linux. Linux is 
already pushed out by *BSD on the professional side, and by 

Are you mad?
Have you looked at top500 lately?
As of June 2012, 462 out of 500 systems are linux based.

Anton
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Wojciech Puchar]

desktop environment" or similar ideas?


Tell you what:

When you have at least 75% of the user population of FreeBSD agreeing
on which window manager we should offer as the default, we can talk
about this.


so if 76% would decide that FreeBSD should have KDE included in system - 
it means that it should?

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: sound programming

[Pieter de Goeje]

Op 18-9-2012 10:15, Wojciech Puchar schreef:
where can i find up to day beginner howto of programming sound (C 
language) in FreeBSD?


particularly how to record/process/mix/play sound realtime - i mean 
having low delays.


Check out 4front's OSS documentation
http://manuals.opensound.com/developer/

In particular (recording and playback with low latency):
http://manuals.opensound.com/developer/fulldup.c.htm

Barring implementation differences, this should all apply to FreeBSD.
To get really low delays you should probably adjust the hw.snd.latency 
sysctls.


Regards,
Pieter de Goeje

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Erich Dollansky]

Hi,

On Tue, 18 Sep 2012 10:39:43 +0200 (CEST)
Wojciech Puchar  wrote:

> >> desktop environment" or similar ideas?
> >
> > Tell you what:
> >
> > When you have at least 75% of the user population of FreeBSD
> > agreeing on which window manager we should offer as the default, we
> > can talk about this.
> 
> so if 76% would decide that FreeBSD should have KDE included in
> system - it means that it should?

and let the project wonder one year later that all users moved to
Windows as the better KDE?

Erich
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: My explanation to "a default DE" (was "Providing a default graphical environment on FreeBSD")

[Mike Meyer]

Zhihao Yuan  wrote:
>As u can see, it's a stand alone app. But the most widely used wifi
>managers are always taskbar applets -- something bound to a specific
>DE. --

There are a number of taskbar applications not bound to a DE (dzen2, mobar, 
gkrellm2) which have plugins for managing the kinds of things you're talking 
about. The X protocols provide a way for such applications to declare 
themselves as "bars", and for other interested applications (usually WM's) to 
discover the screen geometry both with and without bars.

>we must standardize a ${DE}!

Why? So far, the only advantage you've cited is that users can get a supported, 
documented desktop on FreeBSD (except there's nobody to support or document it) 
and developers could get a documented desktop to write apps for (except there's 
again nobody to document it, and you're the only developer asking for it), and 
you haven't said anything bad would happen if this doesn't happen.

  http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: [RFC] Add *.orig/*.rej to svn:ignore in src?

[Tom Evans]

On Sun, Sep 16, 2012 at 10:11 AM, Garrett Cooper  wrote:
> I noticed that we have a handful of patterns currently ignored in 
> svn:ignore (at least at the top-level… the lower levels don't appear to be 
> set in any particular manner):
>
> $ svn propget svn:ignore
> _.tinderbox.*
> _.amd64.*
> _.arm.*
> _.i386.*
> _.ia64.*
> _.mips.*
> _.pc98.*
> _.powerpc.*
> _.sparc64.*
> _.sun4v.*
>
> $ svn info | grep URL:
> URL: http://svn.freebsd.org/base/head
>
> I was wondering if *.orig and *.rej should be added to the list as 
> well to avoid checking in patch `artifact` files?
> Thanks!

svn:ignore is not recursive, so this would only prevent orig/rej files
being committed if they are at the top level, which they are unlikely
to be.

This is best handled personally in ~/.subversion/config

Cheers

Tom
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Changing `iostat -Ix' output

[Mikolaj Golub]

Hi,

I don't like very much what `iostat -Ix' outputs and would like to
change this.

A typical output:

% iostat -Ix
extended device statistics  
device r/i   w/ikr/ikw/i qlen svc_t  %b  
ada0 5599136.0 3953193.0 39982760.0 86819866.50  15.1  16 
cd0  552.0   0.0 4.3 0.00   0.0   0 
pass0333.0   0.0   166.5 0.00   0.0   0 
pass1  2.0   0.0 1.0 0.00   0.0   0 

Parameters like r/i, kr/i (total io operations/kbytes) are very
useful. They allow to use `iostat -Ix' to collect IO statistics
running it periodically (from cron or some monitoring tool) and
calculate average amount of operations or bytes per second at the
specified period subtracting the current value from the previous one
and dividing by time period.

But you can't do the same with % busy, which is very useful IO
characteristics. Average % busy at the specified period could be
calculated storing total busy time for the device at time t1, total
busy time at t2 and then subtracting the last value from the first (to
get busy time at this period) and dividing by the time period.

Currently iostat(8) does not provide 'total busy time' statistics. I
use sysutils/devstat for this but it would be nice if iostat(8) itself
provide such functionality.

I propose for `iostat -Ix` to output total busy time instead of %
busy, and also total duration of transactions instead of average
duration (to be able to calculate average duration for the period
between two iostat runs).

http://people.freebsd.org/~trociny/iostat.total_busy_time.1.patch

Average duration and % busy are still available via `iostat -x`.

Here is an output example

% ./iostat -Ix; sleep 60; ./iostat -Ix
extended device statistics  
device   r/i w/i kr/i kw/i qlen   tdur  
  sb  
ada0   5599785.0   3961913.0   39985960.5   86902385.50   144055.5   
35966.5 
cd0554.0 0.0  4.3  0.000.0  
 9.5 
pass0  336.0 0.0168.0  0.000.0  
17.5 
pass12.0 0.0  1.0  0.000.0  
 0.0 
extended device statistics  
device   r/i w/i kr/i kw/i qlen   tdur  
  sb  
ada0   5599922.0   3963177.0   40002608.0   86958230.50   144074.4   
35970.5 
cd0554.0 0.0  4.3  0.000.0  
 9.5 
pass0  336.0 0.0168.0  0.000.0  
17.5 
pass12.0 0.0  1.0  0.000.0  
 0.0 

So, for ada0, % busy for that period was 100 * (35970.5 - 35966.5) / 60 = 6.

And service time (assuming that only read and write operations were
serviced) was

1000 * (144074.4 - 144055.5) / (5599922 - 5599785 + 3963177 - 3961913)
= 13.4 msec.

What do you think about this?

-- 
Mikolaj Golub
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Poul-Henning Kamp]

In message , Wojci
ech Puchar writes:

>That's all true. But do anyone understand why there is still so much 
>pressure for every open source OS and specifically *BSD on "default 
>desktop environment" or similar ideas?

Tell you what:

When you have at least 75% of the user population of FreeBSD agreeing
on which window manager we should offer as the default, we can talk
about this.

Until such a consensus exists, this discussion is just a waste of time.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Poul-Henning Kamp]

In message , Wojci
ech Puchar writes:
>>> desktop environment" or similar ideas?
>>
>> Tell you what:
>>
>> When you have at least 75% of the user population of FreeBSD agreeing
>> on which window manager we should offer as the default, we can talk
>> about this.
>
>so if 76% would decide that FreeBSD should have KDE included in system - 
>it means that it should?

Just to clarify: when I write "offer by default" I do not mean "cram
down peoples throat".

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
p...@freebsd.org | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Chris Rees]

On 18 Sep 2012 09:41, "Wojciech Puchar" 
wrote:
>>>
>>> desktop environment" or similar ideas?
>>
>>
>> Tell you what:
>>
>> When you have at least 75% of the user population of FreeBSD agreeing
>> on which window manager we should offer as the default, we can talk
>> about this.
>
>
> so if 76% would decide that FreeBSD should have KDE included in system -
it means that it should?

No.  Read the thread properly.

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

cpu_spinwait in cngetc

[Andriy Gapon]

(Why[*]) Would anyone object to a change like this?

diff --git a/sys/kern/kern_cons.c b/sys/kern/kern_cons.c
index 5346bc3..d17846a 100644
--- a/sys/kern/kern_cons.c
+++ b/sys/kern/kern_cons.c
@@ -384,7 +384,7 @@ cngetc(void)
if (cn_mute)
return (-1);
while ((c = cncheckc()) == -1)
-   ;
+   cpu_spinwait();
if (c == '\r')
c = '\n';   /* console input is always ICRNL */
return (c);

[*] :-)
-- 
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

ule+smp: small optimization for turnstile priority lending

[Andriy Gapon]

Here is a snippet that demonstrates the issue on a supposedly fully loaded
2-processor system:

136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"running", attributes: prio:122

136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid 111916",
state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"

136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid 14",
state:"running", attributes: prio:255

136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load", counter:0,
attributes: none

136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid 113473",
state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"

136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load", counter:2,
attributes: none

136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid 113473",
point:"wokeup", attributes: linkedto:"Xorg tid 102818"

136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"

136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load", counter:1,
attributes: none

136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"runq rem", attributes: prio:176

136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid 113473"

136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid 113473",
state:"running", attributes: prio:104

See how how the Xorg thread was forced from CPU 1 to CPU 0 where it preempted
cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero 
load.

Here is a proposed solution:

turnstile_wait: optimize priority lending to a thread on a runqueue

As the current thread is definitely going into mi_switch, it now removes
its load before doing priority propagation which can potentially result
in sched_add.  In the SMP && ULE case the latter searches for the
least loaded CPU to place a boosted thread, which is supposedly about
to run.

diff --git a/sys/kern/sched_ule.c b/sys/kern/sched_ule.c
index 8e466cd..3299cae 100644
--- a/sys/kern/sched_ule.c
+++ b/sys/kern/sched_ule.c
@@ -1878,7 +1878,10 @@ sched_switch(struct thread *td, struct thread *newtd, int
flags)
/* This thread must be going to sleep. */
TDQ_LOCK(tdq);
mtx = thread_lock_block(td);
-   tdq_load_rem(tdq, td);
+#if defined(SMP)
+   if ((flags & SW_TYPE_MASK) != SWT_TURNSTILE)
+#endif
+   tdq_load_rem(tdq, td);
}
/*
 * We enter here with the thread blocked and assigned to the
@@ -2412,6 +2415,21 @@ sched_rem(struct thread *td)
tdq_setlowpri(tdq, NULL);
 }

+void
+sched_load_rem(struct thread *td)
+{
+   struct tdq *tdq;
+
+   KASSERT(td == curthread,
+   ("sched_rem_load: only curthread is supported"));
+   KASSERT(td->td_oncpu == td->td_sched->ts_cpu,
+   ("thread running on cpu different from ts_cpu"));
+   tdq = TDQ_CPU(td->td_sched->ts_cpu);
+   TDQ_LOCK_ASSERT(tdq, MA_OWNED);
+   MPASS(td->td_lock == TDQ_LOCKPTR(tdq));
+   tdq_load_rem(tdq, td);
+}
+
 /*
  * Fetch cpu utilization information.  Updates on demand.
  */
diff --git a/sys/kern/subr_turnstile.c b/sys/kern/subr_turnstile.c
index 31d16fe..d1d68e9 100644
--- a/sys/kern/subr_turnstile.c
+++ b/sys/kern/subr_turnstile.c
@@ -731,6 +731,13 @@ turnstile_wait(struct turnstile *ts, struct thread *owner,
int queue)
LIST_INSERT_HEAD(&ts->ts_free, td->td_turnstile, ts_hash);
}
thread_lock(td);
+#if defined(SCHED_ULE) && defined(SMP)
+   /*
+* Remove load earlier so that it does not affect cpu selection
+* for a thread waken up due to priority lending, if any.
+*/
+   sched_load_rem(td);
+#endif
thread_lock_set(td, &ts->ts_lock);
td->td_turnstile = NULL;

diff --git a/sys/sys/sched.h b/sys/sys/sched.h
index 4b8387c..b1ead1b 100644
--- a/sys/sys/sched.h
+++ b/sys/sys/sched.h
@@ -110,6 +110,9 @@ voidsched_preempt(struct thread *td);
 void   sched_add(struct thread *td, int flags);
 void   sched_clock(struct thread *td);
 void   sched_rem(struct thread *td);
+#if defined(SCHED_ULE) && defined(SMP)
+void   sched_load_rem(struct thread *td);
+#endif
 void   sched_tick(int cnt);
 void   sched_relinquish(struct thread *td);
 struct thread *sched_choose(void);

-- 
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: sound programming

[Reid Linnemann]

On Tue, Sep 18, 2012 at 3:33 AM, Pieter de Goeje  wrote:
> Op 18-9-2012 10:15, Wojciech Puchar schreef:
>
>> where can i find up to day beginner howto of programming sound (C
>> language) in FreeBSD?
>>
>> particularly how to record/process/mix/play sound realtime - i mean having
>> low delays.
>
>
> Check out 4front's OSS documentation
> http://manuals.opensound.com/developer/
>
> In particular (recording and playback with low latency):
> http://manuals.opensound.com/developer/fulldup.c.htm
>
> Barring implementation differences, this should all apply to FreeBSD.
> To get really low delays you should probably adjust the hw.snd.latency
> sysctls.
>
> Regards,
> Pieter de Goeje
>
>
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

FYI, I think you've got a cut and paste error. That last link should
be http://manuals.opensound.com/developer/fulldup.c.html
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

fdgrowtable() cleanup

[Dag-Erling Smørgrav]

The patch below my signature improves the legibility of fdgrowtable(),
and adds comments explaining the hairier bits.  The only functional
change is that the code no longer overwrites the old fileflags array
when the old table is placed on the freelist; instead, it uses the space
actually set aside for that purpose.

(I assume that the old behavior was harmless, since it has persisted for
decades, but it was certainly confusing.)

The slightly repetitive nature of the new code is intentional.

DES
-- 
Dag-Erling Smørgrav - d...@des.no

Index: sys/kern/kern_descrip.c
===
--- sys/kern/kern_descrip.c (revision 240654)
+++ sys/kern/kern_descrip.c (working copy)
@@ -148,11 +148,6 @@
 #defineNDSLOTS(x)  (((x) + NDENTRIES - 1) / NDENTRIES)
 
 /*
- * Storage required per open file descriptor.
- */
-#define OFILESIZE (sizeof(struct file *) + sizeof(char))
-
-/*
  * Storage to hold unused ofiles that need to be reclaimed.
  */
 struct freetable {
@@ -1436,7 +1431,7 @@
struct freetable *fo;
struct file **ntable;
struct file **otable;
-   char *nfileflags;
+   char *nfileflags, *ofileflags;
int nnfiles, onfiles;
NDSLOTTYPE *nmap;
 
@@ -1447,33 +1442,46 @@
 
/* compute the size of the new table */
onfiles = fdp->fd_nfiles;
+   otable = fdp->fd_ofiles;
+   ofileflags = fdp->fd_ofileflags;
nnfiles = NDSLOTS(nfd) * NDENTRIES; /* round up */
if (nnfiles <= onfiles)
/* the table is already large enough */
return;
 
-   /* allocate a new table and (if required) new bitmaps */
-   ntable = malloc((nnfiles * OFILESIZE) + sizeof(struct freetable),
+   /*
+* Allocate a new table.  We need enough space for a) the file
+* entries themselves, b) the file flags, and c) the struct
+* freetable we will use when we decommission the table and place
+* it on the freelist.
+*/
+   ntable = malloc(nnfiles * sizeof(*ntable) +
+   nnfiles * sizeof(*nfileflags) +
+   sizeof(struct freetable),
M_FILEDESC, M_ZERO | M_WAITOK);
nfileflags = (char *)&ntable[nnfiles];
+
+   /* allocate new bitmaps if necessary */
if (NDSLOTS(nnfiles) > NDSLOTS(onfiles))
nmap = malloc(NDSLOTS(nnfiles) * NDSLOTSIZE,
M_FILEDESC, M_ZERO | M_WAITOK);
else
nmap = NULL;
 
-   bcopy(fdp->fd_ofiles, ntable, onfiles * sizeof(*ntable));
-   bcopy(fdp->fd_ofileflags, nfileflags, onfiles);
-   otable = fdp->fd_ofiles;
+   /* copy the old data over and point at the new tables */
+   bcopy(otable, ntable, onfiles * sizeof(*otable));
+   bcopy(ofileflags, nfileflags, onfiles * sizeof(*ofileflags));
fdp->fd_ofileflags = nfileflags;
fdp->fd_ofiles = ntable;
+
/*
 * We must preserve ofiles until the process exits because we can't
 * be certain that no threads have references to the old table via
 * _fget().
 */
if (onfiles > NDFILE) {
-   fo = (struct freetable *)&otable[onfiles];
+   fo = (struct freetable *)(char *)otable +
+   onfiles * sizeof(*otable) + onfiles * sizeof(*ofileflags);
fdp0 = (struct filedesc0 *)fdp;
fo->ft_table = otable;
SLIST_INSERT_HEAD(&fdp0->fd_free, fo, ft_next);
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: ule+smp: small optimization for turnstile priority lending

[Attilio Rao]

On 9/18/12, Andriy Gapon  wrote:
>
> Here is a snippet that demonstrates the issue on a supposedly fully loaded
> 2-processor system:
>
> 136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
> state:"running", attributes: prio:122
>
> 136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid
> 111916",
> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"
>
> 136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid
> 14",
> state:"running", attributes: prio:255
>
> 136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load",
> counter:0,
> attributes: none
>
> 136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid
> 113473",
> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"
>
> 136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load",
> counter:2,
> attributes: none
>
> 136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid
> 113473",
> point:"wokeup", attributes: linkedto:"Xorg tid 102818"
>
> 136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"
>
> 136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load",
> counter:1,
> attributes: none
>
> 136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
> state:"runq rem", attributes: prio:176
>
> 136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid
> 113473"
>
> 136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid
> 113473",
> state:"running", attributes: prio:104
>
> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it
> preempted
> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero
> load.

I think that the idea is bright, but I have reservations against the
implementation because it seems to me there are too many layering
violations.

What is suggest is somewhat summarized like that:
- Add a new SRQ_WILLSLEEP or the name you prefer
- Add a new "flags" argument to sched_lend_prio() (both ule and 4bsd)
and sched_thread_priority (ule only)
- sched_thread_priority() will pass down the new flag to sched_add()
which passed down to sched_pickcpu().

This way sched_pickcpu() has the correct knowledge of what is going on
and it can make the right decision. You likely don't need to lower the
tdq_load at that time either this way, because sched_pickcpu() can
just adjust it locally for its decision.

What do you think?

Thanks,
Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: ule+smp: small optimization for turnstile priority lending

[Andriy Gapon]

on 18/09/2012 19:50 Attilio Rao said the following:
> On 9/18/12, Andriy Gapon  wrote:
>>
>> Here is a snippet that demonstrates the issue on a supposedly fully loaded
>> 2-processor system:
>>
>> 136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>> state:"running", attributes: prio:122
>>
>> 136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid
>> 111916",
>> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"
>>
>> 136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid
>> 14",
>> state:"running", attributes: prio:255
>>
>> 136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load",
>> counter:0,
>> attributes: none
>>
>> 136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid
>> 113473",
>> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"
>>
>> 136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load",
>> counter:2,
>> attributes: none
>>
>> 136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid
>> 113473",
>> point:"wokeup", attributes: linkedto:"Xorg tid 102818"
>>
>> 136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"
>>
>> 136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load",
>> counter:1,
>> attributes: none
>>
>> 136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>> state:"runq rem", attributes: prio:176
>>
>> 136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid
>> 113473"
>>
>> 136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid
>> 113473",
>> state:"running", attributes: prio:104
>>
>> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it
>> preempted
>> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero
>> load.
> 
> I think that the idea is bright, but I have reservations against the
> implementation because it seems to me there are too many layering
> violations.

Just one - for a layer between tunrstile and scheduler :-)
But I agree.

> What is suggest is somewhat summarized like that:
> - Add a new SRQ_WILLSLEEP or the name you prefer
> - Add a new "flags" argument to sched_lend_prio() (both ule and 4bsd)
> and sched_thread_priority (ule only)
> - sched_thread_priority() will pass down the new flag to sched_add()
> which passed down to sched_pickcpu().
> 
> This way sched_pickcpu() has the correct knowledge of what is going on
> and it can make the right decision. You likely don't need to lower the
> tdq_load at that time either this way, because sched_pickcpu() can
> just adjust it locally for its decision.
> 
> What do you think?

This sounds easy but it is not quite so given the implementation of
sched_pickcpu and sched_lowest.  This is probably more work than I am able to
take now.

-- 
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: ule+smp: small optimization for turnstile priority lending

[Jeff Roberson]

On Tue, 18 Sep 2012, Andriy Gapon wrote:


on 18/09/2012 19:50 Attilio Rao said the following:

On 9/18/12, Andriy Gapon  wrote:


Here is a snippet that demonstrates the issue on a supposedly fully loaded
2-processor system:

136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"running", attributes: prio:122

136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid
111916",
state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"

136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid
14",
state:"running", attributes: prio:255

136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load",
counter:0,
attributes: none

136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid
113473",
state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"

136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load",
counter:2,
attributes: none

136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid
113473",
point:"wokeup", attributes: linkedto:"Xorg tid 102818"

136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"

136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load",
counter:1,
attributes: none

136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
state:"runq rem", attributes: prio:176

136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid
113473"

136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid
113473",
state:"running", attributes: prio:104

See how how the Xorg thread was forced from CPU 1 to CPU 0 where it
preempted
cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero
load.


I think that the idea is bright, but I have reservations against the
implementation because it seems to me there are too many layering
violations.


Just one - for a layer between tunrstile and scheduler :-)
But I agree.


What is suggest is somewhat summarized like that:
- Add a new SRQ_WILLSLEEP or the name you prefer
- Add a new "flags" argument to sched_lend_prio() (both ule and 4bsd)
and sched_thread_priority (ule only)
- sched_thread_priority() will pass down the new flag to sched_add()
which passed down to sched_pickcpu().

This way sched_pickcpu() has the correct knowledge of what is going on
and it can make the right decision. You likely don't need to lower the
tdq_load at that time either this way, because sched_pickcpu() can
just adjust it locally for its decision.

What do you think?


This sounds easy but it is not quite so given the implementation of
sched_pickcpu and sched_lowest.  This is probably more work than I am able to
take now.


I agree with Attillio's assessment.  I have tried to do similar things 
before for ithreads etc.  I think a more generic approach would be good. 
I will put it on my list of things to look at and we'll see who gets time 
first.


Thanks,
Jeff



--
Andriy Gapon


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: Providing a default graphical environment on FreeBSD

[Matthias Andree]

Am 17.09.2012 21:52, schrieb Lorenzo Cogotti:

> Even the userbase/time spent developing ratio matters. What also matters
> is the interest that a system shows in something, I think it's obvious
> that FreeBSD can't get much attention as a desktop system if no effort
> is put into it. It is not a bad thing being tied to the server concept,
> but I just think FreeBSD would also be an excellent desktop system with
> a little effort.

There is Debian/kFreeBSD - Debian on a FreeBSD kernel.  Not sure how
useful it is, and it probably lacks FreeBSD's userland, but well, just a
point.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: ule+smp: small optimization for turnstile priority lending

[Attilio Rao]

On Tue, Sep 18, 2012 at 8:07 PM, Andriy Gapon  wrote:
> on 18/09/2012 19:50 Attilio Rao said the following:
>> On 9/18/12, Andriy Gapon  wrote:
>>>
>>> Here is a snippet that demonstrates the issue on a supposedly fully loaded
>>> 2-processor system:
>>>
>>> 136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>>> state:"running", attributes: prio:122
>>>
>>> 136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid
>>> 111916",
>>> state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"
>>>
>>> 136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid
>>> 14",
>>> state:"running", attributes: prio:255
>>>
>>> 136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load",
>>> counter:0,
>>> attributes: none
>>>
>>> 136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid
>>> 113473",
>>> state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"
>>>
>>> 136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load",
>>> counter:2,
>>> attributes: none
>>>
>>> 136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid
>>> 113473",
>>> point:"wokeup", attributes: linkedto:"Xorg tid 102818"
>>>
>>> 136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>>> state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"
>>>
>>> 136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load",
>>> counter:1,
>>> attributes: none
>>>
>>> 136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>>> state:"runq rem", attributes: prio:176
>>>
>>> 136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
>>> point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid
>>> 113473"
>>>
>>> 136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid
>>> 113473",
>>> state:"running", attributes: prio:104
>>>
>>> See how how the Xorg thread was forced from CPU 1 to CPU 0 where it
>>> preempted
>>> cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero
>>> load.
>>
>> I think that the idea is bright, but I have reservations against the
>> implementation because it seems to me there are too many layering
>> violations.
>
> Just one - for a layer between tunrstile and scheduler :-)
> But I agree.
>
>> What is suggest is somewhat summarized like that:
>> - Add a new SRQ_WILLSLEEP or the name you prefer
>> - Add a new "flags" argument to sched_lend_prio() (both ule and 4bsd)
>> and sched_thread_priority (ule only)
>> - sched_thread_priority() will pass down the new flag to sched_add()
>> which passed down to sched_pickcpu().
>>
>> This way sched_pickcpu() has the correct knowledge of what is going on
>> and it can make the right decision. You likely don't need to lower the
>> tdq_load at that time either this way, because sched_pickcpu() can
>> just adjust it locally for its decision.
>>
>> What do you think?
>
> This sounds easy but it is not quite so given the implementation of
> sched_pickcpu and sched_lowest.  This is probably more work than I am able to
> take now.

I think actually that the attached patch should do what you need. Of
course there is more runqueue locking, due to the tdq_load fondling,
but I'm not sure it is really a big deal.
I didn't test it yet, as I understand you have a test case, so maybe
you can. However if Jeff agrees I can send the patch to flo@ for
performance evaluation.

Thoughts?

Attilio

Index: sys/sys/sched.h
===
--- sys/sys/sched.h (revisione 240672)
+++ sys/sys/sched.h (copia locale)
@@ -91,7 +91,7 @@ void  sched_nice(struct proc *p, int nice);
  */
 void   sched_exit_thread(struct thread *td, struct thread *child);
 void   sched_fork_thread(struct thread *td, struct thread *child);
-void   sched_lend_prio(struct thread *td, u_char prio);
+void   sched_lend_prio(struct thread *td, u_char prio, int flags);
 void   sched_lend_user_prio(struct thread *td, u_char pri);
 fixpt_tsched_pctcpu(struct thread *td);
 void   sched_prio(struct thread *td, u_char prio);
@@ -161,6 +161,7 @@ sched_unpin(void)
 #defineSRQ_INTR0x0004  /* It is probably urgent. */
 #defineSRQ_PREEMPTED   0x0008  /* has been
preempted.. be kind */
 #defineSRQ_BORROWING   0x0010  /* Priority updated
due to prio_lend */
+#defineSRQ_WILLSWITCH  0x0020  /* curthread will be
switched out */

 /* Scheduler stats. */
 #ifdef SCHED_STATS
Index: sys/kern/sched_ule.c
===
--- sys/kern/sched_ule.c(revisione 240672)
+++ sys/kern/sched_ule.c(copia locale)
@@ -290,7 +290,7 @@ static struct tdq   tdq_cpu;
 #defineTDQ_LOCKPTR(t)  (&(t)->tdq_lock)

 static void sched_priority(struct thread *);
-static void sched_thread_priority(struct thread *, u_char);
+static void sched_thread_prio

Re: ule+smp: small optimization for turnstile priority lending

[Andriy Gapon]

on 19/09/2012 02:01 Attilio Rao said the following:
> On Tue, Sep 18, 2012 at 8:07 PM, Andriy Gapon  wrote:
>> on 18/09/2012 19:50 Attilio Rao said the following:
>>> On 9/18/12, Andriy Gapon  wrote:

 Here is a snippet that demonstrates the issue on a supposedly fully loaded
 2-processor system:

 136794   0 3670427870244462 KTRGRAPH group:"thread", id:"Xorg tid 102818",
 state:"running", attributes: prio:122

 136793   0 3670427870241000 KTRGRAPH group:"thread", id:"cc1plus tid
 111916",
 state:"yielding", attributes: prio:183, wmesg:"(null)", lockname:"(null)"

 136792   1 3670427870240829 KTRGRAPH group:"thread", id:"idle: cpu1 tid
 14",
 state:"running", attributes: prio:255

 136791   1 3670427870239520 KTRGRAPH group:"load", id:"CPU 1 load",
 counter:0,
 attributes: none

 136790   1 3670427870239248 KTRGRAPH group:"thread", id:"firefox tid
 113473",
 state:"blocked", attributes: prio:122, wmesg:"(null)", lockname:"unp_mtx"

 136789   1 3670427870237697 KTRGRAPH group:"load", id:"CPU 0 load",
 counter:2,
 attributes: none

 136788   1 3670427870236394 KTRGRAPH group:"thread", id:"firefox tid
 113473",
 point:"wokeup", attributes: linkedto:"Xorg tid 102818"

 136787   1 3670427870236145 KTRGRAPH group:"thread", id:"Xorg tid 102818",
 state:"runq add", attributes: prio:122, linkedto:"firefox tid 113473"

 136786   1 3670427870235981 KTRGRAPH group:"load", id:"CPU 1 load",
 counter:1,
 attributes: none

 136785   1 3670427870235707 KTRGRAPH group:"thread", id:"Xorg tid 102818",
 state:"runq rem", attributes: prio:176

 136784   1 3670427870235423 KTRGRAPH group:"thread", id:"Xorg tid 102818",
 point:"prio", attributes: prio:176, new prio:122, linkedto:"firefox tid
 113473"

 136783   1 3670427870202392 KTRGRAPH group:"thread", id:"firefox tid
 113473",
 state:"running", attributes: prio:104

 See how how the Xorg thread was forced from CPU 1 to CPU 0 where it
 preempted
 cc1plus thread (I do have preemption enabled) only to leave CPU 1 with zero
 load.
>>>
>>> I think that the idea is bright, but I have reservations against the
>>> implementation because it seems to me there are too many layering
>>> violations.
>>
>> Just one - for a layer between tunrstile and scheduler :-)
>> But I agree.
>>
>>> What is suggest is somewhat summarized like that:
>>> - Add a new SRQ_WILLSLEEP or the name you prefer
>>> - Add a new "flags" argument to sched_lend_prio() (both ule and 4bsd)
>>> and sched_thread_priority (ule only)
>>> - sched_thread_priority() will pass down the new flag to sched_add()
>>> which passed down to sched_pickcpu().
>>>
>>> This way sched_pickcpu() has the correct knowledge of what is going on
>>> and it can make the right decision. You likely don't need to lower the
>>> tdq_load at that time either this way, because sched_pickcpu() can
>>> just adjust it locally for its decision.
>>>
>>> What do you think?
>>
>> This sounds easy but it is not quite so given the implementation of
>> sched_pickcpu and sched_lowest.  This is probably more work than I am able to
>> take now.
> 
> I think actually that the attached patch should do what you need. Of
> course there is more runqueue locking, due to the tdq_load fondling,
> but I'm not sure it is really a big deal.
> I didn't test it yet, as I understand you have a test case, so maybe
> you can. However if Jeff agrees I can send the patch to flo@ for
> performance evaluation.
> 
> Thoughts?

Originally I had a similar patch with a small difference that I did tdq_load
manipulations in sched_thread_priority around sched_add call.  That patch
produced a deadlock because of the need for TDQ_LOCK.

> Index: sys/sys/sched.h
> ===
> --- sys/sys/sched.h (revisione 240672)
> +++ sys/sys/sched.h (copia locale)
> @@ -91,7 +91,7 @@ void  sched_nice(struct proc *p, int nice);
>   */
>  void   sched_exit_thread(struct thread *td, struct thread *child);
>  void   sched_fork_thread(struct thread *td, struct thread *child);
> -void   sched_lend_prio(struct thread *td, u_char prio);
> +void   sched_lend_prio(struct thread *td, u_char prio, int flags);
>  void   sched_lend_user_prio(struct thread *td, u_char pri);
>  fixpt_tsched_pctcpu(struct thread *td);
>  void   sched_prio(struct thread *td, u_char prio);
> @@ -161,6 +161,7 @@ sched_unpin(void)
>  #defineSRQ_INTR0x0004  /* It is probably urgent. */
>  #defineSRQ_PREEMPTED   0x0008  /* has been
> preempted.. be kind */
>  #defineSRQ_BORROWING   0x0010  /* Priority updated
> due to prio_lend */
> +#defineSRQ_WILLSWITCH  0x0020  /* curthread will be
> switched out */
> 
>  /* Scheduler stats. */
>  #ifdef SCHED_STATS
> Index: sys/kern/sched_ule.c
> =

Re: fdgrowtable() cleanup

[Konstantin Belousov]

On Tue, Sep 18, 2012 at 05:46:23PM +0200, Dag-Erling Sm??rgrav wrote:
> The patch below my signature improves the legibility of fdgrowtable(),
> and adds comments explaining the hairier bits.  The only functional
> change is that the code no longer overwrites the old fileflags array
> when the old table is placed on the freelist; instead, it uses the space
> actually set aside for that purpose.
> 
> (I assume that the old behavior was harmless, since it has persisted for
> decades, but it was certainly confusing.)
> 
> The slightly repetitive nature of the new code is intentional.
> 
> DES
> -- 
> Dag-Erling Sm??rgrav - d...@des.no
> 
> Index: sys/kern/kern_descrip.c
> ===
> --- sys/kern/kern_descrip.c   (revision 240654)
> +++ sys/kern/kern_descrip.c   (working copy)
> @@ -148,11 +148,6 @@
>  #define  NDSLOTS(x)  (((x) + NDENTRIES - 1) / NDENTRIES)
>  
>  /*
> - * Storage required per open file descriptor.
> - */
> -#define OFILESIZE (sizeof(struct file *) + sizeof(char))
> -
> -/*
>   * Storage to hold unused ofiles that need to be reclaimed.
>   */
>  struct freetable {
> @@ -1436,7 +1431,7 @@
>   struct freetable *fo;
>   struct file **ntable;
>   struct file **otable;
> - char *nfileflags;
> + char *nfileflags, *ofileflags;
>   int nnfiles, onfiles;
>   NDSLOTTYPE *nmap;
>  
> @@ -1447,33 +1442,46 @@
>  
>   /* compute the size of the new table */
>   onfiles = fdp->fd_nfiles;
> + otable = fdp->fd_ofiles;
> + ofileflags = fdp->fd_ofileflags;
These two new calculations could be unused if the function return early.

>   nnfiles = NDSLOTS(nfd) * NDENTRIES; /* round up */
>   if (nnfiles <= onfiles)
>   /* the table is already large enough */
>   return;
>  
> - /* allocate a new table and (if required) new bitmaps */
> - ntable = malloc((nnfiles * OFILESIZE) + sizeof(struct freetable),
> + /*
> +  * Allocate a new table.  We need enough space for a) the file
> +  * entries themselves, b) the file flags, and c) the struct
> +  * freetable we will use when we decommission the table and place
> +  * it on the freelist.
> +  */
> + ntable = malloc(nnfiles * sizeof(*ntable) +
> + nnfiles * sizeof(*nfileflags) +
> + sizeof(struct freetable),
>   M_FILEDESC, M_ZERO | M_WAITOK);
Please use the horizontal space less lavishly.

I think that this calculation, as well as fo calculation below, does not
take a required alignment of struct freetable into consideration.

>   nfileflags = (char *)&ntable[nnfiles];
> +
> + /* allocate new bitmaps if necessary */
>   if (NDSLOTS(nnfiles) > NDSLOTS(onfiles))
>   nmap = malloc(NDSLOTS(nnfiles) * NDSLOTSIZE,
>   M_FILEDESC, M_ZERO | M_WAITOK);
>   else
>   nmap = NULL;
>  
> - bcopy(fdp->fd_ofiles, ntable, onfiles * sizeof(*ntable));
> - bcopy(fdp->fd_ofileflags, nfileflags, onfiles);
> - otable = fdp->fd_ofiles;
> + /* copy the old data over and point at the new tables */
> + bcopy(otable, ntable, onfiles * sizeof(*otable));
> + bcopy(ofileflags, nfileflags, onfiles * sizeof(*ofileflags));
>   fdp->fd_ofileflags = nfileflags;
>   fdp->fd_ofiles = ntable;
> +
>   /*
>* We must preserve ofiles until the process exits because we can't
>* be certain that no threads have references to the old table via
>* _fget().
>*/
>   if (onfiles > NDFILE) {
> - fo = (struct freetable *)&otable[onfiles];
> + fo = (struct freetable *)(char *)otable +
> + onfiles * sizeof(*otable) + onfiles * sizeof(*ofileflags);
>   fdp0 = (struct filedesc0 *)fdp;
>   fo->ft_table = otable;
>   SLIST_INSERT_HEAD(&fdp0->fd_free, fo, ft_next);
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"


pgpE294LaSxZb.pgp
Description: PGP signature