calendar(1) regressions

[Jaakko Heinonen]

Hi,

Unfortunately r205821 [1] has caused several regressions to calendar(1).
Relevant PRs:

bin/157718
bin/162211
bin/168785
bin/170930

Some regressions were fixed in summer 2011 but they are still lacking
MFCs.

Is anyone aware of possible problems that reverting calendar(1) to
pre-r205821 version would cause? (Of course excluding the actual
calendar files.)

[1] http://svnweb.freebsd.org/base?view=revision&revision=205821

-- 
Jaakko
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: KVERIFY for non-debug invariants?

[Robert Watson]

On Wed, 5 Dec 2012, Vijay Singh wrote:

All. KASSERT() is a really need way of expressing invariants when INVARIANTS 
is defined. However for regular, non-INVARIANTS code folks have the typical 
if() panic() combos, or private macros. Would a KVERIFY() that does this in 
non-INVARIANTS code make sense?


I'd certainly be fine with something like this.  It might be worth posting to 
arch@ with a code example, as hackers@ has a subset of the potentially 
interested audience.  INVARIANTS has got a bit heavier-weight over the years 
-- the main thing I run into in higher-performance scenarios is its additional 
UMA debugging, which causes a global lock to be acquired during sanity checks. 
It might be worth our pondering adding a new configure option for particularly 
slow invariant tests -- e.g., INVARIANTS_SLOW ... or maybe just 
INVARIANTS_UMA.  However, that's a different issue.


(I sort of feel that things labeled "assert" should be something we can turn 
on in production... so maybe INVARIANTS/KASSERT mission-creep is the issue.)


Robert
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

huge ktr buffer

[Andriy Gapon]

So I configured a kernel with the following option:
options   KTR_ENTRIES="(1024UL*1024)"
then booted the kernel and did
$ sysctl debug.ktr.clear=1
and got an insta-reboot.

No panic, nothing, just a reset.

I suspect that the huge static buffer resulting from the above option could be a
cause.  But I would like to understand the details, if possible.

Also, perhaps ktr could be a little bit more sophisticated with its buffer than
just using a static array.

-- 
Andriy Gapon
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: huge ktr buffer

[Davide Italiano]

On Thu, Dec 6, 2012 at 4:18 PM, Andriy Gapon  wrote:
>
> So I configured a kernel with the following option:
> options   KTR_ENTRIES="(1024UL*1024)"
> then booted the kernel and did
> $ sysctl debug.ktr.clear=1
> and got an insta-reboot.
>
> No panic, nothing, just a reset.
>
> I suspect that the huge static buffer resulting from the above option could 
> be a
> cause.  But I would like to understand the details, if possible.
>
> Also, perhaps ktr could be a little bit more sophisticated with its buffer 
> than
> just using a static array.
>
> --
> Andriy Gapon
> ___
> freebsd-hackers@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

It was a while ago, but running r238886 built using the following
kernel configuration file:
http://people.freebsd.org/~davide/DEBUG I found a similar issue.
The machine  paniced: fatal trap 12 with interrupt disable in early boot
(even before the appareance of the Berkley logo).
Basically, my configuration file is just GENERIC with slight
modifications, in particular debugging options (WITNESS, INVARIANTS,
etc..) turned on and the following KTR options enabled:

options KTR
options KTR_COMPILE=(KTR_CALLOUT|KTR_PROC)
options KTR_MASK=(KTR_CALLOUT|KTR_PROC)
options KTR_ENTRIES=524288

It seems the issue is related to KTR itself, and in particular to the
value of KTR_ENTRIES. As long as this value is little (e.g. 2048)
everything works fine and the boot sequence ends. If I choose 524288
(the value you can also see from the kernel conf file) the fatal trap
occurs.

Even though it was really difficult to me to get some informations
because the fail happens too early, I put some printf() within the
code and I isolated the point in which the kernel dies:
(sys/amd64/amd64/machdep.c, in getmemsize())

1540/*
1541* map page into kernel: valid, read/write,non-cacheable
1542*/
1543*pte = pa | PG_V | PG_RW | PG_N;


As also Alan suggested, a way to workaround the problem is to increase
NKPT value (e.g. from 32 to 64). Obviously, this is not a proper fix.
For a proper fix the kernel needs to be able to dynamically set the
size of NKPT.  In this particular case, this wouldn't be too hard, but
there is a different case, where people preload a large memory disk
image at boot time that isn't so easy to fix.

Thanks,

Davide
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: huge ktr buffer

[Alan Cox]

On 12/06/2012 09:43, Davide Italiano wrote:
> On Thu, Dec 6, 2012 at 4:18 PM, Andriy Gapon  wrote:
>> So I configured a kernel with the following option:
>> options   KTR_ENTRIES="(1024UL*1024)"
>> then booted the kernel and did
>> $ sysctl debug.ktr.clear=1
>> and got an insta-reboot.
>>
>> No panic, nothing, just a reset.
>>
>> I suspect that the huge static buffer resulting from the above option could 
>> be a
>> cause.  But I would like to understand the details, if possible.
>>
>> Also, perhaps ktr could be a little bit more sophisticated with its buffer 
>> than
>> just using a static array.
>>
>> --
>> Andriy Gapon
>> ___
>> freebsd-hackers@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
> It was a while ago, but running r238886 built using the following
> kernel configuration file:
> http://people.freebsd.org/~davide/DEBUG I found a similar issue.
> The machine  paniced: fatal trap 12 with interrupt disable in early boot
> (even before the appareance of the Berkley logo).
> Basically, my configuration file is just GENERIC with slight
> modifications, in particular debugging options (WITNESS, INVARIANTS,
> etc..) turned on and the following KTR options enabled:
>
> options KTR
> options KTR_COMPILE=(KTR_CALLOUT|KTR_PROC)
> options KTR_MASK=(KTR_CALLOUT|KTR_PROC)
> options KTR_ENTRIES=524288
>
> It seems the issue is related to KTR itself, and in particular to the
> value of KTR_ENTRIES. As long as this value is little (e.g. 2048)
> everything works fine and the boot sequence ends. If I choose 524288
> (the value you can also see from the kernel conf file) the fatal trap
> occurs.
>
> Even though it was really difficult to me to get some informations
> because the fail happens too early, I put some printf() within the
> code and I isolated the point in which the kernel dies:
> (sys/amd64/amd64/machdep.c, in getmemsize())
>
> 1540/*
> 1541* map page into kernel: valid, read/write,non-cacheable
> 1542*/
> 1543*pte = pa | PG_V | PG_RW | PG_N;
>
>
> As also Alan suggested, a way to workaround the problem is to increase
> NKPT value (e.g. from 32 to 64). Obviously, this is not a proper fix.
> For a proper fix the kernel needs to be able to dynamically set the
> size of NKPT.  In this particular case, this wouldn't be too hard, but
> there is a different case, where people preload a large memory disk
> image at boot time that isn't so easy to fix.


Andriy makes a good suggestion.  One that I think should be easy to
implement.  The KTR code already supports the use of a dynamically
allocated KTR buffer.  (See sysctl_debug_ktr_entries().)  Let's take
advantage of this.  Place a cap on the size of the (compile-time)
statically allocated buffer.  However, use this buffer early in the
kernel initialization process, specifically, up until SI_ORDER_KMEM has
completed.  At that point, switch to a dynamically allocated buffer and
copy over the entries from the statically allocated buffer.  Relatively
speaking, SI_ORDER_KMEM is early enough in the boot process, that I
doubt many people wanting an enormous KTR buffer will be impacted by the
cap.  In fact, I think you could implement overflow detection without
pessimizing the KTR code.

Alan

P.S. There are other reasons that having an enormous statically
allocated array in the kernel is undesirable.  The first that comes to
mind is that it eats up memory at low physical addresses, which is
sometimes needed for special purposes.  So, I think there are good
reasons besides the NKPT issue to shift the KTR code to dynamic allocation.

___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: kernel module parallel build?

[John Baldwin]

On Wednesday, December 05, 2012 6:51:17 pm Damien Fleuriot wrote:
> 
> On 5 Dec 2012, at 18:39, Warner Losh  wrote:
> 
> > 
> > On Dec 5, 2012, at 9:42 AM, John Baldwin wrote:
> > 
> >> On Tuesday, December 04, 2012 2:41:32 pm Ryan Stone wrote:
> >>> On Tue, Dec 4, 2012 at 10:52 AM, John Baldwin  wrote:
> >>> 
>  Hmm, I certainly see the module directories being built in parallel.  
>  Some
>  of
>  the make jobs may not be as obvious since links are silent (no output
>  unless
>  there is an error).
>  
>  
> >>> This is definitely not the behaviour that I see trying to build any 
> >>> version
> >>> of FreeBSD.  I see the same behaviour as Andre: the depend and all targets
> >>> both iterate through the module directories sequentially.  It never builds
> >>> two module subdirectories concurrently.
> >> 
> >> Hmm, I think I was confused by seeing kernel builds intermingle with the 
> >> associated modules.  sys/modules/Makefile uses bsd.subdir.mk.  I think I 
> >> see 
> >> similar things in world builds where I will see parallel builds of bin vs 
> >> sbin 
> >> vs usr.bin vs usr.sbin, but within each of those directories the builds go 
> >> sequentially.  I think you would need to change bsd.subdir.mk if you want 
> >> to 
> >> fix this.
> > 
> > The builds are in parallel, just that the parallelism is low because it is 
> > only parallel within the module being built. Would love to see a fix.
> > 
> > Warner
> > 
> 
> All trolling aside, I believe an awesome fix to be setting module override in 
> /etc/make.conf to only build the 4-5 specific modules one needs.
> 
> To be honest I think this configuration tweak should be advertised a bit more 
> as it definitely speeds up kernel builds.
> 
> I would be happy to check if this is advertised in the handbook in the 
> "rebuilding kernel" section and enhance its visibility if required.
> 
> I can provide en_US and fr_FR.

Better than doing it in /etc/make.conf (or /etc/src.conf) is doing it direclty
in the kernel config file itself via

makeoptions MODULES_OVERRIDE="foo"

You can use multiple of these (with +=) in a config file as well.

-- 
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

any arch not pack uint32_t x[2]?

[Rick Macklem]

Hi,

The subject line pretty well says it. I am about ready
to commit the NFSv4.1 client patches, but I had better
ask this dump question first.

Is there any architecture where:
  uint32_t x[2];
isn't packed? (Or, sizeof(x) != 8, if you prefer.)

As you might have guessed, if the answer is "yes", I have
some code fixin to do, rick
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"

Re: ELF symtab and ddbsymtab difference

[Konstantin Belousov]

On Wed, Dec 05, 2012 at 12:13:24PM +0530, Shrikanth Kamath wrote:
> This is regarding the fields in the structure "elf_file_t" in link_elf.c.
> For some kernel modules the symtab field is different from the ddbsymtab
> field for some it is the same, would like to know what is the difference
> between the two and how to enable ddbsymtab?
Assuming we are talking about the link_elf.c and not about link_elf_obj.c.
The symtab and ddbsymtab are first initialized from the dynamic symbol
table in the module, and later, in the process of loading the module, if
the non-dynamic symbol table is present, ddbsymtab is rewritten to point
to the table.

> 
> Does enabling "-g" in CFLAGS make the binary build the ddbsymtab different
> from symtab?
No. It is strip done on the module which could result in the removal of the
non-dynamic symtab.

> 
> The problem is lookup for some symbols in the kernel module that I built
> returns with undefined, on inspecting it was getting a ENOENT from the
> function
> link_elf_lookup_symbol()
> {
>  ...
>  /* If we have not found it, look at the full table (if loaded) */
>  if (ef->symtab == ef->ddbsymtab)
>  return (ENOENT);
>  ...
> }

It is not the problem. It just means that you dynamic symbol table does
not contain the needed symbol, which is the problem. As a coincident,
you also stripped your module, making the problem to be exposed.

I guess that you should look at the EXPORT_SYMS feature of the module
Makefiles. But I also remember there were some bugs.


pgpYNFsprvOlA.pgp
Description: PGP signature

Re: ELF symtab and ddbsymtab difference

[Shrikanth Kamath]

Thanks Konstantin, yeah I think there were two levels of strip
happening, one removing the debug sections and another
was removing the .strtab and .symtab. I have EXPORT_SYMS = YES in my
Makefile but that was not helping as the
variables in context are declared static (they are going into the
.bss). Making it un-static helps or avoiding the second
level strip helps too..

--
Shrikanth R K

On Fri, Dec 7, 2012 at 4:41 AM, Konstantin Belousov  wrote:
> On Wed, Dec 05, 2012 at 12:13:24PM +0530, Shrikanth Kamath wrote:
>> This is regarding the fields in the structure "elf_file_t" in link_elf.c.
>> For some kernel modules the symtab field is different from the ddbsymtab
>> field for some it is the same, would like to know what is the difference
>> between the two and how to enable ddbsymtab?
> Assuming we are talking about the link_elf.c and not about link_elf_obj.c.
> The symtab and ddbsymtab are first initialized from the dynamic symbol
> table in the module, and later, in the process of loading the module, if
> the non-dynamic symbol table is present, ddbsymtab is rewritten to point
> to the table.
>
>>
>> Does enabling "-g" in CFLAGS make the binary build the ddbsymtab different
>> from symtab?
> No. It is strip done on the module which could result in the removal of the
> non-dynamic symtab.
>
>>
>> The problem is lookup for some symbols in the kernel module that I built
>> returns with undefined, on inspecting it was getting a ENOENT from the
>> function
>> link_elf_lookup_symbol()
>> {
>>  ...
>>  /* If we have not found it, look at the full table (if loaded) */
>>  if (ef->symtab == ef->ddbsymtab)
>>  return (ENOENT);
>>  ...
>> }
>
> It is not the problem. It just means that you dynamic symbol table does
> not contain the needed symbol, which is the problem. As a coincident,
> you also stripped your module, making the problem to be exposed.
>
> I guess that you should look at the EXPORT_SYMS feature of the module
> Makefiles. But I also remember there were some bugs.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"