Dear Dan, Martin and Nate,
Thank you for looking into it. I guess that is pointing to a problem
within bowtie.
It looks like the EXC_BAD_ACCESS you see on petty in "ebwt.h" is not
reproducible on the other Mac or Linux machines we tried. Is it possible
to run valgrind on petty? That may confirm/rule out if the memory
(de-)allocation issues reported on Linux are related.
I would like to submit a bug-report to the bowtie developers, but am
reluctant to do that without being able to reproduce the problem or test
potential fixed. I would have the options to go through Rbowtie build
cycles, but would have to rely on the assumption that petty will keep
hitting this hickup even with modified bowtie code. The minor
differences between bowtie 1.0.1 and bowtie 1.0.1-bug-312 argue against
that.
I am tempted to stay with the current situtation:
- OS X before 10.9 needs to use Rbowtie <= 1.4.4
(based on bowtie 1.0.1)
- OS X 10.9 onwards and everything else uses Rbowtie >= 1.4.5
(based on bowtie 1.0.1 /patched bugs-312).
Thanks again for your efforts,
Michael
On 14.06.2014 01:31, Dan Tenenbaum wrote:
> Hi Michael,
>
>
>
> ----- Original Message -----
>> From: "Michael Stadler" <michael.stad...@fmi.ch>
>> To: "Dan Tenenbaum" <dtene...@fhcrc.org>, bioc-devel@r-project.org
>> Sent: Friday, June 13, 2014 12:32:52 AM
>> Subject: differences between petty and perceval (OS X 10.6.8 build machines
>> for release/devel)
>>
>> Hi Dan,
>>
>> I'm cc'ing the list; maybe somebody else has experienced differences
>> between petty and perceval.
>>
>> Rbowtie release (1.4.5) is not building under OS X 10.6.8 (petty).
>>
>> Rbowtie release (1.4.5) and development (1.5.5) are virtually
>> identical
>> (only DESCRIPTION and NEWS differ).
>>
>> The development version builds without problems on perceval, but the
>> release version fails on petty:
>> http://bioconductor.org/checkResults/devel/bioc-LATEST/Rbowtie/perceval-buildsrc.html
>> http://bioconductor.org/checkResults/release/bioc-LATEST/Rbowtie/petty-buildsrc.html
>>
>> The only difference I can make out from the node info pages is that
>> perceval has an additional section on "C++11 compiler" that is
>> lacking
>> from petty's NodeInfo page.
>>
>> Unfortunately, I cannot reproduce the issue, both Rbowtie 1.4.5 and
>> 1.5.5 build successfully under OS X 10.6.8 and 10.7.5 using
>> llvm-gcc-4.2.
>>
>> Do you have any idea what else could be different between petty and
>> perceval?
>
> Martin and Nate and I took a look at this. I managed to come up with a bowtie
> command line that would reliably reproduce the segfault on petty.
>
> Then we ran that under gdb (and turned off compiler optimizations) and came
> up with this, which may or may not help you:
>
> petty:vignettes biocbuild$ gdb --args
> '/Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rbowtie/bowtie'
> -y -S -k 10 -m 10 -v 2 -r -p 4 --best --strata 'doit/refsIndex/index'
> 'doit/SpliceMapTemp_876c378e20ac/25mers.map'
> 'doit/SpliceMapTemp_876c378e20ac/25mers.map_unsorted'
> GNU gdb 6.3.50-20050815 (Apple version gdb-1708) (Mon Aug 15 16:03:10 UTC
> 2011)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for details.
> This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared
> libraries ... done
>
> (gdb) run
> Starting program:
> /Library/Frameworks/R.framework/Versions/3.1/Resources/library/Rbowtie/bowtie
> -y -S -k 10 -m 10 -v 2 -r -p 4 --best --strata doit/refsIndex/index
> doit/SpliceMapTemp_876c378e20ac/25mers.map
> doit/SpliceMapTemp_876c378e20ac/25mers.map_unsorted
> Reading symbols for shared libraries ++. done
>
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x23d0d92d
> [Switching to process 36144 thread 0x20f]
> 0x000478b1 in Ebwt<seqan::String<seqan::SimpleType<unsigned char,
> seqan::_Dna>, seqan::Alloc<void> > >::rowL (this=0xbfffda10, l=@0xa300e14) at
> ebwt.h:1816
> 1816 return unpack_2b_from_8b(l.side(this->_ebwt)[l._by], l._bp);
> (gdb) l
> 1811 inline int Ebwt<TStr>::rowL(const SideLocus& l) const {
> 1812 // Extract and return appropriate bit-pair
> 1813 #ifdef SIXTY4_FORMAT
> 1814 return (((uint64_t*)l.side(this->_ebwt))[l._by >> 3] >> ((((l._by
> & 7) << 2) + l._bp) << 1)) & 3;
> 1815 #else
> 1816 return unpack_2b_from_8b(l.side(this->_ebwt)[l._by], l._bp);
> 1817 #endif
> 1818 }
> 1819
> 1820 /**
> (gdb) p this ->_ebwt
> $1 = (uint8_t *) 0x4804a00 "\b<2"
> (gdb) p *this ->_ebwt
> $2 = 8 '\b'
> (gdb) p l._by
> $3 = 45
> (gdb) p l.side
> $4 = &SideLocus::side(unsigned char const*) const
> (gdb) p l.side(this->_ebwt)
> $5 = (uint8_t *) 0x23d0d900 <Address 0x23d0d900 out of bounds>
> (gdb) p l.side(this->_ebwt)[l._by]
> Cannot access memory at address 0x23d0d92d
> (gdb) p this ->_ebwt
> $6 = (uint8_t *) 0x4804a00 "\b<2"
> (gdb)
>
> Running the same command line under valgrind (on a linux box) produces the
> following:
>
> ==14916== Memcheck, a memory error detector
> ==14916== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
> ==14916== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
> ==14916== Command: /home/biocbuild/bbs-2.14-bioc/R/library/Rbowtie/bowtie -y
> -S -k 10 -m 10 -v 2 -r -p 4 --best --strata doit/refsIndex/index
> doit/SpliceMapTemp_876c378e20ac/25mers.map
> doit/SpliceMapTemp_876c378e20ac/25mers.map_unsorted
> ==14916==
> # reads processed: 24
> # reads with at least one reported alignment: 18 (75.00%)
> # reads that failed to align: 6 (25.00%)
> Reported 18 alignments to 1 output stream(s)
> ==14916==
> ==14916== HEAP SUMMARY:
> ==14916== in use at exit: 804 bytes in 7 blocks
> ==14916== total heap usage: 647 allocs, 640 frees, 278,139,442 bytes
> allocated
> ==14916==
> ==14916== 4 bytes in 1 blocks are definitely lost in loss record 1 of 3
> ==14916== at 0x52FB1C7: operator new(unsigned long) (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14916== by 0x4315DB: HitSink::HitSink(OutFileBuf*, std::string const&,
> std::string const&, std::string const&, bool, bool, RecalTable*,
> std::vector<std::string, std::allocator<std::string> >*) (hit.h:332)
> ==14916== by 0x414742:
> _Z6driverIN5seqan6StringINS0_10SimpleTypeIhNS0_4_DnaEEENS0_5AllocIvEEEEEvPKcRKSsSB_RKSt6vectorISsSaISsEESG_SB_.isra.1161.constprop.1242
> (sam.h:53)
> ==14916== by 0x41719A: bowtie (ebwt_search.cpp:2864)
> ==14916== by 0x4060B9: main (bowtie_main.cpp:60)
> ==14916==
> ==14916== 224 bytes in 4 blocks are definitely lost in loss record 2 of 3
> ==14916== at 0x52FB1C7: operator new(unsigned long) (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14916== by 0x40C4D4: void
> twoOrThreeMismatchSearchFull<seqan::String<seqan::SimpleType<unsigned char,
> seqan::_Dna>, seqan::Alloc<void> > >(PairedPatternSource&, HitSink&,
> Ebwt<seqan::String<seqan::SimpleType<unsigned char, seqan::_Dna>,
> seqan::Alloc<void> > >&, Ebwt<seqan::String<seqan::SimpleType<unsigned char,
> seqan::_Dna>, seqan::Alloc<void> > >&,
> std::vector<seqan::String<seqan::SimpleType<unsigned char, seqan::_Dna5>,
> seqan::Alloc<void> >, std::allocator<seqan::String<seqan::SimpleType<unsigned
> char, seqan::_Dna5>, seqan::Alloc<void> > > >&, bool) (ebwt_search.cpp:1873)
> ==14916== by 0x413A20:
> _Z6driverIN5seqan6StringINS0_10SimpleTypeIhNS0_4_DnaEEENS0_5AllocIvEEEEEvPKcRKSsSB_RKSt6vectorISsSaISsEESG_SB_.isra.1161.constprop.1242
> (ebwt_search.cpp:2694)
> ==14916== by 0x41719A: bowtie (ebwt_search.cpp:2864)
> ==14916== by 0x4060B9: main (bowtie_main.cpp:60)
> ==14916==
> ==14916== 576 bytes in 2 blocks are possibly lost in loss record 3 of 3
> ==14916== at 0x52F9DB4: calloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==14916== by 0x46E2074: _dl_allocate_tls (dl-tls.c:297)
> ==14916== by 0x550AABC: pthread_create@@GLIBC_2.2.5 (allocatestack.c:571)
> ==14916== by 0x46D358: tthread::thread::thread(void (*)(void*), void*)
> (tinythread.cpp:207)
> ==14916== by 0x40C4E7: void
> twoOrThreeMismatchSearchFull<seqan::String<seqan::SimpleType<unsigned char,
> seqan::_Dna>, seqan::Alloc<void> > >(PairedPatternSource&, HitSink&,
> Ebwt<seqan::String<seqan::SimpleType<unsigned char, seqan::_Dna>,
> seqan::Alloc<void> > >&, Ebwt<seqan::String<seqan::SimpleType<unsigned char,
> seqan::_Dna>, seqan::Alloc<void> > >&,
> std::vector<seqan::String<seqan::SimpleType<unsigned char, seqan::_Dna5>,
> seqan::Alloc<void> >, std::allocator<seqan::String<seqan::SimpleType<unsigned
> char, seqan::_Dna5>, seqan::Alloc<void> > > >&, bool) (ebwt_search.cpp:1873)
> ==14916== by 0x413A20:
> _Z6driverIN5seqan6StringINS0_10SimpleTypeIhNS0_4_DnaEEENS0_5AllocIvEEEEEvPKcRKSsSB_RKSt6vectorISsSaISsEESG_SB_.isra.1161.constprop.1242
> (ebwt_search.cpp:2694)
> ==14916== by 0x41719A: bowtie (ebwt_search.cpp:2864)
> ==14916== by 0x4060B9: main (bowtie_main.cpp:60)
> ==14916==
> ==14916== LEAK SUMMARY:
> ==14916== definitely lost: 228 bytes in 5 blocks
> ==14916== indirectly lost: 0 bytes in 0 blocks
> ==14916== possibly lost: 576 bytes in 2 blocks
> ==14916== still reachable: 0 bytes in 0 blocks
> ==14916== suppressed: 0 bytes in 0 blocks
> ==14916==
> ==14916== For counts of detected and suppressed errors, rerun with: -v
> ==14916== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 2 from 2)
>
> So maybe those are places to look? Apparently these sorts of memory errors
> can occur on one machine even when it does not occur on an identically
> configured machine.
>
> Dan
>
>
>
>>
>> Thank you,
>> Michael
>>
_______________________________________________
Bioc-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel
