[Bug rtl-optimization/52629] New: global buffer overflow in gcc/reload1.c

[konstantin.s.serebryany at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52629

             Bug #: 52629
           Summary: global buffer overflow in gcc/reload1.c
    Classification: Unclassified
           Product: gcc
           Version: 4.8.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: rtl-optimization
        AssignedTo: unassig...@gcc.gnu.org
        ReportedBy: konstantin.s.serebry...@gmail.com


Building gcc trunk (r185531, linux, x86_64) with AddressSanitizer (a memory
error detector)
shows a global buffer overflow in gcc/reload1.c while compiling
libgcc/libgcc2.c with ./gcc/xgcc

==21687== ERROR: AddressSanitizer global-buffer-overflow on address
0x000002ddc551 at pc 0xf98a5f bp 0x7fffdcbe1050 sp 0x7fffdcbe1048               
READ of size 1 at 0x000002ddc551 thread T0                                      
    #0 0xf98a5f in count_spilled_pseudo gcc/reload1.c:1830                      
    #1 0xf715a6 in bmp_iter_set_init gcc/bitmap.h:385                           
    #2 0xd55dc2 in do_reload gcc/ira.c:3733                                     
    #3 0xea9e0d in execute_one_pass gcc/passes.c:2084                           
    #4 0xeaaf4d in execute_pass_list gcc/passes.c:2139                          
    #5 0xeaaf71 in execute_pass_list gcc/passes.c:2141                          
    #6 0x1200802 in invoke_plugin_callbacks gcc/plugin.h:57                     
    #7 0x81917d in cgraph_expand_function gcc/cgraphunit.c:1840                 
    #8 0x820425 in cgraph_expand_all_functions gcc/cgraphunit.c:1904            
    #9 0x81c655 in timevar_pop gcc/timevar.h:110                                
    #10 0x4ebc2e in c_write_global_declarations gcc/c-decl.c:10053              
    #11 0x10b82c5 in compile_file gcc/toplev.c:575                              
    #12 0x2b1cbf568c4d in __libc_start_main
/build/buildd/eglibc-2.11.1/csu/libc-start.c:258                                
0x000002ddc551 is located 17 bytes to the right of global variable
'default_target_hard_regs (../../gcc_trunk/gcc/reginfo.c)' (0x2dd9d80) of size
10176  

The buggy lines: 

static void                                                                     
count_spilled_pseudo (int spilled, int spilled_nregs, int reg)                  
{                                                                               
  int freq = REG_FREQ (reg);                                                    
  int r = reg_renumber[reg];                                                    
  int nregs = hard_regno_nregs[r][PSEUDO_REGNO_MODE (reg)];             


the value of 'r' can be -1, thus the buffer overflow. 

One can confirm this bug by changing the code like this and performing full gcc
build:

count_spilled_pseudo (int spilled, int spilled_nregs, int reg)
{
  int freq = REG_FREQ (reg);
  int r = reg_renumber[reg];
  int nregs;
  gcc_assert(r >= 0);  /*<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<*/
  nregs = hard_regno_nregs[r][PSEUDO_REGNO_MODE (reg)];