Dear Maintainer,
I tried to have a look at this crash and could it reproduce it
inside a minimal testing amd64 VM.

A backtrace with full debug symbols in [1].

The crash itself happens because of a stack exhaustion, because in
_celt_autocorr a 2015 MB array should be constructed.

But this seems to be just a consequence of function compute_frame_features
calling function pitch_downsample, which is unfortunately first found by
the dynamic linker in libcodec2.so.0.9 instead of obs-filters.so.
And the function signatures are quite different.

Therefore a cheap workaround currently might be to start obs-studio with
obs-filters preloaded. That way at least the denoise filter can be added.

    LD_PRELOAD=/usr/lib/x86_64-linux-gnu/obs-plugins/obs-filters.so obs

But might break when libcodec2.so.0.9 wants to call its function 
pitch_downsample.
Another more reliable option might be to rename
that and the other common functions.
The last part of the attached file shows all common function
between obs-filters.so and libcodec2.so.0.9.

Kind regards,
Bernhard


[1]
    (gdb) bt
    #0  0x00007f1bee6e4b71 in _celt_autocorr (x=x@entry=0x7f1aacfee8e8, 
ac=ac@entry=0x7f1aacfee7c0, window=window@entry=0x0, overlap=overlap@entry=0, 
lag=lag@entry=4, n=n@entry=-1392576048) at ./lpcnet/src/celt_lpc.c:216
    #1  0x00007f1bee6e4d2a in pitch_downsample (x_lp=x_lp@entry=0x7f1aacfee8e8, 
len=len@entry=-1392576048) at ./lpcnet/src/pitch.c:160
    #2  0x00007f1b30293006 in compute_frame_features (in=0x7f1aacfeeac0, 
features=0x7f1aacfeea10, Exp=0x7f1aacfee9b0, Ep=0x7f1aacfee950, 
Ex=0x7f1aacfee8f0, P=0x7f1aacff2560, X=0x7f1aacff1650, st=0x562d5f38cff0) at 
./plugins/obs-filters/rnnoise/src/denoise.c:326
    #3  rnnoise_process_frame (st=0x562d5f38cff0, out=<optimized out>, 
in=<optimized out>) at ./plugins/obs-filters/rnnoise/src/denoise.c:471
    #4  0x00007f1b302a5f4c in process_rnnoise (ng=0x562d5f373f40) at 
./plugins/obs-filters/noise-suppress-filter.c:343
    #5  process (ng=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:392
    #6  noise_suppress_filter_audio (data=0x562d5f373f40, audio=<optimized 
out>) at ./plugins/obs-filters/noise-suppress-filter.c:468
    #7  0x00007f1bf1beab8b in filter_async_audio (in=<optimized out>, 
source=0x562d5f26bf00) at ./libobs/obs-source.c:3068
    #8  obs_source_output_audio (source=0x562d5f26bf00, 
audio=audio@entry=0x7f1aacff81f0) at ./libobs/obs-source.c:3246
    #9  0x00007f1bcc002774 in _alsa_listen (attr=0x7f1be000dc20) at 
./plugins/linux-alsa/alsa-input.c:574
    #10 0x00007f1bf0c26ea7 in start_thread (arg=<optimized out>) at 
pthread_create.c:477
    #11 0x00007f1bf0b56d4f in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:95

# Bullseye/testing amd64 qemu VM 2020-11-14


apt update
apt dist-upgrade


apt install systemd-coredump mc htop psmisc net-tools strace xdm xserver-xorg 
openbox xterm gdb obs-studio obs-studio-dbgsym obs-plugins-dbgsym 
libobs0-dbgsym libcodec2-0.9-dbgsym
apt build-dep libcodec2-0.9
apt build-dep obs-studio


reboot




mkdir /home/benutzer/source/libcodec2-0.9/orig -p
cd    /home/benutzer/source/libcodec2-0.9/orig
apt source libcodec2-0.9
cd

mkdir /home/benutzer/source/obs-studio/orig -p
cd    /home/benutzer/source/obs-studio/orig
apt source obs-studio
cd






export LANG=C
export DISPLAY=:0

obs

- Cancel autoconfiguration
- Sources: Add source, Audio Capture Device Alsa
- Sources: Right click new entry - "Filters"
- Audio Filters: Add "Noise suppresion"





benutzer@debian:~$ obs
...
info: alsa-input: PCM 'default' rate set to 44100
info: alsa-input: PCM 'default' channels set to 2
info: User added source 'Audio Capture Device (ALSA)' (alsa_input_capture) to 
scene 'Scene'
info: adding 64 milliseconds of audio buffering, total audio buffering is now 
64 milliseconds (source: Audio Capture Device (ALSA))

info: User added filter 'Noise Suppression' (noise_suppress_filter_v2) to 
source 'Audio Capture Device (ALSA)'
Segmentation fault (core dumped)




root@debian:~# journalctl --no-pager
...
Nov 14 14:37:13 debian kernel: obs[891]: segfault at 7f1bf902fff8 ip 
00007f1bee6e4b71 sp 00007f1bf9030000 error 6 in 
libcodec2.so.0.9[7f1bee6b4000+38000]
Nov 14 14:37:13 debian kernel: Code: d9 00 00 00 48 8d 35 8e 01 01 00 48 8d 3d 
0b 02 01 00 e8 12 f4 ff ff ba d8 00 00 00 48 8d 35 76 01 01 00 48 8d 3d dd 01 
01 00 <e8> fa f3 ff ff e8 95 fa fc ff 0f 1f 44 00 00 48 b8 00 00 00 00 01
Nov 14 14:37:13 debian systemd[1]: Created slice 
system-systemd\x2dcoredump.slice.
Nov 14 14:37:13 debian systemd[1]: Started Process Core Dump (PID 896/UID 0).
Nov 14 14:37:15 debian kernel: snd_hda_intel 0000:00:05.0: IRQ timing 
workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
Nov 14 14:37:17 debian systemd-coredump[897]: [🡕] Process 705 (obs) of user 
1000 dumped core.
                                              
                                              Stack trace of thread 891:
                                              #0  0x00007f1bee6e4b71 
_celt_autocorr (libcodec2.so.0.9 + 0x3cb71)
                                              #1  0x00007f1bee6e4d2a 
pitch_downsample (libcodec2.so.0.9 + 0x3cd2a)
                                              #2  0x00007f1b30293006 
rnnoise_process_frame (obs-filters.so + 0x9006)
                                              #3  0x00007f1b302a5f4c n/a 
(obs-filters.so + 0x1bf4c)
                                              #4  0x00007f1bf1beab8b 
obs_source_output_audio (libobs.so.0 + 0x74b8b)
                                              #5  0x00007f1bcc002774 n/a 
(linux-alsa.so + 0x2774)
                                              #6  0x00007f1bf0c26ea7 
start_thread (libpthread.so.0 + 0x8ea7)
                                              #7  0x00007f1bf0b56d4f __clone 
(libc.so.6 + 0xfdd4f)
...
                                              Stack trace of thread 727:
                                              #0  0x00007f1bf0c2d7b2 
pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                                              #1  0x00007f1bcdaef80b n/a 
(swrast_dri.so + 0x69b80b)
                                              #2  0x00007f1bcdaef1a7 n/a 
(swrast_dri.so + 0x69b1a7)
                                              #3  0x00007f1bf0c26ea7 
start_thread (libpthread.so.0 + 0x8ea7)
                                              #4  0x00007f1bf0b56d4f __clone 
(libc.so.6 + 0xfdd4f)
Nov 14 14:37:17 debian systemd[1]: systemd-coredump@0-896-0.service: Succeeded.



root@debian:~# coredumpctl list
TIME                            PID   UID   GID SIG COREFILE  EXE
Sat 2020-11-14 14:37:17 CET     705  1000  1000  11 present   /usr/bin/obs


root@debian:~# coredumpctl gdb 705
           PID: 705 (obs)
           UID: 1000 (benutzer)
           GID: 1000 (benutzer)
        Signal: 11 (SEGV)
     Timestamp: Sat 2020-11-14 14:37:13 CET (1min 52s ago)
  Command Line: obs
    Executable: /usr/bin/obs
 Control Group: /user.slice/user-1000.slice/session-4.scope
          Unit: session-4.scope
         Slice: user-1000.slice
       Session: 4
     Owner UID: 1000 (benutzer)
       Boot ID: f5db46c8509344d280e4efdb1cd5ecda
    Machine ID: 33f18f39d2a9438eb75b0ed52848afcd
      Hostname: debian
       Storage: 
/var/lib/systemd/coredump/core.obs.1000.f5db46c8509344d280e4efdb1cd5ecda.705.1605361033000000.zst
       Message: Process 705 (obs) of user 1000 dumped core.
                
                Stack trace of thread 891:
...
Core was generated by `obs'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f1bee6e4b71 in _celt_autocorr () from 
/lib/x86_64-linux-gnu/libcodec2.so.0.9
[Current thread is 1 (Thread 0x7f1aacff9700 (LWP 891))]
(gdb) set width 0
(gdb) set pagination off
(gdb) bt
#0  0x00007f1bee6e4b71 in _celt_autocorr () from 
/lib/x86_64-linux-gnu/libcodec2.so.0.9
#1  0x00007f1bee6e4d2a in pitch_downsample () from 
/lib/x86_64-linux-gnu/libcodec2.so.0.9
#2  0x00007f1b30293006 in rnnoise_process_frame () from 
/usr//lib/x86_64-linux-gnu/obs-plugins/obs-filters.so
#3  0x00007f1b302a5f4c in ?? () from 
/usr//lib/x86_64-linux-gnu/obs-plugins/obs-filters.so
#4  0x00007f1bf1beab8b in obs_source_output_audio () from 
/lib/x86_64-linux-gnu/libobs.so.0
#5  0x00007f1bcc002774 in ?? () from 
/usr//lib/x86_64-linux-gnu/obs-plugins/linux-alsa.so
#6  0x00007f1bf0c26ea7 in start_thread (arg=<optimized out>) at 
pthread_create.c:477
#7  0x00007f1bf0b56d4f in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)



Core was generated by `obs'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f1bee6e4b71 in _celt_autocorr (x=x@entry=0x7f1aacfee8e8, 
ac=ac@entry=0x7f1aacfee7c0, window=window@entry=0x0, overlap=overlap@entry=0, 
lag=lag@entry=4, n=n@entry=-1392576048)
    at ./lpcnet/src/celt_lpc.c:216
216     ./lpcnet/src/celt_lpc.c: Datei oder Verzeichnis nicht gefunden.
[Current thread is 1 (Thread 0x7f1aacff9700 (LWP 891))]
(gdb) set width 0
(gdb) set pagination off
(gdb) bt
#0  0x00007f1bee6e4b71 in _celt_autocorr (x=x@entry=0x7f1aacfee8e8, 
ac=ac@entry=0x7f1aacfee7c0, window=window@entry=0x0, overlap=overlap@entry=0, 
lag=lag@entry=4, n=n@entry=-1392576048) at ./lpcnet/src/celt_lpc.c:216
#1  0x00007f1bee6e4d2a in pitch_downsample (x_lp=x_lp@entry=0x7f1aacfee8e8, 
len=len@entry=-1392576048) at ./lpcnet/src/pitch.c:160
#2  0x00007f1b30293006 in compute_frame_features (in=0x7f1aacfeeac0, 
features=0x7f1aacfeea10, Exp=0x7f1aacfee9b0, Ep=0x7f1aacfee950, 
Ex=0x7f1aacfee8f0, P=0x7f1aacff2560, X=0x7f1aacff1650, st=0x562d5f38cff0) at 
./plugins/obs-filters/rnnoise/src/denoise.c:326
#3  rnnoise_process_frame (st=0x562d5f38cff0, out=<optimized out>, 
in=<optimized out>) at ./plugins/obs-filters/rnnoise/src/denoise.c:471
#4  0x00007f1b302a5f4c in process_rnnoise (ng=0x562d5f373f40) at 
./plugins/obs-filters/noise-suppress-filter.c:343
#5  process (ng=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:392
#6  noise_suppress_filter_audio (data=0x562d5f373f40, audio=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:468
#7  0x00007f1bf1beab8b in filter_async_audio (in=<optimized out>, 
source=0x562d5f26bf00) at ./libobs/obs-source.c:3068
#8  obs_source_output_audio (source=0x562d5f26bf00, 
audio=audio@entry=0x7f1aacff81f0) at ./libobs/obs-source.c:3246
#9  0x00007f1bcc002774 in _alsa_listen (attr=0x7f1be000dc20) at 
./plugins/linux-alsa/alsa-input.c:574
#10 0x00007f1bf0c26ea7 in start_thread (arg=<optimized out>) at 
pthread_create.c:477
#11 0x00007f1bf0b56d4f in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) display/i $pc
1: x/i $pc
=> 0x7f1bee6e4b71 <_celt_autocorr+449>: call   0x7f1bee6e3f70 <_celt_fatal>







segfault at 7f1bf902fff8 ip 00007f1bee6e4b71 sp 00007f1bf9030000 error 6

https://wiki.debian.org/InterpretingKernelOutputAtProcessCrash

decimal 6 == binary 110
    1: user-mode access
    1: write access
    0: no page found
.



(gdb) print/x $rsp
$1 = 0x7f1bf9030000
(gdb) up
#1  0x00007f1bee6e4d2a in pitch_downsample (x_lp=x_lp@entry=0x7f1aacfee8e8, 
len=len@entry=-1392576048) at ./lpcnet/src/pitch.c:160
160     ./lpcnet/src/pitch.c: Datei oder Verzeichnis nicht gefunden.
(gdb) print/x $rsp
$2 = 0x7f1aacfee7a0
(gdb) up
#2  0x00007f1b30293006 in compute_frame_features (in=0x7f1aacfeeac0, 
features=0x7f1aacfeea10, Exp=0x7f1aacfee9b0, Ep=0x7f1aacfee950, 
Ex=0x7f1aacfee8f0, P=0x7f1aacff2560, X=0x7f1aacff1650, st=0x562d5f38cff0) at 
./plugins/obs-filters/rnnoise/src/denoise.c:326
326     ./plugins/obs-filters/rnnoise/src/denoise.c: Datei oder Verzeichnis 
nicht gefunden.
(gdb) print/x $rsp
$3 = 0x7f1aacfee810
(gdb) up
#3  rnnoise_process_frame (st=0x562d5f38cff0, out=<optimized out>, 
in=<optimized out>) at ./plugins/obs-filters/rnnoise/src/denoise.c:471
471     in ./plugins/obs-filters/rnnoise/src/denoise.c
(gdb) print/x $rsp
$4 = 0x7f1aacfee810
(gdb) up
#4  0x00007f1b302a5f4c in process_rnnoise (ng=0x562d5f373f40) at 
./plugins/obs-filters/noise-suppress-filter.c:343
343     ./plugins/obs-filters/noise-suppress-filter.c: Datei oder Verzeichnis 
nicht gefunden.
(gdb) print/x $rsp
$5 = 0x7f1aacff7fb0
(gdb) up
#5  process (ng=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:392
392     in ./plugins/obs-filters/noise-suppress-filter.c
(gdb) print/x $rsp
$6 = 0x7f1aacff7fb0
(gdb) up
#6  noise_suppress_filter_audio (data=0x562d5f373f40, audio=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:468
468     in ./plugins/obs-filters/noise-suppress-filter.c
(gdb) print/x $rsp
$7 = 0x7f1aacff7fb0
(gdb) up
#7  0x00007f1bf1beab8b in filter_async_audio (in=<optimized out>, 
source=0x562d5f26bf00) at ./libobs/obs-source.c:3068
3068    ./libobs/obs-source.c: Datei oder Verzeichnis nicht gefunden.
(gdb) print/x $rsp
$8 = 0x7f1aacff8080
(gdb) up
#8  obs_source_output_audio (source=0x562d5f26bf00, 
audio=audio@entry=0x7f1aacff81f0) at ./libobs/obs-source.c:3246
3246    in ./libobs/obs-source.c
(gdb) print/x $rsp
$9 = 0x7f1aacff8080
(gdb) up
#9  0x00007f1bcc002774 in _alsa_listen (attr=0x7f1be000dc20) at 
./plugins/linux-alsa/alsa-input.c:574
574     ./plugins/linux-alsa/alsa-input.c: Datei oder Verzeichnis nicht 
gefunden.
(gdb) print/x $rsp
$10 = 0x7f1aacff81f0
(gdb) up
#10 0x00007f1bf0c26ea7 in start_thread (arg=<optimized out>) at 
pthread_create.c:477
477     pthread_create.c: Datei oder Verzeichnis nicht gefunden.
(gdb) print/x $rsp
$11 = 0x7f1aacff8280
(gdb) up
#11 0x00007f1bf0b56d4f in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
95      ../sysdeps/unix/sysv/linux/x86_64/clone.S: Datei oder Verzeichnis nicht 
gefunden.
(gdb) print/x $rsp
$12 = 0x7f1aacff8340
(gdb) up
Initial frame selected; you cannot go up.


--> bogus value in $rsp ???







gdb -q --args obs
set width 0
set pagination off
directory /home/benutzer/source/libcodec2-0.9/orig/codec2-0.9.2
directory /home/benutzer/source/obs-studio/orig/obs-studio-26.0.2+dfsg1
display/i $pc
display/x $rsp
b pitch_downsample
y
run
next
step






160        _celt_autocorr(x_lp, ac, NULL, 0,
1: x/i $pc
=> 0x7ffff277ed1d <pitch_downsample+45>:        lea    0x20(%rsp),%r12
2: /x $rsp = 0x7fff1f7f37a0
(gdb) step
0x00007ffff277e9be in _celt_autocorr (x=<optimized out>, ac=<optimized out>, 
window=<optimized out>, overlap=<optimized out>, lag=<optimized out>, 
n=<optimized out>) at ./lpcnet/src/celt_lpc.c:209
209     {
1: x/i $pc
=> 0x7ffff277e9be <_celt_autocorr+14>:  push   %r13
2: /x $rsp = 0x7fff1f7f3780
(gdb) next
[Thread 0x7fffe9187700 (LWP 5610) exited]
[Thread 0x7fffe9988700 (LWP 5609) exited]
[Thread 0x7fffea189700 (LWP 5608) exited]
[Thread 0x7fffea98a700 (LWP 5607) exited]
215        opus_val16 xx[n];
1: x/i $pc
=> 0x7ffff277e9e3 <_celt_autocorr+51>:  lea    0xf(%rdx),%rax
2: /x $rsp = 0x7fff1f7f3740
(gdb) stepi
0x00007ffff277e9e7      215        opus_val16 xx[n];
1: x/i $pc
=> 0x7ffff277e9e7 <_celt_autocorr+55>:  and    $0xfffffffffffffff0,%rax
2: /x $rsp = 0x7fff1f7f3740
(gdb) bt
#0  0x00007ffff277e9e7 in _celt_autocorr (x=<optimized out>, ac=<optimized 
out>, window=<optimized out>, overlap=<optimized out>, lag=<optimized out>, 
n=<optimized out>) at ./lpcnet/src/celt_lpc.c:215
#1  0x00007ffff277ed2a in pitch_downsample (x_lp=x_lp@entry=0x7fff1f7f38e8, 
len=len@entry=528435664) at ./lpcnet/src/pitch.c:160
#2  0x00007fff24330006 in compute_frame_features (in=0x7fff1f7f3ac0, 
features=0x7fff1f7f3a10, Exp=0x7fff1f7f39b0, Ep=0x7fff1f7f3950, 
Ex=0x7fff1f7f38f0, P=0x7fff1f7f7560, X=0x7fff1f7f6650, st=0x555556cac720) at 
./plugins/obs-filters/rnnoise/src/denoise.c:326
#3  rnnoise_process_frame (st=0x555556cac720, out=<optimized out>, 
in=<optimized out>) at ./plugins/obs-filters/rnnoise/src/denoise.c:471
#4  0x00007fff24342f4c in process_rnnoise (ng=0x555556c4e980) at 
./plugins/obs-filters/noise-suppress-filter.c:343
#5  process (ng=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:392
#6  noise_suppress_filter_audio (data=0x555556c4e980, audio=<optimized out>) at 
./plugins/obs-filters/noise-suppress-filter.c:468
#7  0x00007ffff5c84b8b in filter_async_audio (in=<optimized out>, 
source=0x55555691a480) at ./libobs/obs-source.c:3068
#8  obs_source_output_audio (source=0x55555691a480, 
audio=audio@entry=0x7fff1f7fd1f0) at ./libobs/obs-source.c:3246
#9  0x00007fffd0006774 in _alsa_listen (attr=0x5555565a4b20) at 
./plugins/linux-alsa/alsa-input.c:574
#10 0x00007ffff4cc0ea7 in start_thread (arg=<optimized out>) at 
pthread_create.c:477
#11 0x00007ffff4bf0d4f in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) stepi
0x00007ffff277e9eb      215        opus_val16 xx[n];
1: x/i $pc
=> 0x7ffff277e9eb <_celt_autocorr+59>:  sub    %rax,%rsp
2: /x $rsp = 0x7fff1f7f3740
(gdb) 
216        celt_assert(n>0);
1: x/i $pc
=> 0x7ffff277e9ee <_celt_autocorr+62>:  test   %r12d,%r12d
2: /x $rsp = 0x7ffea1821000
(gdb) print/x $rax
$1 = 0x7dfd2740
(gdb) disassemble _celt_autocorr
Dump of assembler code for function _celt_autocorr:
   0x00007ffff277e9b0 <+0>:     push   %rbp
   0x00007ffff277e9b1 <+1>:     mov    %rsp,%rbp
   0x00007ffff277e9b4 <+4>:     push   %r15
   0x00007ffff277e9b6 <+6>:     push   %r14
   0x00007ffff277e9b8 <+8>:     mov    %rdx,%r14
   0x00007ffff277e9bb <+11>:    movslq %r9d,%rdx
   0x00007ffff277e9be <+14>:    push   %r13
   0x00007ffff277e9c0 <+16>:    mov    %edx,%r15d
   0x00007ffff277e9c3 <+19>:    push   %r12
   0x00007ffff277e9c5 <+21>:    mov    %rdx,%r12
   0x00007ffff277e9c8 <+24>:    shl    $0x2,%rdx
   0x00007ffff277e9cc <+28>:    sub    %r8d,%r15d
   0x00007ffff277e9cf <+31>:    push   %rbx
   0x00007ffff277e9d0 <+32>:    sub    $0x28,%rsp
   0x00007ffff277e9d4 <+36>:    mov    %fs:0x28,%rax
   0x00007ffff277e9dd <+45>:    mov    %rax,-0x38(%rbp)
   0x00007ffff277e9e1 <+49>:    xor    %eax,%eax
   0x00007ffff277e9e3 <+51>:    lea    0xf(%rdx),%rax
   0x00007ffff277e9e7 <+55>:    and    $0xfffffffffffffff0,%rax
   0x00007ffff277e9eb <+59>:    sub    %rax,%rsp
=> 0x00007ffff277e9ee <+62>:    test   %r12d,%r12d
   0x00007ffff277e9f1 <+65>:    jle    0x7ffff277eb5e <_celt_autocorr+430>
   0x00007ffff277e9f7 <+71>:    test   %ecx,%ecx
.

(gdb) list
201
202     int _celt_autocorr(
203                        const opus_val16 *x,   /*  in: [0...n-1] samples x   
*/
204                        opus_val32       *ac,  /* out: [0...lag-1] ac values 
*/
205                        const opus_val16       *window,
206                        int          overlap,
207                        int          lag,
208                        int          n)
209     {
210        opus_val32 d;
211        int i, k;
212        int fastN=n-lag;
213        int shift;
214        const opus_val16 *xptr;
215        opus_val16 xx[n];
216        celt_assert(n>0);
217        celt_assert(overlap>=0);
218        if (overlap == 0)
219        {
220           xptr = x;
221        } else {
222           for (i=0;i<n;i++)
223              xx[i] = x[i];
224           for (i=0;i<overlap;i++)
225           {
226              xx[i] = MULT16_16_Q15(x[i],window[i]);
227              xx[n-i-1] = MULT16_16_Q15(x[n-i-1],window[i]);
228           }
229           xptr = xx;
230        }
231        shift=0;
232        celt_pitch_xcorr(xptr, xptr, ac, fastN, lag+1);
233        for (k=0;k<=lag;k++)
234        {
235           for (i = k+fastN, d = 0; i < n; i++)
236              d = MAC16_16(d, xptr[i], xptr[i-k]);
237           ac[k] += d;
238        }
239        return shift;
240     }

(gdb) up
#1  0x00007ffff277ed2a in pitch_downsample (x_lp=x_lp@entry=0x7fff1f7f38e8, 
len=len@entry=528435664) at ./lpcnet/src/pitch.c:160
160        _celt_autocorr(x_lp, ac, NULL, 0,

(gdb) print len
$2 = 528435664
(gdb) print/x len
$3 = 0x1f7f49d0
(gdb) down
#0  _celt_autocorr (x=<optimized out>, ac=<optimized out>, window=<optimized 
out>, overlap=<optimized out>, lag=<optimized out>, n=<optimized out>) at 
./lpcnet/src/celt_lpc.c:216
216        celt_assert(n>0);
(gdb) print/x sizeof(opus_val16)
$4 = 0x4
(gdb) print/x sizeof(opus_val16) * 0x1f7f49d0
$5 = 0x7dfd2740
(gdb) print sizeof(opus_val16) * 0x1f7f49d0
$6 = 2113742656
(gdb) print sizeof(opus_val16) * 0x1f7f49d0 / 1024
$7 = 2064201
(gdb) print sizeof(opus_val16) * 0x1f7f49d0 / 1024 / 1024
$8 = 2015

... tries to allocate 2015 MB on the stack in line 215 ???





(gdb) list lpcnet/src/pitch.c:149
144        mem[3]=mem3;
145        mem[4]=mem4;
146     }
147
148
149     void pitch_downsample(opus_val16 *x_lp,
150           int len)
151     {
152        int i;
153        opus_val32 ac[5];


(gdb) list plugins/obs-filters/rnnoise/src/denoise.c:307,327
307
308     static int compute_frame_features(DenoiseState *st, kiss_fft_cpx *X, 
kiss_fft_cpx *P,
309                                       float *Ex, float *Ep, float *Exp, 
float *features, const float *in) {
310       int i;
311       float E = 0;
312       float *ceps_0, *ceps_1, *ceps_2;
313       float spec_variability = 0;
314       float Ly[NB_BANDS];
315       float p[WINDOW_SIZE];
316       float pitch_buf[PITCH_BUF_SIZE>>1];
317       int pitch_index;
318       float gain;
319       float *(pre[1]);
320       float tmp[NB_BANDS];
321       float follow, logMax;
322       frame_analysis(st, X, Ex, in);
323       RNN_MOVE(st->pitch_buf, &st->pitch_buf[FRAME_SIZE], 
PITCH_BUF_SIZE-FRAME_SIZE);
324       RNN_COPY(&st->pitch_buf[PITCH_BUF_SIZE-FRAME_SIZE], in, FRAME_SIZE);
325       pre[0] = &st->pitch_buf[0];
326       pitch_downsample(pre, pitch_buf, PITCH_BUF_SIZE, 1);
327       pitch_search(pitch_buf+(PITCH_MAX_PERIOD>>1), pitch_buf, 
PITCH_FRAME_SIZE,







LD_PRELOAD=/usr/lib/x86_64-linux-gnu/obs-plugins/obs-filters.so obs



nm -D /lib/x86_64-linux-gnu/libcodec2.so.0.9 | cut -c 20- | sort > 
/tmp/libcodec2.so.txt
nm -D /usr/lib/x86_64-linux-gnu/obs-plugins/obs-filters.so | cut -c 20- | sort 
> /tmp/obs-filters.so.txt
comm -12 /tmp/obs-filters.so.txt /tmp/libcodec2.so.txt | grep -v GLIBC_

_celt_autocorr
celt_fir
celt_iir
_celt_lpc
celt_pitch_xcorr
common
compute_band_corr
compute_band_energy
__gmon_start__
interp_band_gain
_ITM_deregisterTMCloneTable
_ITM_registerTMCloneTable
opus_fft_alloc
opus_fft_alloc_arch_c
opus_fft_alloc_twiddles
opus_fft_c
opus_fft_free
opus_fft_free_arch_c
opus_fft_impl
opus_ifft_c
pitch_downsample
pitch_search
remove_doubling

Reply via email to