Specific configuration leads to memory violations

[guidovranken]

The following command:

./sphinx_fe -smoothspec yes -nfilt 10 -i 001.wav -o x

where 001.wav is https://github.com/cmusphinx/pocketsphinx/blob/master/test/data/cards/001.wav

results in memory violations (out of bounds reads and writes)

If you acknowledge and fix this, could you please credit 'ForAllSecure Mayhem'?

Thanks

[guidovranken]

Valgrind output:

==5160== Memcheck, a memory error detector
==5160== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==5160== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==5160== Command: ./sphinx_fe -smoothspec yes -nfilt 10 -i 001.wav -o x
==5160== 
Current configuration:
[NAME]			[DEFLT]		[VALUE]
-alpha			0.97		9.700000e-01
-argfile				
-blocksize		2048		2048
-build_outdirs		yes		yes
-c					
-cep2spec		no		no
-di					
-dither			no		no
-do					
-doublebw		no		no
-ei					
-eo					
-example		no		no
-frate			100		100
-help			no		no
-i					001.wav
-input_endian		little		little
-lifter			0		0
-logspec		no		no
-lowerf			133.33334	1.333333e+02
-mach_endian		little		little
-mswav			no		no
-ncep			13		13
-nchans			1		1
-nfft			512		512
-nfilt			40		10
-nist			no		no
-npart			0		0
-nskip			0		0
-o					x
-ofmt			sphinx		sphinx
-part			0		0
-raw			no		no
-remove_dc		no		no
-remove_noise		yes		yes
-remove_silence		yes		yes
-round_filters		yes		yes
-runlen			-1		-1
-samprate		16000		1.600000e+04
-seed			-1		-1
-smoothspec		no		yes
-spec2cep		no		no
-sph2pipe		no		no
-transform		legacy		legacy
-unit_area		yes		yes
-upperf			6855.4976	6.855498e+03
-vad_postspeech		50		50
-vad_prespeech		20		20
-vad_startspeech	10		10
-vad_threshold		3.0		3.000000e+00
-verbose		no		no
-warp_params				
-warp_type		inverse_linear	inverse_linear
-whichchan		0		0
-wlen			0.025625	2.562500e-02

INFO: sphinx_fe.c(791): Converting 001.wav to x
==5160== Invalid write of size 2
==5160==    at 0x4C36753: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E61493: memcpy (string_fortified.h:34)
==5160==    by 0x4E61493: fe_prespch_read_cep (fe_prespch_buf.c:119)
==5160==    by 0x4E5F575: fe_copy_from_prespch (fe_interface.c:394)
==5160==    by 0x4E5F575: fe_check_prespeech (fe_interface.c:412)
==5160==    by 0x4E5FF3C: fe_process_frames_ext (fe_interface.c:530)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid write of size 8
==5160==    at 0x4C367E3: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E61493: memcpy (string_fortified.h:34)
==5160==    by 0x4E61493: fe_prespch_read_cep (fe_prespch_buf.c:119)
==5160==    by 0x4E601F5: fe_copy_from_prespch (fe_interface.c:394)
==5160==    by 0x4E601F5: fe_process_frames_ext (fe_interface.c:492)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid write of size 2
==5160==    at 0x4C36753: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E61493: memcpy (string_fortified.h:34)
==5160==    by 0x4E61493: fe_prespch_read_cep (fe_prespch_buf.c:119)
==5160==    by 0x4E601F5: fe_copy_from_prespch (fe_interface.c:394)
==5160==    by 0x4E601F5: fe_process_frames_ext (fe_interface.c:492)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44408 is 8 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid write of size 4
==5160==    at 0x4E625F2: fe_dct2 (fe_sigproc.c:1147)
==5160==    by 0x4E62CF1: fe_mel_cep (fe_sigproc.c:1086)
==5160==    by 0x4E62CF1: fe_write_frame (fe_sigproc.c:1190)
==5160==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid write of size 4
==5160==    at 0x4E62639: fe_dct2 (fe_sigproc.c:1149)
==5160==    by 0x4E62CF1: fe_mel_cep (fe_sigproc.c:1086)
==5160==    by 0x4E62CF1: fe_write_frame (fe_sigproc.c:1190)
==5160==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid write of size 4
==5160==    at 0x4E62651: fe_dct2 (fe_sigproc.c:1151)
==5160==    by 0x4E62CF1: fe_mel_cep (fe_sigproc.c:1086)
==5160==    by 0x4E62CF1: fe_write_frame (fe_sigproc.c:1190)
==5160==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== Invalid read of size 4
==5160==    at 0x4E62749: fe_dct3 (fe_sigproc.c:1176)
==5160==    by 0x4E62CFF: fe_mel_cep (fe_sigproc.c:1087)
==5160==    by 0x4E62CFF: fe_write_frame (fe_sigproc.c:1190)
==5160==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5160==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5160==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5160==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160==  Address 0x5a44400 is 0 bytes after a block of size 400 alloc'd
==5160==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5160==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5160==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5160==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5160==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5160== 
==5160== 
==5160== HEAP SUMMARY:
==5160==     in use at exit: 0 bytes in 0 blocks
==5160==   total heap usage: 512 allocs, 512 frees, 60,482 bytes allocated
==5160== 
==5160== All heap blocks were freed -- no leaks are possible
==5160== 
==5160== For counts of detected and suppressed errors, rerun with: -v
==5160== ERROR SUMMARY: 413 errors from 7 contexts (suppressed: 0 from 0)

[guidovranken]

A different configuration that leads to different memory violations:

./sphinx_fe -logspec yes -nfilt 1 -lifter 71 -i 001.wav -o x

==5308== Memcheck, a memory error detector
==5308== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==5308== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==5308== Command: ./sphinx_fe -logspec yes -nfilt 1 -lifter 71 -i 001.wav -o x
==5308== 
Current configuration:
[NAME]			[DEFLT]		[VALUE]
-alpha			0.97		9.700000e-01
-argfile				
-blocksize		2048		2048
-build_outdirs		yes		yes
-c					
-cep2spec		no		no
-di					
-dither			no		no
-do					
-doublebw		no		no
-ei					
-eo					
-example		no		no
-frate			100		100
-help			no		no
-i					001.wav
-input_endian		little		little
-lifter			0		71
-logspec		no		yes
-lowerf			133.33334	1.333333e+02
-mach_endian		little		little
-mswav			no		no
-ncep			13		13
-nchans			1		1
-nfft			512		512
-nfilt			40		1
-nist			no		no
-npart			0		0
-nskip			0		0
-o					x
-ofmt			sphinx		sphinx
-part			0		0
-raw			no		no
-remove_dc		no		no
-remove_noise		yes		yes
-remove_silence		yes		yes
-round_filters		yes		yes
-runlen			-1		-1
-samprate		16000		1.600000e+04
-seed			-1		-1
-smoothspec		no		no
-spec2cep		no		no
-sph2pipe		no		no
-transform		legacy		legacy
-unit_area		yes		yes
-upperf			6855.4976	6.855498e+03
-vad_postspeech		50		50
-vad_prespeech		20		20
-vad_startspeech	10		10
-vad_threshold		3.0		3.000000e+00
-verbose		no		no
-warp_params				
-warp_type		inverse_linear	inverse_linear
-whichchan		0		0
-wlen			0.025625	2.562500e-02

INFO: sphinx_fe.c(791): Converting 001.wav to x
==5308== Invalid read of size 4
==5308==    at 0x4E626A0: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E5FE99: fe_process_frames_ext (fe_interface.c:522)
==5308==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5308==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== Invalid write of size 4
==5308==    at 0x4E626AF: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E5FE99: fe_process_frames_ext (fe_interface.c:522)
==5308==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5308==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== Invalid read of size 4
==5308==    at 0x4E626A0: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5308==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5308==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== Invalid write of size 4
==5308==    at 0x4E626AF: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E5FF17: fe_process_frames_ext (fe_interface.c:528)
==5308==    by 0x4E6025F: fe_process_frames (fe_interface.c:384)
==5308==    by 0x10C2D6: decode_pcm (sphinx_fe.c:411)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== Invalid read of size 4
==5308==    at 0x4E626A0: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E60354: fe_end_utt (fe_interface.c:614)
==5308==    by 0x10C334: decode_pcm (sphinx_fe.c:421)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== Invalid write of size 4
==5308==    at 0x4E626AF: fe_lifter (fe_sigproc.c:1164)
==5308==    by 0x4E62C2D: fe_write_frame (fe_sigproc.c:1191)
==5308==    by 0x4E60354: fe_end_utt (fe_interface.c:614)
==5308==    by 0x10C334: decode_pcm (sphinx_fe.c:421)
==5308==    by 0x10C739: sphinx_wave2feat_convert_file (sphinx_fe.c:842)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308==  Address 0x5a43928 is 0 bytes after a block of size 40 alloc'd
==5308==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5308==    by 0x4E4B068: __ckd_calloc__ (ckd_alloc.c:157)
==5308==    by 0x4E4B1F6: __ckd_calloc_2d__ (ckd_alloc.c:229)
==5308==    by 0x10C6DD: sphinx_wave2feat_convert_file (sphinx_fe.c:827)
==5308==    by 0x10AC5B: main (sphinx_fe.c:1038)
==5308== 
==5308== 
==5308== HEAP SUMMARY:
==5308==     in use at exit: 0 bytes in 0 blocks
==5308==   total heap usage: 513 allocs, 513 frees, 42,036 bytes allocated
==5308== 
==5308== All heap blocks were freed -- no leaks are possible
==5308== 
==5308== For counts of detected and suppressed errors, rerun with: -v
==5308== ERROR SUMMARY: 1092 errors from 6 contexts (suppressed: 0 from 0)

[dhdaines]

The front-end code in 5prealpha is known to have various issues, so this does not surprise me. It will probably get reverted to the previously released version (with some fixes).