Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-15 Thread Michael Niedermayer
On Sun, Jun 14, 2020 at 12:47:30AM +0200, Michael Niedermayer wrote: > On Sat, Jun 13, 2020 at 05:11:05PM +0100, Kieran Kunhya wrote: [...] > Problem 2 > We do not meassure speed this way because its not realiable nor practical > just look at this, especially the difference and variation > ./ffmpeg

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
> > The one-tailed (we are not considering whether "fast" is slower than > "normal") homoscedastic ("equal variance sampled, not paired") student t > test has a p value of 0.009638597, which for a typical p value threshold of > 0.05 would reject the null hypothesis ("they are the same speed") and >

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Ronald S. Bultje
Hi, On Sat, Jun 13, 2020 at 12:11 PM Kieran Kunhya wrote: > Hello, > > I have run the following commands on the following CPU which is 7 years > old: > Intel(R) Xeon(R) CPU E3-1265L v3 > > ffmpeg_g -i matrixbench_mpeg2.mpg -f null - > and > ffmpeg_g -flags2 fast -i matrixbench_mpeg2.mpg -f null

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
> > > Also the speed benefits are tiny overall, sub-1%. > > This claim is not supported by the statistics you posted > For the overall decoding of MPEG-2 the VLC/dequant functions take up roughly 30% of decoding time. FAST vs normal is about 2% faster on the VLC/dequant functions, therefore the to

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Michael Niedermayer
On Sun, Jun 14, 2020 at 06:50:04PM +0100, Kieran Kunhya wrote: > On Sun, 14 Jun 2020 at 18:47, Carl Eugen Hoyos wrote: > > > Am So., 14. Juni 2020 um 02:33 Uhr schrieb Kieran Kunhya : > > > > > Logical fallacy. > > > > Could you please stop this? > > > > Apart from the personal attack, we have fa

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Michael Niedermayer
On Sun, Jun 14, 2020 at 06:32:31PM +0100, Kieran Kunhya wrote: > On Sun, 14 Jun 2020 at 17:32, Michael Niedermayer > wrote: > > > On Sun, Jun 14, 2020 at 05:05:35PM +0100, Kieran Kunhya wrote: > > > In addition here are results when precomputing the quant tables: > > > > > > Precalculated > > > 6

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
On Sun, 14 Jun 2020 at 18:47, Carl Eugen Hoyos wrote: > Am So., 14. Juni 2020 um 02:33 Uhr schrieb Kieran Kunhya : > > > Logical fallacy. > > Could you please stop this? > > Apart from the personal attack, we have far too many known, reproducible > bugs to spend time on discussing code that you d

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Carl Eugen Hoyos
Am So., 14. Juni 2020 um 02:33 Uhr schrieb Kieran Kunhya : > Logical fallacy. Could you please stop this? Apart from the personal attack, we have far too many known, reproducible bugs to spend time on discussing code that you don't like (for whatever reason) but fortunately don't have to maintai

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
On Sun, 14 Jun 2020 at 17:32, Michael Niedermayer wrote: > On Sun, Jun 14, 2020 at 05:05:35PM +0100, Kieran Kunhya wrote: > > In addition here are results when precomputing the quant tables: > > > > Precalculated > > 6439 > > 6424 > > 6407 > > 6436 > > 6437 > > Mean 6428.6 > > Stdev 13.42758355 >

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Michael Niedermayer
On Sun, Jun 14, 2020 at 05:05:35PM +0100, Kieran Kunhya wrote: > In addition here are results when precomputing the quant tables: > > Precalculated > 6439 > 6424 > 6407 > 6436 > 6437 > Mean 6428.6 > Stdev 13.42758355 does that work on a wider range of videos and CPUs ? from what i remember really

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
In addition here are results when precomputing the quant tables: Precalculated 6439 6424 6407 6436 6437 Mean 6428.6 Stdev 13.42758355 Kieran On Sun, 14 Jun 2020 at 16:05, Kieran Kunhya wrote: > Did you remove the branch for FLAGS2_FAST and the large amount of inlined >> code when making this m

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-14 Thread Kieran Kunhya
> > Did you remove the branch for FLAGS2_FAST and the large amount of inlined > code when making this measurement? > I used the attached patch to benchmark this: Normal Removed 6616 6553 6640 6514 6684 6498 6588 6505 6613 6493 Mean 6628.2 6512.6 Stdev 36.22430124 23.92279248 This means there i

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-13 Thread Kieran Kunhya
Hi, > Problem 1 > you have a mean of around 2100 and Stdev of about between 55 and 80 so if > by > statistically significant you man 2 Stdev, then with the mean you have. > You would declare every optimization of less than 6% to be statistically > insignificant. > So by what you say here, it seem

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-13 Thread Michael Niedermayer
On Sat, Jun 13, 2020 at 05:11:05PM +0100, Kieran Kunhya wrote: > Hello, > > I have run the following commands on the following CPU which is 7 years old: > Intel(R) Xeon(R) CPU E3-1265L v3 > > ffmpeg_g -i matrixbench_mpeg2.mpg -f null - > and > ffmpeg_g -flags2 fast -i matrixbench_mpeg2.mpg -f nul

Re: [FFmpeg-devel] MPEG-2 FLAGS2_FAST benchmarks

2020-06-13 Thread Derek Buitenhuis
On 13/06/2020 17:11, Kieran Kunhya wrote: > It can therefore be shown that the difference is statistically > insignificant. I really think the onus of proving FAST is useful should be on the people who claim it is useful - something I haven't seen any evidence for. - Derek ___