On Fri, Feb 22, 2019 at 01:09:12PM +0100, Tomas Härdin wrote: > fre 2019-02-22 klockan 00:11 +0100 skrev Michael Niedermayer: > > On Thu, Feb 21, 2019 at 09:26:44PM +0000, Matthew Fearnley wrote: > > > - To clear up the effects: the change from 'i=1' to 'i=0' was the only > > > change that should make any difference in the output. > > > The rest of the changes were to improve the speed of an inner loop, and to > > > fix two bugs that happily cancelled each other out, but would have broken > > > if the code were changed to emit larger blocks. These should not lead to > > > changes in the blocks chosen or the final compression size. > > > > > > - Regarding the worse compression: let me start by stating that scoring on > > > byte frequency alone will not always predict well how Deflate will work, > > > since Deflate also uses pattern matching as well. > > > Frequency-based block scoring will only ever be a heuristic. There may be > > > scope for improving the scoring, but it would add a lot of complexity, and > > > I think it would be easy to make things worse rather than better. (I > > > found > > > this in my brief experiments with including the frequencies from the > > > previous block.) > > > > implementing the exact scoring by using Deflate itself would allow us to > > understand how much there is between the heuristic and what is achievable. > > If there is a gap that is significant then it may be interresting to > > look at improving the heuristic if the gap is small than the whole > > heuristic improvment path can be discarded. > > The obvious improvment for the heuristic would be to move to higher > > order entropy estimation. ATM IIUC this uses a 0th order. 1st order could > > be tried or any fast compression algorithm could also be used to estimate > > entropy. > > We tried a sliding window order-0 model, which did worse. An order-1 > model might be useful, but would need quite a bit of context for it to > be meaningful >
> > It also would be interresting to see how a higher order heuristic or > > some simple fast LZ like compressor would perform with the previous block. > > That is if the issue you saw with the previous block inclusion was because > > of the very simplistic entropy estimation. > > zlib at level 1 maybe? You could also have a multi-pass thing where it > first does a coarse pass then tries to jiggle MVs and greedily accept > any change that results in smaller output. yes, but i would avoid using external code like zlib for this. Its not designed for fast entropy estimation of many blocks. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Opposition brings concord. Out of discord comes the fairest harmony. -- Heraclitus
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
