On Wed, 2023-10-25 at 21:00 +0200, Paul B Mahol wrote:
> On Wed, Oct 25, 2023 at 8:39 PM Tomas Härdin <g...@haerdin.se> wrote:
>
> >
> > > if (c) {
> > > e[0] = 1 << 14;
> > > e[1] = 0 << 14;
> > > e[2] = v[1];
> > > e[3] = v[0];
> > > } else {
> > > e[0] = v[0];
> > > e[1] = v[1];
> > > e[2] = 0 << 14;
> > > e[3] = 1 << 14;
> > > }
> > >
> > > if (invert2x2(e, d)) {
> > > sum = UINT64_MAX;
> > > goto next;
> > > }
> > >
> >
> > You can make use of the properties of e to simplify calculating the
> > inverse. The determinant is always v[0]<<14, so you can just do if
> > (!v[0]) continue; and skip the determinant check altogether.
> >
>
> Even for real 2x2 matrix case? (Once one of rows is not 1, 0) ?
> May added such cases later.
You can just work the math out on paper. Inverse of
1 0
v[1] v[0]
is
1 0
-v[1]/v[0] 1/v[0]
not accounting for shifts.
Also RE: my other comments, you are right. I didn't take into account
that MLP is lossless and that there may be off-by-one errors.
And as I said on IRC you can formulate this as a least squares problem,
then solve it using a linear system solve. This patch seems finds a
solution that minimizes L1 rather than L2 though. Not sure what the
implications of that are compressionwise. What happens if you replace
FFABS() with a square for scoring?
/Tomas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
