This is an automated email from the git hooks/post-receive script.
git pushed a commit to branch fix-release-build
in repository efl.
View the commit online.
commit 5f3e2c7523a86b0e35a38389d2361b2723c47a7e
Author: Cedric BAIL <[email protected]>
AuthorDate: Sun Aug 2 19:42:43 2026 -0600
evas: vectorise the gaussian blur with NEON
The gaussian blur had no NEON at all - not even a stub, unlike the box
blur - yet it is what actually runs. EVAS_FILTER_BLUR_DEFAULT resolves to
gaussian and the filter DSL's plain blur({...}) takes that path, so this
is the blur any ordinary filter program exercises.
It is also a far better vectorisation target than the box blur. Box blur
is a sliding window with a serial dependency along the span; the gaussian
middle section is a straight convolution, (len - 2 * radius) outputs each
summing diameter weighted taps, with no dependency between outputs.
The arithmetic maps onto vmlal_n_u16 directly: the weights are normalised
so that sum(weights) == 1 << pow2_divider and are a few thousand at most,
which fits the 16 bit scalar operand, and the largest accumulator value
is 255 * sum(weights), which fits 32 bits for any radius evas uses.
Which axis can be vectorised differs per direction, because the two
instantiations of the template disagree about what has stride one:
- horizontal (STEP == 1): the taps of one output are contiguous, so
walk the line four pixels (rgba) or sixteen (alpha) at a time.
- vertical (STEP == loops): one output's window strides a row per tap,
but adjacent *columns* are contiguous. That axis belongs to the
caller's outer loop, so the middle is done in a separate pass that
inverts the loop nest, and the per line loop then only fills the ramps
for the columns it covered.
The ramps stay scalar: they recompute a divider per output and only run
radius times per line against the middle's len - 2 * radius.
This deliberately does not add a second copy of the file. The vectorised
middle sits inside the existing template behind BLUR_NEON, which the
caller defines for the NEON instantiation, so the ramps exist once. Two
copies of a kernel is what produced four of the bugs fixed earlier in
this branch. For the same reason the weight curve moves to its own file,
shared by the blur and by its test, so the test cannot end up validating
against weights production does not use.
Throughput at 720x480, Cortex-A72, best of 9, pinned:
radius 2 5 10 20 40
horiz rgba 2.62x 2.57x 2.26x 1.98x 1.74x
vert rgba 2.88x 2.93x 3.85x 3.11x 3.24x
horiz alpha 6.84x 5.91x 2.99x 2.75x 2.04x
vert alpha 6.14x 6.05x 4.87x 4.50x 3.47x
The accompanying differential test instantiates each template twice, once
with BLUR_NEON and once without, and sweeps radius, line length and line
count. That last one matters more than it looks: the vertical kernel
vectorises across columns, so a fixed small line count leaves that path
unexercised entirely - deliberately breaking it with three lines produced
no failure at all, which is how the gap was found. Every vectorised path
is mutation checked.
Source buffers sit against an unmapped page, so a read past the end of a
line faults rather than quietly returning heap contents. Removing the
ramp bounds added in "fix out of bounds read in the gaussian blur line
ramps" makes the test segfault, which is what makes that fix verified
rather than merely asserted.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
---
.../filters/blur/blur_gaussian_alpha_.c | 105 ++++++
.../filters/blur/blur_gaussian_rgba_.c | 125 +++++++
.../software_generic/filters/evas_filter_blur.c | 82 +++--
src/tests/evas/evas_test_neon_blur.c | 390 +++++++++++++++++++++
src/tests/evas/meson.build | 16 +
5 files changed, 682 insertions(+), 36 deletions(-)
diff --git a/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_alpha_.c b/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_alpha_.c
index ecee0a4b14..0eca0decbe 100644
--- a/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_alpha_.c
+++ b/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_alpha_.c
@@ -2,6 +2,12 @@
* Should define the functions:
* - _gaussian_blur_horiz_alpha_step
* - _gaussian_blur_vert_alpha_step
+ *
+ * Define BLUR_NEON as well to get the same kernel with its middle section
+ * vectorised. See blur_gaussian_rgba_.c for the reasoning, including why there
+ * is no separate NEON copy of this file. The only difference here is that an
+ * alpha plane is one byte per pixel, so sixteen outputs fit in a register
+ * instead of four.
*/
/* Datatypes and MIN macro */
@@ -27,6 +33,56 @@ FUNCTION_NAME(const DATA8* restrict srcdata, DATA8* restrict dstdata,
const DATA8* restrict s;
const DATA8* restrict src;
DATA8* restrict dst;
+#ifdef BLUR_NEON
+ const int32x4_t shift = vdupq_n_s32(-pow2_divider);
+ int done_cols = 0;
+
+ /* Vertical only: the middle of every column, sixteen columns at a time.
+ * Each tap is one contiguous 16 byte load spanning sixteen columns of one
+ * row, and the tap loop walks down by STEP. */
+ if ((STEP != 1) && (mid > 0))
+ {
+ int c;
+
+ for (c = 0; c + 16 <= loops; c += 16)
+ {
+ const DATA8* restrict scol = srcdata + c;
+ DATA8* restrict dcol = dstdata + c + (size_t) left * STEP;
+
+ for (k = 0; k < mid; k++, scol += STEP, dcol += STEP)
+ {
+ uint32x4_t a0 = vdupq_n_u32(0), a1 = vdupq_n_u32(0);
+ uint32x4_t a2 = vdupq_n_u32(0), a3 = vdupq_n_u32(0);
+ const DATA8* restrict sc = scol;
+
+ for (j = 0; j < diameter; j++, sc += STEP)
+ {
+ const uint8x16_t px = vld1q_u8(sc);
+ const uint16x8_t lo = vmovl_u8(vget_low_u8(px));
+ const uint16x8_t hi = vmovl_u8(vget_high_u8(px));
+ const uint16_t w = (uint16_t) weights[j];
+
+ a0 = vmlal_n_u16(a0, vget_low_u16(lo), w);
+ a1 = vmlal_n_u16(a1, vget_high_u16(lo), w);
+ a2 = vmlal_n_u16(a2, vget_low_u16(hi), w);
+ a3 = vmlal_n_u16(a3, vget_high_u16(hi), w);
+ }
+
+ a0 = vshlq_u32(a0, shift);
+ a1 = vshlq_u32(a1, shift);
+ a2 = vshlq_u32(a2, shift);
+ a3 = vshlq_u32(a3, shift);
+
+ vst1q_u8(dcol,
+ vcombine_u8(vmovn_u16(vcombine_u16(vmovn_u32(a0),
+ vmovn_u32(a1))),
+ vmovn_u16(vcombine_u16(vmovn_u32(a2),
+ vmovn_u32(a3)))));
+ }
+ }
+ done_cols = c;
+ }
+#endif
for (i = loops; i; --i)
{
@@ -53,6 +109,52 @@ FUNCTION_NAME(const DATA8* restrict srcdata, DATA8* restrict dstdata,
// middle
k = radius;
+#ifdef BLUR_NEON
+ if (STEP != 1)
+ {
+ /* already done by the vectorised column pass, for the columns it
+ * covered; step over it */
+ if (i > (loops - done_cols))
+ {
+ src += roff * STEP;
+ dst += roff * STEP;
+ k = len - radius;
+ }
+ }
+ else
+ {
+ for (; k + 16 <= (len - radius); k += 16, src += 16, dst += 16)
+ {
+ uint32x4_t a0 = vdupq_n_u32(0), a1 = vdupq_n_u32(0);
+ uint32x4_t a2 = vdupq_n_u32(0), a3 = vdupq_n_u32(0);
+
+ s = src;
+ for (j = 0; j < diameter; j++, s++)
+ {
+ const uint8x16_t px = vld1q_u8(s);
+ const uint16x8_t lo = vmovl_u8(vget_low_u8(px));
+ const uint16x8_t hi = vmovl_u8(vget_high_u8(px));
+ const uint16_t w = (uint16_t) weights[j];
+
+ a0 = vmlal_n_u16(a0, vget_low_u16(lo), w);
+ a1 = vmlal_n_u16(a1, vget_high_u16(lo), w);
+ a2 = vmlal_n_u16(a2, vget_low_u16(hi), w);
+ a3 = vmlal_n_u16(a3, vget_high_u16(hi), w);
+ }
+
+ a0 = vshlq_u32(a0, shift);
+ a1 = vshlq_u32(a1, shift);
+ a2 = vshlq_u32(a2, shift);
+ a3 = vshlq_u32(a3, shift);
+
+ vst1q_u8(dst,
+ vcombine_u8(vmovn_u16(vcombine_u16(vmovn_u32(a0),
+ vmovn_u32(a1))),
+ vmovn_u16(vcombine_u16(vmovn_u32(a2),
+ vmovn_u32(a3)))));
+ }
+ }
+#endif
for (; k < (len - radius); k++, src += STEP, dst += STEP)
{
acc = 0;
@@ -92,3 +194,6 @@ error:
#undef FUNCTION_NAME
#undef STEP
+#ifdef BLUR_NEON
+# undef BLUR_NEON
+#endif
diff --git a/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_rgba_.c b/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_rgba_.c
index 77a328f034..fd624a58ca 100644
--- a/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_rgba_.c
+++ b/src/modules/evas/engines/software_generic/filters/blur/blur_gaussian_rgba_.c
@@ -2,6 +2,28 @@
* Should define the functions:
* - _gaussian_blur_horiz_rgba_step
* - _gaussian_blur_vert_rgba_step
+ *
+ * Define BLUR_NEON as well to get the same kernel with its middle section
+ * vectorised. There is deliberately no separate NEON copy of this file: the
+ * ramps are subtle enough that two copies would drift, and a differential test
+ * can only compare what the two copies still agree to compute.
+ *
+ * Which axis the middle can be vectorised along depends on the direction,
+ * because the two instantiations disagree about what has stride one:
+ *
+ * horizontal (STEP == 1): the taps of one output are contiguous, so walk the
+ * line four pixels at a time, one unaligned load per tap.
+ *
+ * vertical (STEP == loops): one output's window strides a row per tap, but
+ * adjacent *columns* are contiguous. That axis belongs to the
+ * caller's outer loop, so the middle is done in a separate pass
+ * that inverts the loop nest, and the per line loop below then
+ * only fills in the ramps for the columns it covered.
+ *
+ * Accumulators are 32 bit: the largest value is 255 * sum(weights), and the
+ * weights are normalised so that sum(weights) == 1 << pow2_divider, which for
+ * any radius evas uses stays far inside 32 bits. The weights themselves are a
+ * few thousand at most, so they fit the 16 bit scalar operand of vmlal_n_u16.
*/
#include "evas_filter_private.h"
@@ -25,6 +47,56 @@ FUNCTION_NAME(const DATA32* restrict srcdata, DATA32* restrict dstdata,
const DATA32* restrict src;
DATA32* restrict dst;
int i, j, k;
+#ifdef BLUR_NEON
+ const int32x4_t shift = vdupq_n_s32(-pow2_divider);
+ int done_cols = 0;
+
+ /* Vertical only: the middle of every column, four columns at a time. Each
+ * tap is one contiguous 16 byte load spanning four columns of one row. */
+ if ((STEP != 1) && (mid > 0))
+ {
+ int c;
+
+ for (c = 0; c + 4 <= loops; c += 4)
+ {
+ const DATA32* restrict scol = srcdata + c;
+ DATA32* restrict dcol = dstdata + c + (size_t) left * STEP;
+
+ for (k = 0; k < mid; k++, scol += STEP, dcol += STEP)
+ {
+ uint32x4_t a0 = vdupq_n_u32(0), a1 = vdupq_n_u32(0);
+ uint32x4_t a2 = vdupq_n_u32(0), a3 = vdupq_n_u32(0);
+ const DATA32* restrict s = scol;
+
+ for (j = 0; j < diameter; j++, s += STEP)
+ {
+ const uint8x16_t px = vreinterpretq_u8_u32(vld1q_u32(s));
+ const uint16x8_t lo = vmovl_u8(vget_low_u8(px));
+ const uint16x8_t hi = vmovl_u8(vget_high_u8(px));
+ const uint16_t w = (uint16_t) weights[j];
+
+ a0 = vmlal_n_u16(a0, vget_low_u16(lo), w);
+ a1 = vmlal_n_u16(a1, vget_high_u16(lo), w);
+ a2 = vmlal_n_u16(a2, vget_low_u16(hi), w);
+ a3 = vmlal_n_u16(a3, vget_high_u16(hi), w);
+ }
+
+ a0 = vshlq_u32(a0, shift);
+ a1 = vshlq_u32(a1, shift);
+ a2 = vshlq_u32(a2, shift);
+ a3 = vshlq_u32(a3, shift);
+
+ vst1q_u32(dcol,
+ vreinterpretq_u32_u8(
+ vcombine_u8(vmovn_u16(vcombine_u16(vmovn_u32(a0),
+ vmovn_u32(a1))),
+ vmovn_u16(vcombine_u16(vmovn_u32(a2),
+ vmovn_u32(a3))))));
+ }
+ }
+ done_cols = c;
+ }
+#endif
for (i = loops; i; --i)
{
@@ -59,6 +131,56 @@ FUNCTION_NAME(const DATA32* restrict srcdata, DATA32* restrict dstdata,
// middle
k = mid;
+#ifdef BLUR_NEON
+ if (STEP != 1)
+ {
+ /* already done by the vectorised column pass, for the columns it
+ * covered; step over it */
+ if (i > (loops - done_cols))
+ {
+ src += roff * STEP;
+ dst += roff * STEP;
+ k = 0;
+ }
+ }
+ else
+ {
+ /* four output pixels per iteration. Tap j contributes
+ * src[k + j .. k + 3 + j], one unaligned 16 byte load, widened to
+ * four 32 bit lanes per pixel and scaled by weights[j]. */
+ for (; k >= 4; k -= 4, src += 4, dst += 4)
+ {
+ uint32x4_t a0 = vdupq_n_u32(0), a1 = vdupq_n_u32(0);
+ uint32x4_t a2 = vdupq_n_u32(0), a3 = vdupq_n_u32(0);
+ const DATA32* restrict s = src;
+
+ for (j = 0; j < diameter; j++, s++)
+ {
+ const uint8x16_t px = vreinterpretq_u8_u32(vld1q_u32(s));
+ const uint16x8_t lo = vmovl_u8(vget_low_u8(px));
+ const uint16x8_t hi = vmovl_u8(vget_high_u8(px));
+ const uint16_t w = (uint16_t) weights[j];
+
+ a0 = vmlal_n_u16(a0, vget_low_u16(lo), w);
+ a1 = vmlal_n_u16(a1, vget_high_u16(lo), w);
+ a2 = vmlal_n_u16(a2, vget_low_u16(hi), w);
+ a3 = vmlal_n_u16(a3, vget_high_u16(hi), w);
+ }
+
+ a0 = vshlq_u32(a0, shift);
+ a1 = vshlq_u32(a1, shift);
+ a2 = vshlq_u32(a2, shift);
+ a3 = vshlq_u32(a3, shift);
+
+ vst1q_u32(dst,
+ vreinterpretq_u32_u8(
+ vcombine_u8(vmovn_u16(vcombine_u16(vmovn_u32(a0),
+ vmovn_u32(a1))),
+ vmovn_u16(vcombine_u16(vmovn_u32(a2),
+ vmovn_u32(a3))))));
+ }
+ }
+#endif
for (; k > 0; k--, src += STEP, dst += STEP)
{
int acc[4] = {0};
@@ -114,3 +236,6 @@ error:
#undef FUNCTION_NAME
#undef STEP
+#ifdef BLUR_NEON
+# undef BLUR_NEON
+#endif
diff --git a/src/modules/evas/engines/software_generic/filters/evas_filter_blur.c b/src/modules/evas/engines/software_generic/filters/evas_filter_blur.c
index b410437291..cb92ab717f 100644
--- a/src/modules/evas/engines/software_generic/filters/evas_filter_blur.c
+++ b/src/modules/evas/engines/software_generic/filters/evas_filter_blur.c
@@ -331,42 +331,8 @@ _box_blur_vert_apply_rgba(Evas_Filter_Command *cmd)
/* Gaussian blur */
-static void
-_sin_blur_weights_get(int *weights, int *pow2_divider, int radius)
-{
- const int diameter = 2 * radius + 1;
- double x, divider, sum = 0.0;
- double dweights[diameter];
- int k, nextpow2, isum = 0;
- const int FAKE_PI = 3.0;
+#include "./blur/blur_weights_.c"
- /* Base curve:
- * f(x) = sin(x+pi/2)/2+1/2
- */
-
- for (k = 0; k < diameter; k++)
- {
- x = ((double) k / (double) (diameter - 1)) * FAKE_PI * 2.0 - FAKE_PI;
- dweights[k] = ((sin(x + M_PI_2) + 1.0) / 2.0) * 1024.0;
- sum += dweights[k];
- }
-
- // Now we need to normalize to have a 2^N divider.
- nextpow2 = log2(2 * sum);
- divider = (double) (1 << nextpow2);
-
- for (k = 0; k < diameter; k++)
- {
- weights[k] = round(dweights[k] * divider / sum);
- isum += weights[k];
- }
-
- // Final correction. The difference SHOULD be small...
- weights[radius] += (int) divider - isum;
-
- if (pow2_divider)
- *pow2_divider = nextpow2;
-}
#define FUNCTION_NAME _gaussian_blur_horiz_alpha_step
#define STEP 1
@@ -385,6 +351,30 @@ _sin_blur_weights_get(int *weights, int *pow2_divider, int radius)
#define STEP loops
#include "./blur/blur_gaussian_rgba_.c"
+#ifdef BUILD_NEON
+# include <arm_neon.h>
+
+# define FUNCTION_NAME _gaussian_blur_horiz_alpha_step_neon
+# define STEP 1
+# define BLUR_NEON 1
+# include "./blur/blur_gaussian_alpha_.c"
+
+# define FUNCTION_NAME _gaussian_blur_vert_alpha_step_neon
+# define STEP loops
+# define BLUR_NEON 1
+# include "./blur/blur_gaussian_alpha_.c"
+
+# define FUNCTION_NAME _gaussian_blur_horiz_rgba_step_neon
+# define STEP 1
+# define BLUR_NEON 1
+# include "./blur/blur_gaussian_rgba_.c"
+
+# define FUNCTION_NAME _gaussian_blur_vert_rgba_step_neon
+# define STEP loops
+# define BLUR_NEON 1
+# include "./blur/blur_gaussian_rgba_.c"
+#endif
+
static Eina_Bool
_gaussian_blur_apply(Evas_Filter_Command *cmd, Eina_Bool vert, Eina_Bool rgba)
{
@@ -401,11 +391,31 @@ _gaussian_blur_apply(Evas_Filter_Command *cmd, Eina_Bool vert, Eina_Bool rgba)
h = cmd->input->h;
weights = alloca((2 * radius + 1) * sizeof(int));
- _sin_blur_weights_get(weights, &pow2_div, radius);
+ evas_blur_weights_get(weights, &pow2_div, radius);
if (src && dst)
{
DEBUG_TIME_BEGIN();
+#ifdef BUILD_NEON
+ if (evas_common_cpu_has_neon_for(NEON_PART_BLUR))
+ {
+ if (rgba)
+ {
+ if (!vert)
+ _gaussian_blur_horiz_rgba_step_neon(src, dst, radius, w, h, w, weights, pow2_div);
+ else
+ _gaussian_blur_vert_rgba_step_neon(src, dst, radius, h, w, 1, weights, pow2_div);
+ }
+ else
+ {
+ if (!vert)
+ _gaussian_blur_horiz_alpha_step_neon(src, dst, radius, w, h, w, weights, pow2_div);
+ else
+ _gaussian_blur_vert_alpha_step_neon(src, dst, radius, h, w, 1, weights, pow2_div);
+ }
+ }
+ else
+#endif
if (rgba)
{
if (!vert)
diff --git a/src/tests/evas/evas_test_neon_blur.c b/src/tests/evas/evas_test_neon_blur.c
new file mode 100644
index 0000000000..2d60f2aa80
--- /dev/null
+++ b/src/tests/evas/evas_test_neon_blur.c
@@ -0,0 +1,390 @@
+/* Differential test: C reference vs NEON for the gaussian blur steps.
+ *
+ * The blur steps are static inline templates instantiated per direction and
+ * per format, so unlike the op tables there is no dispatch table to walk.
+ * Instead this instantiates each template twice under different names - once
+ * from the C file and once from the NEON one - and runs the pair over
+ * identical buffers.
+ *
+ * Buffers are over allocated and compared in full, so a step that writes
+ * outside the region it was given is caught as well as one that computes the
+ * wrong pixel.
+ */
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <math.h>
+#include <time.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#ifdef BUILD_NEON
+# include <arm_neon.h>
+#endif
+
+#include "evas_common_private.h"
+#include "evas_private.h"
+#include "evas_filter.h"
+
+int _evas_filter_log_dom = -1;
+
+/* the templates log through CRI on a division by zero; keep them satisfied
+ * without dragging in the whole filter machinery */
+#undef CRI
+#define CRI(...) do { printf(" kernel reported: " __VA_ARGS__); printf("\n"); } while (0)
+
+#define FUNCTION_NAME blur_h_rgba_c
+#define STEP 1
+#include "blur/blur_gaussian_rgba_.c"
+
+#define FUNCTION_NAME blur_v_rgba_c
+#define STEP loops
+#include "blur/blur_gaussian_rgba_.c"
+
+#define FUNCTION_NAME blur_h_alpha_c
+#define STEP 1
+#include "blur/blur_gaussian_alpha_.c"
+
+#define FUNCTION_NAME blur_v_alpha_c
+#define STEP loops
+#include "blur/blur_gaussian_alpha_.c"
+
+#ifdef BUILD_NEON
+# define FUNCTION_NAME blur_h_rgba_neon
+# define STEP 1
+# define BLUR_NEON 1
+# include "blur/blur_gaussian_rgba_.c"
+
+# define FUNCTION_NAME blur_v_rgba_neon
+# define STEP loops
+# define BLUR_NEON 1
+# include "blur/blur_gaussian_rgba_.c"
+
+# define FUNCTION_NAME blur_h_alpha_neon
+# define STEP 1
+# define BLUR_NEON 1
+# include "blur/blur_gaussian_alpha_.c"
+
+# define FUNCTION_NAME blur_v_alpha_neon
+# define STEP loops
+# define BLUR_NEON 1
+# include "blur/blur_gaussian_alpha_.c"
+#endif
+
+/*--------------------------------------------------------------------------*/
+
+#include "blur/blur_weights_.c"
+
+#define GUARD 0xA5
+#define PAD 64 /* words of slack either side of every buffer */
+
+static unsigned int rng_state = 1;
+
+static unsigned int
+rnd(void)
+{
+ unsigned int x = rng_state;
+ x ^= x << 13; x ^= x >> 17; x ^= x << 5;
+ rng_state = x;
+ return x;
+}
+
+/* Source buffers are placed hard against an unmapped page so that any read
+ * past the end of the data faults instead of quietly returning whatever was
+ * next in the heap. The ramps used to walk off the end of a line whenever
+ * len < 2 * radius + 1, and that is invisible to a plain malloc. */
+static void *
+guarded_alloc(size_t bytes, size_t *out_off)
+{
+ const size_t page = (size_t) sysconf(_SC_PAGESIZE);
+ const size_t body = ((bytes + page - 1) / page) * page;
+ char *p = mmap(NULL, body + page, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+
+ if (p == MAP_FAILED) { perror("mmap"); exit(2); }
+ mprotect(p + body, page, PROT_NONE);
+ *out_off = body - bytes; /* butt the data up against the guard */
+ return p;
+}
+
+static void
+guarded_free(void *p, size_t bytes)
+{
+ const size_t page = (size_t) sysconf(_SC_PAGESIZE);
+
+ munmap(p, ((bytes + page - 1) / page) * page + page);
+}
+
+typedef void (*Blur_Rgba)(const DATA32 *, DATA32 *, int, int, int, int,
+ const int *, int);
+typedef void (*Blur_Alpha)(const DATA8 *, DATA8 *, int, int, int, int,
+ const int *, int);
+
+static int verbose = 0;
+static int reported = 0;
+static int max_report = 8;
+
+/* Both directions address the buffer as len * loops elements: horizontal walks
+ * a row of len and steps loopstep = len between rows, vertical walks a column
+ * of len with STEP = loops and steps one column at a time. */
+static int
+compare(const char *name, int radius, int len, int loops,
+ const unsigned char *ref, const unsigned char *got, size_t bytes)
+{
+ size_t i;
+ int bad = 0, worst = 0;
+
+ for (i = 0; i < bytes; i++)
+ {
+ int d;
+
+ if (ref[i] == got[i]) continue;
+ bad++;
+ d = ref[i] > got[i] ? ref[i] - got[i] : got[i] - ref[i];
+ if (d > worst) worst = d;
+ if ((verbose || bad == 1) && reported < max_report)
+ {
+ reported++;
+ printf(" %s r=%d len=%d loops=%d: byte %zd ref=%02x neon=%02x\n",
+ name, radius, len, loops, (ssize_t)i - PAD, ref[i], got[i]);
+ }
+ }
+ if (bad)
+ printf(" %-12s radius=%-3d len=%-4d loops=%-4d differing bytes=%d worst=%d\n",
+ name, radius, len, loops, bad, worst);
+ return bad;
+}
+
+static int
+run_rgba(const char *name, Blur_Rgba fc, Blur_Rgba fn,
+ int radius, int len, int loops, int loopstep)
+{
+ const size_t words = (size_t)len * loops + 2 * PAD;
+ const size_t bytes = words * sizeof(DATA32);
+ size_t soff;
+ char *sbase = guarded_alloc(bytes, &soff);
+ DATA32 *src = "" *)(sbase + soff);
+ DATA32 *ref = malloc(bytes), *got = malloc(bytes);
+ int *weights = malloc((2 * radius + 1) * sizeof(int));
+ int pow2 = 0, bad;
+ size_t i;
+
+ memset(ref, GUARD, bytes);
+ for (i = 0; i < words; i++) src[i] = rnd();
+ memcpy(got, ref, bytes);
+ evas_blur_weights_get(weights, &pow2, radius);
+
+ fc(src + PAD, ref + PAD, radius, len, loops, loopstep, weights, pow2);
+ fn(src + PAD, got + PAD, radius, len, loops, loopstep, weights, pow2);
+
+ bad = compare(name, radius, len, loops,
+ (unsigned char *)ref, (unsigned char *)got, bytes);
+ guarded_free(sbase, bytes); free(ref); free(got); free(weights);
+ return bad;
+}
+
+static int
+run_alpha(const char *name, Blur_Alpha fc, Blur_Alpha fn,
+ int radius, int len, int loops, int loopstep)
+{
+ const size_t bytes = (size_t)len * loops + 2 * PAD;
+ size_t soff;
+ char *sbase = guarded_alloc(bytes, &soff);
+ DATA8 *src = "" *)(sbase + soff);
+ DATA8 *ref = malloc(bytes), *got = malloc(bytes);
+ int *weights = malloc((2 * radius + 1) * sizeof(int));
+ int pow2 = 0, bad;
+ size_t i;
+
+ memset(ref, GUARD, bytes);
+ for (i = 0; i < bytes; i++) src[i] = (DATA8)rnd();
+ memcpy(got, ref, bytes);
+ evas_blur_weights_get(weights, &pow2, radius);
+
+ fc(src + PAD, ref + PAD, radius, len, loops, loopstep, weights, pow2);
+ fn(src + PAD, got + PAD, radius, len, loops, loopstep, weights, pow2);
+
+ bad = compare(name, radius, len, loops, ref, got, bytes);
+ guarded_free(sbase, bytes); free(ref); free(got); free(weights);
+ return bad;
+}
+
+/*--------------------------------------------------------------------------*/
+
+static double
+now_sec(void)
+{
+ struct timespec ts;
+ clock_gettime(CLOCK_MONOTONIC, &ts);
+ return (double)ts.tv_sec + (double)ts.tv_nsec * 1e-9;
+}
+
+/* C and NEON are timed alternately inside one trial loop and the best of N is
+ * reported, for the same reasons as the op table benchmark: on a loaded or
+ * big.LITTLE machine separate batches are not comparable. Pin with taskset. */
+static void
+bench_rgba(const char *name, Blur_Rgba fc, Blur_Rgba fn,
+ int radius, int len, int loops, int loopstep, int trials)
+{
+ const size_t words = (size_t)len * loops + 2 * PAD;
+ DATA32 *src = "" * sizeof(DATA32));
+ DATA32 *dst = malloc(words * sizeof(DATA32));
+ int *weights = malloc((2 * radius + 1) * sizeof(int));
+ double bc = 1e30, bn = 1e30;
+ int pow2 = 0, t;
+ size_t i;
+
+ for (i = 0; i < words; i++) src[i] = rnd();
+ evas_blur_weights_get(weights, &pow2, radius);
+
+ fc(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ fn(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+
+ for (t = 0; t < trials; t++)
+ {
+ double t0 = now_sec(), d;
+
+ fc(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ d = now_sec() - t0;
+ if (d < bc) bc = d;
+
+ t0 = now_sec();
+ fn(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ d = now_sec() - t0;
+ if (d < bn) bn = d;
+ }
+
+ printf(" %-12s radius=%-3d %4dx%-4d C %7.1f Mpx/s NEON %7.1f Mpx/s %5.2fx\n",
+ name, radius, len, loops,
+ (double)len * loops / bc / 1e6, (double)len * loops / bn / 1e6, bc / bn);
+ free(src); free(dst); free(weights);
+}
+
+static void
+bench_alpha(const char *name, Blur_Alpha fc, Blur_Alpha fn,
+ int radius, int len, int loops, int loopstep, int trials)
+{
+ const size_t bytes = (size_t)len * loops + 2 * PAD;
+ DATA8 *src = "" *dst = malloc(bytes);
+ int *weights = malloc((2 * radius + 1) * sizeof(int));
+ double bc = 1e30, bn = 1e30;
+ int pow2 = 0, t;
+ size_t i;
+
+ for (i = 0; i < bytes; i++) src[i] = (DATA8)rnd();
+ evas_blur_weights_get(weights, &pow2, radius);
+
+ fc(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ fn(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+
+ for (t = 0; t < trials; t++)
+ {
+ double t0 = now_sec(), d;
+
+ fc(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ d = now_sec() - t0;
+ if (d < bc) bc = d;
+
+ t0 = now_sec();
+ fn(src + PAD, dst + PAD, radius, len, loops, loopstep, weights, pow2);
+ d = now_sec() - t0;
+ if (d < bn) bn = d;
+ }
+
+ printf(" %-12s radius=%-3d %4dx%-4d C %7.1f Mpx/s NEON %7.1f Mpx/s %5.2fx\n",
+ name, radius, len, loops,
+ (double)len * loops / bc / 1e6, (double)len * loops / bn / 1e6, bc / bn);
+ free(src); free(dst); free(weights);
+}
+
+int
+main(int argc, char **argv)
+{
+#ifndef BUILD_NEON
+ (void)argc; (void)argv;
+ printf("built without BUILD_NEON - nothing to compare\n");
+ return 77;
+#else
+ /* radii around the edges of the middle loop, lengths around the vector
+ * width and around 2*radius where the middle section vanishes */
+ static const int radii[] = { 1, 2, 3, 4, 5, 7, 8, 9, 15, 16, 17, 31, 40 };
+ static const int lens[] = { 1, 2, 3, 4, 5, 7, 8, 9, 15, 16, 17, 31, 32,
+ 33, 63, 64, 65, 100, 127, 128, 129, 256, 720 };
+ int failures = 0, cases = 0, i, j;
+ unsigned int seed = 1;
+
+ for (i = 1; i < argc; i++)
+ {
+ if (!strcmp(argv[i], "-v")) verbose = 1;
+ else if (!strncmp(argv[i], "--seed=", 7)) seed = (unsigned)atoi(argv[i] + 7);
+ }
+ rng_state = seed ? seed : 1;
+
+ for (i = 1; i < argc; i++)
+ if (!strcmp(argv[i], "--bench"))
+ {
+ static const int br[] = { 2, 5, 10, 20, 40 };
+ int b;
+
+ printf("gaussian blur throughput, 720x480, best of 9\n\n");
+ for (b = 0; b < (int)(sizeof(br) / sizeof(br[0])); b++)
+ {
+ bench_rgba("horiz rgba", blur_h_rgba_c, blur_h_rgba_neon,
+ br[b], 720, 480, 720, 9);
+ bench_rgba("vert rgba", blur_v_rgba_c, blur_v_rgba_neon,
+ br[b], 480, 720, 1, 9);
+ bench_alpha("horiz alpha", blur_h_alpha_c, blur_h_alpha_neon,
+ br[b], 720, 480, 720, 9);
+ bench_alpha("vert alpha", blur_v_alpha_c, blur_v_alpha_neon,
+ br[b], 480, 720, 1, 9);
+ }
+ return 0;
+ }
+
+ printf("gaussian blur C vs NEON differential test (seed=%u)\n\n", seed);
+
+ /* loops must straddle the column block width too: the vertical kernel
+ * vectorises across columns, so a fixed small value would leave that path
+ * unexercised entirely */
+ for (i = 0; i < (int)(sizeof(radii) / sizeof(radii[0])); i++)
+ for (j = 0; j < (int)(sizeof(lens) / sizeof(lens[0])); j++)
+ {
+ const int r = radii[i], len = lens[j];
+ static const int loopvals[] = { 1, 3, 4, 5, 8, 17 };
+ int li;
+
+ if (len < 1) continue;
+
+ for (li = 0; li < (int)(sizeof(loopvals) / sizeof(loopvals[0])); li++)
+ {
+ const int loops = loopvals[li];
+
+ failures += run_rgba("horiz rgba", blur_h_rgba_c, blur_h_rgba_neon,
+ r, len, loops, len);
+ failures += run_rgba("vert rgba", blur_v_rgba_c, blur_v_rgba_neon,
+ r, len, loops, 1);
+ failures += run_alpha("horiz alpha", blur_h_alpha_c, blur_h_alpha_neon,
+ r, len, loops, len);
+ failures += run_alpha("vert alpha", blur_v_alpha_c, blur_v_alpha_neon,
+ r, len, loops, 1);
+ cases += 4;
+ }
+ }
+
+ printf("\n--- summary ---\n");
+ printf("configurations : %d\n", cases);
+ printf("differing bytes: %d\n", failures);
+ if (failures)
+ {
+ printf("RESULT: FAIL\n");
+ return 1;
+ }
+ printf("RESULT: PASS (bit exact)\n");
+ return 0;
+#endif
+}
diff --git a/src/tests/evas/meson.build b/src/tests/evas/meson.build
index a404fb5e8a..e5758c2bb7 100644
--- a/src/tests/evas/meson.build
+++ b/src/tests/evas/meson.build
@@ -53,4 +53,20 @@ if cpu_neon
env : test_env,
timeout : master_timeout
)
+
+ # Same idea for the gaussian blur steps. They are static inline templates
+ # rather than table entries, so this instantiates each one twice - from the
+ # C file and from the NEON one - and diffs the pair.
+ evas_neon_blur = executable('evas_neon_blur',
+ ['evas_test_neon_blur.c'],
+ dependencies: [evas_bin, evas, evas_ext_none_static_deps, eet, m],
+ include_directories: include_directories(
+ join_paths('..', '..', 'modules', 'evas', 'engines', 'software_generic', 'filters')),
+ c_args : ['-DEVAS_BUILD']
+ )
+
+ test('evas-neon-blur', evas_neon_blur,
+ env : test_env,
+ timeout : master_timeout
+ )
endif
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
