Dear Pierre,Thanks for digging into it. Indeed the tests check whether unaligned memory access works. Unfortunately, that is used in the NativeTaggedArray for 64bit primitive types and even ranks, see e.g. line 267 in NativeTaggedArray.
As I cannot reproduce the error myself, not even on a machine with aarch64 architecture, can you please tell me if the tests pass when you apply the attached patch?
Best regards, Bernd On 12/13/20 9:51 PM, Pierre Gruet wrote:
Dear Bernd,I have looked again at the issue we discussed last Friday; as pointed out in the companion bug reporthttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=929530#5I see an unaligned address is passed to GetXxxArrayRegion() in the native part of the code, when running the tests testXxxToByteToXxx, for Xxx in {Char, Short, Ing, Long, Float, Double}: this happens when the argument targetOfs of the test is not a multiple of the size of Xxx.I suggest to remove all pairs (i,j) from getOfs() in NativeDataTests where j is not zero. I think the pairs (i,0) could remain even when i is not zero, but I would remove them as well because the only place, in the main source, where copyXxxToByte or copyByteToXxx are used is in NativeTaggedArray, and there it never happens that inStart and outStart are not multiples of the size of Xxx.Do you agree or am I missing something? If you think my suggestions are correct, I will upload the package with a patch leaving only the pair (0,0) in getOfs() in NativeDataTests.Warm thanks for your help, Pierre On Thu, 10 Dec 2020 18:45:37 +0100 Bernd Rinn <br...@ethz.ch> wrote: > Dear Pierre, > > No, I've not yet seen this bus error. I can see two changes to my test > setup: > > - JDK 11 instead of JDK 8 > - 64bit OS instead of 32 bit OS > > The 64bit platoform is more likely to be the change that uncovers this > bug. Which hardware did you get the error on? > > All the best, > Bernd > > On 12/10/20 5:45 PM, Pierre Gruet wrote: > > Control: forwarded -1 br...@ethz.ch > > > > > > Dear Bernd, > >> > In Debian we have received a bug report you may find below: there seems > > to be an unaligned memory access in (seemingly) the JNI part of sis-base > > version 18.09, leading to a failure on the architecture armhf running on> > a 64bit kernel. > > > > Have you already met this issue and do you see how it might be fixed? > > > > Thanks a lot, > > Pierre Gruet > > > >> > On Thu, 10 Dec 2020 07:15:24 +0100 Matthias Klose <d...@debian.org> wrote:> > > Package: src:libsis-base-java > > > Version: 18.09~pre1+git20180928.45fbd31+dfsg-2 > > > Severity: important > > > Tags: sid bullseye > > >> > > building libsis-base-java (or running the jni) leads to a bus error,> > usually > > > caused by unaligned memory accesses. > > > > > > [...] > > > LC_ALL=C java -Djava.library.path=source/c/.libs -classpath > > sis-base-test.jar > > > ch.systemsx.cisd.base.AllTests > > > Application: base > > > Version: UNKNOWN* > > > Java VM: OpenJDK Server VM (v11.0.9.1+1-Debian-1) > > > CPU Architecture: arm > > > OS: Linux (v4.15.0-126-generic) > > > Test class: NativeDataTests > > > > > > Running testFloatToByteNonNativeByteOrderPartialOutputArray > > > Running testIntToByteToInt > > > Arguments: [0, 0] > > > Arguments: [0, 1] > > > # > > > # A fatal error has been detected by the Java Runtime Environment: > > > # > > > # SIGBUS (0x7) at pc=0xf74b1d1c, pid=10186, tid=10187
diff --git a/source/java/ch/systemsx/cisd/base/convert/NativeData.java b/source/java/ch/systemsx/cisd/base/convert/NativeData.java
index f962af5..ec85500 100644
--- a/source/java/ch/systemsx/cisd/base/convert/NativeData.java
+++ b/source/java/ch/systemsx/cisd/base/convert/NativeData.java
@@ -15,8 +15,6 @@ package ch.systemsx.cisd.base.convert;
import java.nio.ByteBuffer;
-import ch.systemsx.cisd.base.utilities.NativeLibraryUtilities;
-
/**
* This class encapsulates native methods to deal with arrays of numbers, converting from numbers to
* bytes and bytes to numbers.
@@ -28,26 +26,10 @@ import ch.systemsx.cisd.base.utilities.NativeLibraryUtilities;
* primitive numbers (int, short, ...)
* <p>
* Variant interfaces convert only a sub-array.
- * <p>
- * The class has optimized methods using jni-libraries for some common platforms and a pure-java
- * implementation (called <i>javamode</i> if the jni-libraries are not available). If you want to
- * enforce <i>javamode</i>, you need to pass the property <code>nativedata.javamode=true</code> to
- * the JRE.
*/
public class NativeData
{
- private static final boolean useNativeLib;
-
- static
- {
- if (Boolean.getBoolean("nativedata.javamode"))
- {
- useNativeLib = false;
- } else
- {
- useNativeLib = NativeLibraryUtilities.loadNativeLibrary("nativedata");
- }
- }
+ private static final boolean useNativeLib = false;
/** Size of a <code>short</code> value in <code>byte</code>s. */
public final static int SHORT_SIZE = 2;
diff --git a/sourceTest/java/ch/systemsx/cisd/base/convert/NativeDataTests.java b/sourceTest/java/ch/systemsx/cisd/base/convert/NativeDataTests.java
index e343acb..6198ee5 100644
--- a/sourceTest/java/ch/systemsx/cisd/base/convert/NativeDataTests.java
+++ b/sourceTest/java/ch/systemsx/cisd/base/convert/NativeDataTests.java
@@ -162,7 +162,6 @@ public class NativeDataTests
@Test(dataProvider = "getOfs")
public void testLongToByteToLong(int sourceOfs, int targetOfs)
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 8;
final long[] orignalArr = new long[]
{ -1, 17, 100000, -1000000 };
@@ -180,7 +179,6 @@ public class NativeDataTests
@Test(dataProvider = "getOfs")
public void testShortToByteToShort(int sourceOfs, int targetOfs)
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 2;
final short[] orignalArr = new short[]
{ -1, 17, 20000, (short) -50000 };
@@ -198,7 +196,6 @@ public class NativeDataTests
@Test(dataProvider = "getOfs")
public void testCharToByteToChar(int sourceOfs, int targetOfs)
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 2;
final char[] orignalArr = new char[]
{ 'c', ';', '\u0222', '\u1000' };
@@ -216,7 +213,6 @@ public class NativeDataTests
@Test(dataProvider = "getOfs")
public void testFloatToByteToFloat(int sourceOfs, int targetOfs)
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 4;
final float[] orignalArr = new float[]
{ -1, 17, 3.14159f, -1e6f };
@@ -234,7 +230,6 @@ public class NativeDataTests
@Test(dataProvider = "getOfs")
public void testDoubleToByteToDouble(int sourceOfs, int targetOfs)
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 8;
final double[] orignalArr = new double[]
{ -1, 17, 3.14159, -1e42 };
@@ -252,7 +247,6 @@ public class NativeDataTests
@Test
public void testShortEndianConversion()
{
- assertTrue(NativeData.isUseNativeLib());
final short[] values = new short[]
{ 1, 2, 4, 8, 16, 256, 512 };
final short[] convertedValuesExpected = new short[]
@@ -266,7 +260,6 @@ public class NativeDataTests
@Test
public void testIntEndianConversion()
{
- assertTrue(NativeData.isUseNativeLib());
final int[] values = new int[]
{ 1, 2, 4, 8, 16, 256, 1 << 16 };
final int[] convertedValuesExpected = new int[]
@@ -280,7 +273,6 @@ public class NativeDataTests
@Test
public void testLongEndianConversion()
{
- assertTrue(NativeData.isUseNativeLib());
final long[] values = new long[]
{ 1, 2, 4, 8, 16, 256, 1L << 16, 1L << 24 };
final long[] convertedValuesExpected = new long[]
@@ -294,7 +286,6 @@ public class NativeDataTests
@Test
public void testFloatLittleEndianRoundtrip()
{
- assertTrue(NativeData.isUseNativeLib());
final float[] values = new float[]
{ 1.1f, 2.2f, 3.3f, 1e-25f, 1e25f };
final float[] convertedValuesFound =
@@ -306,7 +297,6 @@ public class NativeDataTests
@Test
public void testFloatBigEndianRoundtrip()
{
- assertTrue(NativeData.isUseNativeLib());
final float[] values = new float[]
{ 1.1f, 2.2f, 3.3f, 1e-25f, 1e25f };
final float[] convertedValuesFound =
@@ -318,7 +308,6 @@ public class NativeDataTests
@Test
public void testDoubleLittleEndianRoundtrip()
{
- assertTrue(NativeData.isUseNativeLib());
final double[] values = new double[]
{ 1.1f, 2.2f, 3.3f, 1e-25f, 1e25f };
final double[] convertedValuesFound =
@@ -330,7 +319,6 @@ public class NativeDataTests
@Test
public void testDoubleBigEndianRoundtrip()
{
- assertTrue(NativeData.isUseNativeLib());
final double[] values = new double[]
{ 1.1, 2.2, 3.3, 1e-25, 1e25 };
final double[] convertedValuesFound =
@@ -342,28 +330,24 @@ public class NativeDataTests
@Test(expectedExceptions = NullPointerException.class)
public void testNPE()
{
- assertTrue(NativeData.isUseNativeLib());
NativeData.copyByteToLong(null, 0, null, 0, 0, ByteOrder.NATIVE);
}
@Test(expectedExceptions = IndexOutOfBoundsException.class)
public void testIOOB()
{
- assertTrue(NativeData.isUseNativeLib());
NativeData.copyByteToLong(new byte[] {}, -1, new long[] {}, 0, 0, ByteOrder.NATIVE);
}
@Test(expectedExceptions = IndexOutOfBoundsException.class)
public void testIOOB2()
{
- assertTrue(NativeData.isUseNativeLib());
NativeData.copyByteToLong(new byte[] {}, 0, new long[] {}, 10, 0, ByteOrder.NATIVE);
}
@Test
public void testPlatformEndiness()
{
- assertTrue(NativeData.isUseNativeLib());
final double[] values = new double[]
{ 1.1, 2.2, 3.3, 1e-200, 1e200 };
final double[] valuesLE =
@@ -387,7 +371,6 @@ public class NativeDataTests
@Test
public void testFloatToByteNonNativeByteOrderPartialOutputArray()
{
- assertTrue(NativeData.isUseNativeLib());
final int sizeOfTarget = 4;
final ByteOrder nonNativeByteOrder =
(NativeData.getNativeByteOrder() == ByteOrder.LITTLE_ENDIAN) ? ByteOrder.BIG_ENDIAN
smime.p7s
Description: S/MIME Cryptographic Signature
