Hi Bob, Thanks for some feedback, I don't think a lot of people are developing on windows. Some answers in line:
* Build does require Java 8, not "8 or later" as stated in java/README.md > There's a reference to sun.misc.Unsafe > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > which of course went away in JDK 9. Is this the case even using "-Dio.netty.tryReflectionSetAccessible=true" as mentioned in the README? * The build won't work on Windows because the java/format POM downloads a > binary flatc executable, but there's no artifact for Windows, just Linux > and OSX. I wound up downloading Visual Studio and building the flatbuffers > project. This unfortunately sounds familiar, I think this indicate the popularity on windows. I also think the hosting of the mac and linux are not exactly official (they are hosted by a former contributor). Updating the Readme might be a good first step with instructions on how to do this. I see in the pom.xml that user.timezone is set to UTC. I have seen these > types of errors in tests before; I know there are ways to insulate the test > from the user's current timezone but maybe someone knows what's going on This is somewhat surprising. I would thought we had the user.timezone set for exactly this reason. There might have been a regression, this might make a good second contribution if you wanted to look into fixing it. * I bumped into what looks like a spurious checkstyle error: it reports > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no > linefeed at the end when it definitely does. I set up Git not to do Windows > conversions, and I checked with various editors and binary dump utilities. > One source says that this because I'm running on Windows, checkstyle > actually expects a CR-LF and throws an error if it doesn't find it! I've > worked around this by disabling the check. It looks like we can force the checker to assume linux line feeds: https://stackoverflow.com/questions/997021/how-to-get-rid-of-checkstyle-message-file-does-not-end-with-a-newline (second answer). This would also make a good contribution for someone new. Cheers, Micah On Thu, Mar 11, 2021 at 7:14 PM bobtins <bobti...@gmail.com> wrote: > My mail client took out all the linefeeds, so let me reformat; sorry about > that! > > > In the process of slogging through the build, I've bumped into various > issues. I'm happy to document them in java/README.md or make any other > changes that might be helpful to others. > > I'm pretty experienced with Java and Maven, so I think these are not > beginner's mistakes, but let me know if I'm missing something. > > A lot of these may be Windows-specific. I normally prefer Linux but just > got a new laptop and haven't set it up, but this experience is giving me a > lot of incentive to run screaming back to Linux ;-) > > Environment details: > * Windows 10 > * Java 8 > here's the output of java -version: > openjdk version "1.8.0_282" > OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_282-b08) > OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08, mixed mode) > > * Cygwin environment > * Maven 3.6.2 > > > Issues encountered thus far: > > * Build does require Java 8, not "8 or later" as stated in java/README.md > There's a reference to sun.misc.Unsafe > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > which of course went away in JDK 9. > * The build won't work on Windows because the java/format POM downloads a > binary flatc executable, but there's no artifact for Windows, just Linux > and OSX. I wound up downloading Visual Studio and building the flatbuffers > project. > > * I bumped into what looks like a spurious checkstyle error: it reports > memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java having no > linefeed at the end when it definitely does. I set up Git not to do Windows > conversions, and I checked with various editors and binary dump utilities. > One source says that this because I'm running on Windows, checkstyle > actually expects a CR-LF and throws an error if it doesn't find it! I've > worked around this by disabling the check. > > * The one thing that I'm stuck on now is failures on the jdbc module: > [INFO] > [INFO] Results: > [INFO] > [ERROR] Failures: > [ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209 > expected:<45935000> but was:<74735000> > [ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213 > expected:<1518439535000> but was:<1518468335000> > [ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205 > expected:<-365> but was:<-364> > [ERROR] > > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209 > expected:<17574> but was:<17573> > [ERROR] JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 > expected:<17574> but was:<17573> > [ERROR] > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277 > expected:<17574> but was:<17573> > [ERROR] > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277 > expected:<17574> but was:<17573> > [INFO] > [ERROR] Tests run: 93, Failures: 7, Errors: 0, Skipped: 0 > > I attached the full build output. > Looking more closely at these errors, they seem to be due to the timezone > difference; for example, the difference between 74735000 (actual value) and > 45935000 (expected) is 2880000, or 8 hours in milliseconds, which is the > PST timezone offset. > > I see in the pom.xml that user.timezone is set to UTC. I have seen these > types of errors in tests before; I know there are ways to insulate the test > from the user's current timezone but maybe someone knows what's going on. > > Thanks for any input! > > > On 2021/03/11 23:49:37, Bob Tinsman <bobt...@pacbell.net> wrote: > > I've been mostly lurking for awhile, but I would like to start picking > off some bugs in the Java implementation.In the process of slogging through > the build, I've bumped into various issues. I'm happy to document them in > java/README.md or make any other changes that might be helpful to others. > I'm pretty experienced with Java and Maven, so I think these are not > super-obvious, but let me know if I'm missing something.A lot of these may > be Windows-specific. I normally prefer Linux but just got a new laptop and > haven't set it up, but this experience is giving me a lot of incentive to > run screaming back to Linux ;-) > > Environment details:- Windows 10- Java 8:openjdk version > "1.8.0_282"OpenJDK Runtime Environment (AdoptOpenJDK)(build > 1.8.0_282-b08)OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.282-b08, > mixed mode)- Cygwin environment- Maven 3.6.2 > > Issues encountered thus far:- Build does require Java 8, not "8 or > later" as stated in java/README.md There's a reference to > sun.misc.Unsafe > in > memory/memory-core/src/main/java/org/apache/arrow/memory/util/MemoryUtil.java > which of course went away in JDK 9. I can update the build > instructions.- The build won't work on Windows because the java/format POM > downloads a binary flatc executables; when I looked, there was no version > for Windows, just Linux and OSX. I wound up downloading Visual Studio and > building the flatbuffers project.- I bumped into what looks like a spurious > checkstyle error: it > reports memory/src/test/java/io/netty/buffer/TestExpandableByteBuf.java > having no linefeed at the end when it definitely does. I set up Git not to > do Windows conversions, and I checked with various editors and binary dump > utilities. One source says that this because I'm running on Windows, > checkstyle actually expects a CR-LF and throws an error if it doesn't find > it! I've worked > around this by disabling the check.- The one thing that I'm stuck on now > is failures on jdbc: > > [INFO] Results:[INFO][ERROR] Failures:[ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:209 > expected:<45935000> but was:<74735000>[ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:213 > expected:<1518439535000> but was:<1518468335000>[ERROR] > JdbcToArrowDataTypesTest.testJdbcToArrowValues:146->testDataSets:205 > expected:<-365> but was:<-364>[ERROR] > > JdbcToArrowNullTest.testJdbcToArrowValues:123->testDataSets:165->testAllVectorValues:209 > expected:<17574> but was:<17573>[ERROR] > JdbcToArrowTest.testJdbcToArrowValues:138->testDataSets:206 > expected:<17574> but was:<17573>[ERROR] > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValuesNoLimit:107->validate:199->assertDateDayVectorValues:277 > expected:<17574> but was:<17573>[ERROR] > > JdbcToArrowVectorIteratorTest.testJdbcToArrowValues:95->validate:199->assertDateDayVectorValues:277 > expected:<17574> but was:<17573>[INFO][ERROR] Tests run: 93, Failures: 7, > Errors: 0, Skipped: 0 > > I attached the full build output.Looking more closely at these errors, > they seem to be due to the timezone difference; for example, the difference > between 74735000 (actual value) and 45935000 (expected) is 2880000, or 8 > hours in milliseconds, which is the PST timezone offset. I see in the > pom.xml that user.timezone is set to UTC. I have seen these types of errors > in tests before; I know there are ways to insulate the test from the user's > current timezone but maybe someone knows what's going on. > > Thanks for any input! > > >
