Alan Burlison created HADOOP-12926:
--
Summary: lz4.c does not detect 64-bit mode properly
Key: HADOOP-12926
URL: https://issues.apache.org/jira/browse/HADOOP-12926
Project: Hadoop Common
Alan Burlison created HADOOP-12925:
--
Summary: Checks for SPARC architecture need to include 64-bit SPARC
Key: HADOOP-12925
URL: https://issues.apache.org/jira/browse/HADOOP-12925
Project: Hadoop
Alan Burlison created HADOOP-12727:
--
Summary: Minor cleanups needed for CMake 3.X
Key: HADOOP-12727
URL: https://issues.apache.org/jira/browse/HADOOP-12727
Project: Hadoop Common
Issue Type
Alan Burlison created HADOOP-12720:
--
Summary: Misuse of sun.misc.Unsafe by
org.apache.hadoop.io.FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo
causes misaligned memory access coredumps
Key: HADOOP
Alan Burlison created HADOOP-12630:
--
Summary: Mususe of sun.misc.Unsafe causes midaligned memory access
coredumps
Key: HADOOP-12630
URL: https://issues.apache.org/jira/browse/HADOOP-12630
Project
Alan Burlison created HADOOP-12629:
--
Summary: Misaligned memory accesses in CRC32 native code causes
coredumps
Key: HADOOP-12629
URL: https://issues.apache.org/jira/browse/HADOOP-12629
Project
Alan Burlison created HADOOP-12583:
--
Summary: Sundry symlink problems on Solaris
Key: HADOOP-12583
URL: https://issues.apache.org/jira/browse/HADOOP-12583
Project: Hadoop Common
Issue Type
Alan Burlison created HADOOP-12581:
--
Summary: ShellBasedIdMapping needs suport for Solaris
Key: HADOOP-12581
URL: https://issues.apache.org/jira/browse/HADOOP-12581
Project: Hadoop Common
Alan Burlison created HADOOP-12580:
--
Summary: Hadoop needs a SysInfo class for Solaris
Key: HADOOP-12580
URL: https://issues.apache.org/jira/browse/HADOOP-12580
Project: Hadoop Common
Issue
[
https://issues.apache.org/jira/browse/HADOOP-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Burlison reopened HADOOP-12344:
This fix introduced multiple sprintf format warnings which need attention.
{code}
[exec
Alan Burlison created HADOOP-12488:
--
Summary: DomainSocket: Solaris does not support timeouts on
AF_UNIX sockets
Key: HADOOP-12488
URL: https://issues.apache.org/jira/browse/HADOOP-12488
Project
Alan Burlison created HADOOP-12487:
--
Summary: DomainSocket.close() assumes incorrect Linux behaviour
Key: HADOOP-12487
URL: https://issues.apache.org/jira/browse/HADOOP-12487
Project: Hadoop Common
On 07/10/2015 22:05, Alan Burlison wrote:
I'll draft up a proposal and attach it to HADOOP-11127.
Attached to HADOOP-11127 as proposal.txt
--
Alan Burlison
--
me up
with a once-and-for-all solution, I think overall that will work out to
be less effort and certainly less risky.
I'll draft up a proposal and attach it to HADOOP-11127.
Thanks,
--
Alan Burlison
--
inSocket on Solaris
with a 'No JNI signature changes' restriction.
Does that sound acceptable? If so I can draft up a proposal for native
library version and platform naming, library search locations etc.
Thanks,
--
Alan Burlison
--
ing about as it changed fields from final to static. I haven't
figured out what impact that has on the classes & shared object. Plus
https://issues.apache.org/jira/browse/HADOOP-12184 which removed some
fields.
--
Alan Burlison
--
out as well . In summary, the bridge is already burned.
We've covered the bridge in petrol but not quite dropped a match on
it.
No, I'm reasonable certain you've already dropped the match, and if you
haven't its just good fortune.
--
Alan Burlison
--
without further complicating the
existing complicated code with a lot of scaffolding.
I don't understand how YARN & multiple Hadoop versions interact, but if
they are all in the same JVM instance then no amount of fiddling with
shared objects will help as you can't have multiple SOs providing the
same APIs within the same process - or at least not without a lot of
complicated, fragile and utterly platform-specific configuration and code.
--
Alan Burlison
--
s I said, I believe that ship has long since sailed. Changes that have
already been let in have I believe broken the backwards binary
compatibility of the Java/JNI interface. Broken is broken, arguing that
this proposal shouldn't be allowed in because it simply adds more
brokenness to the existing brokenness is really missing the point. As
far as I can tell, there already is no backwards compatibility.
--
Alan Burlison
--
have been committed recently would have to be ripped out as well . In
summary, the bridge is already burned.
--
Alan Burlison
--
On 30/09/2015 09:14, Alan Burlison wrote:
The basic idea is to add two new fields to DomainSocket.c to hold the
read/write timeouts. On platforms that support SO_SNDTIMEO and
SO_RCVTIMEO these would be unused as setsockopt() would be used to set
the socket timeouts. On platforms such as Solaris
On 30/09/2015 17:23, Chris Nauroth wrote:
I think file descriptor sharing is a capability of Unix domain
sockets only, and not INET sockets.
Yes, that's correct.
--
Alan Burlison
--
x27;s not much in it, around 10% in both cases,
and I suspect socket throughput isn't the rate-limiting step anyway.
--
Alan Burlison
--
only caveats are that all code that does anything to a PF_UNIX
socket would *always* have to do so via DomainSocket. As far as I can
tell that's not an issue, but it would have to be borne in mind if any
changes were made in this area.
Before I set about doing this, does the approach
[
https://issues.apache.org/jira/browse/HADOOP-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Burlison resolved HADOOP-12300.
Resolution: Duplicate
> BUILDING.txt instructions for skipping tests incomplete/obsol
Alan Burlison created HADOOP-12261:
--
Summary: Surefire needs to make sure the JVMs it fires up are
64-bit
Key: HADOOP-12261
URL: https://issues.apache.org/jira/browse/HADOOP-12261
Project: Hadoop
--
Alan Burlison
--
Alan Burlison created HADOOP-12170:
--
Summary: hadoop-common's JNIFlags.cmake is redundant and can be
removed
Key: HADOOP-12170
URL: https://issues.apache.org/jira/browse/HADOOP-12170
Project: H
is pretty old, it's a 4-core Opteron with 8Gb mem. I
haven't attempted test runs on Solaris yet as I know they won't complete
successfully.
--
Alan Burlison
--
Normally I end up with a timeout error somewhere in hadoop-common, but
it's taking an hour before it fails. Is that normal?
I think parts of the ASF infra is sick. I’ve been having problems too.
I'm running the tests locally, is the ASF infrastructure sill involved?
--
Alan Burlison
--
ewhere
in hadoop-common, but it's taking an hour before it fails. Is that normal?
--
Alan Burlison
--
Alan Burlison created HADOOP-12114:
--
Summary: Make hadoop-tools/hadoop-pipes Native code -Wall-clean
Key: HADOOP-12114
URL: https://issues.apache.org/jira/browse/HADOOP-12114
Project: Hadoop Common
Alan Burlison created HADOOP-12112:
--
Summary: Make hadoop-common-project Native code -Wall-clean
Key: HADOOP-12112
URL: https://issues.apache.org/jira/browse/HADOOP-12112
Project: Hadoop Common
rowse/MAPREDUCE-6407
Migrate YARN native build to new CMake framework
https://issues.apache.org/jira/browse/YARN-3827
--
Alan Burlison
--
don't bother. I agree that doing a mass
clean-up would make a patch noisy and more difficult to review.
There are some known issues with false positives on the whitespace check
right now. Fixes are in progress.
Will do as you suggest, thanks.
--
Alan Burlison
--
While running the dev-support/test-patch.sh I got some warnings about
trailing whitespace. I'll fix the 2 that I introduced but what about all
the others in the files I've touched, fix or leave? I can fix easily
enough but it makes the patches rather noisy.
--
Alan Burlison
--
at I can gather, cross-platform support in PB breaks
fairly regularly,
--
Alan Burlison
--
s hadoop-mapreduce-client-nativetask is built without them at the
moment, and presumably works just fine.
--
Alan Burlison
--
Alan Burlison created HADOOP-12104:
--
Summary: Migrate Hadoop Pipes native build to new CMake framework
Key: HADOOP-12104
URL: https://issues.apache.org/jira/browse/HADOOP-12104
Project: Hadoop Common
lease notes. I think it's worthwhile to
figure this out first. Otherwise, there is a risk that any testing work
turns out to be a wasted effort.
One reason at least: PB 2.5.0 has no support for Solaris SPARC. 2.6.1 does.
--
Alan Burlison
--
On 03/06/2015 21:23, Alan Burlison wrote:
hadoop-common-project/hadoop-common/src/JNIFlags.cmake sets the
CMAKE_LD_FLAGS variable to the flags required for the linker. However,
despite the name, CMAKE_LD_FLAGS is *not* a valid standard CMake
variable
Confirmed, I've put nonsense fla
t variables are
CMAKE_EXE_LINKER_FLAGS, CMAKE_SHARED_LINKER_FLAGS and
CMAKE_STATIC_LINKER_FLAGS. However it's entirely possible I may be
misunderstanding, does anyone know why CMAKE_LD_FLAGS has been used?
--
Alan Burlison
--
Hadoop to YARN
3. Convert to a YARN sub-task
However I don't seem to be able to assign it (YARN-3724) to myself...
--
Alan Burlison
--
one against YARN, copying everything across manually?
Thanks,
--
Alan Burlison
--
Alan Burlison created HADOOP-12012:
--
Summary: Investigate JNI for improving byte array comparison
performance
Key: HADOOP-12012
URL: https://issues.apache.org/jira/browse/HADOOP-12012
Project
Alan Burlison created HADOOP-12008:
--
Summary: Investigate providing SPARC hardware-accelerated CRC32
code
Key: HADOOP-12008
URL: https://issues.apache.org/jira/browse/HADOOP-12008
Project: Hadoop
without causing
significant regressions. I guess we can continue the discussion on
the JIRA.
Sure, I'm typing up some comments right now :-)
Thanks,
--
Alan Burlison
--
Alan Burlison created HADOOP-11997:
--
Summary: CMake CMAKE_C_FLAGS are non-portable
Key: HADOOP-11997
URL: https://issues.apache.org/jira/browse/HADOOP-11997
Project: Hadoop Common
Issue
On 17/05/2015 21:57, Roman Shaposhnik wrote:
quick question: when you say Solaris, what do you *really* mean?
Solaris 11 latest update, sparc and x86.
--
Alan Burlison
--
t's
obviously a longer-term option.
--
Alan Burlison
--
Alan Burlison created HADOOP-11987:
--
Summary: JNI build should use default cmake FindJNI.cmake
Key: HADOOP-11987
URL: https://issues.apache.org/jira/browse/HADOOP-11987
Project: Hadoop Common
Alan Burlison created HADOOP-11985:
--
Summary: Improve Solaris support in Hadoop
Key: HADOOP-11985
URL: https://issues.apache.org/jira/browse/HADOOP-11985
Project: Hadoop Common
Issue Type
On 16/05/2015 03:24, Mai Haohui wrote:
Just checked the repo of cmake and it turns out that FindJNI.cmake is
available even before cmake 2.4. I think it makes sense to file a bug
to replace it to the standard cmake module. Can you please file a jira
for this?
Will do, thanks.
--
Alan
only correct on Linux whereas I'd expect FindJNI.cmake to be
more platform-independent.
If I'm reading JNIFlags.cmake properly it only exports JNI_INCLUDE_DIRS
and JNI_LIBRARIES, both of which appear to also be exported by
FindJNI.cmake.
Looking for guidance...
--
Alan Burlison
--
side Hadoop and we'd obviously like
to minimise the versions of PB that we ship, and preferably just ship
the latest version. The fact that PB seems to often be incompatible
across releases is an issue as it makes upgrading and dropping older
versions problematic.
--
Alan Burlison
--
Alan Burlison created HADOOP-11975:
--
Summary: Native code needs to be built to match the 32/64 bitness
of the JVM
Key: HADOOP-11975
URL: https://issues.apache.org/jira/browse/HADOOP-11975
Project
Alan Burlison created HADOOP-11974:
--
Summary: FIONREAD is not always in the same header
Key: HADOOP-11974
URL: https://issues.apache.org/jira/browse/HADOOP-11974
Project: Hadoop Common
possible to try a custom build right now
linked against 2.6.1. You'd pass -Dprotobuf.version=2.6.1 and
-Dprotoc.path= when you run the mvn command.
Once I have fixed all the other source portability issues I'll circle
back around and take a look at this.
--
Alan Burlison
--
is still unclear about the need to build
hadoop-maven-plugins first
And assigned it to myself.
--
Alan Burlison
--
Alan Burlison created HADOOP-11968:
--
Summary: BUILDING.txt is still unclear about the need to build
Key: HADOOP-11968
URL: https://issues.apache.org/jira/browse/HADOOP-11968
Project: Hadoop Common
commended target for someone to use when
starting from scratch, test?
Please feel free to file a new issue if you'd like to see this clarified
in BUILDING.txt. We'd welcome a patch if you want to provide one. Thanks!
Yes, I'll do that.
--
Alan Burlison
--
The current version of Protocol Buffers is 2.6.1 but the current version
required by Hadoop is 2.5.0. Is there any reason for this, or should I
log a JIRA to get it updated?
--
Alan Burlison
--
JIRA, presumably because
I'm not blessed enough to do so.
Thanks,
--
Alan Burlison
--
63 matches
Mail list logo
