On Wed, Dec 10, 2014 at 2:31 AM, malcolm <malcolm.kaval...@oracle.com> wrote: > Hi Colin, > > Thanks for the hints around JIRAs. > > You are correct errno still exists, however sys_errlist does not. > > Hadoop uses a function terror (defined in exception.c) which indexes > sys_errlist by errno to return the error message from the array. This > function is called 26 times in various places (in 2.2) > > Originally, I thought to replace all calls to terror with strerror, but > there can be issues with multi-threading (it returns a buffer which can be > overwritten), so it seemed simpler just to recreate the sys_errlist message > array. > > There is also a multi-threaded version strerror_r where you pass the buffer > as a parameter, but this would necessitate changing every call to terror > with mutiple lines of code.
Why don't you just use strerror_r inside terror()? I wrote that code originally. The reason I didn't want to use strerror_r there is because GNU libc provides a non-POSIX definition of strerror_r, and forcing it to use the POSIX one is a pain. But you can do it. You also will require a thread-local buffer to hold the return from strerror_r, since it is not guaranteed to be static (although in practice it is 99% of the time-- another annoyance with the API). > > Sorry, I wasn't clear. > > Also, I have been requested to ensure my port is available on 2.4, perceived > as a more stable release. If I make changes to this branch are they > automatically available for 2.6, or will I need multiple JIRAs ? As Steve commented, 2.4 is pretty much "done." I don't think this kind of thing would get backported there. Even getting it in a 2.6 release seems unlikely to me (but I haven't thought about it that hard). I would just get the work done, and let it show up in the release it's ready in. cheers, Colin > > Thanks, > Malcolm > > > On 12/10/2014 10:45 AM, Colin McCabe wrote: >> >> Hi Malcolm, >> >> In general we file JIRAs for particular issues. So if one issue is >> handling errlist on Solaris, that might be one JIRA. Another issue >> might be handling socket write timeouts on Solaris. And so on. Most >> of these should probably be HADOOP tickets since they sound like they >> are mostly in the generic hadoop-common code. >> >> "solaris does not have errno" seems like a bold statement. errno is >> part of POSIX, and Solaris is a POSIX os, right? Am I way off base on >> this? >> I googled around and one of the first results I found talked about >> errno values on Solaris. >> >> http://www.pixelstech.net/article/1413273556-A-trick-of-building-multithreaded-application-on-Solaris >> Perhaps I misunderstood what you meant by this statement. >> >> Anyway, please file JIRAs for any portability improvements you can think >> of! >> >> best, >> Colin >> >> On Mon, Dec 8, 2014 at 9:09 PM, malcolm <malcolm.kaval...@oracle.com> >> wrote: >>> >>> Hi Colin, >>> >>> A short summary of my changes are as follows: >>> >>> - Native C source files: added 5, modified 6, requiring also changes to >>> CMakeLists.txt. Of course, all changes are "ifdeffed" for Solaris >>> appropriately and new files, are prefixed with solaris_ as well. >>> >>> For example, Solaris does not have errno, or errlist any more which are >>> used >>> quite a lot in hadoop native code. I could have replaced all calls to use >>> strerror instead which would be compatible with Linux, however in the >>> interests of making minimal changes, I recreated and added these files >>> from >>> a running Solaris machine instead. >>> >>> Another issue is that Solaris doesn't have the timeout option for >>> sockets, >>> so I had to write my own solaris_read routine with timeout and added it >>> to >>> DomainSocket.c . A few issues with lz4 on Sparc needed modification, and >>> some other OS specific issues: getgrouplist, container-executer (from >>> yarn). >>> >>> - Some very minor changes were made to some Java source files (mainly >>> tests >>> to get them to pass on Solaris) >>> >>> The above changes were made to 2.2, I will recheck everything against the >>> latest trunk, maybe some fixes aren't needed any more. >>> >>> I have generated a single patch file with all changes. Perhaps it would >>> be >>> better to file multiple JIRAs for each change, perhaps grouped, one per >>> issue ? Or should I file a JIRA for each modified source file ? >>> >>> Thank you, >>> Malcolm >>> >>> >>> On 12/08/2014 09:53 PM, Colin McCabe wrote: >>>> >>>> Hi Malcolm, >>>> >>>> It's great that you are going to contribute! Please make your patches >>>> against trunk. >>>> >>>> 2.2 is fairly old at this point. It hasn't been the focus of >>>> development in more than a year. >>>> >>>> We don't use github or pull requests. >>>> >>>> Check the section on http://wiki.apache.org/hadoop/HowToContribute >>>> that talks about "Contributing your work". Excerpt: >>>> "Finally, patches should be attached to an issue report in Jira via >>>> the Attach File link on the issue's Jira. Please add a comment that >>>> asks for a code review following our code review checklist. Please >>>> note that the attachment should be granted license to ASF for >>>> inclusion in ASF works (as per the Apache License ยง5)." >>>> >>>> As this says, you attach the patch file to a JIRA that you have >>>> created, and then hit "submit patch." >>>> >>>> I don't think a branch is required for this work since it is just >>>> build fixes, right? >>>> >>>> best, >>>> Colin >>>> >>>> >>>> On Mon, Dec 8, 2014 at 3:30 AM, malcolm <malcolm.kaval...@oracle.com> >>>> wrote: >>>>> >>>>> I have ported Hadoop native libraries to Solaris 11 (both Sparc and >>>>> Intel ) >>>>> Oracle have agreed to release my changes to the community so that >>>>> Solaris >>>>> platforms can benefit. >>>>> Reading the HowToContribute and GitandHadoop documents, I am not 100% >>>>> clear >>>>> on how to get my changes into the main tree. I am also a Git(hub) >>>>> newbie, >>>>> and was using svn previously. >>>>> >>>>> Please let me know if I am going the correct path: >>>>> >>>>> 1. I forked Hadoop on Github and downloaded a clone to my development >>>>> machine. >>>>> >>>>> 2. The changes I made were to 2.2.0, can I still add changes to this >>>>> branch, >>>>> and hopefully get them accepted or must I migrate my changes to 2.6 ? >>>>> (On >>>>> the main Hadoop download page, 2.2 is still listed as the GA version ) >>>>> >>>>> 3. I understand that I should create a new branch for my changes, and >>>>> then >>>>> generate pull requests after uploading them to Github. >>>>> >>>>> 4. I also registered at Jira in the understanding that I need to >>>>> generate a >>>>> Jira number for my changes, and to name my branch accordingly ? >>>>> >>>>> Does all this make sense ? >>>>> >>>>> Thanks, >>>>> Malcolm >>>>> >>>>> >
