Re: ./bootstrap: LWP::Protocol::https replaced by java.net.URLConnection
On 07/24/2016 01:24 PM, Damjan Jovanovic wrote: Hi Both we and Infra have been battling with the buildbots for far too long, and the inability to do https:// downloads consistently due to missing LWP::Protocol::https has been a major thorn in our side. Perl in AOO is not going well. CPAN modules don't install easily: in both CentOS 5 and (Infra tells me) in Cygwin some tests fail, preventing installation. If in 5 months INFRA-11296 has not been closed, and we're still having mystery problems on the buildbots where Perl modules are installed but can't be found, and this is even holding up further development, such as being able to test whether removing VCVARS32.BAT from the Windows build script fixes the apr build failure and file lock, then I believe it's past time to tell Perl goodbye! So as of r1753943, Perl's LWP::UserAgent no longer downloads files. I was thinking of using a CLI tool instead, something like wget, but that's another dependency we'd have to document, that may not be available on all platforms (eg. Mac, OS/2), and that is broken on CentOS 5 for https:// which is what we need it for the most. So how do we download files now? Java. Java supports https:// out of the box, is very portable between operating systems and CPUs, uses its own root CA certificates, is already used on all the buildbots, and is documented as being a mandatory build dependency (even though it doesn't seem that way in ./configure). A simple class main/solenv/javadownloader/AOOJavaDownloader.java gets compiled into main/solenv//class/AOOJavaDownloader.class by the ./bootstrap script, and then called from Perl's main/solenv/bin/ download_external_dependencies.pl and main/solenv/bin/modules/ExtensionsLst.pm using system(), in place of LWP::UserAgent. Internally, it uses java.net.URLConnection for http/https support, deals with HTTP redirection, also verifying MD5 and SHA1 hashes like the Perl DownloadFile() function it replaces. The way it's been set up is a bit of a hack, but it's working phenomenally well on both FreeBSD and Windows, and with it the Windows aoo-win7 buildbot has successfully bootstrapped for the first time in memorable history, and with VCVARS32.BAT eliminated it looks like it may even finish building successfully! Damjan That seems like a great solution. Good luck. Carl - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: ./bootstrap: LWP::Protocol::https replaced by java.net.URLConnection
On Sun, Jul 24, 2016 at 7:43 PM, Patricia Shanahan wrote: > On 7/24/2016 10:24 AM, Damjan Jovanovic wrote: > ... > >> So how do we download files now? >> >> Java. Java supports https:// out of the box, is very portable between >> operating systems and CPUs, uses its own root CA certificates, is already >> used on all the buildbots, and is documented as being a mandatory build >> dependency (even though it doesn't seem that way in ./configure). A simple >> class main/solenv/javadownloader/AOOJavaDownloader.java gets compiled into >> main/solenv//class/AOOJavaDownloader.class by the ./bootstrap >> script, and then called from Perl's main/solenv/bin/ >> download_external_dependencies.pl and >> main/solenv/bin/modules/ExtensionsLst.pm using system(), in place of >> LWP::UserAgent. Internally, it uses java.net.URLConnection for http/https >> support, deals with HTTP redirection, also verifying MD5 and SHA1 hashes >> like the Perl DownloadFile() function it replaces. >> >> The way it's been set up is a bit of a hack, but it's working phenomenally >> well on both FreeBSD and Windows, and with it the Windows aoo-win7 >> buildbot >> has successfully bootstrapped for the first time in memorable history, and >> with VCVARS32.BAT eliminated it looks like it may even finish building >> successfully! >> > > This sounds very promising. I hope it works. > Every nightly buildbot bootstraps now and every *nix buildbot among them builds successfully. I consider the patch a complete success. The snapshot buildbots still can't bootstrap unless they have LWP::Protocol::https installed - not until we make another snapshot for them with r1753943 merged. It's just Windows we have to fix now. See the other thread.
The Windows buildbot saga
Hi In build 380, with bootstrapping succeeding and VCVARS32.BAT deleted, aoo-win7 got far with the build, but apparently hung on "sc deliver" until it was killed after 2 seconds with no output. A subsequent build failed to delete apr/Makefile.win which Infra found DEVENV had locked again. It's possible that one build.pl instance hung on that file, while the other compiled everything else, something that only became apparent when sc delivered and there was nothing else left to compile. Thus it's not clear whether this is progress. Another build is running now, and I've asked Infra for the output of "wmic process list" if it hangs again, which will show command line arguments passed to dmake/nmake/sh/perl/cl to help us work out what is going on. Damjan
Re: Editing Download page
On Sun, Jul 24, 2016 at 1:59 PM, JZA wrote: > I would also suggest to revisit the intro programs for development which > were put on hold indefinetly and at the momento there is really no good way > to 'learn' the AOO sourcecode. > "Intro" programs? What are these? I don't think I've seen reference to this before. > Most of the wiki Dev documentation might also need some review. Specially > key pages like conventions, and recent changes. > Yes. We will get to this soon. > Are we still using cws? > No, we are not. Mostly all development is now done in trunk, or rarely, in specialized branches. > > On Sun, Jul 24, 2016 at 3:52 PM, Andrea Pescetti > wrote: > > > On 24/07/2016 Marcus wrote: > > > >> now it's here: > >> http://ooo-site.staging.apache.org/download/index.html > >> > > > > Looks good to me (text and layout). I'd simply replace the title with > > "Help wanted" for consistency with the ASF-wide site that Dennis > mentioned > > (and also because we are already using the box for recruiting not only > > developers, so "Developers needed" is incomplete). > > > > Regards, > > Andrea. > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > > For additional commands, e-mail: dev-h...@openoffice.apache.org > > > > > > > -- > Alexandro Colorado > Apache OpenOffice Contributor > 9060 55AB FFD2 2F02 0E1A 3409 599C 14FC 9450 D3CF > -- Kay Schenk@Apache OpenOffice “Things work out best for those who make the best of how things work out.” -- John Wooden
Re: Editing Download page
On 7/24/2016 1:59 PM, JZA wrote: I would also suggest to revisit the intro programs for development which were put on hold indefinetly and at the momento there is really no good way to 'learn' the AOO sourcecode. Even if we were hiring full time programmers, AOO is too big for learning the AOO source code to be a reasonable objective for people new to it. Recruits are just going to have to live with reading and studying the area around a particular issue. Patricia - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Editing Download page
It seems I had a problem with receiving mails from MLs. E.g., I've seen Andrea's mail only in the archives. I'll work them in. Anybody else? If not, then I would publish in a few hours what we have now. Marcus Am 07/24/2016 11:23 PM, schrieb Marcus: Am 07/24/2016 06:25 PM, schrieb Dennis E. Hamilton: Marcus, A little correction here and there ... -Original Message- From: Marcus [mailto:marcus.m...@wtnet.de] My technical additions need a bit more time but in the meantime here are my text suggestions: Box headline: "Developers Needed" Box text: "Are you a software developer with C++ skills or have you deeper knowledge about building software? Do you like to work in open source projects and you are familar with the English language? Then come to us and help Apache OpenOffice to grow." [orcmid] s/you are familiar/are you familiar/ or maybe better, "are you proficient in English? s/Then come/Come/ s/to grow/expand/ Box mouse-over: "The Apache OpenOffice project need software developers to grow its base" [orcmid] s/need/needs/ Or how about, "The Apache OpenOffice project seeks developers to take part in expanding its activities." OK, sounds better. I've also changed the other things. and I would like to see more discussion too. Then I'll keep the things in staging. When someone thinks it's ready, then just publish it in the CMS. Not perfect but a start - for discussion and improvement. For further ideas you just need to change the "orange" properties in "w.oo.o/dl/msg_prop_l10n_en.js". Or I can do it tomorrow - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Officially releasing a patch for CVE-2016-1513
Thanks for the list. Apart from the differences thing it looks good to me. Marcus Am 07/24/2016 11:37 PM, schrieb Andrea Pescetti: While the severity of the security bug we disclosed http://www.openoffice.org/security/cves/CVE-2016-1513.html is not particularly high (it is classified as "Medium" with no known exploits and anti-virus software can detect malicious documents), we should release an update incorporating the -already tested- patch we disclosed in the announcement. I assume we will want to keep the effort minimal. To do so, an outline would be: 1) We commit the patch to the AOO410 branch. This is the branch used for all the 4.1.x series. 4.2.0 isn't out yet, so 4.1.x is still our reference version. 2) We do not make any other changes to the AOO410 branch. This is really meant to be a minimal update. Even the version number in the source package will remain 4.1.2. 3) We tag the release as AOO4121 and build the corresponding source package, which will have 4.1.2.1 in its name (I mean the filename, nowhere else). 4) We don't prepare full end-user release binaries but we do supply repaired libraries for power users - remember the circumstances above. The bugfix modifies one library file, and we have binaries ready for several platforms already. 5) We vote on the source and possibly binaries. We advertise the availability of the new packages on our website, but we don't send out update notifications and we don't put the files on SourceForge. Does this look OK? Once this is done, we will probably want to open another discussion and see how we can coordinate for a release that incorporates more fixes or features and is made available in full form, with all localized installers, to end users. But the above is mostly aimed in having an official way to ship the existing patch. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Officially releasing a patch for CVE-2016-1513
Am 07/25/2016 12:45 AM, schrieb Dennis E. Hamilton: The patched DLL is shipped with an external digital signature. I guess we could ask that to be installed alongside it. That would be a good tell-tale. The web site where the patch is downloadable from will have hashes for the archive containing the patched library and will also have an external signature for that. These are on a secure AOO infrastructure site, the best place to retrieve hashes and signature files. There is no reason not to have a hash of the library inside the downloadable archive for those who, for some reason, cannot check the signature but can verify the hash. In the manual procedure, we will ask users to rename the existing shared-library before copying in the replacement. This will provide a means to revert to the patched library if a regression results. There is a difference in file-creation dates and in the size of the files as well. The procedure for hotfixing with the patched library should provide that information to discourage attempting to patch a different release and also make it easier to tell the patch is there. You're right that different builds by others who look to just extract the shared library will likely end up with a different binary of that library. For a binary distribution from any origin that has the patch compiled-in, I would think something like the static string might be helpful. If we do that in the AOO4121 tag, we'll have to redo the patched libraries we've already built. I was hoping we could avoid that and stick with ones we have done some testing on already. - file size - file date+time - hash value - signature. With this we have IMHO enough differences to give the users at hand so that they can see whats old and new. Then we have enough differences to avoid to touch the source again. However, I appreciate this easy and clever thing to make the difference visible. Is what we're planning enough? For the moment I think so. Marcus -Original Message- From: Don Lewis [mailto:truck...@apache.org] Sent: Sunday, July 24, 2016 15:14 To: dev@openoffice.apache.org Subject: Re: Officially releasing a patch for CVE-2016-1513 On 24 Jul, Don Lewis wrote: At a minimum, we should publish the hash values of buggy and fixed versions of the library. That might not help someone who builds and installs from source since the build not be completely repeatable. For instance the library might contain a timestamp. Adding a static string "CVE-2016-1513 Fixed" to the source is another possibiliy. On *nix, the user/administrator can run: strings whatever.so | grep CVE and look for the above to verify that the fixed library has been installed. Someone would have to figure out how to do the equivalent on Windows. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Officially releasing a patch for CVE-2016-1513
+1 this looks like a good plan On 07/24/2016 02:37 PM, Andrea Pescetti wrote: > While the severity of the security bug we disclosed > http://www.openoffice.org/security/cves/CVE-2016-1513.html is not > particularly high (it is classified as "Medium" with no known exploits > and anti-virus software can detect malicious documents), we should > release an update incorporating the -already tested- patch we disclosed > in the announcement. > > I assume we will want to keep the effort minimal. > > To do so, an outline would be: > > 1) We commit the patch to the AOO410 branch. This is the branch used for > all the 4.1.x series. 4.2.0 isn't out yet, so 4.1.x is still our > reference version. > > 2) We do not make any other changes to the AOO410 branch. This is really > meant to be a minimal update. Even the version number in the source > package will remain 4.1.2. > > 3) We tag the release as AOO4121 and build the corresponding source > package, which will have 4.1.2.1 in its name (I mean the filename, > nowhere else). > > 4) We don't prepare full end-user release binaries but we do supply > repaired libraries for power users - remember the circumstances above. > The bugfix modifies one library file, and we have binaries ready for > several platforms already. > > 5) We vote on the source and possibly binaries. We advertise the > availability of the new packages on our website, but we don't send out > update notifications and we don't put the files on SourceForge. > > Does this look OK? > > Once this is done, we will probably want to open another discussion and > see how we can coordinate for a release that incorporates more fixes or > features and is made available in full form, with all localized > installers, to end users. But the above is mostly aimed in having an > official way to ship the existing patch. > > Regards, > Andrea. > > - > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > For additional commands, e-mail: dev-h...@openoffice.apache.org > -- Kay Schenk Apache OpenOffice "Things work out best for those who make the best of the way things work out." -- John Wooden - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
Re: Officially releasing a patch for CVE-2016-1513
On 24 Jul, Dennis E. Hamilton wrote: > The patched DLL is shipped with an external digital signature. I > guess we could ask that to be installed alongside it. That would be a > good tell-tale. > > The web site where the patch is downloadable from will have hashes for > the archive containing the patched library and will also have an > external signature for that. These are on a secure AOO infrastructure > site, the best place to retrieve hashes and signature files. There is > no reason not to have a hash of the library inside the downloadable > archive for those who, for some reason, cannot check the signature but > can verify the hash. > > In the manual procedure, we will ask users to rename the existing > shared-library before copying in the replacement. This will provide a > means to revert to the patched library if a regression results. > > There is a difference in file-creation dates and in the size of the > files as well. The procedure for hotfixing with the patched library > should provide that information to discourage attempting to patch a > different release and also make it easier to tell the patch is there. > > You're right that different builds by others who look to just extract > the shared library will likely end up with a different binary of that > library. For a binary distribution from any origin that has the patch > compiled-in, I would think something like the static string might be > helpful. If we do that in the AOO4121 tag, we'll have to redo the > patched libraries we've already built. I was hoping we could avoid > that and stick with ones we have done some testing on already. > > Is what we're planning enough? I think that should be OK. - To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org For additional commands, e-mail: dev-h...@openoffice.apache.org
