New docbook-xsl (1.79.2) breaks existing XSL files

[Leonardo Brondani Schenkel]

Ticket: https://trac.macports.org/ticket/55255

DocBook XSLs changed their URIs from `sourceforge.net` to `github.com`, 
which results in existing XML files using the former URIs not able to 
find the local copy installed by the port. This will trigger a download 
from the URI or fail if net access is disabled.


This could break many existing ports and user XML files. Neomutt for 
example can no longer be built: https://trac.macports.org/ticket/55253


I'm writing to the list because this change in upstream might have far 
reaching implications, and this port has no official maintainer, so I 
want to bring as much visibility to the issue as possible and discuss a 
solution.


Thanks,
// Leonardo.

Re: New docbook-xsl (1.79.2) breaks existing XSL files

[Zero King]

On Mon, Nov 06, 2017 at 09:25:47AM +0100, Leonardo Brondani Schenkel wrote:

Ticket: https://trac.macports.org/ticket/55255

DocBook XSLs changed their URIs from `sourceforge.net` to 
`github.com`, which results in existing XML files using the former 
URIs not able to find the local copy installed by the port. This will 
trigger a download from the URI or fail if net access is disabled.


They switched to cdn.docbook.org.

This could break many existing ports and user XML files. Neomutt for 
example can no longer be built: https://trac.macports.org/ticket/55253


I'm writing to the list because this change in upstream might have far 
reaching implications, and this port has no official maintainer, so I 
want to bring as much visibility to the issue as possible and discuss 
a solution.


From https://trac.macports.org/ticket/55255:

make both URIs be supported somehow (not sure if this is even possible)


Fix proposed in https://github.com/macports/macports-ports/pull/1004.
Not sure if this is the best solution though.

--
Best regards,
Zero King


smime.p7s
Description: S/MIME cryptographic signature

Re: New docbook-xsl (1.79.2) breaks existing XSL files

[Leonardo Brondani Schenkel]

On 2017-11-06 10:10, Zero King wrote:

On Mon, Nov 06, 2017 at 09:25:47AM +0100, Leonardo Brondani Schenkel wrote:

Ticket: https://trac.macports.org/ticket/55255

DocBook XSLs changed their URIs from `sourceforge.net` to 
`github.com`, which results in existing XML files using the former 
URIs not able to find the local copy installed by the port. This will 
trigger a download from the URI or fail if net access is disabled.


They switched to cdn.docbook.org.


Yes, my mistake when writing the e-mail. I got it right on the issue, 
though.




This could break many existing ports and user XML files. Neomutt for 
example can no longer be built: https://trac.macports.org/ticket/55253


I'm writing to the list because this change in upstream might have far 
reaching implications, and this port has no official maintainer, so I 
want to bring as much visibility to the issue as possible and discuss 
a solution.


 From https://trac.macports.org/ticket/55255:

make both URIs be supported somehow (not sure if this is even possible)


Fix proposed in https://github.com/macports/macports-ports/pull/1004.
Not sure if this is the best solution though.


I agree with your intent, at least being a pragmatic way of fixing it 
until something better comes up.


It's really strange to me that upstream would simply make this change 
and suddenly affect all existing DocBook XMLs. This is not the kind of 
stuff I would expect in a point release. Later I'll take a look at their 
mailing list because there must have been some discussion about that, 
and I want to understand their reasoning.


// Leonardo.

Re: [MacPorts] ProblemHotlist modified

[Rainer Müller]

On 2017-11-05 21:28, Ryan Schmidt wrote:
> 
> On Nov 5, 2017, at 14:10, MacPorts wrote:
> 
>> Page "ProblemHotlist" was changed by grimreaper
>> Diff URL: 
>> 
>> Revision 136
>> Comment: xcode 7.2 is old and people upgraded
> 
> Well, Xcode 7.2.1 is the last version of Xcode available for OS X Yosemite 
> 10.10; those users cannot upgrade to a fixed version.

In this case the entry could be kept on the YosemiteProblems wiki page
only? There is already a note, although I am not sure if this addresses
the same problem:

https://trac.macports.org/wiki/YosemiteProblems#iphonesimulator

Rainer

interesting tool - deploymate - scans software for MacOS SDK needs

[Ken Cunningham]

I thought I would mention a tool I came across recently that might be of some 
use to people on this list who are sometimes tasked with making software 
runnable on older systems.

Deploymate 

scans source code for SDK dependencies, and will list certain MacOS SDK calls 
that don't meet a certain target. Handy for seeing what will and won't break on 
certain systems, and gives you the option of of #ifdef'ing things out or 
otherwise fixing the builds if you want to do so.

So far seems much easier than the trial and error method I've used to date.

Only real hiccup is that it requires an xcode project to scan, which sometimes 
means making an xcode project for a Makefile project. (A good system I found 
for doing that is here, btw: 

Use HFS compression on installed ports...

[Eric A. Borisch]

Just wondering what everyone would think of applying this patch? It enables
a runtime test during activation to see if (1) bsdtar is in path and (2) if
it supports the --hfsCompression option. (The version from libarchive does.)

The --hfsCompression does what you would expect: applies compression to the
extracted files. The bsdtar implementation of this automatically becomes a
no-op [1] on filesystems that do not support compression, as well as
checking to make sure the compression is beneficial [2] and bailing
(decompressing) otherwise. As such, we can just pass the flag, and if it
can't be used, ~ no harm done.

For systems where it can be used, however, large savings can be realized
(3:1) on some of the largest (e.g. compilers w/ include files) packages.
With more macs being sold with non-user-upgradable storage, this seems like
a win.

Since this is a run-time check, similar to the lbzip2 check a few lines
later, it won't get tripped up on a 'port upgrade libarchive' action, for
example. (By the same token, libarchive itself will never be HFS+
compressed, unless the user explicitly installs yet another modern-enough
bsdtar in path. Direction could be given to users to install libarchive
before activating ports if space is a concern. (Or perhaps we have a
'macports-extras' port group that installs libarchive and lbzip2; two
things that really are nice to have installed, and improve the performance
of MacPorts. We could also prompt during 'port upgrade' actions if it is
not installed, perhaps with a persistent "don't bother me again" y/n/N
option...)

 -- Eric

[1] https://github.com/libarchive/libarchive/blob/c10875db78
b04fd730bb08245b7032baba906815/libarchive/archive_write_disk_posix.c#L710
[2] https://github.com/libarchive/libarchive/blob/c10875db78
b04fd730bb08245b7032baba906815/libarchive/archive_write_disk_posix.c#L1326

diff --git a/src/registry2.0/portimage.tcl b/src/registry2.0/portimage.tcl
index 72dcdfd3..bdab6f8f 100644
--- a/src/registry2.0/portimage.tcl
+++ b/src/registry2.0/portimage.tcl
@@ -348,9 +348,18 @@ proc extract_archive_to_tmpdir {location} {
 t(ar|bz|lz|xz|gz) {
 set tar "tar"
 if {[catch {set tar [macports::findBinary $tar
${macports::autoconf::tar_path}]} errmsg] == 0} {
-ui_debug "Using $tar"
-set unarchive.cmd "$tar"
-set unarchive.pre_args {-xvpf}
+# Opportunistic HFS compression. bsdtar will
automatically
+# ignore if on non-HFS filesystem.
+if {![catch {macports::binaryInPath bsdtar}] &&
+![catch {exec bsdtar -x --hfsCompression <
/dev/null >& /dev/null}]} {
+ui_debug "Using bsdtar with HFS+ compression (if
valid)"
+set unarchive.cmd "bsdtar"
+set unarchive.pre_args {-xvp --hfsCompression -f}
+} else {
+ui_debug "Using $tar"
+set unarchive.cmd "$tar"
+set unarchive.pre_args {-xvpf}
+}
 if {[regexp {z2?$} ${unarchive.type}]} {
 set unarchive.args {-}
 if {[regexp {bz2?$} ${unarchive.type}]} {

Re: [macports-ports] branch master updated: cmake: require C++11 for both release and devel.

[Ryan Schmidt]

On Oct 18, 2017, at 06:25, Michael Dickens wrote:

> Michael Dickens (michaelld) pushed a commit to branch master
> in repository macports-ports.
> 
> 
> https://github.com/macports/macports-ports/commit/c8d94a92cb1d690a3dc052ff1ed7c0107af9b32a
> 
> The following commit(s) were added to refs/heads/master by this push:
> 
>  new c8d94a9  cmake: require C++11 for both release and devel.
> 
> c8d94a9 is described below
> 
> 
> commit c8d94a92cb1d690a3dc052ff1ed7c0107af9b32a
> 
> Author: Michael Dickens 
> AuthorDate: Wed Oct 18 07:25:19 2017 -0400
> 
> 
> cmake: require C++11 for both release and devel.

Wow, this really sucks. It means that on Mountain Lion and earlier, cmake now 
depends on clang-5.0, clang-4.0, clang-3.9, clang-3.8, and clang-3.7, and on 
Lion and earlier, also clang-3.4. It also says all compilers are blacklisted so 
I'm not sure if it builds. Judging by the fact that binary packages were only 
produced for Mavericks and later, I'm guessing it does not build on earlier 
systems.

cmake is in the dependency chain of the ports needed for the buildbot worker 
infrastructure, which I'm just in the process of updating and discovering this 
problem. Not to mention lots of other ports that heretofore built successfully 
with libstdc++. Assuming cmake 3.9.5 does not build on systems with libstdc++, 
maybe you could have the port drop back to cmake 3.9.4 on those systems.

Actually, I can't find the "The C++ compiler does not support C++11" message in 
3.9.5. Are you sure it requires C++11? If it does not, please revert the above 
commit.

Re: Use HFS compression on installed ports...

[Ryan Schmidt]

On Nov 6, 2017, at 21:10, Eric A. Borisch wrote:

> Just wondering what everyone would think of applying this patch?

Are you aware of prior discussion in this ticket?

https://trac.macports.org/ticket/36560

Re: [macports-ports] branch master updated: cmake: require C++11 for both release and devel.

[Ken Cunningham]

On 2017-11-06, at 7:13 PM, Ryan Schmidt wrote:

> Wow, this really sucks. It means that on Mountain Lion and earlier, cmake now 
> depends on clang-5.0, clang-4.0, clang-3.9, clang-3.8, and clang-3.7, and on 
> Lion and earlier, also clang-3.4. It also says all compilers are blacklisted 
> so I'm not sure if it builds. Judging by the fact that binary packages were 
> only produced for Mavericks and later, I'm guessing it does not build on 
> earlier systems.


Yeah, Mike and I had a little chat about that a few weeks ago when this came 
through. This is presently a bootstrap issue for new installs on older systems. 
The c++11 requirement is right in the kitware commits, so that looks like a 
real deal. 
.


A cmake 3.8 port that doesn't require c++11 could be held back to bootstrap.

Or clang-3.7 (last one we have that builds with autotools) could be used to 
build cmake, I'm pretty sure that builds on everything 10.6 to 10.8.

Or even gcc6 could be used to build it (which is what we do on PPC).

But it has to be resolved.


Ken

Re: [macports-ports] branch master updated: cmake: require C++11 for both release and devel.

[Ryan Schmidt]

On Nov 6, 2017, at 21:34, Ken Cunningham wrote:

> On 2017-11-06, at 7:13 PM, Ryan Schmidt wrote:
> 
>> Wow, this really sucks. It means that on Mountain Lion and earlier, cmake 
>> now depends on clang-5.0, clang-4.0, clang-3.9, clang-3.8, and clang-3.7, 
>> and on Lion and earlier, also clang-3.4. It also says all compilers are 
>> blacklisted so I'm not sure if it builds. Judging by the fact that binary 
>> packages were only produced for Mavericks and later, I'm guessing it does 
>> not build on earlier systems.
> 
> 
> Yeah, Mike and I had a little chat about that a few weeks ago when this came 
> through.

Yes I just saw your comments on GitHub:

https://github.com/macports/macports-ports/commit/c8d94a92cb1d690a3dc052ff1ed7c0107af9b32a


> This is presently a bootstrap issue for new installs on older systems. The 
> c++11 requirement is right in the kitware commits, so that looks like a real 
> deal. 
> .

I know cmake master requires C++11. What I'm saying is that I don't think 3.9.5 
requires C++11. I'm checking on that now.

Re: [macports-ports] branch master updated: cmake: require C++11 for both release and devel.

[Ryan Schmidt]

On Nov 6, 2017, at 21:40, Ryan Schmidt wrote:

> On Nov 6, 2017, at 21:34, Ken Cunningham wrote:
> 
>> On 2017-11-06, at 7:13 PM, Ryan Schmidt wrote:
>> 
>>> Wow, this really sucks. It means that on Mountain Lion and earlier, cmake 
>>> now depends on clang-5.0, clang-4.0, clang-3.9, clang-3.8, and clang-3.7, 
>>> and on Lion and earlier, also clang-3.4. It also says all compilers are 
>>> blacklisted so I'm not sure if it builds. Judging by the fact that binary 
>>> packages were only produced for Mavericks and later, I'm guessing it does 
>>> not build on earlier systems.
>> 
>> 
>> Yeah, Mike and I had a little chat about that a few weeks ago when this came 
>> through.
> 
> Yes I just saw your comments on GitHub:
> 
> https://github.com/macports/macports-ports/commit/c8d94a92cb1d690a3dc052ff1ed7c0107af9b32a
> 
> 
>> This is presently a bootstrap issue for new installs on older systems. The 
>> c++11 requirement is right in the kitware commits, so that looks like a real 
>> deal. 
>> .
> 
> I know cmake master requires C++11. What I'm saying is that I don't think 
> 3.9.5 requires C++11. I'm checking on that now.

Yup, 3.9.5 builds fine with libstdc++. c8d94a92cb1d690a3dc052ff1ed7c0107af9b32a 
should be reverted.

Re: Use HFS compression on installed ports...

[Eric A. Borisch]

On Mon, Nov 6, 2017 at 9:24 PM, Ryan Schmidt 
wrote:

>
> On Nov 6, 2017, at 21:10, Eric A. Borisch wrote:
>
> > Just wondering what everyone would think of applying this patch?
>
> Are you aware of prior discussion in this ticket?
>
> https://trac.macports.org/ticket/36560


Yes. I thought I'd start here to cast a wider net.

I actually had been running with my own patch to run afsctool (in parallel)
over extracted files, and then discovered that bsdtar has the
--hfsCompression option; we can use _a_ tar, just like we usually do, with
one extra flag, and it's done. Unfortunately even on High Sierra the native
bsdtar is 2.8.3, and it looks [1] like this went in in 3.1.0.

As you can see from the patch there are only four new actions: two tests
(bsdtar in path? bsdtar accepts --hfsCompression?) and two assignments if
those are both true (use tar=bsdtar, and use --hfsCompression in the flags).

It's a slam dunk from my perspective. Compared with five years ago, the
things that make it beneficial (trading CPU cores sitting idle for disk
space) are just more in abundance, and with more systems with soldered-on
storage, drive capacity is still at a premium. The only question (to me) is
if we should bundle (~2MB installed, ~900k packaged) libarchive, or test
for it at runtime similar to lbzip2. (Which is another
everyone-should-use-it tool in the multicore age...) I can see arguments
for both, but the test-at-activation-time is certainly lower impact to
begin with, and reduces the required footprint.

The (by far) biggest downside is that it negatively impacts activation
time. This as due to the extra (expensive) compression step performed done
during activation, but it doesn't have to write the file and then
re-process it (afsctool route) -- bsdtar does it in-line. (If a file
doesn't compress well, it ends up writing the compressed one, and then
re-writing the decompressed one, from my quick read of the code.)

Also of (new) note, it appears to still work on APFS volumes. There were
mixed messages in early reviews of APFS if this would be the case, but at
least the finder is reporting a file extracted via bsdtar with
--hfsCompression clearly shows the file as compressed and reports both the
size and space used on disk -- and they're vastly different for a trivially
compressible (repetitive) text file.

Some quick sizes (High Sierra)

boost: 290MB w/o compression; 190MB with
qt5-qtbase: 56MB / 30MB
llvm-4.0: 135MB / 60MB

I'm sure there will be those that will question how it will impact
performance. I'm sure there are tests one could devise where it is not
beneficial, but I'll exit with one final note: Apple HFS+ compresses just
about everything I've checked (again High Sierra) in the system / default
applications. Take a look for yourself; 'open /usr/include' and then use
the finder to get info on a header file. curses.h as a nice large one to
see.

 - Eric

(I'll post a link to this thread in the ticket for good measure.)

[1] https://github.com/libarchive/libarchive/commit/fb67a295
4f6ad067a272373eca797278e8232223

[Sorry for the duplicate Ryan; didn't hit reply-all; edited third paragraph
from previous email that just went to you, FWIW]

Help needed: fix build issue with xdxf_makedict in older platforms

[Leonardo Brondani Schenkel]

Hello all,

I recently introduced a new port, xdxf_makedict, which built 
successfully on my box (macOS 10.13) on first try. However, after it got 
merged I started getting notifications from the buildbots about build 
failures on 10.12 and older.


The first problem, which I now have solved, was that the code was using 
the EXIT_SUCCESS/EXIT_FAILURE macros in some files without including 
 first.


However, now I started getting other, different failures:
- 10.8: 
https://build.macports.org/builders/ports-10.8_x86_64_legacy-builder/builds/42330/steps/install-port/logs/stdio

- 10.7: same problem as 10.8
- 10.6: 
https://build.macports.org/builders/ports-10.6_x86_64_legacy-builder/builds/46675/steps/install-port/logs/stdio


The problem is that I cannot replicate any of this, and I don't have any 
older environment to test. So far I have been fixing the issues "blind" 
but I have committed two fixes already and I don't want to be committing 
fixes on top of fixes that might or might not solve the issue, polluting 
the history in the process.  Could somebody help?


This is kind of a fringe program, I have decided to add the port since I 
could build it on first try and thought it might be useful to others. 
But now I'm kinda disappointed with the state of the codebase and I'm 
starting to wonder if adding this was the right call...


// Leonardo.