Re: how to handle this element in Makefile.config ?

[Rainer Müller]

On 2018-03-26 14:31, macpo...@parvis.nl wrote:
> I'm working on a port for munin 2.0.x
> 
> the munin-node component builds and installs java plugins if the Makefile can 
> find the javac compiler.
> 
> on my system I need to have java but I want to ignore this feature.
> 
> code is:
> -
> # Java compiler stuff - only needed on the buildhost
> JC := javac
> JFLAGS := -g -source 1.5 -target 1.5 -Xlint
> JAR:= jar
> 
> # Check if the java compiler works
> #Note that we defer JCVALID evaluation to runtime,
> #since $(JC) can be redefined later in a specific Makefile.config
> #The core Makefile.config is then used as a Makefile.default
> JCVALID= $(shell $(JC) -version >/dev/null 2>/dev/null && echo "yes")
> -
> 
> I've tried:
> - build.env JCVALID=no
> - build.env JC=false
> but both don't work.

make will not use these variables from the environment, as they are set
explicitly in the Makefile. You can only override the variables on the
make command line.

Try setting the variable in build.args instead:

build.args-append JCVALID=no

Rainer

Mirror size & completeness of binaries

[Rainer Müller]

On 2018-03-27 13:56, Mojca Miklavec wrote:
> I don't know about the best way to do it, but I would like to suggest to
> provide macports mirorring in two different sizes: a small one and a
> complete one.
> 
> While I'm a heavy supporter of providing support for legacy systems, I
> see no reason to mirror files for them on all of our mirrors and cause
> troubles to them. I would suggest to mirror by default just the latest
> version of any given source and binary and only support the latest three
> OSes there. Then we could have additional files to support older systems
> on a smaller set of mirrors, just on those where it would not cause any
> additional troubles to them. Since the number of users of legacy systems
> is much smaller, this should not have a heavy impact on bandwidth to
> that smaller number of mirrors either.
> 
> I'm not saying this should be implemented immediately, but I would
> certainly start thinking about that before we add additional four
> mirrors (three legacy ones and 10.14).

It would probably help if we had a top-level directory for the macOS
version. Mirroring a specific subset of the archives would then be trivial.

I know it is nice to see all packages for a port in one place and it is
easier to check what has already been built. But hopefully we would have
this information on individual port index pages soon.

Rainer

Re: Mirror size & completeness of binaries

[Rainer Müller]

On 2018-03-29 16:19, Ryan Schmidt wrote:
> At the same time, we need to consider how MacPorts base will respond to the 
> situation. Currently, it assumes all packages are available on all mirrors. 
> Therefore, it tries to find a package on three mirrors before giving up. If 
> many mirror operators choose to exclude an OS version, this could leave a 
> user building from source, even when a binary exists on a more-distant 
> mirror; I want to avoid that. One workaround is to recommend that mirror 
> operators who exclude some packages should also add http redirects for 
> excluded packages to their web server configuration, redirecting those 
> requests back to the main (full) mirror. We can update our sample 
> configurations on the Mirroring wiki page to reflect this. 

We can easily split our definition of mirror_sites and only list the
URLs that will provide packages for a particular version.

I started looking at #56053 [1] yesterday and I think it would actually
be helpful if we would use a configuration file with the same syntax as
archive_sites.conf instead of the messy mirror_sites.tcl we have right
now, as it makes assumptions about the internals of base.

I am not sure either how we could do such a reorganisation without
wasting a lot of bandwidth, but we could at least start with the new
scheme as of 10.14.

>> I'm not saying this should be implemented immediately, but I would certainly 
>> start thinking about that before we add additional four mirrors (three 
>> legacy ones and 10.14).
> 
> As I understood our libstdc++ to libc++ transition plan for 10.6-10.8 
> systems, we will not create new builders. We will instead repurpose existing 
> builders for libc++, and delete the old libstdc++ archives for 10.6-10.8 
> system when we do so.

As far as I see based on the ticket [2], we do not have a decision on
that yet.

Rainer

[1] https://trac.macports.org/ticket/56053
[2] https://trac.macports.org/ticket/50448

Re: Mirror size & completeness of binaries

[Rainer Müller]

On 2018-03-30 21:20, Ryan Schmidt wrote:
> I have not yet heard any reason why we need to change the file structure of 
> the packages server.

The start of the discussion was to have mirrors that only provide a
subset of the packages, for example the latest release. It would be a
lot easier to see what a mirror provides if that was visible through a
top-level directory and it would also make it clear where the mirror
should be listed.

For example, in a future archive_sites.conf:

name   macports
platform   darwin 18
urls   https://packages.macports.org/10.14/ \
   https://foo.bar.packages.macports.org/10.14/

name   macports
platform   darwin 17
urls   https://packages.macports.org/10.13/ \
   https://bar.baz.packages.macports.org/10.13/

... and so on.

With this layout, it would be trivial to spot if one of the mirrors does
not actually provide packages for this version. When just excluding
*darwin_X*, not so easy.

>> PS: if we gradually move packages.macports.org to a different
>> subdomain, we could, say, one year after the move, reuse that
>> subdomain for package index :)
> 
> If by "package index" you mean the one-web-page-per-port system that we've 
> long desired but not yet implemented, then I had always assumed it would be 
> located under www.macports.org/ports; we should stop creating separate 
> hostnames for every new thing and instead try to create a single cohesive web 
> experience.

There has been no major update to the website in the last 10 years.
Whoever is going to create such a ports index should not have the burden
to redesign the rest of the website. Every second link on
www.macports.org is already going to another domain. I am not against
having the same design for all content on the web, but it is a larger
project than just getting such a ports index at all and I would not care
if it did not match any of the others at first.

Rainer

Re: Python ports: should we use wheel files?

[Rainer Müller]

On 2018-03-31 12:11, Enrico Maria Crisostomo wrote:
> The definition of a wheel file is the following:
> 
>> Wheel (in this context) is a project that adds the bdist_wheel command to 
>> distutils/setuptools. This produces a cross platform binary packaging format 
>> (called “wheels” or “wheel files” and defined in PEP 427) that allows Python 
>> libraries, even those including binary extensions, to be installed on a 
>> system without needing to be built locally.  In the case in point: one 
>> TensorFlow dependency has a native component that needs to be built locally.

I see two reasons why python modules need natively compiled binary files:

a) for performance
b) they link against another library

In case of b), using pre-compiled binary files is out of the question,
because we want to link to our own libraries and not against libraries
in /usr/lib.

Can we rely on Portfile authors to check what is linked?
Are they aware which libraries this python module links to before
submitting the Portfile?

It would probably always be safe to use wheels for platform "any", but
there is also no advantage over setup.py for them.

If we had a working pypi2port, this would probably be easy to test on a
wide range of python ports...

> Now, the problem.  macOS is a platform where TensorFlow is _built_ and 
> _tested_ by upstream.  Some of its dependencies (e.g.: grpcio) are packaged 
> and distributed as wheel files.  Hence, what upstream supports is the state 
> of a system after the dependencies are installed from wheel files.
> 
> Given the above assumption, my impression is that we have an opportunity to 
> leverage wheel files instead of rebuild each dependency from source.  The 
> advantages in my opiion are:
> 
>   * The _state_ of the system after the installation of the wheel files shall 
> be _exactly_ se seen, tested and validated by upstream.

Might as well be a downside, as it takes away all options to patch or
adapt...

We would also lose +universal variants for python ports, but I did not
check if we actually have such ports.

>   * We're not duplicating effort already put in by upstream.
> 
>   * We may not be introducing subtle problems.

However, nothing would change for users. We already provide
(pre-compiled) archives for most python ports, so there is no immediate
benefit of switching a python module port to use a wheel.

> I'm well aware that the philosophy of MacPorts (and BSD ports in general) is 
> to rebuild every package from source.  But I'm wondering, for the sake of 
> argument, whether a parallel can be drawn between (Python, wheel file, PyPi) 
> and (Java, *AR files, Maven Central).  After all we already ship Java 
> byte-code without recompiling it, and sometimes Java projects ship 
> precompiled native libraries.

The situation for Java is not fully comparable, as we do not provide the
runtime environment, but rely on the JRE of the system. Whether we can
safely ship pre-compiled Java code is still not clear to me.

Have any other package managers picked up wheels to build packages?

Rainer

Re: [macports-ports] branch boost_update created (now b06095e)

[Rainer Müller]

Hello Mark,

On 2016-12-27 20:15, Mark Evenson wrote:
> Mark Evenson (easye) pushed a change to branch boost_update
> in repository macports-ports.
> 
>   at b06095e  boost: update to 1.63.0; remove $id$ line.
> 
> This branch includes the following new commits:
> 
>  new b06095e  boost: update to 1.63.0; remove $id$ line.
> 
> The 1 revisions listed above as "new" are entirely new to this
> repository and will be described in separate emails.  The revisions
> listed as "add" were already present in the repository and have only
> been added to this reference.

You restored this branch multiple times, although it was always deleted
it in the macports-ports repository afterwards. I do not know with which
tool or command you are pushing, but please do not re-add this branch again.

The best option would be that you just delete it on your local
repository, so it will not accidentally be pushed again:

$ git branch -D boost_update

Rainer

xcodebuild wants to write to $HOME/Library/Developer (was: Re: Editing Git history with GitUp)

[Rainer Müller]

On 2018-03-23 11:17, Mojca Miklavec wrote:
> I just stumbled upon this app (I haven't tested it yet) which might be
> able to avoid some pain when editing git history before pushing your
> changes upstream:
> http://gitup.co

Thank you for the this tip. I especially like the visualization of
branches in GitUp a lot.

As GitUp is open source software, I tried to create a port for it.
However, I ran into strange problems with xcodebuild.

Here is the work in progress Portfile:
https://github.com/raimue/macports-ports/blob/gitup/devel/GitUp/Portfile

The build fails with this error message (ignoring other messages from
com.apple.CoreSimulator.SimError, which I have seen before and seem
harmless):

---
Build Preparation
Couldn't create workspace arena folder 
'/opt/local/var/macports/home/Library/Developer/Xcode/DerivedData/GitUp-fnbahhvyzxfjjddhqkxlurazjuhj':
 You don’t have permission to save the file 
“GitUp-fnbahhvyzxfjjddhqkxlurazjuhj” in the folder “DerivedData”.
Couldn't create module cache folder 
'/opt/local/var/macports/home/Library/Developer/Xcode/DerivedData/ModuleCache': 
Unable to create directory: 
/opt/local/var/macports/home/Library/Developer/Xcode/DerivedData/ModuleCache 
(Operation not permitted)
Couldn't update module cache session file 
'/opt/local/var/macports/home/Library/Developer/Xcode/DerivedData/ModuleCache/Session.modulevalidation':
 The folder “Session.modulevalidation” doesn’t exist.
---

My system is currently running:
macOS 10.12.6 16G1212
Xcode 9.2 9C40

The problem here is that xcodebuild is not respecting $HOME from the
environment, but uses the home directory of the macports user as set in
directory service. This path is not writable, as it is shared between
all port builds and we do not want builds to influence each other.

As you can see in the Portfile linked above, I already tried to pass an
explicit HOME variable and also some other variables that the Xcode
build settings reference [1] suggested.

The commented line can be used to make xcodebuild dump all build
settings and their values.

Do we have any Xcode experts with another idea which path setting needs
to be changed to avoid accessing $HOME/Library/Developer?

Rainer

[1] Current: http://help.apple.com/xcode/mac/9.3/#/itcaec37c2a6 (could not find 
the same for 9.2)
Retired: 
https://developer.apple.com/legacy/library/documentation/DeveloperTools/Reference/XcodeBuildSettingRef/1-Build_Setting_Reference/build_setting_ref.html#//apple_ref/doc/uid/TP40003931-CH3-SW51
 (more detailed)

Re: [macports-base] branch master updated: portbuild: Pass -j for parallel builds to cargo

[Rainer Müller]

On 2018-04-02 22:13, Jeremy Huddleston Sequoia wrote:
> This causes cargo to fail to build with -j being unrecognized.
> 
> --->  Building cargo
> Executing:  cd
> "/opt/local/var/macports/build/_Users_jeremy_src_macports_macports-ports_devel_cargo/cargo/work/cargo-0.25.0"
> && 
> /opt/local/var/macports/build/_Users_jeremy_src_macports_macports-ports_devel_cargo/cargo/work/cargo-0.24.0-x86_64-apple-darwin/cargo/bin/cargo
> -j4 --release -j4 
> error: Unknown flag: '-j'

Indeed, thanks for noticing.

The -jX argument would only be valid after the sub-command, for example
'cargo build -j4'. It worked with the cargo port group, because there I
set it up like this:

build.cmd   cargo build
build.target--release

I guess I will rather add -jX to the port group and revert the change in
base as it seems unreliable to add it this way.

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-03 01:18, db wrote:
> On 2 Apr 2018, at 23:09, Clemens Lang  wrote:
>> On Mon, Apr 02, 2018 at 08:43:52PM +0200, db wrote:
>>> GitLab 10.6 released with CI/CD for GitHub
>> Interesting, but I couldn't find any indication that this would support 
>> builds on macOS. Do you have more info?
> 
> I think you and Mojca were referring to the shared runners, then 
> https://gitlab.com/gitlab-com/infrastructure/issues/3183. But you don't need 
> them.

Then please explain what this would offer us at all?

Rainer

Re: Trac upgraded to 1.2

[Rainer Müller]

On 2018-04-01 09:35, Ryan Schmidt wrote:
> So far, I dislike the "cosmetic improvements for ticket changelog".
> 
> https://trac.edgewall.org/ticket/11835
> 
> I'm pretty sure I don't like the property changes floated to the right, and I 
> don't like that the box they're in is yellow.

I agree the floated box might look a bit odd when no comment was
entered. But it also allows to quickly separate comments from status
changes when scrolling through the changes to the ticket.

> We already override the yellow color of the ticket description box, so maybe 
> we could at least override the yellow color of the property change box too.

Yes, please, the colors should be consistent. The yellow of the property
change box and also the :target background color (after adding a new
comment) should be replaced with the same gray as the summary box at the
top.

Our custom CSS is here, additional rules would be appreciated:

https://github.com/macports/trac.macports.org/blob/master/htdocs/style.css

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-03 12:31, db wrote:
> On 3 Apr 2018, at 02:20, Rainer Müller  wrote:
>> Then please explain what this would offer us at all?
> 
> I'll try. GitLab let's you have external runners 
> (https://docs.gitlab.com/ee/ci/runners/README.html), while TravisCI offers a 
> certain amount of pipeline minutes and only supports the currently supported 
> macOS versions (AFAIK from what others posted not long ago on the subject). 
> Now, since version 10.6 you can use an external repo hosted on GitHub 
> (https://about.gitlab.com/2018/03/22/gitlab-10-6-released/#gitlab-cicd-for-external-repos).
>  And once a build succeeds, I guess it's also deliverable, whereas I'm not 
> sure it's now the case.

As far as I understood it, gitlab-runner can be installed on a machine
and is then able to spawn VMs on-demand with snapshots with an executor
(docker, virtualbox, parallels, ...). The VM image also has to have
gitlab-runner installed.

But what exactly do you think would be the benefit from such a
complicated setup (GitHub -> GitLab -> External Runner)?

We can already trigger our Buildbot directly from GitHub and could spawn
a VM from a snapshot using libvirt. In any case, the the same amount of
work is required to prepare the VM image, just with different software
in it.

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-03 13:37, Mojca Miklavec wrote:
>> We can already trigger our Buildbot directly from GitHub and could spawn
>> a VM from a snapshot using libvirt.
> 
> My (potentially wrong) impression was that libvirt only works:
> - from 10.11 on
I think you are mixing things up here. libvirt offers an abstraction
layer to control various hypervisors, such as qemu, VMware, VirtualBox,
ESX, etc. It would be used as a mechanism to start/stop VMs and manage
snapshots.

What you could possibly mean by the limitations is the
Hypervisor.framework (actually as of macOS 10.10). But that is only
needed on the host machine, not for the VMs running on that hypervisor.

> - with a bit of legal questionability

>From my understanding, it is legal to virtualize macOS, as long as the
hypervisor is running on a Mac as well, which is possible.

Rainer

Re: port policy questions for munin

[Rainer Müller]

On 2018-04-04 01:38, macpo...@parvis.nl wrote:
> Questions about policy:
> - how to submit Portfile (unified diff against the previous version?)

If your new Portfile is significantly different from the previous
version, a unified diff would not be helpful. Just attach the full
Portfile in this case.

> - how to submit patch files (complete file as diff on diff is unreadable ?)

Indeed, attach the full patch files.

> - how to remove patch files (submit empty file + note in comment ?)

Just a note should do.

> - where to submit (in trac ?)

It is fine to attach a port update to a Trac ticket.

The preferred way, however, would be to submit a pull request against
the macports-ports repository on GitHub [1], as it offers automatic
checks and a better interface for reviewing.

> - port lint nitpicking complains about patchfile filenames, not ending in 
> '.diff'. should i keep the previous names or change according to lint?
Please use either .diff or .patch, so syntax highlighting works
correctly when viewing the file.

As a side note, we had a recent discussion on relaxing the rules for
patchfile naming [2] and consensus seems to be that we no longer want to
require the patch-* prefix.

> - munin has 2 cron jobs: munin-node and munin-cron. currently munin-node is 
> implemented as a normal startupitem, munin-cron (1.4.7 munin-update) as a 
> separate file because StartInterval=300 isn't supported by macports 
> startupitem. wouldn't it be more logical to use seperate files for both as 
> that is more clear to the user? or perhaps create a real crontab?

Users would not care how the .plist files were created when the end
result is the same.

Using the startupitem.* in MacPorts for munin-node seems fine, as this
will automatically handle the interaction with launchd through daemondo.

> - munin stores html files in ${prefix}/www/munin. shouldn't that be 
> /var/www/munin instead?

I do not have that directory on my system. Other ports are also using
${prefix}/www/ for HTML files.

$ ls -d /var/www
ls: /var/www: No such file or directory

Rainer

[1] https://github.com/macports/macports-ports/tree/master/net/munin
[2] https://lists.macports.org/pipermail/macports-dev/2018-March/037676.html

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Rainer Müller]

On 2018-04-04 16:36, Andrew Moore wrote:
> If I’m reading correctly the attached report from build...@macports.org
> , port net/mosquitto fails for OS X <=
> 10.6 due to indirect dependency: libpsl.  If so, this is presumably a
> known issue, since many ports depend upon libpsl?

The failed libpsl build is this one (you can find it in the log of the
install-dependencies step). The build failed because libpsl cannot be
fetched on systems that do not support modern TLS protocol versions.
This is a well known problem.

https://build.macports.org/builders/ports-10.6_x86_64_legacy-builder/builds/61220

Now that we execute the mirror step before attempting a build, we should
make the buildbot use a master_site_local pointing to a local mirror, as
we cannot expect the upload to be completed at the time the build runs.
Right now we are downloading all files twice.

Rainer

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Rainer Müller]

On 2018-04-04 17:19, Rainer Müller wrote:
> Now that we execute the mirror step before attempting a build, we should
> make the buildbot use a master_site_local pointing to a local mirror, as
> we cannot expect the upload to be completed at the time the build runs.
> Right now we are downloading all files twice.

I split the fetch phase into a separate command. This should make it
easier to see that the dependency failed because it was unable to fetch
the distfiles.

https://github.com/macports/mpbb/commit/639def5ad990ecf1e227e377e5a46124c447ca94

This will also no longer update the failcache, as fetching could be a
temporary problem.

This should also remedy the situation on legacy systems for now.
Installation as a dependency will be retried even if a previous run
failed due to fetching. By that time, the distfile was hopefully
uploaded to the mirror.

We should still investigate in adding a master_site_local, as this
involves more fetches from the internet than necessary at the moment.

Rainer

Re: port & root

[Rainer Müller]

On 2018-04-05 18:26, Bjarne D Mathiesen wrote:
> When running './diffPorts php72-' as root I get this :

Just to be specific, did you open a root shell first or did you run it
with sudo?

> Error: /macports/bin/port: Failed to initialize MacPorts, error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < while executing
> < "file delete -force "${target_dir}/com.apple.dt.Xcode.plist""
> < (procedure "macports::copy_xcode_plist" line 5)
> < invoked from within
> < "macports::copy_xcode_plist $env(HOME)"
> < (procedure "mportinit" line 540)
> < invoked from within
> < "mportinit ui_options global_options global_variations"

This looks like the error we have also seen on Travis, but were never
able to reproduce:

https://lists.macports.org/pipermail/macports-dev/2017-June/035945.html

Rainer

Re: port & root

[Rainer Müller]

On 2018-04-05 19:14, Joshua Root wrote:
> My guess is that this is a bug in Tcl's file command. Deleting a
> nonexistent file is not supposed to be considered an error. But by
> starting two instances of port(1) at the same time, a race condition
> arises. The 'file delete' command probably checks whether the file
> exists, returns TCL_OK if not, and then attempts to delete the file. If
> the file has been deleted by the other running instance (or indeed
> anything else) between these two steps, then the actual deletion fails,
> and it incorrectly returns an error.

You are right. We hit a race condition in the implementation of 'file
delete' in Tcl.

It is missing another check for ENOENT here, as done in line 381:

https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415


@l2dy,
Is our Travis job executing multiple 'port lint' commands in parallel?
Then this is the same cause.

Rainer

Re: py36 sub-ports fail to build

[Rainer Müller]

On 2018-04-05 19:05, Bjarne D Mathiesen wrote:
> port -q search py36- \
> | xargs -n1 sudo port -Nup -f install

Just a small hint here. The port command supports wildcards for port
names. Just remember to escape for your shell:

$ sudo port -Nup -f install 'py36-*'

> diff <( port -q search py36- ) \
>  <( port -q installed name:^py36- | awk '{print $1}' ) \
> | grep '<'

The port command also supports operators to merge the results of
wildcard expansion and pseudo-ports. This would be equivalent to:

$ port -q echo 'py36-*' and not installed

Although the syntax is nicer, the downside is that this is also slower.
This is due to the inefficient implementation in the port client, which
first builds both lists completely before calculating the intersection.

However, if you need that interactively, the time requirement for typing
the port expression and its execution together will probably still be
less than for the shell pipeline above.

Rainer

Re: [macports-ports] branch master updated: librsvg: also use pre-cargo version for 10.8

[Rainer Müller]

On 2018-04-05 22:32, David B. Evans wrote:
> +# cargo build also fails on 10.8 due to known SSL/TLS issues with
> github.com +# Updating registry
> `https://github.com/rust-lang/crates.io-index` +# Downloading
> shell-escape v0.1.4 +# error: unable to get packages from source +#
> Caused by:[35] SSL connect error (error:14077410:SSL
> routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure) +#
> https://trac.macports.org/ticket/56195
cargo is using git for this operation and the failure is coming from
/usr/bin/git.

cargo should declare

    if {${os.subplatform} eq "macosx" && [vercmp ${macosx_version} 10.8]
<= 0} {
    depends_run port:git
    }

for legacy platforms, as cargo is unusable if it cannot update its registry.

I'll leave the change to someone who is interested and willing to test this.

Rainer

Re: [MacPorts] ProblemHotlist modified

[Rainer Müller]

On 2018-04-06 07:54, Ryan Schmidt wrote:
> 
> On Apr 5, 2018, at 20:54, MacPorts Wiki wrote:
> 
>> Page "ProblemHotlist" was changed by raimue
>> Diff URL: 
>> 
>> Revision 138
>> Comment: No longer hot, have not even heard of this problem
>> Changes:
>> ---8<--8<--8<--8<--8<--8<--8<--8<
>> Index: ProblemHotlist
>> =
>> --- ProblemHotlist (version: 137)
>> +++ ProblemHotlist (version: 138)
>> @@ -28,20 +28,6 @@
>>
>> Note you might have an older version of the `xcode-select` tool in case you 
>> installed Xcode 4.2 after installing the Mac OS X 10.7.3 update. If the 
>> command `xcode-select -version` returns `version 2003`, this is from Xcode 
>> 4.2 and we recommend to install the update once again with the 
>> [http://support.apple.com/kb/DL1484 Mac OS X 10.7.3 combo updater]. The 
>> current `xcode-select` version that comes with Mac OS X 10.7.3 identifies 
>> itself as `version 2307`.
>>
>> -== Port installation fails with "no destroot found" == #nodestrootfound
> 
> It happens all the time:
> 
> https://trac.macports.org/query?summary=~no+destroot+found
> 
> As far as I know, we've changed nothing in MacPorts base to fix it.

Okay... I restored this section.

Rainer

Re: port & root

[Rainer Müller]

On 2018-04-05 19:39, Joshua Root wrote:
> On 2018-4-6 03:25 , Rainer Müller wrote:
>> On 2018-04-05 19:14, Joshua Root wrote:
>>> My guess is that this is a bug in Tcl's file command. Deleting a
>>> nonexistent file is not supposed to be considered an error. But by
>>> starting two instances of port(1) at the same time, a race condition
>>> arises. The 'file delete' command probably checks whether the file
>>> exists, returns TCL_OK if not, and then attempts to delete the file. If
>>> the file has been deleted by the other running instance (or indeed
>>> anything else) between these two steps, then the actual deletion fails,
>>> and it incorrectly returns an error.
>>
>> You are right. We hit a race condition in the implementation of 'file
>> delete' in Tcl.
>>
>> It is missing another check for ENOENT here, as done in line 381:
>>
>> https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415
> 
> Still seems broken in 8.6 at first glance too.

I filed a patch with Tcl upstream:
https://core.tcl.tk/tcl/tktview?name=27b6822849

Rainer

Re: [macports-ports] 01/04: aften: An A/52 audio encoder

[Rainer Müller]

On 2018-04-06 14:34, Michael Dickens wrote:
> The issue being that if a port's configure checks for the build type (e.g., 
> {{{
> if(${CMAKE_BUILD_TYPE} STREQUAL "Release") then
> ...
> elseif
> }}}
> and so forth, and if "MacPorts" is not in BUILD_TYPEs list -- no matter 
> whether it has settings available for use by CMake already --, then cmake 
> errors out. See, e.g., < 
> https://github.com/gnuradio/gnuradio/blob/master/cmake/Modules/GrBuildTypes.cmake
>  >. Yes, I know I can always add "MacPorts" to the list and/or tweak the 
> CMAKE_BUILD_TYPE in the portfile to be something acceptable. Again: That 
> takes (a little) work and testing /debugging to make sure it's correct. It 
> also removes some of the point of updating to cmake 1.1 PG: to simplify 
> Portfiles and add MP-specific options for building. Although the cmake 1.0 PG 
> works out of the for my ports (per design), I will update those ports that 
> don't check for the BUILD_TYPE to the 1.1 PG as I find time. I'm not sure 
> what I'll do with my ports that do check the BUILD_TYPE. 

I think the problem was that -DCMAKE_BUILD_TYPE=Release would already
add some compiler flags like "-O3 -DNDEBUG", but we specify our own
flags in CFLAGS and CXXFLAGS and we end up with both on the command line.

Sounds like it should actually be -DCMAKE_BUILD_TYPE=None in the
cmake-1.1 port group and the wanted compiler flags can then be passed
via CFLAGS and CXXFLAGS in the environment (as we already do).

https://cmake.org/Wiki/CMake_Useful_Variables#Compilers_and_Tools

As "None" is generated by default by CMake, it should work with all
projects.

Rainer

CI system for PR builds

[Rainer Müller]

On 2018-04-07 07:40, Mojca Miklavec wrote:
>>> - with a bit of legal questionability
>>
>> From my understanding, it is legal to virtualize macOS, as long as the
>> hypervisor is running on a Mac as well, which is possible.
> 
> I'm aware that it's legal to virtualize macOS on mac hardware, it's
> just not 100% clear to me that all of the steps involved in setting up
> libvirt images are 100% OK. (Can those same images be used on Linux
> unmodified? If so, I would at least imagine that preparing them
> requires the knowledge of a special "key" that's available in the
> above mentioned repo. I don't know though.)

They are not "libvirt images". This refers to using qemu as hypervisor.
On a Mac, you might as well use Parallels or VMware Fusion as hypervisor
and control this hypervisor via libvirt.

The general idea is that if you program against the libvirt API, you can
transparently switch the hypervisor backend and everything keeps
working. Of course here are limitations in practice, as not all
hypervisors offer the same functionality or expose everything with a
scriptable interface. But the basics such as starting and stopping, or
creating new VMs based on a template should work for all of them.

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-07 19:44, Clemens Lang wrote:
> For these resons, we want to reset the machines to a clean state before
> every build, which we could do with buildbot, but requires some python
> magic that hasn't been written yet.

I think this is mostly written. What is needed is only a little bit of
coding and a lot of testing, which is very time consuming.

So here is the full plan in detail:

Buildbot supports libvirt with "latent buildslaves", even in 0.8.12:

http://docs.buildbot.net/0.8.12/manual/cfg-buildslaves.html#libvirt

When a build is about to be executed on this buildslave, Buildbot
instructs clones the base image to a new (temporary) disk image and then
uses this new disk image to spin up a VM from a clean state.

To run this on ESX or any other hypervisor than qemu, we would need to
implement the disk cloning manually as buildbot only uses qemu-img or cp
on the local machine. Probably we would want to use snapshots instead of
cloning full disks. We can customize this in parts by inheriting from
LibVirtSlave.

This VM will then boot and it is supposed to have a buildslave installed
that will connect back to the buildbot master. From then on, the build
runs as usual. After the build finished, buildbot will shutdown the VM
and discard the modified disk image.

The hardest part is to prepare this VM image. At best, it should be
automated so we can easily recreate it for macOS updates.

1) Install macOS
   Not sure how much can be automated here. For a basic version [1], it
   can just be the instructions that need to be clicked through.
2) Install MacPorts into toolsprefix
3) Install buildslave (and other tools) into the toolsprefix
4) Install configuration for buildslave
   Also set buildslave to start automatically after booting and to
   connect to the master.
5) Install MacPorts into /opt/local

As far as I know, Ryan already has a script that could be used for some
of the steps.

When we have such an, we need to configure a builder in master.cfg that
uses LibVirtSlave. This builder needs a modified portbuilder_factory
that skips the deployment of archives. And finally a changesource that
listens to pull requests via a GitHub WebHook (or just some branch for
testing at first).

At this point the setup would already be working.


If the goal is just to get anything working, you could stop reading
here. But I imagine this setup to be slow, as the VM cannot keep any
persistent state. Therefore, I already made up my mind how this could be
optimized.

As the buildbot has no local packages installed, operations like
checking for the existence of a package over HTTP and then fetching the
file will be slow. I expect this to be one major bottleneck. Our usual
portbuilders have all previously built packages available locally already.

One possible optimization would be to mount distfiles and packages in
the VM over (read-only) NFS from another machine that provides the local
mirror.

mount -t nfs -o ro mirror:.../allpackages \
/opt/local/var/macports/incoming/verified

Now a 'port install' would just directly use the package from this
directory, which is much faster than doing the usual archivefetch.
However, this expects a flat hierarchy; a single directory containing
all packages. This may impose limitations we have yet to find.

For distfiles, it would be possibile to apply a trick with a union mount
to create a local overlay that allows writing new files:

mount -t nfs -o ro,union mirror:.../distfiles \
/opt/local/var/macports/distfiles
hdiutil create -o /tmp/distfiles-overlay.dmg -size 20g -fs HFS+J
hdiutil attach -nomount /tmp/distfiles-overlay.dmg  # diskX in output
mount -t hfs -o rw,union /dev/diskXs1 /opt/local/var/macports/distfiles

With this, all file lookups are first attempted in the overlay and if
the file does not exist there, it will be looked up in the layer below,
which is our mirror. New files will always be written to the top layer.
Therefore existing distfiles will be used from the mirror, but new
distfiles can be written to the overlay.


This email got a bit long, but apparently there is an interest in
getting a detailed explanation of how this could be implemented. It is
also good I finally took the time to write these things down, as I have
been thinking about it for a while.

Rainer

[1] For an advanced version of automated macOS installation:
  In the High Sierra installer, the startosinstall script has an
  additional --installpackage option that can point to a .pkg to be
  executed after the installation finished. If, and only if, you also
  disable SIP, you can use the option --volume to install onto a
  different disk than the one in your Mac. Maybe this could be combined
  to fully automate all steps... but that is really advanced.

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-08 12:06, Ryan Schmidt wrote:
>> As the buildbot has no local packages installed, operations like
>> checking for the existence of a package over HTTP and then fetching the
>> file will be slow. I expect this to be one major bottleneck. Our usual
>> portbuilders have all previously built packages available locally already.
>>
>> One possible optimization would be to mount distfiles and packages in
>> the VM over (read-only) NFS from another machine that provides the local
>> mirror.
> 
> I'm not convinced that fetching via HTTP from the private mirror would be 
> significantly slower than fetching via NFS from the private mirror. Either 
> way, the files are coming from the fairly slow RAID card in the master Xserve.

The point is that the approach below would not "fetch" the file. It uses
the package directly from NFS, without making a local copy first.

>> mount -t nfs -o ro mirror:.../allpackages \
>>/opt/local/var/macports/incoming/verified
>>
>> Now a 'port install' would just directly use the package from this
>> directory, which is much faster than doing the usual archivefetch.
>> However, this expects a flat hierarchy; a single directory containing
>> all packages. This may impose limitations we have yet to find.
> 
> As you say, we don't have the packages in a flat hierarchy. How would you 
> propose that we create that? Are we talking about yet another parallel 
> hierarchy of hardlinks for our archive deployment script to maintain, like 
> the one we've been talking about in another thread for rearranging the 
> contents of the packages server?

I am just throwing out ideas. Yes, right now this would require to
create yet another hierarchy. Maybe with changes in base it could reuse
a normal hierarchy.

As I said, that is just for optimization once we have it running.
We can ignore it for now.

Maybe it turns out this is not necessary at all and I am overthinking.
I would be fine with that outcome as well. :-)

> If we are thinking about having PR test machines running OS X 10.8 or 
> earlier, then we have to consider the HTTPS fetch failure situation again. I 
> don't think we want to mirror distfiles for unapproved PRs to the main 
> distfiles server, so we may need a separate PR distfiles server that could 
> house the files temporarily and delete them later.

I would not mirror distfiles for PRs at all.

Testing with just supported versions of macOS should be enough. Anything
on OS X 10.8 and earlier is expected to fail. We need to stop investing
so much effort into legacy systems.

Even just testing with one version would be able to detect the most
common problems, like missing dependencies.

Rainer

Re: [macports-ports] branch master updated: aacgain: Update dirname in post-extract phase

[Rainer Müller]

On 2018-04-08 12:56, Jackson Isaac wrote:
> On Sun, Apr 8, 2018 at 3:33 PM, Jackson Isaac  wrote:
>> I did find one at "https://github.com/mecke/aacgain"; but I was not so
>> sure if they both are by same author since 'alvarez' was mentioned at
>> http://aacgain.altosdesign.com and on github it was 'mecke'. Also the
>> commit log didn't help much either.
>>
>> Looking at the README, it looks like this might possibly be the 1.9 version.
>>
>> I would like to know what to do in such a scenario. Should I go
>> forward with the github version ?

It looks like someone just took some source snapshot (maybe from
upstream CVS), imported it into a Git repository, and pushed it to
GitHub. There is no history attached to it, so it could also contain
other modifications.

Definitely not a maintained version, so it should not be used.

> Github version is just code files and does not contain files for Unix
> build which was part of the tarball created
> by Alvarez.
> 
> Digging into the issue I came across a very old ticket for aacgain
> reported by Ryan at https://trac.macports.org/ticket/25942.
> 
> Looks like someone had created the tarball for 1.9 and hosted it one
> their own website since 'Alvarez' didn't create the
> official 1.9 tarball yet. But the port was never updated.

Option a: Leave it at version 1.8

Option b: Update to a "CVS" snapshot

The repository was converted to git, the tag would be R1_9:
https://sourceforge.net/p/mp3gain/code/ci/R1_9/tree/

Rainer

Re: Online MacPorts Meeting on 21st April at 13:00 UTC - suggestion

[Rainer Müller]

On 2018-04-07 14:18, William H. Magill wrote:
> IRC is still quite popular in Europe, but is little used in the US.
IRC on our #macports channel is fine to ask questions for which you do
not want to write a lengthy email or to debug problems with quick
back-and-forth on what to check or try.

However, meetings on IRC are slow, as you have to wait for others to
type, you have no notifications whether others still have something to
say, etc.

For such a scheduled meeting, I would prefer audio chat as that would
make the communication more direct and faster.

> One alternative to IRC is “Discord” — it has the same “type text” 
> capabilities as IRC, but as far as I’m concerned, it is MUCH easier to use.
> It also supports voice channels.
> 
> And, yes, there is a Mac client - OSX and IOS - as well as access through the 
> browser.
> 
> https://discordapp.com
> 
> It is pitched as “for gamers,” mainly because of its voice capabilities.

Discord looks fine to me. Apparently we would not even be the first open
source project to use it [1]. Apparently you can use it directly in a
browser without registration ("claiming the account").

The other well-known option would of course be Skype. However, with
Skype one person would have to collect interested developers first and
then invite them to a group call. With Discord it seems like you can
just create a room that everyone can join, which would be a lot simpler.

There is also Jitsi [2], an open source solution for video conferencing
via WebRTC in the browser. However, browser support seems to be limited,
especially with Safari.

Rainer

[1] https://discordapp.com/open-source
[2] https://jitsi.org/

Re: [macports-base] branch master updated: Honour startupitem.install at activation time

[Rainer Müller]

On 2018-04-08 03:26, Joshua Root wrote:
> Joshua Root (jmroot) pushed a commit to branch master
> in repository macports-base.
>
> https://github.com/macports/macports-base/commit/3dde77d7d6c66dbb8fcc1d4b1b4972a2ac8b1947
>
> The following commit(s) were added to refs/heads/master by this push:
> new 3dde77d Honour startupitem.install at activation time 3dde77d is 
> described below
>
> commit 3dde77d7d6c66dbb8fcc1d4b1b4972a2ac8b1947 Author: Joshua Root 
> 
> AuthorDate: Sun Apr 8 11:16:22 2018 +1000
>
> Honour startupitem.install at activation time When activating files,
> move plists from /Library/Launch* to ${prefix}/etc/Launch* if the port
> was built with startupitem_install on but it is currently off, and
> vice versa in the opposite case.
Somehow this seems complicated. Wouldn't it be simpler to always ship
the .plists in ${prefix}/etc/Launch* and only add symlinks in
/Library/Launch* on activation if startupitem_install allows it?

But I guess you did it this way to be compatible with older archives?

Rainer

Re: [macports-base] branch master updated: Honour startupitem.install at activation time

[Rainer Müller]

On 2018-04-09 00:06, Joshua Root wrote:
> On 2018-4-9 07:55 , Rainer Müller wrote:
>> On 2018-04-08 03:26, Joshua Root wrote:
>>> Joshua Root (jmroot) pushed a commit to branch master
>>> in repository macports-base.
>>>
>>> https://github.com/macports/macports-base/commit/3dde77d7d6c66dbb8fcc1d4b1b4972a2ac8b1947
>>>
>>> The following commit(s) were added to refs/heads/master by this push:
>>> new 3dde77d Honour startupitem.install at activation time 3dde77d is 
>>> described below
>>>
>>> commit 3dde77d7d6c66dbb8fcc1d4b1b4972a2ac8b1947 Author: Joshua Root 
>>> 
>>> AuthorDate: Sun Apr 8 11:16:22 2018 +1000
>>>
>>> Honour startupitem.install at activation time When activating files,
>>> move plists from /Library/Launch* to ${prefix}/etc/Launch* if the port
>>> was built with startupitem_install on but it is currently off, and
>>> vice versa in the opposite case.
>> Somehow this seems complicated. Wouldn't it be simpler to always ship
>> the .plists in ${prefix}/etc/Launch* and only add symlinks in
>> /Library/Launch* on activation if startupitem_install allows it?
> 
> Not that much simpler, strangely enough.

Maybe not in the implementation, but simpler for me to grasp what's
going on. :-)

>> But I guess you did it this way to be compatible with older archives?
> 
> Indeed. It doesn't matter how startupitem.install was set when the
> archives were built, this will DTRT either way.

This handles ports that create .plist files via startupitem.* or the new
startupitems. However, there are also ports that install custom .plist
file to ${destroot}/Library/Launch*. Some ports even do that
unconditionally.

Maybe we could rename all *.plist files in these directories to avoid
recreating this logic in many ports? If we handled this automatically at
activation time, ports would no longer have to deal with
${startupitem.install} at all.

Rainer

Re: [macports-base] branch master updated: Honour startupitem.install at activation time

[Rainer Müller]

On 2018-04-09 01:20, Joshua Root wrote:
> On 2018-4-9 08:43 , Rainer Müller wrote:
>> This handles ports that create .plist files via startupitem.* or the new
>> startupitems. However, there are also ports that install custom .plist
>> file to ${destroot}/Library/Launch*. Some ports even do that
>> unconditionally.
> 
> Ports that do this should set 'startupitem.create no' and
> 'startupitem.type launchd', plus whatever other variables like
> startupitem.name are needed based on what they called their plist.

I found no port in or ports tree that does it that way.

There are only exactly two ports that use 'startupitem.create no':
mpstats and certsync.

But these do not set startupitem.type either. And even they create the
symlink in /Library/Launch* manually...

Other ports just install a .plist to
${prefix}/etc/Launch*/org.macports.${name}/*.plist and creates the
symlink at /Library/Launch*/*.plist without setting any startupitem.*
option. For example, mysql*-server (and the percona and mariadb forks).

As far as I can see, 'startupitem.create no' is not required right now
and not used. I do not think Portfile authors will understand why it
would be necessary. We should find a solution without it.

> I've been wondering if we should also have an option called something
> like startupitem.file that could be set to a path, e.g.
> ${filespath}/my_job.plist, and then base would install it to the right
> place.

This could work, except for ports where the upstream build system
already installs the .plist. For example, xinit.

https://github.com/macports/macports-ports/blob/master/x11/xinit/Portfile#L53

Rainer

Releasing 2.4.3

[Rainer Müller]

Hello,

according to our milestone on Trac, there is nothing left to be fixed or
backported to the release-2.4 branch for MacPorts 2.4.3.

The ChangeLog was already updated and reflects the changes this release
will contain:
https://github.com/macports/macports-base/blob/release-2.4/ChangeLog

See also the full details in the commit log and corresponding tickets:

https://github.com/macports/macports-base/compare/v2.4.2...release-2.4
https://trac.macports.org/milestone/MacPorts%202.4.3

If nobody objects, I am going to tag a new release on Tuesday and
prepare tarballs.

Should I wait with publishing the update over selfupdate until we also
have installer packages ready?

Rainer

Re: Releasing 2.4.3

[Rainer Müller]

On 2018-04-10 19:07, Ryan Schmidt wrote:
> 
> On Apr 9, 2018, at 17:15, Rainer Müller wrote:
> 
>> If nobody objects, I am going to tag a new release on Tuesday and
>> prepare tarballs.
> 
> Please merge my other two commits from this ticket:
> 
> https://trac.macports.org/ticket/55492
> 
> My first commit, which you merged, was buggy and needed fixing.

Thanks for noticing. I cherry-picked the other two commits now.

https://github.com/macports/macports-base/commit/47d219119281a5f574ee6298c9e731293bca464e

https://github.com/macports/macports-base/commit/a98fe3b89fba99f8bb873663f85a04831718583b

>> Should I wait with publishing the update over selfupdate until we also
>> have installer packages ready?
> 
> Sure?

New releases could be made available via selfupdate immediately. Old
installers will already always selfupdate to the latest version
automatically, so the the result is the same anyway when using on older
installer.

Building package installers is the most time-consuming operation in our
release process. Therefore I was asking whether it is actually worth to
wait for them to appear before releasing to selfupdate.

However, for 2.4.3, Joshua will tag the release instead, as he is also
going to build installers right away.

Rainer

Re: Releasing 2.4.3

[Rainer Müller]

On 2018-04-10 19:39, Ryan Schmidt wrote:
> 
> On Apr 10, 2018, at 12:07, Ryan Schmidt wrote:
> 
>> On Apr 9, 2018, at 17:15, Rainer Müller wrote:
>>
>>> If nobody objects, I am going to tag a new release on Tuesday and
>>> prepare tarballs.
>>
>> Please merge my other two commits from this ticket:
>>
>> https://trac.macports.org/ticket/55492
>>
>> My first commit, which you merged, was buggy and needed fixing.
> 
> Do you want to merge this performance enhancement too:

At some point we should make a cut and not add many changes in the last
minute...

I do not want to delay 2.4.3 any longer to finally release the fix for
the broken trace mode on High Sierra.

I would save reviewing and backporting of these fixes for a 2.4.4 soon
after.

Rainer

Re: Releasing 2.4.3

[Rainer Müller]

On 2018-04-10 20:26, Ryan Schmidt wrote:
> 
> On Apr 10, 2018, at 12:54, Rainer Müller wrote:
>>
>> At some point we should make a cut and not add many changes in the last
>> minute...
> 
> Well I can't force you to merge them.

I cannot stop you either... I am just afraid that by merging changes in
the last minute we are likely to introduce problems we did not think of
before. Not all changes in backported commits can be applied cleanly as
master and release-2.4 diverged, so they might need to be modified.

The decision whether to merge them or not would be by Joshua now, as he
is going to tag the release.

>> I do not want to delay 2.4.3 any longer to finally release the fix for
>> the broken trace mode on High Sierra.
> 
> I don't want to release 2.4.3 with the libstdc++ ABI issue unanswered. I 
> believe it will cause problems.

Then we should revert this change for 2.4.3 and we have one more reason
to go for a 2.4.4 release.

Rainer

Re: Releasing 2.4.3

[Rainer Müller]

On 2018-04-10 21:06, Ryan Schmidt wrote:
> I'm not going to merge them, because I don't know how, and don't have the 
> energy to attempt to learn how to do so, given the zillion other things I'm 
> trying to do right now.

This is even documented:
https://trac.macports.org/wiki/WorkingWithGit#cherrypick

> I proposed a list of commits to backport for 2.4.3 back in December. You've 
> merged those now; thank you, but it's now 4 months later. I don't want 
> another 4 months to go by before we bring the other critical fixes we've 
> already committed to master to our users. The use of MacPorts git and svn on 
> older systems spring to mind as pretty important fixes for older systems.

I do not want to wait months for a release either. I think that the long
time between releases is mainly caused by our release process.

I cannot sign package installers. As long as we insist on having them
for each and every release, I am not in the position to cut a full
release anyway.

So, what are we going to do? I already proposed not to wait for .pkg
installers and just release to selfupdate whenever the release is ready.

Rainer

Re: Releasing 2.4.3

[Rainer Müller]

On 2018-04-10 19:39, Ryan Schmidt wrote:
> Do you want to merge this performance enhancement too:

[...]

> And these bugfixes:

I assigned this to the MacPorts 2.4.4 milestone, so we do not forget
about them:

https://trac.macports.org/ticket/56267

Rainer

Re: port version reports 2.4.2 when built from tag v2.4.3

[Rainer Müller]

On 2018-04-11 10:52, Enrico Maria Crisostomo wrote:
> I've just created a new installation of macports-base from tag v2.4.3 
> following the instructions in the documentation (basically git checkout 
> v2.4.3, ./configure ..., make and make install) and I've just noticed that 
> `port` reports 2.4.2:

Hm, I cannot reproduce this problem. The configure script is supposed to
update the relevant file with the version number.

Check "macports_version" in src/macports1.0/macports_autoconf.tcl in
your working tree.

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-11 14:11, db wrote:
> I won't address every single point and just say that it might be interpreted 
> as finger-pointing, but I'm actually curious about what's the state of the 
> project, how it got where it is, where is it going, should I build always 
> from source, should I use another package manager or complement it with 
> another one, etc. And all these I seem to learn only in the mailing list, 
> which might be annoying, I admit.

I do not understand what you mean by "state of the project". These are
questions that most users just do not seem to consider at all.

As said before, if you think information that is only on the mailing
list should be elsewhere, please make it available. Publish it in
whatever form you think would be appropriate.

>>> That streamlined process is what keeps new and updated portfiles in my 
>>> local repo…
>> I have no clue what you wanted to say with this.
> 
> It means that ports I submitted like stem and ipfs are not further reviewed, 
> so new portfiles I write I just keep in my local repo and don't bother 
> submitting.

There are more than 400 pending submissions for new ports on Trac, all
being in various states. If your submission does not make progress,
please ask on macports-dev@ for a review with links to the corresponding
tickets.

You might also want to consider submitting them as pull requests,
because Perry Metzger is going to make sure no pull request will be
forgotten. :-)

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-11 16:43, Joshua Root wrote:
> On 2018-4-12 00:37 , Mojca Miklavec wrote:
>> We could move them to something like "changesneeded" (not sure where
>> exactly; they could get a special status, even if closed, but it
>> should be easy enough to find them should anyone have motivation to
>> fix the remaining issues). Just because none of us took the time to
>> review the changes for long enough that all dependency and the
>> software itself became completely outdated in the meantime, doesn't
>> mean that we should make the submissions non-discoverable and give the
>> signal to users that first of all we don't care for 5 years, and when
>> we start caring, we boldly close all the tickets.
>>
>> Very often I see tickets getting status "upstream", "infoneeded" etc.
>> which basically means that developers would ignore such tickets until
>> something happens. Or perhaps "helpneeded" which means that they are
>> genuinely interested in fixing the issue, but have no resources to fix
>> it. We need something similar.
> 
> New ticket resolutions are something we can do.

Those are applied when closing the ticket, which kind of buries the
ticket as it will no longer be returned in queries. That does not seem
appropriate when the underlying issue was in fact not yet resolved,
because you would then miss this ticket when searching for an issue.

However, we could have a new resolution to indicate that it was closed
because there was no response from the submitter, because none of the
existing options seems to fit. Would a resolution of "timeout" likely be
confused with the "maintainer timeout"?

By the way, the Trac ticket workflow can be customized a lot. At the
moment, we are using the default Trac ticket workflow [1]. For example,
it would be possible to have a state like "changesneeded" and the
reporter could be allowed to change the state back to "new".

Rainer

[1] https://trac.edgewall.org/wiki/TracWorkflow#TheDefaultTicketWorkflow

Re: Builds with "fakeroot" (was Re: CI system for PR builds)

[Rainer Müller]

On 2018-04-11 18:55, Perry E. Metzger wrote:
> Just a side note: other package building systems have dealt (in
> various ways) with being able to build things without privileges.
> 
> For example, the Debian project has a cool tool called "fakeroot"
> which uses an LD_PRELOADed library to make things being built _think_
> they have root privileges when they don't -- actions like chmods and
> chowns are recorded by the libraries and subsequent "fakerooted" calls
> that read permissions or ownership or what have you replay what was
> set earlier.
> 
> This allows the construction of installation packages that have
> arbitrary users and permissions set in them without root having been
> involved in builds.
> 
> See: https://manpages.debian.org/stretch/fakeroot-ng/fakeroot.1.en.html
> 
> I believe that "fakeroot" (or at least forks of it) have already been
> ported to MacOS for use by other projects
Note that fakeroot itself is licensed as GPL-3+ and therefore not
suitable for inclusion into MacPorts base.

We already have the functionality to hook library functions which is
used for tracemode. This could as well be used for a "fakeroot"
functionality. Especially the destroot target would benefit from this.

For destroot, this could hook file operations (chown, chmod, etc.) and
divert that to a database. This can then be used to put the correct
information into the archive that is created from the destroot directory.

Note that this should not mean you could use MacPorts without sudo, but
the isolation between Portfile execution and your user account would be
stronger.

As a side note, at the MacPorts Meeting last month we had a session on
documentation were we wrote down why MacPorts needs root privileges and
what they are used for. This information still has to find a place in
the guide or FAQ.

https://trac.macports.org/wiki/Meetings/MacPortsMeeting2018/Documentation#PrivilegeSeparationakaMacPortsrequiressudoroot

Rainer

Re: Dealing with False Positive during Livecheck

[Rainer Müller]

On 2018-04-11 21:18, Jackson Isaac wrote:
> One possible way as suggested on IRC is to add a comment in the Portfile
> as to why it cannot and/or won't be updated further (in case of
> inactive upstream)
> and removing the livecheck regex.

Be careful that removing the options does not mean disabling. For
aacgain it worked, because the default livecheck will now match 1.8
only. If you really want to disable livecheck, set livecheck.type.

  livecheck.type none

Rainer

Re: CI system for PR builds

[Rainer Müller]

On 2018-04-12 20:14, db wrote:
> On 12 Apr 2018, at 18:52, Ryan Schmidt  wrote:
>> 1. MacPorts does not have a method of declaring that a port does not build 
>> on a version of macOS. Such a feature is being discussed:
>> https://trac.macports.org/ticket/15712
>> In the absence of this feature, we write pre-fetch blocks that manually 
>> check the OS version and print an error.
> 
> Opened 10 years ago. Fat chance, I guess.

Do I have to explain again that we run on volunteers only?
Apparently, nobody volunteered to implement this.

Priority and focus shifts with what people are willing to work on.
This was not that important before. It is now becoming important as we
want to have it on the buildbot.

>> 3. There is no plan to have any automated process add such blocks.
> 
> Precisely that I wanted to know. I'd like the powers that be to consider this 
> — delivering portfiles that won't build (in certain platforms) doesn't make 
> much sense.

Please think about the implications first. When a build fails on all
versions, do you automatically want to list them all as unsupported?

It is up to the maintainer (or any contributor) to add the appropriate
list of supported platforms after determining the cause of build
failures, so we will not run the check again next time.

Rainer

Re: port version reports 2.4.2 when built from tag v2.4.3

[Rainer Müller]

On 2018-04-13 00:42, Jack Howarth wrote:
> This issue might also be impacting the High Sierra macports 2.4.3
> installer image as well. After installing it on a machine with no
> /opt/local, I get the following from 'sudo port self update'...
> 
> --->  Updating MacPorts base sources using rsync
> 
> MacPorts base version 2.4.3 installed,
> 
> MacPorts base version 2.4.2 downloaded.

It is actually an unrelated problem. The new version is not yet
available over selfupdate. The script that is supposed to update the
rsync server is failing.

Check this and following messages in the thread on macports-users:
https://lists.macports.org/pipermail/macports-users/2018-April/044969.html

Rainer

Re: New guide section on Git and GitHub

[Rainer Müller]

On 2018-04-13 15:31, Jonathan Stickel wrote:
> As a long time contributor of patches using the ticket interface (Trac),
> I think it would be a mistake to discourage submission of patches there.
> While it is fine to encourage use of git pull requests, the barrier of
> entry is much higher. In fact, I just made my first pull request this
> week, and, after a nontrivial amount of effort, I still managed to mess
> it up. Git is quite powerful for sure, but it definitely has a learning
> curve. Pull requests makes use of multiple layers of abstract concepts
> (forks and branches) and multiple steps.
> 
> I think it would be reasonable to expect casual users to still upload
> patches to trac, which then port maintainers ought to turn into PRs. My
> 2 cents.

Maybe we could revisit the idea that once existed way back with the
MacPorts Web Application, but was never realized: a 'port submit' action.

I envision it to work like this:

$ port submit 
or:
$ port submit 

The second form would only be needed when also updating patches.
This would use credentials stored in Keychain (a GitHub API auth token),
which would be asked for when used for the first time, guiding the user
through creation of such a API token that has the necessary scope. Of
course there should also be commands to delete them again or use
different credentials.

1. Run 'port lint' on the given port, stop on errors
   On warnings, ask for confirmation or require a --force option
   to continue
2. Get path from the Portfile as / (PORTPATH)
1. Create a new unique branch name on "/macports-ports" (BRANCH)
2. Update BRANCH to the latest master of macports/macports-ports
3. Commit new Portfile to PORTPATH in BRANCH
   Ask user for a commit message, if implemented with a temporary
   shallow clone, can just launch 'git commit --verbose'.
4. Open URL of new commit in browser, ask user to review and confirm
5.1 If confirmed, open a pull request against macports/macports-ports
5.2 If not confirmed, delete BRANCH

Rainer

Re: port version reports 2.4.2 when built from tag v2.4.3

[Rainer Müller]

On 2018-04-13 23:55, Joshua Root wrote:
> On 2018-4-14 07:30 , Enrico Maria Crisostomo wrote:
>> Hence, the macports installation that was built from the dirty tree is 
>> referring to the installation directory of the previous build.  That also 
>> explains why removing that directory makes port crash, as I said in a 
>> previous mail.
> 
> OK, so the issue is not with switching tags without cleaning at all, but
> configuring for a different prefix without cleaning. The bug is in one
> of Tcl's Makefiles, which compiles tclPkgConfig.c with several -D flags
> using values that are directly substituted into the Makefile by
> autoconf. There should probably be a dependency declared on the Makefile
> itself.

FWIW, I am usually work with multiple worktrees instead of switching
branches in the same directory. That way I can have directories for both
master and release-2.4 side-by-side.

$ pwd
/Users/raimue/src/macports/base
$ git worktree add ../base-2.4 release-2.4
$ cd ../base-2.4
$ cat config/macports_version
2.4.3

See the git manual for the details:
https://git-scm.com/docs/git-worktree

The copy in base-2.4 is also configured to use a different prefix. As
the files of the build system can stay in the filesystem, I do not have
to re-configure all the time. When I want to work on release-2.4, I only
have to change directory in my shell.

Rainer

Re: [macports-base] 03/03: fetch: Do not redirect stderr

[Rainer Müller]

On 2018-04-16 01:31, Ryan Schmidt wrote:
> 
> On Apr 15, 2018, at 11:34, Rainer Müller wrote:
> 
>> Rainer Müller (raimue) pushed a commit to branch vcs-fetch
>> in repository macports-base.
>>
>>
>> https://github.com/macports/macports-base/commit/600c6b2c7e03d208eefda5ae25d1f1d0ac58933e
>>
>> commit 600c6b2c7e03d208eefda5ae25d1f1d0ac58933e
>>
>> Author: Rainer Müller
>> AuthorDate: Sun Apr 15 18:34:48 2018 +0200
>>
>> fetch: Do not redirect stderr
>> 
>> There is no need to redirect stderr to stdout with 2>&1, because the
>> system command will already do this by default for us.
> 
> Why does the system command do this?

We want to record the output of commands in our log files, both stdout
and stderr. Therefore these channels will already be merged when we
redirect the output of the child process in the system command.

The set up for this is here:
https://github.com/macports/macports-base/blob/42e7ce792faaebb5e3aec958d2936c5fb8f02216/src/pextlib1.0/system.c#L239-L247

Now that I added the -o/-e options to system to capture stdout and
stderr in separate variables, this redirect would defeat this
functionality. When the shell already merges the channels with 2>&1, we
would not be able to tell them apart anymore as all output would only
arrive on stdout.

Rainer

Re: [macports-base] 01/06: fetch: Always use ${prefix}/bin/git for gitfetch

[Rainer Müller]

On 2018-04-16 22:29, Ryan Schmidt wrote:
> 
> On Apr 14, 2018, at 22:09, Rainer Müller wrote:
> 
>> Rainer Müller (raimue) pushed a commit to branch vcs-fetch
>> in repository macports-base.
>>
>>
>> https://github.com/macports/macports-base/commit/d28ec4564f13b59afd439ee3fbf05f9c43a76bb9
>>
>> commit d28ec4564f13b59afd439ee3fbf05f9c43a76bb9
>>
>> Author: Rainer Müller
>> AuthorDate: Thu Apr 12 16:12:06 2018 +0200
>>
>> fetch: Always use ${prefix}/bin/git for gitfetch
>> 
>> See also 784fee88.
>>
>> ---
>>  configure | 41 -
>>  configure.ac  |  1 -
>>  src/port1.0/portfetch.tcl | 14 ++
>>  3 files changed, 2 insertions(+), 54 deletions(-)
> 
> Again: Why?
> 
> https://lists.macports.org/pipermail/macports-dev/2018-March/037750.html

The previous code always used /usr/bin/git, even if the git port was
installed. Only old versions of macOS got to use the latest version of
git, while recent versions of macOS were stuck with /usr/bin/git.

That does not seem to be a useful default to me, therefore it made sense
to me to always use the latest version from ${prefix}/bin/git.

Is always adding the git port as a dependency too heavy?

One of the remaining points on my to-do list for this branch would be to
figure out a way to get rid of the dependency on the VCS tools if the
fetch is tarballable, as the distfile would be available from our mirrors.

Rainer

Re: Online MacPorts Meeting on 21st April at 13:00 UTC - suggestion

[Rainer Müller]

On 2018-04-13 13:34, Mojca Miklavec wrote:
> If we find three volunteers (I volunteer, so only two more needed), we
> could perhaps do a technology test run tomorrow at 13:00 UTC (that's
> precisely 7 days earlier)
> ... (or at any other potentially more suitable hour depending on who
> wants to participate).
> 
> We could then test:
> - Hangouts
> - Discord
> - (potentially Skype, but it needs personal invitations etc.)
> - any other video conferencing tool, in case there are better suggestions
> and see which one fits best, but only if we get some proposal early enough.
> 
> We could have IRC as a backup for coordination in case of technical
> problems etc.

As nobody replied, I assume the test run did not happen...
Unfortunately, I could not make it either.

In any case, we are approaching the deadline to suggest topics.
I created a wiki page to track the agenda:

https://trac.macports.org/wiki/Meetings/2018-04-21

Rainer

Re: [macports-base] 01/06: fetch: Always use ${prefix}/bin/git for gitfetch

[Rainer Müller]

On 2018-04-18 05:37, Ryan Schmidt wrote:
> 
> On Apr 17, 2018, at 07:33, Rainer Müller wrote:
> 
>> On 2018-04-16 22:29, Ryan Schmidt wrote:
>>>
>>> On Apr 14, 2018, at 22:09, Rainer Müller wrote:
>>>
>>>> Rainer Müller (raimue) pushed a commit to branch vcs-fetch
>>>> in repository macports-base.
>>>>
>>>>
>>>> https://github.com/macports/macports-base/commit/d28ec4564f13b59afd439ee3fbf05f9c43a76bb9
>>>>
>>>> commit d28ec4564f13b59afd439ee3fbf05f9c43a76bb9
>>>>
>>>> Author: Rainer Müller
>>>> AuthorDate: Thu Apr 12 16:12:06 2018 +0200
>>>>
>>>>fetch: Always use ${prefix}/bin/git for gitfetch
>>>>
>>>>See also 784fee88.
>>>>
>>>> ---
>>>> configure | 41 -
>>>> configure.ac  |  1 -
>>>> src/port1.0/portfetch.tcl | 14 ++
>>>> 3 files changed, 2 insertions(+), 54 deletions(-)
>>>
>>> Again: Why?
>>>
>>> https://lists.macports.org/pipermail/macports-dev/2018-March/037750.html
>>
>> The previous code always used /usr/bin/git, even if the git port was
>> installed. Only old versions of macOS got to use the latest version of
>> git, while recent versions of macOS were stuck with /usr/bin/git.
>>
>> That does not seem to be a useful default to me, therefore it made sense
>> to me to always use the latest version from ${prefix}/bin/git.
> 
> But if /usr/bin/git works... why not use that and save the user the time and 
> disk space? I know we go back and forth on this, whether we should use OS 
> ports when available or always use ports for consistency, but this is one of 
> the situations where thus far we've used the OS binary if it works.
> 
> 
>> Is always adding the git port as a dependency too heavy?
> 
> It does take a long time to build from source. Building git, in order to 
> check out the infrastructure ports, is, for example, one of the most 
> time-consuming parts of setting up a new buildbot worker.

Okay, from the feedback I get we want to keep the old behavior.
I reverted the changes to only use /usr/bin/git.

https://github.com/macports/macports-base/commit/eef07c3ce77b31c55af69107d1e591571ae7b088

>> One of the remaining points on my to-do list for this branch would be to
>> figure out a way to get rid of the dependency on the VCS tools if the
>> fetch is tarballable, as the distfile would be available from our mirrors.
> 
> Absolutely. Not sure how you'll do it!

I think of adding another port target for VCS fetch, so we can assign a
different list of dependencies to it. That would only be used internally.

Rainer

Re: Buiid/install stages documentation?

[Rainer Müller]

On 2018-04-18 15:20, Perry E. Metzger wrote:
> On Tue, 17 Apr 2018 22:33:10 -0500 Ryan Schmidt
>  wrote:
>> On Apr 17, 2018, at 12:39, Perry E. Metzger wrote:
>>
>>> Just to be clear: I know what is in:
>>> https://guide.macports.org/#reference.phases
>>>
>>> but I'm trying to figure out a bit more detail. In particular, I
>>> _suspect_ what I want is to trick opam into installing its build
>>> products into the destdir, but I'm not sure that's the right
>>> approach.  
>>
>> Certainly, at minimum, that would be needed. 
>>
>> What else do you want to know?
>>
> 
> Maybe I should ask this a bit differently. Are there other precedents
> for using a "foreign" packaging or build system and tricking it into
> building stuff for macports that I could study and possibly steal
> pieces of?

Not that I am aware of. Most package managers would also install all the
dependencies, something we do not want in a port.

The usual approach is to generate a Portfile from the definition of the
"foreign" package manager and also create ports for all dependencies.

For example, cpan2port or pypi2port in macports-contrib [1,2] do that
for Perl or Python, respectively. They are far from perfect, but are a
great help to get an initial Portfile.

Rainer

[1] https://github.com/macports/macports-contrib/tree/master/cpan2port
[2] https://github.com/macports/macports-contrib/tree/master/pypi2port

Re: Online MacPorts Meeting on 21st April at 13:00 UTC - suggestion

[Rainer Müller]

On 2018-04-18 01:11, Rainer Müller wrote:
> On 2018-04-13 13:34, Mojca Miklavec wrote:
>> If we find three volunteers (I volunteer, so only two more needed), we
>> could perhaps do a technology test run tomorrow at 13:00 UTC (that's
>> precisely 7 days earlier)
>> ... (or at any other potentially more suitable hour depending on who
>> wants to participate).
>>
>> We could then test:
>> - Hangouts
>> - Discord
>> - (potentially Skype, but it needs personal invitations etc.)
>> - any other video conferencing tool, in case there are better suggestions
>> and see which one fits best, but only if we get some proposal early enough.
>>
>> We could have IRC as a backup for coordination in case of technical
>> problems etc.
> 
> As nobody replied, I assume the test run did not happen...
> Unfortunately, I could not make it either.

We should figure out the technical side, so we can focus on the
discussion on Saturday.

Let's give it another try today at 17:00 UTC, meet on #macports IRC.
Who else would be willing to join the test run?

https://www.timeanddate.com/worldclock/fixedtime.html?msg=MacPorts+Online+Meeting+Test+Run&iso=20180419T17

I know this time of day is not ideal for anyone not in Europe, but the
time is scarce until the meeting on Saturday.

Rainer

Re: Can't use Expect in a portfile

[Rainer Müller]

On 2018-04-19 13:31, Ryan Schmidt wrote:
> I can't seem to use Expect in a portfile. Using the following minimal 
> portfile:
> 
> 
> PortSystem 1.0
> name foo
> version 1
> fetch {
> system "expect -c 'spawn echo hello; interact;'"
> }
> 
> 
> And running:
> 
> 
> sudo port fetch
> 
> 
> I get this unexpected output:
> 
> 
> The system has no more ptys.  Ask your system administrator to create more.
> 
> 
> The system, of course, has ptys available, and running the expect command 
> outside of MacPorts works. Expect seems to be experiencing some problem, and 
> erroneously claims that the problem is no more ptys. How can I get Expect to 
> tell me what the real problem is? I've tried using Expect's "-d" flag to 
> print more information, but it doesn't illuminate the situation for me.
> 
> I used to use Expect successfully in a portfile, but it was years ago and 
> much has changed in MacPorts base since then.

This is probably due to sandboxing. I guess Expect wants to open some
device such as /dev/ptmx read/write, but is not allowed to do so.

Rainer

Re: Can't use Expect in a portfile

[Rainer Müller]

On 2018-04-19 13:58, Ryan Schmidt wrote:
> 
> On Apr 19, 2018, at 06:49, Rainer Müller wrote:
> 
>> On 2018-04-19 13:31, Ryan Schmidt wrote:
>>> I can't seem to use Expect in a portfile. [...]
>>>
>>> The system has no more ptys.  Ask your system administrator to create more.
>>
>> This is probably due to sandboxing. I guess Expect wants to open some
>> device such as /dev/ptmx read/write, but is not allowed to do so.
> 
> Maybe. The error message is printed when exp_getptymaster() (in pty_unicos.c) 
> returns -1. It's hard to follow exactly what that function does. It has a log 
> of diagnostic log lines (expDiagLog(...)) but I can't figure out how to get 
> Expect to print those diagnostics.
> 
> I could try adding some paths to the sandbox whitelist.

You can check in Console.app for sandbox violations.

Or run this command in another terminal window parallel to the port process:

$ log stream --predicate '(process == "sandboxd")' --level error

Rainer

Re: Allow access to a file in trace mode

[Rainer Müller]

On 2018-04-19 12:44, Ryan Schmidt wrote:
> If I try to build minivmac in trace mode (sudo port -t install minivmac) it 
> fails because:
> 
> Warning: The following existing files were hidden from the build system by 
> trace mode:
>   /Applications/MacPorts/Mini vMac/vMac.ROM
> 
> The minivmac build requires access to this file. The file is not provided by 
> any port; it must be provided by the user. How can I allow access to this 
> file in trace mode in this port?

There is no way to extend the trace mode bounds from a Portfile.

Files inside ${prefix} not provided by any port may be accessed, but not
outside ${prefix}. This is a workaround to allow access to cache files
such as the XML catalog or the GTK icon cache.

If these ROM files for the build are not distributable, how can the
result of this port be distributable under a GPL-2 license? That would
imply that the ROM does not end up in the build results at all.

Rainer

Re: Online MacPorts Meeting on 21st April at 13:00 UTC - suggestion

[Rainer Müller]

On 2018-04-19 11:24, Rainer Müller wrote:
> On 2018-04-18 01:11, Rainer Müller wrote:
>> On 2018-04-13 13:34, Mojca Miklavec wrote:
>>> If we find three volunteers (I volunteer, so only two more needed), we
>>> could perhaps do a technology test run tomorrow at 13:00 UTC (that's
>>> precisely 7 days earlier)
>>> ... (or at any other potentially more suitable hour depending on who
>>> wants to participate).
>>>
>>> We could then test:
>>> - Hangouts
>>> - Discord
>>> - (potentially Skype, but it needs personal invitations etc.)
>>> - any other video conferencing tool, in case there are better suggestions
>>> and see which one fits best, but only if we get some proposal early enough.
>>>
>>> We could have IRC as a backup for coordination in case of technical
>>> problems etc.
>>
>> As nobody replied, I assume the test run did not happen...
>> Unfortunately, I could not make it either.
> 
> We should figure out the technical side, so we can focus on the
> discussion on Saturday.
> 
> Let's give it another try today at 17:00 UTC, meet on #macports IRC.
> Who else would be willing to join the test run?
> 
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=MacPorts+Online+Meeting+Test+Run&iso=20180419T17
> 
> I know this time of day is not ideal for anyone not in Europe, but the
> time is scarce until the meeting on Saturday.

Thanks everyone who joined the test run!
We tested today:

* Google Hangouts (Video)
  + worked out of the box for everyone
  - requires Google account
  - has a limit of maximum 10 people

* Discord (Audio)
  + does not require an account
  + runs in the browser
  - some people could only hear a subset of participants

* Zoom (Video)
  - forces you to install a .pkg, which already installs an app to
/Applications in the preflight script; we aborted the test

* Mumble (Audio)
   - requires a native app
   - very difficult to set up with certificate-based auth
   - default settings for voice activation did not work

* talky.io (Video)
   + does not require an account
   + runs in the browser
   - some people could only hear a subset of participants


As it was the only option that worked out of the box for everyone,
we decided to use Google Hangouts on Saturday.

We assume (and hope) that we will not be more than 10 people who want to
participate in the meeting. The link to join the call will be posted to
the corresponding wiki page for the meeting.

Rainer

Releasing 2.4.4

[Rainer Müller]

Hello,

as discussed in today's online meeting [1], there is nothing left to be
fixed or backported to the release-2.4 branch for MacPorts 2.4.4.

The ChangeLog shows the changes this release will contain:
https://github.com/macports/macports-base/blob/release-2.4/ChangeLog

See also the full details in the commit log and corresponding tickets:

https://github.com/macports/macports-base/compare/v2.4.3...release-2.4
https://trac.macports.org/milestone/MacPorts%202.4.4

Josh,
would you be ready to tag this release and create new pkg installers?

Rainer

[1]
https://trac.macports.org/wiki/Meetings/2018-04-21#PlansforMacPortsReleases1

Re: Online MacPorts Meeting on 21st April at 13:00 UTC

[Rainer Müller]

On 2018-04-07 10:31, Mojca Miklavec wrote:
> I would like to announce the first online MacPorts meeting happening
> two weeks from now, on April 21st at 13:00 - 14:00 UTC (if it takes
> much longer than one hour, we are probably doing something wrong).

The minutes of the meeting can be found on the corresponding wiki page:
https://trac.macports.org/wiki/Meetings/2018-04-21#Minutes

The next online meeting was set to be held on May 26th, 13:00 UTC.
An invitation with a call for topics will follow shortly before
the event.

Rainer

Failure in port diagnose (was: Re: Releasing 2.4.4)

[Rainer Müller]

On 2018-04-21 23:23, Michael wrote:
> I took a look at the change log, and saw there is a command, port diagnose

This was first released with 2.4.0 last year, but no changes were made
for 2.4.4.

> bash-3.2# port -v diagnose
> Error: process_cmd failed: dsAttrTypeNative:shell: /bin/bash
> launchctl: Couldn't 
> stat("/System/Library/LaunchDaemons/com.apple.DirectoryServicesLocal.plist"): 
> No such file or directory
> nothing found to load
> bash-3.2# 
> 
> It seems to not work properly.

I cannot reproduce this here running macOS 10.12 Sierra. There is no
launchctl invoked by the port action at all, so this must be coming from
/usr/bin/dscl. I do not have the mentioned file on my system either.

If you see this on an old version of macOS, it could be that this is the
same problem described in the ticket below. The fix would then be to
ignore this error message with 'exec -ignorestderr ...'.

https://trac.macports.org/ticket/42347

Rainer

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-22 09:44, Andrew Moore wrote:
> Just to be clear, asciidoc is the format, but asciidoctor is the text 
> processor (a superset of python asciidoc).  Is that correct?

Yes. AsciiDoc is both the name of the format and also the name of the
original reference implementation in Python, but development on that
ceased a few years ago. AsciiDoctor is an alternative implementation in
Ruby and actively being developed.

We are already using the AsciiDoc format for our man pages, and that has
proved to be a good choice, because the format is quite extensible
(custom macros, includes, etc.).

Rainer

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-23 10:42, Jan Stary wrote:
> Also, having manpages written in asciidoc means we need
> the abominable docbook to actually produce the man(7) pages, right?
> At least that's what it says at the end of install:
> 
>   * Warning: Using pre-generated man pages only.
>   * asciidoc, xsltproc (port libxslt) and docbook-xsl are required
> to generate man pages from source.

That is correct, DocBook XML is used because AsciiDoc has no native
backend to produce man pages.

However, AsciiDoctor has a native backend for man pages [1], which does
not require the additional step through DocBook XML. If you, or anyone
else, wants to look into switching the man page generation in base from
AsciiDoc to AsciiDoctor, that would be appreciated.

Rainer

[1]
https://github.com/asciidoctor/asciidoctor/blob/master/lib/asciidoctor/converter/manpage.rb

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-23 10:12, Jan Stary wrote:
>>> We are already using the AsciiDoc format for our man pages,
>>
>> Manpages are supposed to be written in mdoc(7), the language of
>> manpages, just like e.g. /usr/share/man/man1/ls.1 is.
>>
>> The current port-* manpages are horrible
>> (not talking about content, but markup).
>>
>> Look at e.g. port-diagnose.1; read it, line by line.
>> Note the 2008 workarounds to debian bugs.
>> Note the 2009 workarouns to GNU roff.
>> Note the pointless low-level roff(7) tweaks.
> 
> 
> https://lists.gnu.org/archive/html/groff/2014-02/msg00109.html

I do not care about the raw input, I read man pages in rendered form
with man(1).

These quirks are added by DocBook XSLT and are apparently required to
make the output compatible. If you disagree with that, talk to DocBook
why they added them.

>> Then read the port-diagnose.1 below, written in mdoc(7).
>> Which one is cleaner, simpler, more readable, more writable, shorter?
>>
>> I offer to rewrite them.

We moved away from mdoc(7) with the NewHelpSystem [1]. It took us about
five years to write all man pages, including conversion of the old man
pages.

>> .Dd April 23, 2018
>> .Dt PORT-DIAGNOSE 1
>> .Os
>> .Sh NAME
>> .Nm port diagnose
>> .Nd detects common issues with macports
>> .Sh SYNOPSIS
>> .Nm port
>> .Ic diagnose
>> .Op Fl -quiet
>> .Sh DESCRIPTION
>> .Nm port
>> .Ic diagnose
>> will check a list of common issues that could
>> affect the user of MacPorts in one way or another.
>> If any issues are found, a warning will be
>> shown to the user included with a possible fix.
>> .Pp
>> With the
>> .Fl -quiet
>> option, only warnings and errors are displayed.
>> .Sh SEE ALSO
>> .Xr port 1
>> .Sh AUTHORS
>> .An Kyle Sammons Aq Mt ksamm...@macports.org

I really cannot see the benefits of this. To me, writing this by hand
would be even more terrible than writing XML. I have no idea what any of
these two-character commands mean. This syntax is far from being intuitive.

Yes, it seems to be possible to encode more semantics than with
AsciiDoc. But do we really need that in a man page? At least for
terminal output, almost everything of that is lost with rendering anyway.

If you really care that much about the non-rendered file on disk,
I would recommend to rewrite the DocBook XSLT to use the mdoc(7) format
or contribute a native backend for it to AsciiDoctor.

Rainer

[1] https://trac.macports.org/wiki/NewHelpSystem

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-23 14:27, Jan Stary wrote:
>> These quirks are added by DocBook XSLT and are apparently required to
>> make the output compatible. If you disagree with that, talk to DocBook
>> why they added them.
> 
> You don't care that a 2008 workaround to a Debian bug
> is included in every manpage of a MacPorts package system?
> Or a 2009 workaround to a Solaris bug in GNU roff?
> Don't you find it a prime example of clutter
> that simply does not belong there?

No, I really do not care about these workarounds. Why should I care
about workarounds in intermediate formats? I do not see anything of this
in my terminal.

Again, this is added to every man page produced by DocBook and not
specific to the man pages generated for MacPorts. We are not maintaining
the XSLT stylesheets that produce this output. If you think these
workarounds are not necessary, the DocBook project is the right address
for complaints.

 Then read the port-diagnose.1 below, written in mdoc(7).
 Which one is cleaner, simpler, more readable, more writable, shorter?

 I offer to rewrite them.
>>
>> We moved away from mdoc(7) with the NewHelpSystem [1]. It took us about
>> five years to write all man pages, including conversion of the old man
>> pages.
> 
> $ cd /opt/mports/macports-base/doc
> $ wc -l *.txt *.1 *.5 *.7
> 
> That would make it about nine lines per day.>
> BTW, was it mdoc(7) or man(7)?

mdoc(7).

You seem to count both the input and the output files. Also, some of the
old files, such as portfile.7 and porthier.7, still have not been
converted. The real progress was even slower than that.

> How did you write the mdoc(7) manpages then?

Those had been written before I even joined the project. Apparently
someone wrote the original man pages, then left the project and no
longer contributed. Nobody updated the man pages because they were
written in a format nobody wanted to learn. Therefore the plan came up
to replace that format with something else that is easier to pick up for
new contributors.

Although there was already some work to use DocBook XML for the man
pages, I proposed to use AsciiDoc in the NewHelpSystem. This format was
already used by other projects to write man pages. Just to name popular
examples, git and mercurial still use it. I do not think that the HTML
version of their man pages is missing semantics. I do not think that our
HTML version [1] is missing semantics, either.

Back then, I did not even know there was a difference between man(7) and
mdoc(7). And even today, neither looks like a syntax new contributors
without previous knowledge want to write in my opinion.

I see you are strongly attached to mdoc(7), but take a look around:
except some *BSD projects nobody uses mdoc(7) to write man pages. Most
projects use lightweight markup languages such as AsciiDoc, Markdown, or
restructuredText, or even pod2man for this task.

$ find /opt/local/share/man | wc -l
   15591
$ find /opt/local/share/man -exec zgrep -l '^.Dt' {} + | wc -l
 588

Rainer

[1] https://man.macports.org/port-diagnose.1.html

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-22 09:29, Mojca Miklavec wrote:
> Hi,
> 
> As requested during yesterday's meeting, here's the result of
> automatic conversion of our guide from xml (docbook) to asciidoc:
> 
> https://github.com/macports/macports-guide/tree/master/guide/adoc
> 
> This was initially explored by Aljaž and gives quite nice results
> out-of-the-box, but there are still some issues with conversion that
> will have to be addressed either by fixing docbookrx itself or
> manually.

Ruby and gems are really strange...

Apparently, installing nokogiri with bundle fails because it tries to
execute pkg-config with DYLD_LIBRARY_PATH set, for whatever reason. This
leads to crashes as it draws in incompatible versions of libjpeg. I was
not even able to determine where this DYLD_LIBRARY_PATH in the
environment is coming from.

Here is a way to get it working:

$ git clone https://github.com/asciidoctor/docbookrx
$ cd docbookrx
$ sudo port install rb23-bundler
$ bundle-2.3 config --local build.nokogiri --use-system-libraries 
--with-xml2-dir=/opt/local --with-xml2-include=/opt/local/include/libxml2 
--with-xslt-dir=/opt/local --with-zlib-dir=/opt/local 
--with-exslt-dir=/opt/local --without-pkg-config
$ bundle-2.3 install --path=.bundle/gems

Then go to the guide/xml/ directory of macports-guide:

$ BUNDLE_GEMFILE=~/src/docbookrx/Gemfile bundle-2.3 exec docbookrx guide.xml
$ mv *.adoc ../adoc/

Rainer

Re: [macports-ports] branch master updated: collectd: update to 5.8.0

[Rainer Müller]

On 2018-04-23 19:47, Perry E. Metzger wrote:
> Perry E. Metzger (pmetzger) pushed a commit to branch master
> in repository macports-ports.
>
> https://github.com/macports/macports-ports/commit/5b70112c367fdc957b751f1ca6b8f1d1314f3b46
>
> The following commit(s) were added to refs/heads/master by this push:
> new 5b70112 collectd: update to 5.8.0 5b70112 is described below
>
> commit 5b70112c367fdc957b751f1ca6b8f1d1314f3b46 Author: Perry E. Metzger 
> 
> AuthorDate: Mon Apr 23 13:47:10 2018 -0400
>
> collectd: update to 5.8.0 ---
>  sysutils/collectd/Portfile | 10 +-
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/sysutils/collectd/Portfile b/sysutils/collectd/Portfile
> index 589255c..978b1e6 100644 --- a/sysutils/collectd/Portfile +++
> b/sysutils/collectd/Portfile @@ -3,7 +3,7 @@  PortSystem  1.0
>  
>  namecollectd
> -version 5.7.2 +version 5.8.0  categories  sysutils net
>  platforms   darwin
>  license GPL-2
> @@ -20,10 +20,10 @@ homepage http://collectd.org/  master_sites
> http://collectd.org/files/
>  use_bzip2 yes
>  
> -checksums sha1 b27074fc363d942c8aee2f05baed0395a20c86d0 \ - sha256
> 9d20a0221569a8d6b80bbc52b86e5e84965f5bafdbf5dfc3790e0fed0763e592 \ -
> rmd160 81860b4fba7cf05b8eca19913ffe05d06b37e1a2 \ - size 1798777
> +checksums sha1 eb3567563de471e1950b7388e8883b911be3756a \ + sha256
> b06ff476bbf05533cb97ae6749262cc3c76c9969f032bd8496690084ddeb15c9 \ +
> rmd160 7b812b1046d57a9047213bb182b8744d98606547 \ + size 1686017  
>  depends_build   port:pkgconfig
>  

It is a bit suspicious that you made no changes to files/dep-gen.sh in
the port directory.

The second half of the Portfile is fully generated by that shell script,
as collectd has *a lot* of variants for each of its plugins. Updating
that script is mostly manual work, based on the output of './configure
--help'. I somehow doubt that this version of collectd does not have any
new plugins or updated dependencies...

Rainer

Installing nokogiri with --use-system-libraries (was: Re: Migrating the guide to AsciiDoc)

[Rainer Müller]

On 2018-04-23 20:35, Andrew Moore wrote:
> You might try:
> 
> $ sudo gem update —system
> 
> which should install the latest version of bundler (you really want that), 
> and go from there.
> In the particular case of nokogiri, you might also try running `gem install 
> nokogiri` as user root.

I am very cautious about running third-party package managers as root
and let them write to ${prefix}. That is supposed to be managed by
MacPorts and no other package manager should update files.

If this overwrites any file provided by the ruby port, just running
a deactivate/activate cycle will bring back the old version. Or an
inconsistent state, if it also wrote new files to keep the version
information.

If there is ever going to be a rb23-nokogiri port, I expect this to
cause conflicts later with files not known to MacPorts.

When I uninstall the ruby port, this might leave traces behind
in ${prefix} that can be cleaned up, but I would not be aware of that.

nokogiri uses vendored versions of zlib/libxml2/libxslt and other
dependencies by default. I already have them installed in ${prefix}
and I do not want to build them again from source, therefore I used
--use-system-libraries (as the README of docbookrx suggested).

I even dared to give this a try as root (again, not recommended),
but this also fails in the same way:

$ sudo gem2.5 install nokogiri -- --use-system-libraries
...
pkg-config could not be used to find libxml-2.0
...

>From .../mkmf.log:

"pkg-config --exists libxml-2.0"
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: 
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /opt/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
package configuration for libxml-2.0 is not found

This pkg-config command works fine outside of 'gem install'.

The crash report in Console.log shows that the invocation of pkg-config
included the environment variable: DYLD_LIBRARY_PATH=.:/opt/local/lib

Rainer

Re: [macports-ports] branch master updated: collectd: update to 5.8.0

[Rainer Müller]

On 2018-04-24 00:48, Perry E. Metzger wrote:
> On Mon, 23 Apr 2018 23:58:23 +0200 Rainer Müller
>  wrote:
>> It is a bit suspicious that you made no changes to files/dep-gen.sh
>> in the port directory.
>>
>> The second half of the Portfile is fully generated by that shell
>> script, as collectd has *a lot* of variants for each of its
>> plugins. Updating that script is mostly manual work, based on the
>> output of './configure --help'. I somehow doubt that this version
>> of collectd does not have any new plugins or updated dependencies...
> 
> Apologies. I didn't see any instructions to that effect. Might I
> suggest that we remove this from being openmaintainer given that
> simple version bumps won't work well without inside knowledge?

There is a box with a "WARNING" near the end, but probably we should
also place another one at the top.

> I'll go in now and try to fix what I damaged.

No worries, I am looking into it. The update is long overdue anyway.

> (As for why I was playing with this, it's the only client of
> protobuf-c...)
Rainer

Re: Installing nokogiri with --use-system-libraries

[Rainer Müller]

On 2018-04-24 07:50, Mojca Miklavec wrote:
> On 23 April 2018 at 17:42, Rainer Müller wrote:
>>
>> Ruby and gems are really strange...
>>
>> Apparently, installing nokogiri with bundle fails
> 
> Wasn't it you who said that we don't need a port for it as it's just a
> one-time conversion? :)
> (I'm honestly not sure who said that.)

Yes, that was me... :-)

I thought we could just edit unprocessed elements in in *.adoc, but as
I looked at them, some seemed really simple to be added to docbookrx in
order to avoid manual work.

> I have huge philosophical problems with this line:
> --use-system-libraries --with-xml2-dir=/opt/local
> --with-xml2-include=/opt/local/include/libxml2
> --with-xslt-dir=/opt/local --with-zlib-dir=/opt/local
> --with-exslt-dir=/opt/local
> because this will lead to broken software as soon as one updates some
> libraries under MacPorts. (Does HomeBrew ever delete old versions or
> do their old versions of libraries remain installed forever until the
> user removes them?)

It is just a shortcut to avoid pkg-config and tell it the paths
manually. I assume the result would be the same if pkg-config worked.

This is not that different from compiling software outside of MacPorts,
but linking to MacPorts libraries. This can break with updates to ports,
but it will work as long as the name of the .dylib does not change and
the ABI is compatible. If any of this changes, ports linking to the
library would also break.

> I did not use any of those workarounds this time for once (I didn't
> use external libraries).

Indeed, the external libraries caused the problems for me. That's what I
get for trying to save time by not compiling the libraries again...

Rainer

Re: Migrating the guide to AsciiDoc

[Rainer Müller]

On 2018-04-22 09:29, Mojca Miklavec wrote:
> As requested during yesterday's meeting, here's the result of
> automatic conversion of our guide from xml (docbook) to asciidoc:
> 
> https://github.com/macports/macports-guide/tree/master/guide/adoc
> 
> This was initially explored by Aljaž and gives quite nice results
> out-of-the-box, but there are still some issues with conversion that
> will have to be addressed either by fixing docbookrx itself or
> manually.

I fixed some of the issues by slightly modifying the XML. None of these
should change the output produced by DocBook, but help docbookrx to
understand the XML.

I started to document the issues with the conversion in the README. This
is not exhaustive, please add more things to this list if you spot them.

https://github.com/macports/macports-guide/blob/master/guide/adoc/README.md

On one hand, some issues can only be fixed manually once we are sure we
are not going to run docbookrx again. On the other hand, I noticed that
it would be easier to fix other issues in docbookrx instead of
post-processing the *.adoc files.

However, some of the issues require tweaks to docbookrx that are
specific to the MacPorts guide, as they depend on assumptions how
certain elements are used.

I pushed some commits with such changes that are specific to the
MacPorts guide to my fork of docbookrx on the macports-guide branch:

https://github.com/raimue/docbookrx/tree/macports-guide

Should I move this repository to the MacPorts organization on GitHub for
collaboration? This would only be temporary until we are finished with
the conversion.

Rainer

Re: What "openmaintainer" means

[Rainer Müller]

On 2018-04-25 17:10, Joshua Root wrote:
> On 2018-4-26 00:25 , Perry E. Metzger wrote:
>> On Wed, 25 Apr 2018 04:43:12 +1000 Joshua Root 
>> wrote:
>>> On 2018-4-25 03:56 , Ken Cunningham wrote:
 Waiting for the maintainer to review the ticket submission
 someday often resulted in months of nothing happening, or years.  
>>>
>>> The maintainer timeout was 72 hours all along, so that's not really
>>> relevant to a discussion about the limits of openmaintainer.
>>
>> I think if you don't feel a clean version update falls in the limits
>> of openmaintainer (that is, just bumping the version and the
>> checksums), then I'm not sure what does fall under "openmaintainer"
>> for you.
> 
> Minor, uncontroversial changes. Something is broken or suboptimal and
> the fix is obvious. Specific examples:
> 
> * Typos

If a port is broken by an obvious typo, I would expect that everyone is
always allowed to commit the change to fix the port.

Requiring approval of the maintainer for a non-openmaintainer port just
leaves the port longer in the broken state.

> * Using eval when {*} could be used

OK under openmaintainer policy.

> * Rev bump needed when a dependency's ABI changed

The port would be broken without this change. Bumping the revision only
cannot do any harm. I would always allow this, even for ports without
openmaintainer.

> * Add --disable-werror when the build starts failing when a new version
> of clang adds a new warning

While the port would be broken in this case, this might also indicate
there is a need to submit a fix upstream. In many cases better left to
the maintainer.

> * Fix bundled libtool that thinks 10.10 is 10.1
> * Build fails on a new OS version because of something like a missing
> #include

These should also be reported and fixed upstream, so should go through
the maintainer.

> * Build is missing the correct -arch flags and adding them in the right
> place is simple

OK under openmaintainer policy.

> Some version bumps may be minor, others are definitely not. I would
> suggest considering the size of upstream changes in addition to those
> made to the port.
Agreed. It depends on the port what should be considered a minor update.

Rainer

Re: Agility

[Rainer Müller]

On 2018-04-25 21:02, Perry E. Metzger wrote:
> On Thu, 26 Apr 2018 01:19:59 +1000 Joshua Root 
> wrote:
>> On 2018-4-26 00:44 , Perry E. Metzger wrote:
>>> I tend to think it's a better way of working but it's up to the
>>> community as a whole to say if it's really the right way to do
>>> things. (If people really don't like it, I can hang back more.)  
>>
>> To be clear, I have no problem at all with you (or anyone else)
>> merging things if the maintainer has not said otherwise after 72
>> hours.
> 
> But apparently 72 hours is the rule in general, even without
> "openmaintainer" being set. It still doesn't tell us what the
> practical difference is between "openmaintainer" and a closed
> port.

This seems a problem that only occurs in the context of pull requests.

I think the openmaintainer policy was intended for changes coming from
other project members and not necessarily for patches coming from
external contributors. openmaintainer allows other project members to
push changes directly to master, without explicit approval by the
maintainer.

Pull requests are like the original workflow in which patches are
attached to a ticket on Trac. That ticket would either be handled by the
maintainer or run into maintainer timeout. It does not seem common that
another project member takes a patch from a ticket and pushes the change
under the openmaintainer policy before the maintainer timeout is over.

Of course, in the original workflow that also involves more than just
the press of a button...

Rainer

Re: Binary packages not rebuilding against updated libraries

[Rainer Müller]

On 2018-04-26 00:11, Jan Stary wrote:
> On Apr 24 09:31:04, ken.cunningham.web...@gmail.com wrote:
>> Portfile authors need to manually "revbump"
>> the library's dependent ports when
>> supporting libraries change significantly.
>> It's not automatically figured out by MacPorts.
> 
> Wait, what? I'm still relatively new to MP
> and this seems a bit weird.
> 
> "Port A depends on library B. The port of library B has
> just been rebuilt/upgraded. So port A needs to be rebuilt
> as well now, if only to record the new dependency."

An update to port B cannot cause a new dependency in port A.

> MP does not have that?
Port A only has to be rebuilt if the compatibility version of port B
changes in a way that makes it incompatible with the previous version
(basically increasing soname on other systems). This is quite seldom.

If port B was updated and the compatibility version changed, the
automatic run of 'port rev-upgrade' after the installation will detect
that a binary/library of port A has broken linkage. It will then rebuild
port A from source, ensuring you have a working installation locally.

To produce new packages, the revision of port A has to be increased.

Rainer

Re: Binary packages not rebuilding against updated libraries

[Rainer Müller]

On 2018-04-26 08:03, Jan Stary wrote:
>>
>> For comparison, the OpenBSD port system has resigned on upstreams'
>> library versioning, and versions the libraries itself. For example,
>> in audio/libsndfile (1.0.28):
>>
>>  SHARED_LIBS +=  sndfile   6.0   # .1.28
>>
>> and it installs /usr/local/lib/libsndfile.so.6.0.
> 
> BTW, what's with the version numbers in e.g.
> otool -L /opt/local/bin/openssl:
> /opt/local/lib/libssl.45.dylib (compatibility version 46.0.0, current version 
> 46.1.0)
> 
> libssl.45.dylib is what the vanilla tarball itself produces.
> Where exactly do the numbers 46.0.0 and 46.1.0 come from?

Check the file ssl/VERSION in the libressl distribution.

Rainer

Re: Binary packages not rebuilding against updated libraries

[Rainer Müller]

On 2018-04-26 07:48, Jan Stary wrote:
> On Apr 26 03:16:37, rai...@macports.org wrote:
>> On 2018-04-26 00:11, Jan Stary wrote:
>>> On Apr 24 09:31:04, ken.cunningham.web...@gmail.com wrote:
 Portfile authors need to manually "revbump"
 the library's dependent ports when
 supporting libraries change significantly.
 It's not automatically figured out by MacPorts.
>>>
>>> Wait, what? I'm still relatively new to MP
>>> and this seems a bit weird.
>>>
>>> "Port A depends on library B. The port of library B has
>>> just been rebuilt/upgraded. So port A needs to be rebuilt
>>> as well now, if only to record the new dependency."
>>
>> An update to port B cannot cause a new dependency in port A.
> 
> I mean dependency on the new lib.

A regular update to port B does not require anything to be changed in
port A. Only incompatible changes in port B require action on port A.

If you meant something else, MacPorts does not record linkage
information in the registry at the moment.

> Say a port produces a package containing 'bin/prog'
> that depends (as in otool -L) on libz.1.2.11.dylib.

'bin/prog' would be supposed to link against libz.1.dylib, not to a
filename with the full version number.

This is an essential concept of shared libraries. You can update them
without having to rebuild dependents.

> If zlib is updated, and provides libz.1.2.12.dylib now,
> bin/prog ca no longer depend on libz.1.2.11.dylib
> (which no longer exists), but on libz.1.2.12.dylib.
> That's what I meant by "new dependency", in the
> installed package of the port.

If this exists, the port providing 'bin/prog' is to blame for linking
the wrong library filename.

'bin/prog' should only break when the update changes from libz.1.dylib
to something like libz.2.dylib. As said before, this should be seldom.

> For comparison, the OpenBSD port system has resigned on upstreams'
> library versioning, and versions the libraries itself. For example,
> in audio/libsndfile (1.0.28):
> 
>   SHARED_LIBS +=  sndfile   6.0   # .1.28
>
> and it installs /usr/local/lib/libsndfile.so.6.0.
> Is that incompatible with the previous? That was libsndfile.so.5.0,
> so yes, it's incompatible; that's why the maintainer named it so.
This should not be the responsibility of the maintainer, upstream
controls the library versioning and decides when to increase it.

OpenBSD seems to need this because they want to be able to bump all
libraries for changes in their base OS. MacPorts produces separate
archives per OS version and on migration to a new version of macOS, a
full rebuild is necessary. Therefore the situation to bump all library
versions will not arise and MacPorts can use the version information
provided by upstream.

> Hence, rebuild everything that depends on libsndfile
> (without bumping _their_ revision).
>
> In an ideal world, the decision could be based on the libfoo.X.Y.Z,
> but often it cannot.

I highly doubt that changes to SHARED_LIBS do not involve changes to
dependents.

As far as I understand it, OpenBSD also does not produce a new binary
package of dependents unless their REVISION is increased. Therefore a
change in the dependent is required when SHARED_LIBS in a dependency is
changed. This is the exact same process we use in MacPorts right now.

>> To produce new packages, the revision of port A has to be increased.
> 
> That's what I don't get: new packages of what?
> The packages that were installed before get rebuilt
> with port rev-upgrade, automatically,
> without anyone touching their Portfile.
> Which ports need to get a new revision in the Portfile, and why?

'port rev-upgrade' can only fix your local system. The archives provided
on packages.macports.org would still contain binaries that link with the
old version.

Users would download the binary archive and install it, after which
'port rev-upgrade' detects is as broken and rebuilds it locally.
Therefore, the revision needs to be increased to rebuild a new archive
from source with the correct linking to be published at
packages.macports.org.

Rainer

Re: Binary packages not rebuilding against updated libraries

[Rainer Müller]

On 2018-04-26 15:36, Perry E. Metzger wrote:
> Not sure. I did come up with a better solution for the ABI bump
> problem, though, and wanted to run it by you.
> 
> So rather than just guessing based on things like major version of a
> library whether dependents need to be bumped, I would suggest we add
> an "abiversion" keyword. Changing it in any way would imply that ports
> depending on that port would be rebuilt.

All recursive dependents? That could be a long list and many unnecessary
rebuilds. You only need to rebuild ports that actually link with the
library, but that information is not available.

> This would imply that there would be a need to be able to query what
> "abiversion" is currently installed.

Users would not get any upgrades for ports unless epoch/version/revision
was increased. How would you recognize a port as outdated after a
dependency increased its abiversion?

We should not rebuild an archive that has the same identifier of
name/version/revision. Any changes to the contents of an archive has to
imply a change in the identifier to be reproducible. Otherwise you would
get bug reports against @1.2_3 which is a different @1.2_3 than you have
on your system.

Rainer

Re: Binary packages not rebuilding against updated libraries

[Rainer Müller]

On 2018-04-26 18:55, Perry E. Metzger wrote:
> On Fri, 27 Apr 2018 02:46:43 +1000 Joshua Root 
> wrote:
>>> So what might work would be trying out installing the archives in
>>> a sandbox and using the same mechanism that's used locally before
>>> triggering a rebuild?  
>>
>> Sounds like a plan?
>> 
>>
>> ;-P
> 
> Pardon my ignorance, but I'm not sure I understand why the rev bump
> would still be necessary if we could tell the build bots to just
> rebuild the package at that point?

If you rebuild the archive, new installations will get the new version.
But what about existing installations?

We should get the updated port to users without forcing them to rebuild
locally from source. That requires to bump the revision.

https://lists.macports.org/pipermail/macports-dev/2018-April/038572.html

Rainer

Re: How do I check what is autobuilding successfully?

[Rainer Müller]

On 2018-04-26 22:31, Clemens Lang wrote:
> | Error: Checksum (rmd160) mismatch for coffee-script-1.3.3.tar.gz
> | Portfile checksum: coffee-script-1.3.3.tar.gz rmd160 
> 22cf20180c06c92f5fdc223180ba94bb96b6ff7b
> | Distfile checksum: coffee-script-1.3.3.tar.gz rmd160 
> e069ff420bc2584015f4a412d5595f04c8be8df6
> | DEBUG: Calculated (sha256) is 
> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
> | Error: Checksum (sha256) mismatch for coffee-script-1.3.3.tar.gz
> | Portfile checksum: coffee-script-1.3.3.tar.gz sha256 
> 21b6782b23e09e768ff8b440eeddb0c42c7ad94990d120a348bfd4ffaf6511e0
> | Distfile checksum: coffee-script-1.3.3.tar.gz sha256 
> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
> | The correct checksum line may be:
> | checksums   rmd160  e069ff420bc2584015f4a412d5595f04c8be8df6 \
> | sha256  
> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
> | Error: Failed to checksum coffee-script: Unable to verify file checksums
> 
> Maybe a stealth update?

coffee-script is one of the ports affected by the git update at GitHub
half a year ago.

https://trac.macports.org/ticket/54839

Rainer

Re: How do I check what is autobuilding successfully?

[Rainer Müller]

On 2018-04-26 22:48, Rainer Müller wrote:
> On 2018-04-26 22:31, Clemens Lang wrote:
>> | Error: Checksum (rmd160) mismatch for coffee-script-1.3.3.tar.gz
>> | Portfile checksum: coffee-script-1.3.3.tar.gz rmd160 
>> 22cf20180c06c92f5fdc223180ba94bb96b6ff7b
>> | Distfile checksum: coffee-script-1.3.3.tar.gz rmd160 
>> e069ff420bc2584015f4a412d5595f04c8be8df6
>> | DEBUG: Calculated (sha256) is 
>> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
>> | Error: Checksum (sha256) mismatch for coffee-script-1.3.3.tar.gz
>> | Portfile checksum: coffee-script-1.3.3.tar.gz sha256 
>> 21b6782b23e09e768ff8b440eeddb0c42c7ad94990d120a348bfd4ffaf6511e0
>> | Distfile checksum: coffee-script-1.3.3.tar.gz sha256 
>> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
>> | The correct checksum line may be:
>> | checksums   rmd160  e069ff420bc2584015f4a412d5595f04c8be8df6 \
>> | sha256  
>> 509797e0bbfa534f7ea1c67d6df9f638c30d28f76f75cc74adbdc10096992f4a
>> | Error: Failed to checksum coffee-script: Unable to verify file checksums
>>
>> Maybe a stealth update?
> 
> coffee-script is one of the ports affected by the git update at GitHub
> half a year ago.

Scratch that, it was fixed just yesterday.

Rainer

Re: guide bug?

[Rainer Müller]

On 2018-04-27 03:23, Perry E. Metzger wrote:
> In one part of the guide, we say not to use "file copy" but to use
> "copy" instead,

Are you sure you read that in the guide?

> but in 4.3.2. we show an example using "file copy"
> rather than "copy". Is this a bug?
"file copy|move|delete" are native Tcl commands, while "copy", "move"
and "delete" are implemented in port1.0 with additional functionality as
documented in the guide [1].

There is no difference between "copy" and "file copy", it is just a
shorthand and a matter of style.

Rainer

[1] https://guide.macports.org/#reference.tcl-extensions

Re: guide bug?

[Rainer Müller]

On 2018-04-27 14:37, Perry E. Metzger wrote:
> "For the above operations provided by Tcl's file command, MacPorts
> provides the following shorthands. These should be used in preference
> to the Tcl commands above, as they may work around certain bugs."

Ah, I even linked to this section, but did not realize it says that...

> So we've discouraged the direct use of "file copy" over using "copy"
> by my reading. "These should be used in preference".

Very old versions of Tcl (on Tiger and before) indeed had a bug in
"file delete" that prompted the addition of "delete". That problem no
longer exists as we include our own version of Tcl, but it is probably
where this is originally coming from.

Do we really have a preference? At least for "file copy" vs. "copy",
both seem to be used to equal amounts in Portfiles.

Rainer

Re: guide bug?

[Rainer Müller]

On 27 April 2018 23:12:31 GMT+02:00, Ryan Schmidt  
wrote:
>
>On Apr 27, 2018, at 16:11, Joshua Root wrote:
>
>> On 2018-4-28 06:25 , Ryan Schmidt wrote:
>> 
>>> On Apr 27, 2018, at 08:10, Joshua Root wrote:
>>> 
>>> 
 'move' still handles some case-sensitivity issues, but indeed
>'copy' and
 'delete' are just aliases these days. I think we don't need to
 discourage the use of 'file', since if you hit the one fairly rare
>case
 where there's a difference you'll know it.
>>> 
>>> Not if the bug only affects an OS version you're not using.
>> 
>> What OS version doesn't need the case-only rename code?
>
>Right now, none of the workaround are OS-specific. But you noted that
>one of them used to be. And it could easily be that a bug affects only
>some OS versions in the future.

That applies to any Tcl command. We are not wrapping them all.

The workaround for move could as well be applied to 'file move' and eliminate 
the wrapper. It is possible to rename builtin commands.

Rainer

Re: [macports-ports] 17/17: fd: use new features of cargo PG

[Rainer Müller]

On 2018-04-28 00:34, Marcus Calhoun-Lopez wrote:
> Marcus Calhoun-Lopez (MarcusCalhoun-Lopez) pushed a commit to branch master
> in repository macports-ports.
>
> https://github.com/macports/macports-ports/commit/24864aeda99bedbc183f882878df70e8b555aa0b
>
> commit 24864aeda99bedbc183f882878df70e8b555aa0b Author: Marcus Calhoun-Lopez 
> 
> AuthorDate: Fri Apr 27 15:33:16 2018 -0700
>
> fd: use new features of cargo PG ---
>  sysutils/fd/Portfile | 35 +++
>  1 file changed, 19 insertions(+), 16 deletions(-)
>
> diff --git a/sysutils/fd/Portfile b/sysutils/fd/Portfile index
> 5100103..c5ca61c 100644 --- a/sysutils/fd/Portfile +++
> b/sysutils/fd/Portfile @@ -22,6 +22,8 @@ checksums fd-${version}.tar.gz \ 
>  sha256  
> e6ba21e66a295b9fb69bfc9a361813b3c7d73b8d29dc819d37d21af750814ee9 \
>  size49655
>  
> +cargo.direct_call yes
"direct_call"? What is this supposed to mean?
> +  cargo.crates \
>  aho-corasick 0.6.4  
> d6531d44de723825aa81398a6415283229725a00fa30713812ab9323faa82fc4 \
>  ansi_term   0.11.0  
> ee49baf6cb617b853aa8d93bf420db2383fab46d314482ca2803b40d5fde979b \
> @@ -79,22 +81,23 @@ pre-build {  
>  build.env-appendSHELL_COMPLETIONS_DIR="${worksrcpath}/shell_completions"
>  
> -destroot { - xinstall -m 755 ${worksrcpath}/target/release/${name}
> ${destroot}${prefix}/bin/ - xinstall -m 444
> ${worksrcpath}/doc/${name}.1 ${destroot}${prefix}/share/man/man1/ +#
> muniversal build requires Makefile +destroot.args-append -f
> Makefile-Install +post-extract { + set makefile [open
> ${worksrcpath}/Makefile-Install "w"] + puts ${makefile} "install:" +
> puts ${makefile} "\tinstall -m 0755
> target/\$(CARGO_BUILD_TARGET)/release/${name}
> \$(DESTDIR)${prefix}/bin" + puts ${makefile} "\tinstall -m 0444
> doc/${name}.1 \$(DESTDIR)${prefix}/share/man/man1/" + + puts
> ${makefile} "\tinstall -d -m 0755
> \$(DESTDIR)${prefix}/share/doc/${name}" + puts ${makefile} "\tinstall
> -m 0644 README.md LICENSE-MIT LICENSE-APACHE
> \$(DESTDIR)${prefix}/share/doc/${name}"  
> - xinstall -d ${destroot}${prefix}/share/doc/${name} - xinstall -m 644
> -W ${worksrcpath} \ - README.md LICENSE-MIT LICENSE-APACHE \ -
> ${destroot}${prefix}/share/doc/${name} + puts ${makefile} "\tinstall
> -d -m 0755 \$(DESTDIR)${prefix}/share/bash-completion/completions" +
> puts ${makefile} "\tinstall -m 644
> ${worksrcpath}/shell_completions/fd.bash
> \$(DESTDIR)${prefix}/share/bash-completion/completions/fd" + puts
> ${makefile} "\tinstall -d -m 0755
> \$(DESTDIR)${prefix}/share/zsh/site-functions" + puts ${makefile}
> "\tinstall -m 644 ${worksrcpath}/shell_completions/_fd
> \$(DESTDIR)${prefix}/share/zsh/site-functions/_fd" + puts ${makefile}
> "\tinstall -d -m 0755 \$(DESTDIR)${prefix}/share/fish/completions" +
> puts ${makefile} "\tinstall -m 644
> ${worksrcpath}/shell_completions/fd.fish
> \$(DESTDIR)${prefix}/share/fish/completions/fd.fish"  
> - xinstall -d ${destroot}${prefix}/share/bash-completion/completions -
> xinstall -m 644 ${worksrcpath}/shell_completions/fd.bash \ -
> ${destroot}${prefix}/share/bash-completion/completions/fd - xinstall
> -d ${destroot}${prefix}/share/zsh/site-functions - xinstall -m 644
> ${worksrcpath}/shell_completions/_fd \ -
> ${destroot}${prefix}/share/zsh/site-functions/_fd - xinstall -d
> ${destroot}${prefix}/share/fish/completions - xinstall -m 644
> ${worksrcpath}/shell_completions/fd.fish \ -
> ${destroot}${prefix}/share/fish/completions/fd.fish + close ${makefile}  }
I do not understand why you would need a Makefile? The destroot phase
seemed totally fine?

If muniversal cannot work with destroot {} please fix muniversal, or
disable +universal for this port. I do not want to maintain such a hack.

Rainer

Re: [macports-ports] 09/17: cargo PG: respect build_arch; allow universal build

[Rainer Müller]

On 2018-04-28 00:34, Marcus Calhoun-Lopez wrote:
> Marcus Calhoun-Lopez (MarcusCalhoun-Lopez) pushed a commit to branch master
> in repository macports-ports.
>
> https://github.com/macports/macports-ports/commit/68a34f577814e3e5b3f444530262b109093da515
>
> commit 68a34f577814e3e5b3f444530262b109093da515 Author: Marcus Calhoun-Lopez 
> 
> AuthorDate: Fri Apr 27 11:47:22 2018 -0700
>
> cargo PG: respect build_arch; allow universal build ---
>  _resources/port1.0/group/cargo-1.0.tcl | 56 
> ++
>  devel/mesalink/Portfile|  2 ++
>  textproc/ripgrep/Portfile  | 31 +++
>  3 files changed, 77 insertions(+), 12 deletions(-)

> diff --git a/textproc/ripgrep/Portfile b/textproc/ripgrep/Portfile
> index c62e042..7dbba25 100644 --- a/textproc/ripgrep/Portfile +++
> b/textproc/ripgrep/Portfile @@ -67,16 +67,23 @@ depends_build-append \
>   port:asciidoc \
>  port:docbook-xsl
>  
> -destroot { - xinstall -m 755 ${worksrcpath}/target/release/rg
> ${destroot}${prefix}/bin/ - ln -s rg ${destroot}${prefix}/bin/ripgrep
> - - set outpath
> ${worksrcpath}/target/release/build/ripgrep-65e5a64db1247a47/out/ -
> xinstall -m 644 ${outpath}/rg.1 ${destroot}${prefix}/share/man/man1/ -
> xinstall -d -m 755
> ${destroot}${prefix}/share/bash-completion/completions - xinstall -m
> 644 ${outpath}/rg.bash
> ${destroot}${prefix}/share/bash-completion/completions/rg - xinstall
> -d -m 755 ${destroot}${prefix}/share/fish/completions - xinstall -m
> 644 ${outpath}/rg.fish ${destroot}${prefix}/share/fish/completions/ -
> xinstall -d -m 755 ${destroot}${prefix}/share/zsh/site-functions -
> xinstall -m 644 ${worksrcpath}/complete/_rg
> ${destroot}${prefix}/share/zsh/site-functions/ +# muniversal build
> requires Makefile +# OUTDIR contains a checksum, so it is not know
> until late in the build process and may +# change with a change in the
> cargo PortGroup
The out dir can be overriden with the environment variable OUT_DIR, used
by build.rs. I was going to change this with the next update.
> +destroot.args-append -f Makefile-Install +post-extract { + set
> makefile [open ${worksrcpath}/Makefile-Install "w"] + # from
> ci/utils.sh + puts ${makefile} "OUTDIR=\$(shell find . -name
> ripgrep-stamp -print0 | xargs -0 ls -t | head -n1 | xargs dirname)" +
> puts ${makefile} "install:" + puts ${makefile} "\tinstall -m 0755
> target/\$(CARGO_BUILD_TARGET)/release/rg \$(DESTDIR)${prefix}/bin" +
> puts ${makefile} "\tln -s rg \$(DESTDIR)${prefix}/bin/ripgrep" + puts
> ${makefile} "\tinstall -m 0644 \$(OUTDIR)/rg.1
> \$(DESTDIR)${prefix}/share/man/man1" + puts ${makefile} "\tinstall -d
> -m 0755 \$(DESTDIR)${prefix}/share/bash-completion/completions" + puts
> ${makefile} "\tinstall -m 0644 \$(OUTDIR)/rg.bash
> \$(DESTDIR)${prefix}/share/bash-completion/completions" + puts
> ${makefile} "\tinstall -d -m 0755
> \$(DESTDIR)${prefix}/share/fish/completions" + puts ${makefile}
> "\tinstall -m 0644 \$(OUTDIR)/rg.fish
> \$(DESTDIR)${prefix}/share/fish/completions" + puts ${makefile}
> "\tinstall -d -m 0755 \$(DESTDIR)${prefix}/share/zsh/site-functions" +
> puts ${makefile} "\tinstall -m 0644 complete/_rg
> \$(DESTDIR)${prefix}/share/zsh/site-functions/" + close ${makefile}  }
Same comment for this Makefile as for the changes to fd. Why write a
Makefile from the Portfile? Even shipping the Makefile in files/ would
be better than this hack.

destroot {} is exactly meant for this kind of code. If muniversal cannot
handle it, supporting +universal is not important enough for me to
maintain this.

I do not consider this a minor change under the openmaintainer policy.
If you change the cargo port group in a way that requires such big
changes in ports, please discuss them first. If you cannot wait to
update your own ports, then commit the updated port group as cargo 1.1
instead.

Rainer

Re: Open Letter of Apology

[Rainer Müller]

Hello Marcus,

On 2018-04-28 01:33, Marcus Calhoun-Lopez wrote:
> Recently, I made some major changes to the cargo PortGroup and the Portfiles 
> of several people.
> My intent was:
>   *) create a branch
>   *) solicit feedback on the branch with the changes
> 
> Instead, I pushed the changes to master.
> It was very careless of me, and I sincerely apologize to all those I have 
> inconvenienced.

I am sorry that my initial reaction to the changes was this harsh.
I also want to apologize for that. Mistakes happen and it is okay that
they happen. I also work on MacPorts because I learn a lot by doing so
and it should be the same for others. Mistakes are part of every
learning process.

I appreciate that you are working on better support for cargo. I know
that the port group that I wrote was probably not able to handle all use
cases. However, I am also proud that the port group allows to write
Portfiles that are simple and elegant.

I agree with some of the changes you made without a doubt and they can
and should be applied. None of these should require changes in ports, so
they are all good to go:

* additional explanations in comments
* ensuring all builds use the right compiler
* set extract.only to all non-crates
* remove unnecessary if clause

I made a pull request out if these and if you approve (since they carry
you as author), they can be merged right away:

https://github.com/macports/macports-ports/pull/1680

Rainer

Re: [MacPorts] Tickets modified

[Rainer Müller]

On 2018-04-30 16:14, MacPorts Wiki wrote:
> Page "Tickets" was changed by pmetzger
> Diff URL: 
> Revision 43
> Comment: Correct report number.
> Changes:
> ---8<--8<--8<--8<--8<--8<--8<--8<
> Index: Tickets
> =
> --- Tickets (version: 42)
> +++ Tickets (version: 43)
> @@ -28,7 +28,7 @@
>  See also:
>  
>   * [/report Ticket Reports]
> - * [/report/10 Most Recent Active Tickets]
> + * [/report/11 Most Recent Active Tickets]

The report uses "order=time" (Created), but the name suggests it would
be supposed to use "order=changetime" (Modified). I just corrected that.

Rainer

Re: Why didn't this PR get labeled?

[Rainer Müller]

On 2018-04-30 15:49, Perry E. Metzger wrote:
> Sometimes, I've noticed, PRs in the queue don't get automatically
> labeled as expected. See this one, for example:
> 
> https://github.com/macports/macports-ports/pull/1687
> 
> Anyone know why?

The log file on braeburn says (times in UTC):

Apr 28 19:14:41 braeburn prbot-current[1214]: 2018/04/28 19:14:41 PR #1687 
opened
Apr 28 19:14:41 braeburn prbot-current[1214]: 2018/04/28 19:14:41 GET 
https://api.github.com/repos/macports/macports-ports/pulls/1687/files?per_page=30:
 404 Not Found []
Apr 28 19:16:40 braeburn prbot-current[1214]: 2018/04/28 19:16:40 sql: no rows 
in result set

Looks like GitHub send out the WebHook, but the API did not knew about the pull 
request yet...?

Rainer

Re: build failures and lack of logs

[Rainer Müller]

On 2018-04-29 14:31, Joshua Root wrote:
> On 2018-4-29 22:04 , Jack Howarth wrote:
>> As far as I can see, the links for the main.log file on these mailed
>> build failure reports is a dead or empty file.
> I see what you mean now. I'm not sure why we have a separate main.log
> link when all the same information is in the stdio, nor why it is not
> populated when the build fails.

That is because mpbb-install-port returned early on error and did not
update statistics or copy log files.

Proposed fix:
https://github.com/macports/mpbb/pull/6

Rainer

Re: Request for discussion: lowering the backlog of old Trac tickets.

[Rainer Müller]

On 2018-04-30 18:16, Perry E. Metzger wrote:
> But, more generally: because if you make your tracking system into a
> graveyard of dead requests that will never be acted on, you end up
> with it being very difficult for someone to look through it and find
> things that actively need fixing.
> 
> The function of a ticket system is to allow people to keep track of
> open issues and to direct their work so that those issues get
> resolved. This requires that most tickets contain actionable
> information, so that someone looking through the tickets can easily
> find tasks that they can quickly perform.

I highly doubt anyone is going to look through open tickets unless they
are looking for tickets filed against a specific port...

A ticket documenting a bug can still be valid after years, even if
nobody commented on the ticket anymore. Unless the bug was either fixed
with a patch or with an update by upstream, the ticket should stay open
to document the bug. Why would you want to remove the documentation that
the bug exists?

Other users with the same problem can find the ticket and see that this
is a known problem. The ticket may include suggestion on how to solve
the problem and work on it. Often, the comments also document
workarounds around the issue, which are helpful for those encountering
the same problem. I do not see how closing a bug report without
resolving the issue helps users.

If you closed such a bug report just because it was old, users
encountering the problem would search for a solution and find nothing,
or they could find a ticket that was closed. I would consider this case
would be even more frustrating. An open ticket is at least the
acknowledgement that the bug actually exists.

> If a ticket system gets overly clogged up with things that cannot be
> acted on and likely never will be acted on, the things that do need
> attention get lost in the fog. You end up with people waiting years
> on things that could be acted on because no one can see that they're
> in need of attention. Then people get depressed that the problem
> keeping them from getting their work done has been ignored, and
> wander off, never to return.

> New people looking at the tickets system also get immensely depressed
> when they see that it is not uncommon for people to report an issue
> and have it sit for many years without action. It is, of course,
> vastly easier to clear out actionable requests when you can find
> them. (And thus, again, it is best not to have a lot of things in the
> system that cannot be acted on, because otherwise you need a lot of
> effort to find the actionable items amidst the unactionable ones,
> with most effort going not to fixing problems but to finding the
> problems that can be fixed.)

The solution to fewer open bug reports is to either show that the bug no
longer exists and close the ticket as "worksforme", or fix the bug and
close the ticket as "resolved".

Note: these comments only apply to bug reports. I fully support closing
open port submissions that have not been worked on for years.

Rainer

Re: Request for discussion: lowering the backlog of old Trac tickets.

[Rainer Müller]

On 2018-04-30 21:09, Chris Jones wrote:
> I would also argue we are using the same system, for two different things. 
> Short term bug reports and longer term 'work goals' are not the same, but we 
> seem to mix them up in the same system. Maybe some way (label ?) to identify 
> tickets that are going to be longer term is required ?

I would assume that bug reports would be short term and requests for new
ports or enhancements in a port are long term, but that distinction
already exists with the "Type" field. In which other cases would you
apply such a label/keyword?

Rainer

Re: [macports-ports] 01/17: cargo PG: modify comments

[Rainer Müller]

On 2018-04-30 05:26, Ryan Schmidt wrote:
> 
> On Apr 28, 2018, at 04:31, Joshua Root wrote:
> 
>> On 2018-4-28 13:04 , Ryan Schmidt wrote:
>>> According to WikiPedia, "All rights reserved" has no effect in any legal 
>>> jurisdiction:
>>>
>>> https://en.wikipedia.org/wiki/All_rights_reserved
>>>
>>> Unless anyone knows otherwise, we may as well remove it everywhere.
>>
>> It's not even true since we explicitly grant rights to others in the
>> license.
> 
> Here's an in-depth post about this problem:
> 
> https://opensource.stackexchange.com/questions/2121/mit-license-and-all-rights-reserved/4403#4403
> 
> It seems the phrase "All rights reserved" is contradictory and confusing in 
> the case of an open-source license, and yet, it is part of the BSD license 
> text. If we remove the phrase, can we still claim that we are using the BSD 
> license?

Interestingly, the phrase is not part of the OSI-approved BSD-3-Clause
license text:
https://opensource.org/licenses/BSD-3-Clause

 @@ -11,7 +12,7 @@
 # 2. Redistributions in binary form must reproduce the above copyright
 #notice, this list of conditions and the following disclaimer in the
 #documentation and/or other materials provided with the distribution.
 -# 3. Neither the name of Apple Computer, Inc. nor the names of its
 +# 3. Neither the name of The MacPorts Project nor the names of its
 #contributors may be used to endorse or promote products derived from
 #this software without specific prior written permission.
>>>
>>> The attribution to Apple in the license text appears in many files, not 
>>> just this one. This file certainly didn't originate from Apple, but some 
>>> MacPorts source files did, and I don't know what the correct treatment of 
>>> those files is. Do we change them to MacPorts, since we maintain the files 
>>> now, or do we need to retain the mention of Apple in those files that Apple 
>>> originated, possibly adding our name to theirs? And if so, do we change 
>>> their name to Apple Inc., since that is their name now? If someone could 
>>> look up what the legally correct thing to do here is, that would be great. 
>>> Whatever the outcome, we should apply it to all files.
>>
>> Apple definitely retains the copyright on any portions of the code that
>> were written at Apple. Their copyright notices and license should not be
>> removed. We add our own notices for the portions we write. We can either
>> choose to distribute those portions under the same license as Apple's
>> portions (including clause 3 which specifically mentions Apple's name)
>> or we can add our own license in addition, and the combined work would
>> be subject to both.

For the cargo port group, the mistake was introduced by me. I think
I copied the license text from another port group, assuming all would
use the same. I should have checked more thoroughly.

Where would I find the license text to use for new files? Looking around
now, shouldn't LICENSE in the top-level directory contain the license
text we want? That file also explicitly mentions Apple and was added
only a year ago [1]. Looking at it now, I would say it is incorrect.

However, why do we add a license header to the port groups at all?
We also do not add it to each Portfile. I think we should drop the
license headers from all port groups.

Rainer

[1] https://github.com/macports/macports-ports/pull/287

Re: Why didn't this PR get labeled?

[Rainer Müller]

On 2018-05-01 03:45, Helmut K. C. Tessarek wrote:
> 
> 
> On 2018-04-30 09:49, Perry E. Metzger wrote:
>> Sometimes, I've noticed, PRs in the queue don't get automatically
>> labeled as expected. See this one, for example:
>>
>> https://github.com/macports/macports-ports/pull/1687
>>
>> Anyone know why?
> 
> Same here:
> https://github.com/macports/macports-ports/pull/1697

Apr 30 23:07:18 braeburn prbot-current[1214]: 2018/04/30 23:07:18 PR #1697 
opened
Apr 30 23:07:18 braeburn prbot-current[1214]: 2018/04/30 23:07:18 GET 
https://api.github.com/repos/macports/macports-ports/pulls/1697/files?per_page=30:
 404 Not Found []

Rainer

Re: trac rrdtool

[Rainer Müller]

On 2018-05-01 15:20, macpo...@parvis.nl wrote:
> searching for open tickets on rrdtool, I don't understand some things.

[...]

> What is happening here? Did these people perhaps made a patch on basis
> of a rrdtool patch and didn't change the filename?

The guide uses "Portfile-rrdtool.diff" as an example filename [1]. Some
users seem to take that literally and submit patches with that name.

When looking for open tickets for a particular port, it is usually
better to filter by the "port" field. You can use the search box on the
Tickets wiki page [2] or a custom query [3].

Rainer

[1] https://guide.macports.org/#development.patches.portfile
[2] https://trac.macports.org/wiki/Tickets#Searchingforexistingtickets
[3] https://trac.macports.org/query?port=~rrdtool&status=!closed

Re: Why didn't this PR get labeled?

[Rainer Müller]

On 2018-05-01 16:28, Perry E. Metzger wrote:
> On Tue, 1 May 2018 13:20:45 + Zero King  wrote:
>> As Rainer replied, GitHub didn't provide the needed information in
>> time. Maybe we should wait 30 seconds and retry?
> 
> That seems like a very good idea. I'd probably wait two minutes,
> though, just to nail it.

This should be reported to GitHub support. I would consider it a bug at
their end if they send out WebHooks but do not actually provide the
referenced data via the API yet.

Rainer

Re: [macports-ports] 01/17: cargo PG: modify comments

[Rainer Müller]

On 2018-05-01 03:16, Rainer Müller wrote:
> However, why do we add a license header to the port groups at all?
> We also do not add it to each Portfile. I think we should drop the
> license headers from all port groups.

https://github.com/macports/macports-ports/pull/1707

Rainer

Re: missing dejavusansmono in rrdtool pango cairo fontconfig

[Rainer Müller]

On 2018-05-02 12:44, macpo...@parvis.nl wrote:

Macports installs fonts in /opt/local/share/fonts.
This directory is seen only by fontconfig (defined in 
/opt/local/etc/fonts/fonts.conf).


Most software uses cairo for advanced font handling and pango for the 
heavy-lifting in cairo. This is called pangocairo. rrdtool has '#include 
'.


Pango has a simple tool to show font examples:
pango-view --backend= --text '     ' --font 
'DejaVuSansMono 36' # execute in XQuartz


I think this should be
  --font 'DejaVu Sans Mono 36'
At least it only works for me with spaces when I manually install the 
font to /Library/Fonts/DejaVuSansMono.ttf.



Note that backend is for pango, not for cairo.
With backend 'xft' or 'ft2' DejaVu shows fine.
With backend 'cairo' the replacement is done ('cairo' is the default 
backend for pango, see pangocairo).


The problem lies in cairo. If 'quartz' is enabled, 'fontconfig' is not used.
The Portfile for cairo explicitely enables 'quartz' and 'x11'.
With 'quartz' disabled everything works as expected.

I cannot oversee the consequences of disabling 'quartz' for cairo.


The native macOS font renderer searches for fonts only in paths such as 
~/Library/Fonts, /Library/Fonts, and /System/Library/Fonts as documented 
in [1]. There does not seem to be a way to extend that.


We could install our fonts to a path such as /Library/Fonts/MacPorts/ to 
make them available to native applications by default. However, this 
would be a long-term solution as that requires changes in base for a 
${fonts_dir} to be handled just like ${applications_dir}.


On the pango mailing list, Behdad Esfahbod offered the solution to set 
environment variable 'env var PANGOCAIRO_BACKEND=fc' and that works fine.


While I agree this is a problem that should be solved for all ports, one 
workaround for rrdtool would be to use a simple shell script wrapper 
that exports this environment variable before executing the real binary.


However, that is only necessary if rrdtool can only work correctly with 
"DejaVu Sans Mono". Would it be acceptable to use a different font that 
is always available for the time being? Even a generic 'monospace' 
should give similar results.


Rainer

Re: missing dejavusansmono in rrdtool pango cairo fontconfig

[Rainer Müller]

On 2018-05-02 22:17, macpo...@parvis.nl wrote:
> 
>> On 2018-05-02, at 16:15, Rainer Müller  wrote:
>>
>> On 2018-05-02 12:44, macpo...@parvis.nl wrote:
>>> (...)
>>> Pango has a simple tool to show font examples:
>>> pango-view --backend= --text '     ' --font 
>>> 'DejaVuSansMono 36' # execute in XQuartz
>>
>> I think this should be
>> --font 'DejaVu Sans Mono 36'
>> At least it only works for me with spaces when I manually install the font 
>> to /Library/Fonts/DejaVuSansMono.ttf.
> 
> It should work with and without spaces, even in lowercase:
> fc-match dejavusansmono
> DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"
> fc-match 'dejavu sans mono'
> DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

Yes, fontconfig accepts that. But when using --backend=cairo and the
quartz renderer, fontconfig is not involved. In this case, the name has
to be specified with spaces, or the font can not be found and will be
substituted. Sorry if I was unclear.

It looks like all unknown fonts are just substituted with Helvetica for
the quartz renderer.

>> However, that is only necessary if rrdtool can only work correctly with 
>> "DejaVu Sans Mono". Would it be acceptable to use a different font that is 
>> always available for the time being? Even a generic 'monospace' should give 
>> similar results.
> 
> Munin, where I am currently working on, explicitly defines DejaVuSans and 
> DejaVuSansMono. Both are substitued by Helvetica. That cannot be handled by a 
> wrapper but requires patches, to Menlo for example.

Ah, I assumed this was about the default in rrdtool, which is specified
in the configure script:

   RRD_DEFAULT_FONT='"DejaVu Sans Mono,Bitstream Vera Sans 
Mono,monospace,Courier"'

That could be overridden with the --with-rrd-default-font option to
something like Menlo.

If you can already identify where Munin specifies the font, then writing
a patch to change that to Menlo should be quite easy.

> I never would have thought that porting Munin was so complicated. I'd prefer 
> to start with ddrescue and foremost, my heart lies at the OS and things like 
> filesystems.

I am sorry your experience with updating Munin to 2.x was not the best.
You really hit a can of worms there.

If you do not want to pursue updating munin further, do you think it
would be worth to attach your work-in-progress Portfile to the Trac
ticket [1] for others to pick up?

Rainer

[1] https://trac.macports.org/ticket/36860

Re: Help with Pull Request

[Rainer Müller]

On 2018-05-03 12:14, Marcus Calhoun-Lopez wrote:
> The only git commands I have been running (not in this order) are
> git pull --rebase upstream master
> git pull --rebase origin cargo-pg
> git pull --rebase origin master
> git push origin cargo-pg
> where
> cargo-pg is the name of my branch and upstream is
> git remote add upstream https://github.com/macports/macports-ports.git

With 'git pull --rebase', you instruct git to take the specified remote
and put all commits that are not in the remote on top of the branch.

git pull --rebase upstream master
This brought in new commits from the master branch and put the local
commits on top. This is what you wanted.

git pull --rebase origin cargo-pg
This started with the cargo-pg branch, but then the new local commits
(that came from master, but are not yet on cargo-pg), were put on top.
Now you have the commits from cargo-pg first, then the commits that were
done on master. This is the wrong order and not what you wanted.

Usually, you never want to rebase with anything else than
upstream/master. You always want to start with the state of upstream,
then put your new commits on top of that. You would first switch to the
branch you want to work on, then run rebase:
$ git checkout cargo-pg
$ git pull --rebase upstream master


After that advice for the future, there are two things to be fixed here.
One is your master branch and one is the cargo-pg branch. Let's start
with the cargo-pg branch.

Switch to the branch and rebase again on top of the current state of
upstream/master. This will start with upstream, then put all local
commits on top. Although you had "cloned" commits from upstream/master
to this branch before, this will still do what you wanted. In case a
commit results in an empty patch (has been applied before), it will be
dropped during rebase. Therefore all the commits that are not meant to
be on this branch will be gone after running these commands.

$ git checkout cargo-pg
$ git pull --rebase upstream master
(verify results in 'git log')
$ git push --force origin cargo-pg

Now, let's go to the master branch. It is diverged because there is a
commit for octave on it, then lots of "cloned" commits from
upstream/master. Again, you can get rid of the "cloned" commits by
running rebase again. As this one commit for octave is also already in
upstream/master, there will be no local commits left after these commands:

$ git checkout master
$ git pull --rebase upstream master
(verify results in 'git log', should be identical to upstream/master)
$ git push --force origin master


'git push --force' is considered dangerous because it allows you push a
branch state that removes commits that were pushed previously. But
sometimes this is exactly what you need to clean up.

'git push --force' can only have unwanted effects when you are
collaborating with others. By changing the branch such that commits are
now missing, others might no longer be able to pull changes from the
branch. However, these are branches in your personal fork, so you can do
with the branches whatever you want. It is also unlikely that anyone
else has a clone of your repository.

Hope this helps!

Rainer

Re: Instructions for creating patches

[Rainer Müller]

On 2018-05-03 09:14, Zero King wrote:
> On Thu, May 03, 2018 at 08:51:49AM +0200, Mojca Miklavec wrote:
>> On 1 May 2018 at 16:11, Rainer Müller wrote:
>>>
>>> The guide uses "Portfile-rrdtool.diff" as an example filename [1]. Some
>>> users seem to take that literally and submit patches with that name.
>>>
>>> [1] https://guide.macports.org/#development.patches.portfile
>>
>> Looking at this part of the guide, I would suggest that we change the
>> instructions. It almost definitely makes more sense to use "git diff
>> ..." instead of "diff ...", but more importantly we probably want to
>> suggest creating a pull request at that point anyway.
> 
> Some contributors would edit the Portfile in
> /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/,
> which isn't a git repository and `git diff` won't work for them.

If users edit the ports tree directly there is no way to diff at all and
the changes will be lost on the next sync. They should never do that.

'git diff' actually works with arbitrary paths when not inside a
repository, as it implies the --no-index flag in this case.

The mix of instructions on how to create a patch against a Portfile and
how to create a patch against upstream sources might also be a bit
confusing and I would recommend to separate that.

Maybe the instructions on how to create a local ports tree should come
first and then the next section should show how to create a diff against
the official ports tree?

Instructions on how to create a pull request would overlap with the
section "Using Git and GitHub" [1]. We should only document this in one
place.

> Maybe "git format-patch -1" is better than "git diff ..."? But that
> would be even closer to a pull request.

That also requires that the user already has a repository available.

I still think that a port action like 'port submit' [2] (or as a start
even just an external script), would help untrained contributors more
than replicating the GitHub documentation on how to make a pull request.

Rainer

[1] https://guide.macports.org/#project.github
[2] https://lists.macports.org/pipermail/macports-dev/2018-April/038318.html

Re: Releasing 2.4.4

[Rainer Müller]

On 2018-04-29 10:17, Mojca Miklavec wrote:
> Are there any news or progress about the 2.4.4 release?

It has been two weeks. How do we want to proceed?

Rainer

Re: Compiler blacklist 'shorthand'

[Rainer Müller]

On 2018-05-06 12:07, Ryan Schmidt wrote:
> 
> On May 5, 2018, at 19:36, Craig Treleaven wrote:
> 
>> A couple of times recently, I’ve noticed boilerplate in ports that require 
>> C++14.  After including the compiler_blacklist_versions portgroup, they then 
>> do some gymnastics like:
>>
>> compiler.blacklist  *gcc-3.* *gcc-4.* {*gcc-5.[0-3]} \
>>{clang < 800} macports-clang-3.4 
>> macports-clang-3.5 macports-clang-3.6 macports-clang-3.7
>>
>> Would it not be easier to use and maintain if we had some shorthand 
>> definitions.  Maybe something like:
>>
>> compiler.blacklist  ${min_cxx14}
>>
>> “min_cxx14” would be defined in the portgroup and then expand to the 
>> above...assuming the above actually does a good job of blacklisting 
>> compilers that don’t support C++14!
>>
>> A major advantage is that if our list of non-C++14 compilers ever changes, 
>> it only needs to be updated in one spot.
>>
>> I suspect there would be a few other shorthand lists that could be 
>> pre-defined.
>>
>> Thoughts?
> 
> Yes, we should have support for specifying the required language standard(s) 
> in Portfile, so that MacPorts could then select a compatible compiler.
> 
> Until we have that, you need to blacklist incompatible compilers.
>
> If you require C++11, include the cxx11 1.1 portgroup which will do what's 
> needed for you, including blacklisting incompatible compilers and ensuring 
> the right C++ standard library is used.
> 
> If you require C++14, include the cxx11 1.1 portgroup and additionally use 
> "compiler.blacklist-append {clang < 602}".

This does not seem very intuitive. Maybe that should be put into a
simple cxx14 port group acting as a thin wrapper as shown below?

# cxx14-1.1.tcl
PortGroup cxx11 1.1
compiler.blacklist-append {clang < 602}


Rainer

New project member: vishnu

[Rainer Müller]

Please join us in welcoming the following new MacPorts project member:

 - Vishnu M (vishnu, @Vishnum98)

We look forward to continued excellent contributions during Google
Summer of Code 2018 and beyond.

- Joshua, Ryan, and Rainer


Do you want to join the MacPorts team? If you would like to be
considered for team membership and commit access, please read this
section of the Guide:

http://guide.macports.org/chunked/project.membership.html