Don Lewis wrote:
On 27 Apr, Michelle Sullivan wrote:
Don Lewis wrote:
On 27 Apr, Michelle Sullivan wrote:
Don Lewis wrote:
On 27 Apr, Michelle Sullivan wrote:
Don Lewis wrote:
On 27 Apr, Rick Miller wrote:
On Wed, Apr 27, 2016 at 12:53 PM, Michelle Sullivan <miche...@sorbs.net>
wrote:
Kevin Oberman wrote:
On Wed, Apr 27, 2016 at 8:06 AM, Michelle Sullivan <miche...@sorbs.net>
wrote:
After a portsnap update it seems all my jails won't build the current tree
returning the following error:
====>> MOVED: sysutils/puppet renamed to sysutils/puppet38
====>> MOVED: textproc/rubygem-augeas renamed to
textproc/rubygem-ruby-augeas
====>> Computing deps for converters/libiconv
====>> Computing deps for archivers/unzip
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for converters/p5-Encode
====>> Computing deps for converters/p5-Convert-BinHex
====>> Computing deps for converters/p5-Encode-Locale
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for converters/p5-JSON-PP
====>> Computing deps for converters/p5-JSON
====>> Computing deps for converters/p5-JSON-XS
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for converters/p5-Text-Iconv
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for databases/ip4r
====>> Computing deps for databases/gdbm
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for databases/p5-Bucardo
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
Terminated
Terminated
Terminated
Terminated
====>> Cleaning up
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for databases/p5-DBD-Pg
====>> Error: Invalid port origin '/usr/local/bin/ccache' not found.
====>> Computing deps for databases/memcached
====>> Error: Invalid port origin '/usr/local/bin/automake-1.15' not
found.
====>> Umounting file systems
Checked updating but don't see anything to suggest that port origins of
'/usr/local/bin/ccache' are normal..
It looks like you're building with Poudriere. I observed similar behavior,
but not the exact message the other day. I don't remember what origin it
was complaining about, but located a post (either on a mailing list or
forums) recommending a `pkg install poudriere`. It did resolve the problem
in this particular scenario.
This is probably caused by the recent change to globally drop
${PORTSDIR} from *_DEPENDS. The framework changes initially were done
in bsd.port.mk r399278, but the the actual removal of ${PORTSDIR} didn't
happen until r411970, r412342, ...
Ok that sorta makes a bit more sense... however as this is a jail and
the tree is updated why did it break? (I have no local mods in the 'ng'
build tree - except an additional (local only) couple of ports which are
copied in manually after the portsnap update)...
Of course the nice thing is my non-ng tree is still working 100% - but
that would be because it didn't get the change... but again that's a
completely separate tree and the 2 are not associated with each other in
any way...
I was assuming that this was your non-ng tree where you have local
framework changes ...
No, completely separate repo as the new trees are constantly breaking my
tree so I keep them entirely separate.
Did you upgrade ports from something older than r411970 (Sun Mar 27
01:23:25 2016 UTC)
It would have been around march I did the last build so yes probably
prior to Mar 27.
to something more recent?
To the latest.
If poudriere on your host
is seriously old, it might not cope with the framework change. It looks
like you need at least 3.1.9, which was released on Wed Oct 14 21:06:00
2015 UTC.
Yeah, 3.1.x changes the base OS without authority and breaks the entire
build system (can't build anything but the official tree in it) so it's
been deemed a security issue (because it "upgrades" the existing
repositories) and therefore cannot be installed or used on any of the
existing build servers.
Sorry, this is all from memory ... my poudriere machine will be offline
for several more hours so I can't use it as a reference.
Poudriere shouldn't be changing anything in the base OS. It probably
creates some temp files under /tmp and puts the package repositories
that I builds and the log files under its own directories under
/var/tmp.
Unfortunately the first thing that 3.1.[012]? did was install all the
pkg stuff and change the pkg_add repo into a pkg repo... or something
like it, which broke everything horribly.. it was a long time ago so no
idea the specifics now... but it (3.1.x) was put on a 'not suitable for
use due to security issues' list.
That could be the point at which support for the old pkg_* tools was
removed from poudriere. That would explain why it wanted to upgrade
things.
Yes, but that's also a security issue in itself. Had I not spotted and
stopped this the changes it tried to make to the repo would have
resulted in production machines being 'reload from scratch' on the first
security patch - because it screwed the build servers and the repos.
I'd thought that poudriere was using the host copy of pkg to do the
final part of the respository build, but since poudriere doesn't list
pkg as a dependency, that appears not to be the case. It looks like
poudriere is running pkg (from the repository being constructed) in the
jail for that.
Yeah.. except something around the time went about "upgrading" the OS to
use pkg as well... which screwed the OS... fortunately I caught the
first VM it tried to do it to and was able to limit the damage just to
that VM so the rebuild was minimum.
You should be able to build as many different ports trees as you want
and they can be downloaded via portsnap or svn, or created by hand.
I think I've got 4-6 ports trees that I use with poudriere.
Which I have 2 currently - one which is 'HEAD' and the other which is my
'pkg_*' tools tree (up to date - mostly).
The repository that gets updated by a poudriere run is named
with a combination of the jail name, the ports tree name, and the set
name (-z option). The latter can be use to select an alternate
make.conf to set different port options.
Yes, however 3.1.x 'updated' the repo from pkg_* to something like pkgng
- it was completely f**ked though... basically had to erase everything,
downgrade and reinstall everything to get it back to a 'will build both
trees' state.
Question is why would it be needed? Surely the tree is the tree in the
jail and has nothing to do with the host? or is it not a case of
everything is done in the jail, just the actual building is and
therefore I need new build servers for the NG tree.. Which basically
means I should just decide to fork or erase the whole system because I
can't "NG" right now and I can't actually continue to build in parallel
because of this breakage?
Only the actual building is done in jails. When poudriere first starts
up, it looks at the list of ports that you want to build and then uses
the Makefiles for each of those ports to determine the dependencies of
each and the proper build order.
That doesn't make sense... the host has for months been well behind
both my tree and the ng trees... all the versions would be way out of
whack and even some not existing (ruby comes to mind.)
As I mentioned previously, poudriere is just a shell script so it isn't
tightly bound to the version of the ports tree. What caused the problem
that you are seeing now is that the ports framework changed in a way
that is not compatible with really old versions of poudriere. Basically
the output of "make -V BUILD_DEPENDS" changed from
blah:/path/to/ports/category/port to blah:category/port, and the old
version of poudriere can't cope with that.
I find that 'odd' at first parse... though will take your word for it
and check the code... maybe make my own version of poudriere that can
handle both...
The ports tree used by each build jail is not related to any ports tree
on the host (unless you do something like "poudriere ports -c -F -f none
-M /usr/ports -p systemports", see
<https://fossil.etoilebsd.net/poudriere/doc/trunk/doc/use_system_ports_tree.wiki>).
It's possible to run poudriere without having /usr/ports installed on
the host.
Will check this - if it mounts the local copy of the tree this would
probably fix it.
If you think this is a security risk, you could run poudriere in a VM.
They already are in VMs.. but if poudriere make modifications to the OS
then it is a security issue. If it modifies/builds packages that's fine.
... let me expand on that ... anything that modifies something in the
base OS unless specifically designed and approved to interact with the
OS (eg puppet) then as far as I am concerned (and my employer) it's a
security risk. Can't have things willy nilly changing the OS, it will
eventually break stuff and that could cause/lead to production
outages... it's just not done.
I don't think it's changing anything in the base OS.
Well bapt swears that it wasn't freebsd-update that screwed the OS, so
must be pourdiere ... don't know both changes happened at the same time,
and it screwed my migration plans to the point I've told $employer to
forget looking at FreeBSD at all for internal systems or embedded
systems and stick with the current OS of RHEL/CentOS and please assign
people to take over my machines and change them to match the corp
standard... So far they have refused to switch them (lack of resources)
and after a long discussion with a person in OPs that also has an
@freebsd.org address it was suggested I just fork the OS and ports
tree... However, after news that i might not have as long as I hoped, I
think this would be folly and a waste of my remaining days.
It sounds to me
like the problem is confined to upgrading old-style pkg repositories to
the NG when you try to update some of the packages contained in them,
which is not what you want in this case.
Yeah that happened as well.
You should be able to install a newer version of poudriere alongside the
old one.
That occurred to me, however unless its changed a lot it'll really not
work unless chrooted elsewhere because of shared scripts... or
rebuilding it to handle the shared stuff in different places (assuming
not already able to cope)
Just call it poudriere-ng or something. Just be sure not to
run it on one of your old pkg repositories.
That's not going to be an issue... will probably just have to put in
new VMs... just about got everything off 9.1 and pretty sure nothing is
on 9.0 now so maybe re-purpose them to be NG only hosts.
You might even be able to
edit the script to bail out if it detects and old-style pkg repository
as an anti-foot-shooting measure.
Think it would need to be physically separated because I know if bapt
was not lying about freebsd-update making the changes then it must have
been poudriere that changed the OS... so not worth taking the risk.
For the time being, just disable to entire NG repo keep my fork going
with the working (mostly) up to date ports tree for pkg_* tools and well
if the worst comes to the worst it'll fall to someone else in my absence
and they can deal with switching to RHEL/CentOS because they'd never
work out how the SORBS FreeBSD build system is put together.
Thanks for the replies though Don, all the best.
--
Michelle Sullivan
http://www.mhix.org/
_______________________________________________
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
