Re: AMI building AMI

2018-01-01 Thread Rafal Lukawiecki

> On 29 Dec 2017, at 02:44, Colin Percival  wrote:
> 
> us-east-1 ami-e6a6ea9c is a FreeBSD 11.1-RELEASE AMI-building AMI.

Many thanks, Colin. And a Happy New Year!

May I ask you if what I am hoping to do makes sense? Taking the gist from 
http://www.daemonology.net/blog/2015-11-21-FreeBSD-AMI-builder-AMI.html I plan 
to make an AMI, install packages, patch and install the kernel, all in one go. 
I am unsure how to “make kernel” so that it ends up in the new AMI, rather than 
just in the builder AMI.

Firs, I create an IAM role with a policy which grants access to the EC2 
CreateImage API. Then I launch your AMI building AMI ami-e6a6ea9c in us-east-1 
using config init script as follows, on a sufficiently large c5:

#!/bin/sh
export ASSUME_ALWAYS_YES=YES
PKGS_TO_INSTALL=lots of packages go here
pkg -r /mnt fetch -d $PKGS_TO_INSTALL
pkg -c /mnt install $PKGS_TO_INSTALL
svnlite co https://svn.freebsd.org/base/releng/11.1/ /usr/src/
cd /usr/src
cat > /tmp/kernel.patch 

Re: AMI building AMI

2018-01-01 Thread Rafal Lukawiecki
PS. Could it be as easy as make DESTDIR=/mnt installkernel in the config init?

> On 29 Dec 2017, at 02:44, Colin Percival  wrote:
> 
> us-east-1 ami-e6a6ea9c is a FreeBSD 11.1-RELEASE AMI-building AMI.

Many thanks, Colin. And a Happy New Year!

May I ask you if what I am hoping to do makes sense? Taking the gist from 
http://www.daemonology.net/blog/2015-11-21-FreeBSD-AMI-builder-AMI.html I plan 
to make an AMI, install packages, patch and install the kernel, all in one go. 
I am unsure how to “make kernel” so that it ends up in the new AMI, rather than 
just in the builder AMI.

Firs, I create an IAM role with a policy which grants access to the EC2 
CreateImage API. Then I launch your AMI building AMI ami-e6a6ea9c in us-east-1 
using config init script as follows, on a sufficiently large c5:

#!/bin/sh
export ASSUME_ALWAYS_YES=YES
PKGS_TO_INSTALL=lots of packages go here
pkg -r /mnt fetch -d $PKGS_TO_INSTALL
pkg -c /mnt install $PKGS_TO_INSTALL
svnlite co https://svn.freebsd.org/base/releng/11.1/ /usr/src/
cd /usr/src
cat > /tmp/kernel.patch 

Re: AMI building AMI

2018-01-01 Thread Rafal Lukawiecki

> On 1 Jan 2018, at 22:39, Colin Percival  wrote:
> 
> Maybe a dumb question, but do you really need to use a configinit script
> for this?  I know I showed that as an example in my blog post, but I
> expected that the main way the AMI builder would be used would be by
> SSHing in and setting things up "manually".

Automation is attractive but not not a primary goal this time. Being able to 
kick it off from an EC2 launch template and come back to a finished AMI, makes 
it more likely I’d have it done the next few times the AMI needs remaking.

I suppose I will need to do it manually the first time to see where the issues 
are. I will share the feedback with you in case it is useful for a future post.

Many thanks,
Rafal
___
freebsd-cloud@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-cloud
To unsubscribe, send any mail to "freebsd-cloud-unsubscr...@freebsd.org"


Re: AMI building AMI

2018-01-01 Thread Julian Elischer

On 2/1/18 12:16 am, Rafal Lukawiecki wrote:

On 29 Dec 2017, at 02:44, Colin Percival  wrote:

us-east-1 ami-e6a6ea9c is a FreeBSD 11.1-RELEASE AMI-building AMI.

Many thanks, Colin. And a Happy New Year!

May I ask you if what I am hoping to do makes sense? Taking the gist from 
http://www.daemonology.net/blog/2015-11-21-FreeBSD-AMI-builder-AMI.html I plan 
to make an AMI, install packages, patch and install the kernel, all in one go. 
I am unsure how to “make kernel” so that it ends up in the new AMI, rather than 
just in the builder AMI.

Firs, I create an IAM role with a policy which grants access to the EC2 
CreateImage API. Then I launch your AMI building AMI ami-e6a6ea9c in us-east-1 
using config init script as follows, on a sufficiently large c5:

#!/bin/sh
export ASSUME_ALWAYS_YES=YES
PKGS_TO_INSTALL=lots of packages go here
pkg -r /mnt fetch -d $PKGS_TO_INSTALL
pkg -c /mnt install $PKGS_TO_INSTALL

please explain to me the  use of -r AND -c?
if you use -r to populate a chroot for -c then should that be pkg add 
rater than pkg install?

svnlite co https://svn.freebsd.org/base/releng/11.1/ /usr/src/
cd /usr/src
cat > /tmp/kernel.patch 

Re: AMI building AMI

2018-01-01 Thread Rafal Lukawiecki

> On 1 Jan 2018, at 17:42, Rafal Lukawiecki  wrote:
> 
> PS. Could it be as easy as make DESTDIR=/mnt installkernel in the config init?

I went ahead and tried the script. It is not working yet, and I need to debug 
it further, which means connecting to the instance etc, which I have not yet. 
So far a few observations are:

- AMI builder does not like c5.4xlarge, it aborts when trying to install the 
base FreeBSD image. It works with c4.4xlarge.
- svnlite needed extra params to avoid hanging on keyboard input
- pkg -c throws error about "no address record unable to update repository”, I 
am trying to force an update of the repository
- kernel compiles, but I am not yet sure if it installed correctly
- mkami failed to create an AMI, unsure yet why, need to rerun with more debug 
output.

My current script is:

#!/bin/sh
set -x
export ASSUME_ALWAYS_YES=YES
PKGS_TO_INSTALL="awscli cli53 lots of other ones"
pkg -r /mnt fetch -d $PKGS_TO_INSTALL
pkg -r /mnt update -f
pkg -c /mnt install $PKGS_TO_INSTALL
svnlite --non-interactive --trust-server-cert-failures=unknown-ca co 
https://svn.freebsd.org/base/releng/11.1/ /usr/src/
cd /usr/src
cat > /tmp/kernel.patch 

Re: AMI building AMI

2018-01-01 Thread Colin Percival
On 01/01/18 11:00, Rafal Lukawiecki wrote:
> On 1 Jan 2018, at 17:42, Rafal Lukawiecki  wrote:
>> PS. Could it be as easy as make DESTDIR=/mnt installkernel in the config 
>> init?

That is indeed how to install the kernel you built into the right place.

> I went ahead and tried the script. [...]

Maybe a dumb question, but do you really need to use a configinit script
for this?  I know I showed that as an example in my blog post, but I
expected that the main way the AMI builder would be used would be by
SSHing in and setting things up "manually".

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
___
freebsd-cloud@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-cloud
To unsubscribe, send any mail to "freebsd-cloud-unsubscr...@freebsd.org"