The Latest Version including those in Subversion

I have been conversing with one of the primary developers, see some of my conversation below. I hope to distill these soon.

Creating kernel and initrd files from a given captured image

Specifically we have an image lws2008-winter along a local.cfg file that specifies network data, in particular, this would be the place to embed the imageserver hostname as that will ensure when we run si_updateclient it will boot with the correct name even if the host has no pre-existing Linux install on it, i.e., /local.cfg does not exist on the file system locally or by installing using the PXE server on asimov.cscf.

By the way, one can have a look inside the initrd by doing, gunzip -c initrd.img >out and then mount -o loop out /mnt

si_mkbootpackage --destination /tmp --image lws2008-winter --config ./local.cfg                                
>>> Copying modules to new initrd from: /var/lib/systemimager/images/lws2008-winter/lib/modules/2.6.20-15-generic...
skipping directory /var/lib/systemimager/images/lws2008-winter/lib/modules/2.6.20-15-generic/initrd
skipping directory /var/lib/systemimager/images/lws2008-winter/lib/modules/2.6.20-15-generic/kernel
skipping directory /var/lib/systemimager/images/lws2008-winter/lib/modules/2.6.20-15-generic/madwifi
skipping directory /var/lib/systemimager/images/lws2008-winter/lib/modules/2.6.20-15-generic/volatile
>>> Appending insmod commands to ./my_modules_dir/INSMOD_COMMANDS...
 >> Using custom kernel: hotplug will be used to autodetect the needed modules...
>>> Including local.cfg into the initrd.img: ./local.cfg
>>> Choosing filesystem for new initrd:  cramfs
>>> Creating new initrd from staging dir:  /tmp/.systemimager.0
 >> mkcramfs /tmp/.systemimager.0 /tmp/initrd
Directory data: 45256 bytes
Everything: 27640 kilobytes
Super block: 76 bytes
CRC: aba84419

 >> gzip -f -9 -S .img /tmp/initrd

 >> ls -l /tmp/initrd.img
-rw-r----- 1 root root 28192843 Jun 16 15:31 /tmp/initrd.img

>> Evaluating initrd size to be added in the kernel boot options
>> (e.g. /etc/systemimager/pxelinux.cfg/syslinux.cfg):
 >>   suggested value -> ramdisk_size=37880

>>> Using kernel from:          /var/lib/systemimager/images/lws2008-winter/boot/vmlinuz-2.6.20-15-generic
 >> ls -l /tmp/kernel
-rw-r----- 1 root root 1726608 Jun 16 15:31 /tmp/kernel


Boot kernel and initrd.img (generated by the initrd_template package)
can be found in /etc/systemimager/boot. If PXE-booting, you can copy the
files to /tftpboot or the specified location.

From righi.andrea@gmail.com  Sat Apr 12 06:47:10 2008
Date: Sat, 12 Apr 2008 12:39:53 +0200
From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
User-Agent: Thunderbird 2.0.0.12 (X11/20080227)
MIME-Version: 1.0
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]

Walter Tautz wrote:
> Andrea Righi wrote:
>> Walter Tautz wrote:
>>   
>>>>> # si_updateclient --server ubuntu-i386.cscf --autoinstall
>>>>> ======================= WARNING =================================
>>>>>  This command will configure this host to be auto re-installed by
>>>>>  SystemImager at the next reboot!!!
>>>>> ======================= WARNING =================================
>>>>>
>>>>> Are you sure to continue? (y / N)? y
>>>>>
>>>>> Here is a list of available flavors:
>>>>>
>>>>>   gl.2008.spring
>>>>>   standard
>>>>>
>>>>> Which flavor would you like to use? [standard]: gl.2008.spring
>>>>> Retrieving SystemImager kernel...
>>>>> Retrieving SystemImager initial ramdisk...
>>>>> Using System Configurator to set boot configuration...ROOTDEV /dev/ram 
>>>>> under [BOOT] does not exist, please check your systemconfigurator config
>>>>> systemconfigurator command to configure autoinstall boot failed! at 
>>>>> /usr/sbin/si_updateclient line 436
>>>>>
>>>>> walter
>>>>>     
>>>>>         
>>>> It seems you need your /etc/systemconfig/systemconfig.conf in your image
>>>> (/var/lib/systemimager/images/<your_image>/). Did you create this image
>>>> using a version of systemager < 4.x?
>>>>
>>>>   
>>>>       
>>> No. It was using the version 4.0.2
>>>
>>>     
>>
>> This seems a bug, when si_updateclient is used with --autoinstall it
>> creates the systemconfig.conf in the client on-the-fly as following:
>>
>> [BOOT]
>> BOOTDEV = $bootdev
>> ROOTDEV = /dev/ram
>> TIMEOUT = 50
>> DEFAULTBOOT = SystemImager
>>
>> [KERNEL0]
>> LABEL = SystemImager
>> PATH = ${relative_bootdir}kernel.systemimager
>> INITRD = ${relative_bootdir}initrd.img.systemimager
>> APPEND = $append
>>
>> But if /dev/ram doesn't exist in the running system systemconfigurator
>> will fail.
>>
>> I'll post a fix ASAP.
>>   
> 
> I just added an additional information to the bug report.
> 
> Any progress on this?
> 

The fix is not so simple. I can try to workaround the problem in
systemimager, but the actual bug is in systemconfigurator. Since I'm not
the maintainer of systemconfigurator I need to do some tests before and
then propose a fix to Erich Focht (the SC maintainer) to speed-up the
whole process. Since I'm the only active developer in these projects
this needs some time...

In the while as a workaround you can image your clients using an
autoinstall media (si_mkautoinstallcd / si_mkautoinstalldisk) or boot
via PXE (si_mkclientnetboot).

BTW why are you using si_updateclient to configure the client(s) to
autoinstall? (probably you don't have control on the dhcp server,
right?)

-Andrea

From righi.andrea@gmail.com  Fri Apr 25 09:26:19 2008
Date: Fri, 25 Apr 2008 15:26:11 +0200
From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]

Walter Tautz wrote:
> Andrea Righi wrote:
>> Walter Tautz wrote:
>>> Warning: '/proc/partitions' does not match '/dev' directory structure.
>>>     Name change: '/dev/dm-0' -> '/dev/evms/hda1'
>>> Fatal: Illegal 'root=' specification: /dev/ram
>>>
>>>  at /usr/lib/systemconfig/Boot.pm line 112
>> Walter, I did some tests trying to reproduce your problem, but I'm not
>> able to get the "illegal root=/dev/ram" error message.
>>
>> I would suggest to try to manually fix the configuration in
>> /etc/systemconfig/systemconfig.conf, remove /boot/grub/devices.map and
>> re-run si_updateclient --autoinstall.
> 
> No go. Is there a way I can run things more verbosely.

Adding --verbose to the systemconfigurator invocation directly
in si_updateclient, I mean from:
...
    ### BEGIN autoinstall boot configuration ###
    $cmd = <<SC_EOF;
systemconfigurator --configboot --runboot --stdin <<EOF
...

Change to:
...
    ### BEGIN autoinstall boot configuration ###
    $cmd = <<SC_EOF;
systemconfigurator --verbose --configboot --runboot --stdin <<EOF
...

> 
> How would I edit the file systemconfig.conf?
> 

Well, I meant, try to do some tests changing the systemconfig.conf
embedded in si_updateclient, but ok, it's not a clean approach...

Now, try to set the verbose flag in systemconfigurator and post the
complete output of si_updateclient. This should allow to get useful
informations to better understand your problem.

-Andrea

From righi.andrea@gmail.com  Mon Apr 28 12:41:21 2008
Date: Mon, 28 Apr 2008 18:41:12 +0200
From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]

Walter Tautz wrote:
> Hi Andrea,
> 
> si_updateclient  created /etc/lilo.conf but I do not have lilo. I use 
> grub. The issue
> seems to be it can't update /boot/grub/menu.lst on Ubuntu/hardy/i386.
> I got the system to boot by manually editing /boot/grub/menu.lst but
> then another problem occurred with an unquoted string in one
> of the init scripts (how do I capture this on the server)
> 
> walter

Walter,

try also to add PREFERED=Boot::Grub to the systemconfig.conf in
si_updateclient.

I've just moved to ubuntu hardy in my PC, so tonight or tomorrow I'll do
a test and I'll try to reproduce the problem.

I'll keep you informed.

-Andrea

From a.righi@cineca.it  Wed May  7 11:35:23 2008
From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]
Date: Wed,  7 May 2008 17:35:02 +0200 (MEST)

Walter Tautz wrote:
> Andrea Righi wrote:
>> Walter Tautz wrote:
>>> Hi Andrea,
>>>
>>> si_updateclient  created /etc/lilo.conf but I do not have lilo. I use 
>>> grub. The issue
>>> seems to be it can't update /boot/grub/menu.lst on Ubuntu/hardy/i386.
>>> I got the system to boot by manually editing /boot/grub/menu.lst but
>>> then another problem occurred with an unquoted string in one
>>> of the init scripts (how do I capture this on the server)
>>>
>>> walter
>> Walter,
>>
>> try also to add PREFERED=Boot::Grub to the systemconfig.conf in
>> si_updateclient.
> 
> I added it to the client I ran si_updateclient on and it does indeed 
> overwrite
> the grub file which is good and bad. It would be nice it it could use
> the update-grub tool to add an entry for systemimager (handy if the
> autoinstall fails before partitioning of the disk occurs) and preserve the
> other entries. Even better if it password protected the entry.
> It could look for ### BEGIN AUTOMAGIC KERNELS LIST
> line and add the entry before this part as update-grub looks for kernels 
> with the
> name vmlinuz.

Yes. But it would be implemented in systemconfigurator as a
"post-install" action after the grub installation. I'll try to look at
it (added to my TODO list).

> The mount command in systemimager does not like -o relatime option.
> It gives: wrong fs type, bad option, bad superblock on /dev/sda2......
> basically it gives up. Without the option the file system mounts no problem
> with ext3 type.

Interesting. The mount implemented in Busybox 1.9.1 doesn't seem to
support this option. I'll check if it has been fixed in recent versions.
Thanks!

> walter
> 
>> I've just moved to ubuntu hardy in my PC, so tonight or tomorrow I'll do
>> a test and I'll try to reproduce the problem.
>>
>> I'll keep you informed.
>>
>> -Andrea

Sorry, I didn't have time to investigate further on this. I'll try next
weekend.

-Andrea

From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]
Date: Wed,  7 May 2008 17:43:17 +0200 (MEST)

Walter Tautz wrote:
> The mount command in systemimager does not like -o relatime option.
> It gives: wrong fs type, bad option, bad superblock on /dev/sda2......
> basically it gives up. Without the option the file system mounts no problem
> with ext3 type.

See also:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=460824

I'll just copy the fix from the debian repository and apply to the
trunk.

-Andrea

From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]
Date: Wed,  7 May 2008 18:01:38 +0200 (MEST)

Walter Tautz wrote:
 > The mount command in systemimager does not like -o relatime option.
> It gives: wrong fs type, bad option, bad superblock on /dev/sda2......
> basically it gives up. Without the option the file system mounts no problem
> with ext3 type.

FYI:
http://trac.systemimager.org/changeset/4459

-Andrea

From: Andrea Righi <a.righi@cineca.it>
Reply-To: a.righi@cineca.it
Organization: CINECA
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070604 Thunderbird/1.5.0.12 Mnenhy/0.7.5.666
To: Walter Tautz <wtautz@cs.uwaterloo.ca>
Subject: Re: [sisuite-users] cannot find .ROOTDEV /dev/ram under [BOOT]

Walter Tautz wrote:
> You're welcome Andrea. By the way, I noticed that si_mkautoinstallcd 
> does not support /local.cfg
> in the root of the cdrom. I tried it with the latest 4.1.6 version. I 
> noticed that the bug
> is reported here: 
> http://trac.systemimager.org/browser/trunk/sbin/si_mkautoinstallcd

You can put the local.cfg directly in the initrd using si_mkbootpackage
--config FILE. This works in any case and doesn't need a BOEL or UYOK
kernel with all the possible CD-ROM / floppy / USB-disk modules
statically built on it.

BTW I've seen that --config option is not documented in si_mkbootpackage
manpage. I'll fix it ASAP. Thanks for asking! ;-)

> 
> Walter
> 
> ps. It would be nice if one could specify the imageserver at the boot: 
> prompt in some manner.

Do mean interactively, right?

> Or when I run si_updateclient it would have been nice if it would assume 
> the --server is obeyed.

Yes, this would be interesting as well. And quite simple to implement,
just putting a file /etc/systemimager/IMAGESERVER in any imaged client
and look at the contents of this file would be enough to automatically
retrieve the image server address. It could be always overwritten by the
--server option anyway.

> Unfortunately our dhcp server is chosen which does not contain the 
> images.....I know it ought
> to but that's another story :-(

OK, now I've probably understood why you asked the local.cfg question...
;-)

-Andrea

-- WalterTautz - 16 Jun 2008