-- MikePatterson - 13 Oct 2004

New machine, I scooped a dual P3 from Phil. I had started noting what I did in XhierOnFreeBSD, but the hard disk barfed and the previous notes were getting cluttered, so I'm starting again.

Partitions set up thusly:

• / 6G
• swap 2G
• /usr 10G
• /fsys1 19G

I may have wanted to reverse /usr and /fsys1. Oh well, I can probably always score more disk.

First, install base minimal system. Mount torres:/usr/ports and install the 44bsd-rdist, cvsup-without-gui, and rsync packages. Then cvsup src and ports from freebsd.uwaterloo.ca. Copy OATES config from torres to IPFILTER, copy torres' make.conf, and do a make buildworld.

16 October 2004

Allow inetd to start in rc.conf (made some other changes too, hurray for RCS), allow rsh/rlogin/rexec and set up .rhosts/.shosts to allow various archmasters in. Set rc.conf to allow linux emulation.

Create the standard xhier directory structure.

Hrm, I'd previously linked /usr/ports to /fsys1/ports. But make index doesn't seem to like this. I'd cvsupped with ports still really living in /usr. It's pissed off about that. So rm -r /fsys1/ports and re-cvsup.

With that done, let's turn our attention to xh-first-timing it. I think debian31 is a better starting point than debian30, so make a temp change on cscf.cs to tell debian31 that it's the master for freebsd5. dist that to debian31. Can't xh-first time, we get "rshd: Login incorrect". This is because we have to change /etc/pam.d/rsh:

#
# $FreeBSD: src/etc/pam.d/rsh,v 1.5 2003/02/10 00:50:03 des Exp $
#
# PAM configuration for the "rsh" service
#

# auth
auth            required        pam_nologin.so          no_warn
auth            required        pam_rhosts.so           no_warn allow_root

# account
account         required        pam_unix.so

# session
session         required        pam_permit.so

# password
password        required        pam_deny.so

ie, note the allow_root on the pam_rhosts.so line.

Now xh-first-time fails like this:

freebsd5# ./xh-check
ELF interpreter /compat/linux/lib/ld-linux.so.2 not found
Abort
freebsd5#

so linux_enable in rc.conf isn't sufficient. cd /usr/ports/emulators/linux_base-debian and make package-recursive. It's still not keen on making packages, but it does create them.

Now xh-first-time will start running, at least, but:

xh-first-time FYI: (if it's not there)
xh-first-time FYI: Crude check for missing NFS mounts.
xh-first-time FYI: Distributing the real "xhier" package.
xh-first-time FYI: Distributing the real "mfcf-basics" package.
xh-first-time FYI: Distributing the real "os-extras" package.
poll: protocol failure in circuit setup
debian31:~#

http://www.redhat.com/archives/redhat-list/2001-August/msg00858.html talks about this a bit. Walter might also know about it; he posted to a debian group in 2002 asking about this.

-- MikePatterson - 26 Oct 2004

somehow some binaries made their way over:

freebsd5# pwd
/fsys1/.software/arch/xhier/bin
freebsd5# ./patch
./patch: /lib/libc.so.6: version `GLIBC_2.3' not found (required by ./patch)
freebsd5# ldd patch
patch:
patch: /lib/libc.so.6: version `GLIBC_2.3' not found (required by patch)
        libc.so.6 => /lib/libc.so.6 (0x28067000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x28051000)
freebsd5#

those are debian31 binaries. But copy patch from debian30, and it runs ok, or at least:

freebsd5# pwd
/usr/home/mpatters
freebsd5# ldd patch
patch:
        libc.so.6 => /lib/libc.so.6 (0x28067000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x28051000)
freebsd5# 

So let's try first-timing this as a client of debian30.

1>(root)@debian30.math[106]% xh-first-time -v freebsd5.cscf
blahblah
xh-distribute FYI: rdist output of the "xhier" package from "debian30.math" follows:
  updating host freebsd5.cscf
    Note: freebsd5.cscf: '/.software/admin/xhier/./config/admin/packages': unknown group 'none'; clearing setgid
bleebloo
    special: /tmp/xh-install_23870/out: No such file or directory. 
    special: xh-install: installation of "mfcf-basics" 
    special: xh-install:   using "/software/mfcf-basics/.admin/Install" 
    special: xh-install:   failed -- exit status 1 
    special: sed: /tmp/xh-install_23870/out: No such file or directory 
    special: xh-install: Cannot echo message into "/software/xhier/data/logs/installation/mfcf-basics"
      
    special: cat: /tmp/xh-install_23870/out: No such file or directory 
xh-first-time FYI: Distributing the real "os-extras" package.
    special: cat: /tmp/xh-install_23912/out: No such file or directory 
xh-first-time FYI: Updating our copy of the requests on "freebsd5.cscf".
xh-transfer-requests: "/software/xhier/servers/xh-gather-requests" failed
xh-transfer-requests:   on host "freebsd5.cscf":
xh-transfer-requests:     /software/xhier/servers/xh-gather-lists: Exec format error. Binary file not executable.
1>(root)@debian30.math[107]% 

So two problems: no group none (minor) and it's still choking on some binaries (major).

freebsd5# file xh-gather-lists 
xh-gather-lists: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.0.30, statically linked, stripped

And indeed, a simple hello world program linked statically hoarks up. I must be missing something with linux binary compatibility.

Read http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/linuxemu-lbc-install.html and it says something about brandelf.

freebsd5# brandelf -t Linux ./hello
freebsd5# ./hello
Hello world!
freebsd5# 

So I need to run brandelf against every single binary xhier distributes. Great. I did:

freebsd5# cd /.software/arch
freebsd5# ls -l
total 6
drwxr-xr-x  7 root  wheel  512 Oct 26 19:31 mfcf-basics
drwxr-xr-x  7 root  wheel  512 Oct 26 19:31 os-extras
drwxr-xr-x  7 root  wheel  512 Oct 26 19:31 xhier
freebsd5# foreach i ( */bin/* */maintenance/* */servers/* ) 
foreach? brandelf -t Linux $i
foreach? end

and it kicked and screamed a lot about config files and such, but hey. I also added a group "none:x:101:". Now, the machine is only half-xh-first-timed. Witness:

I'll try just "xh-distribute -h freebsd5.cscf xhier" - no, that rdists the binaries too. suck.

What if we modified the distribution scripts to call brandelf if the target is a FreeBSD machine? Just temporarily would be sufficient.