2009.5.21 updated by R. Saito

Current status: finished now we can use the machine as Unix machine

Current problem: it seems that /nfsboot/root should be made for each PXE client.

http://pre-dawn.net/hiki/?DisklessCluster

http://vision.kuee.kyoto-u.ac.jp/~nob/doc/diskless/diskless.html

Current known problem:

A new page for Linux cluster (newtube) is now open.

http://flex.phys.tohoku.ac.jp/english/pukiwiki-e/index.php?PXE%20Server%20for%20New%20Tube%20(Open)

http://www15.big.or.jp/~yamamori/sun/pxe/nic.html http://docs.fedoraproject.org/install-guide/f10/

We want to make a diskless computing system so that we need a PXE Server. PXE stands for "Pre-boot eXecution Environment". PXE is a special extension of services provided by the Dynamic Host Configuration Protocol (DHCP). It uses a Trivial File Transfer Protocol (TFTP) server to provide minimal boot to a network client. Let's try to configure it!


Contents

Set a small subnet for testing PXE

What we need

Checking

Network setting

SSH check

Setting for the original Linux machine from which we copy the system

172.17.4.178:# yum install busybox-anaconda

Setting for the PXE server machine

SElinux shoule be permissive mode.

Install DHCP and TFTP Server etc

TFTP Configuration

# default: off
# description: The tftp server serves files using the trivial file transfer \
#       protocol.  The tftp protocol is often used to boot diskless \
#       workstations, download configuration files to network-aware printers, \
#       and to start the installation process for some operating systems.
service tftp
{
       disable                 = no
       socket_type             = dgram
       protocol                = udp
       wait                    = yes
       user                    = root
       server                  = /usr/sbin/in.tftpd
       server_args             = -s /tftpboot
       per_source              = 11
       cps                     = 100 2
       flags                   = IPv4
}
# service xinetd restart

Copy system data for booting

# mkdir /nfsboot
# rsync -v -a 
--exclude='/proc/*' --exclude='/sys/*' --exclude='/dev/*'\
--exclude='/media/*'   --exclude='/tmp/*' --exclude='/misc/*'\
/ nfsboot
# rsync -v -a -e ssh \
--exclude='/proc/*' --exclude='/sys/*' --exclude='/dev/*'\
--exclude='/media/*'   --exclude='/tmp/*' --exclude='/misc/*'\
172.17.4.178:/ /nfsboot

DHCP Configuration

http://d.hatena.ne.jp/adsaria/20080209

# /sbin/service dhcpd restart
# chkconfig dhcpd on
(the last line is for activating dhcp on booting process)

NFS configuration

# Port rpc.mountd should listen on.
  1. MOUNTD_PORT=892

http://d.hatena.ne.jp/setq/20090312/1236853536

[root@rsaito-necPC rsaito]# cat /etc/exports

PXE server configuration

#system-config-netboot
Name fedora_10_32bit
Explanation fedora_10_32bit
server 192.168.1.10
directory /nfsboot
IP address/subnet 255.255.255.0  <- we use subnet information
operating system fedora_10_32bit
Other part should be as it is.
# cd /tftpboot/linux-install/pxelinux.cfg
# mv default default.org
# ln -s FFFFFF00 01-00-21-70-c9-eb-60
[root@rsaito-necPC pxelinux.cfg]# cat 01-00-21-70-c9-eb-60

label fedora_10_32bit
   kernel fedora_10_32bit/vmlinuz
   append  initrd=fedora_10_32bit/initrd.img root=/dev/ram0 init=disklessrc
 NFSROOT=192.168.1.10:/nfsboot ramdisk_size=24753 ETHERNET=eth0  
 SNAPSHOT=255.255.255.0

Set /nfsboot/root/etc

[root@rsaito-necPC sysconfig]# cat network
NETWORKING=yes
HOSTNAME=pxe-fefoda10-dell
[root@rsaito-necPC network-scripts]# cat ifcfg-eth0
# Broadcom Corporation NetXtreme BCM5705_2 Gigabit Ethernet
TYPE=Ethernet
DEVICE=eth0
HWADDR=00:21:70:c9:eb:60  <---- set PXE client Mac address
BOOTPROTO=none            <---- change dhcp to none
NETMASK=255.255.255.0     <---- set subnet mask
IPADDR=192.168.1.30       <---- set fixed IP address
GATEWAY=192.168.1.10      <---- now PXE server is gateway 172.17.4.0
ONBOOT=yes
USERCTL=yes
PEERDNS=yes
USERCTL=no
PEERDNS=yes
IPV6INIT=no
NM_CONTROLLED=no

Set PXE client


Problems and Solutions

P: init is not found. /disklessrc is not found.

S: use system-config-netboot

S: init should be in /tftpboot/linux-install/32bit_fedora_10/initrd.omg

S: the name of "32bit_fedora_10" is specified by system-config-netboot.

P: I can find initrd.img but not init itself.

S: If you can expand initrd.img as above, you will find init.

S: disklessrc is not generated by system-config-netboot

P: nfs is not mountd

S: setting fire wall is important as above.

S: you can check the mount the /nfsboot from the other Linux machine.

P: dhcpd does not work

S: dhcpd is not running. /etc/rc5.d/S65dpcpd restart.

S: after dhcpd restarts, /etc/rc5.d/S56xined restart.

S: check dhcp function by Windows machine.

P: /etc/resolve.conf is not correct.

S: resolve.conf is generated the host computor which is copied by rsync.

P: some file can not be downloaded by Selinux

S: stop SeLinux for a moment and check it again.

S: we should change the file type. checked by ls -Z.

P: How to know the contents initrd.img which is generated by system-config-netboot?

S: The solution is given by using cpio command

P: X window is not running and the OS can not be used.

S: edit /nfsboot/root/etc/inittab then change ranlevel to 1,

P: X window should be adjusted to new computor

S: The following seems to work.

  1. Run X -configure which makes /root/xorg.conf.new
  2. cp /root/xorg.conf.new /etc/X11/xorg.conf
  3. xorgcfg  -texmode
  4. startx
  5  if it does not work, edit /etc/X11/xorg.conf (Change driver from ati to vesa?
  6. startx again 

S: If it does not work, try to do the following, too.

# Xorg -configure

Then /root/xorg.conf.new generates. Check this files works well

# X -config /root/xorg.conf.new

If X starts nicely, press [Ctrl]+[Alt]+[F1] to go back to console.

# mv /etc/X11/xorg.conf /etc/X11/xorg.conf.old
# mv /root/xorg.conf.new /etc/X11/xorg.conf

That is all. Reboot this.

http://argon.bus.osaka-cu.ac.jp/index.php?Xorg%20%A4%CE%C0%DF%C4%EA

P: from PXE client, we can not go to 172.17.4.0 network

S: in PXE server (192.168.1.10) Edit /etc/sysctl.conf

S: IP forwarding is necessary.

net.ipv4.ip_forward = 1
for runntine
# echo 1 > /proc/sys/net/ipv4/ip_forward
for setting
# sysctl -p
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
Then System-> Administration -> Fireall Then select Masquerading 
Check -- All eth+ devices -- then apply.
# service network restart

P: SuperMicro?, X7DCA-L motherboad does not have PXE boot function?

S: Edit BIOS Advanced -> PCI configuration then NIC boot can be enabled.

S: Then set BOOT order by pushing "x" or "+" or "-".

P: What kind NIC does support PXE or IAS

http://www.intel.com/products/desktop/adapters/pro1000gt/pro1000gt-overview.htm

Links for references

http://takedarts.jp/index.php?%A5%C7%A5%A3%A5%B9%A5%AF%A5%EC%A5%B9%B4%C4%B6%AD%A4%CE%B9%BD%C3%DB

http://www.linux.or.jp/JF/JFdocs/Authentication-Gateway-HOWTO/setup.html

http://tomo.ac/goodstream/fedoracore/fc3/fw-fc3.html

http://lumber-mill.co.jp/gallery/view/tips/linux/fedora

Directrories and files

PXE server

/tftpboot

/nfsboot

PXE client

When we try to find "disklessrc fedora", we found the following Web site.

http://d.hatena.ne.jp/adsaria/20080131/1201792574 http://wikiwiki.jp/disklessfun/?FrontPage


The following is the statements which are evetually not used.

Edit initrd.img for NFS mount

http://www.atmarkit.co.jp/flinux/rensai/linuxtips/a021pxediskless.html

http://www.devdrv.co.jp/linux/cpio-initrd-format.htm

http://blogs.yahoo.co.jp/natto_heaven/11513467.html

# cp /tftpboot/linux-install/f9-diskless/initrd.img /tmp
# cd /tmp
# mkdir initrd.test
# cd initrd.test
# cp ../initrd.img .
# mkdir initrd
# cd initrd
# zcat ../initrd.img | cpio -i -c
# cd /sbin
# cp /sbin/mount.nfs .
# cp /sbin/umount.nfs .
# cp /sbin/mount.nfs4 .
# cp /sbin/umount.nfs4 .
# cd ..
# find . | cpio --quiet -c -o | gzip -c > ../initrd-new.img
# cd /tftpboot/linux-install/F9-diskless/
# mv initrd.img initrd.img.org
# cp /tmp/initrd-test/initrd-new.img ./initrd.img

PXE server configuration (old)

[root@rsaito-necPC rc5.d]# cd /tftpboot/f9/snapshot/
[root@rsaito-necPC snapshot]# ls -l
total 12
drwxrwxr-x 8 root root 4096 2009-05-09 08:57 255
-rw-r--r-- 1 root root 1070 2008-08-26 19:09 files
-rw-r--r-- 1 root root    7 2009-05-09 08:52 files.custom
[root@rsaito-necPC snapshot]# cat files.custom
/home/

If Selinux complains something, chcon command can be used.

# cd /tftpboot
# chcon -R -t type .
# ls -Z .

Front page   Edit Freeze Diff Backup Attach Copy Rename Reload   New List of pages Search Recent changes   Help   RSS of recent changes
Last-modified: 2009-05-23 (Sat) 23:09:02