Michal Suszycki mike (at) wizard (dot) ae (dot) krakow (dot) pl 21.07.2005

Solaris unattended installation (PXE and custom JumpStart) with Linux server (TFTP, DHCP and NFS)

This document describes automatic, unattended installation of Solaris (Nevada b14 or later that comes with GRUB and pxegrub) on x86 machines using PXE booting and Linux machine serving DHCP and all necessary configuration files through TFTP and installation files (basically this is a copy of Solaris installation dvd) as NFS export.

Nevada is Sun's code name for the Solaris release after Solaris 10. OpenSolaris and Nevada are built from the same source base. Take a look at open solaris and read more about it.

Solaris has support for networked installation but unfortunately almost all documentation (as well as shell scripts) assumes that your installation server will be Solaris. Preparing Solaris based installation server is rather simple because Solaris comes with scripts, like setup_install_server or add_install_client that can configure DHCP server, copy installation files etc. Unfortunately those scripts cannot be run on Linux machines because they need some solaris specific tools. Fortunately you can prepare your Linux server without those scripts.
How does it work
Machine that is capable of network booting (so called PXE boot) and has it enabled in the BIOS, doesn't try to boot from local hard disk or CD, but instead it uses DHCP to obtain IP address. It sends broadcast asking for IP, netmask, default router, and possibly many other informations that you can specify in the DHCP server configuration file. One of this information can be name of the file that will be downloaded by the machine through TFTP and executed immediately. Usually this file is a NBP (Network Bootstrap Program) that perform rest of the booting process. Whole unattended Solaris installation goes like this:

  • Enable PXE boot (network booting) in the BIOS of the machine and boot it
  • Machine sends DHCP broadcast and waits for a response
  • It gets response from our Linux server with such informations like IP address, netmask and name of the file to download and execute (it's a 'pxegrub' file that is Network Bootstrap Program)
  • It configures network card and then downloads pxegrub from our Linux TFTP server
  • After pxegrub is downloaded it is executed
  • pxegrub then uses TFTP to obtain configuration files (like menu.lst file that contains boot options) from our Linux server. Once appriopriate files are successfully downloaded by pxegrub we should see nice grub menu.
  • After choosing solaris installation from the menu or some timeout (it should have timeout because we want unattended installation) grub loads Solaris kernel with options that tells it to install Solaris system using NFS for getting distribution files and so called "Custom Jumpstart". Custom Jumpstart lets you specify all the necessary information, for example disk layout, network configuration and list of packages in a file that will be readed during installation and will provide all kind of information installer needs to perform his job without asking any question.
  • Solaris kernel boots and performs its magic stuff, like detecting devices etc.
  • Installation script gets running and tries to obtain IP address from DHCP
  • Then it tries to NFS mount directory exported from our Linux server to download Jumpstart configuration files
  • According to information in these files it starts installing Solaris using distribution files that resides on our exported NFS directory.
  • After successfull instalation Solaris reboots and you should see your machine booting from local disk and showing you grub menu.
    What is needed Fortunately everything we need can be obtained from internet (like pxegrub or Solaris installation dvd/cd iso). Linux with DHCP server, TFTP server and NFS (can be version 3) server. pxegrub file, Solaris kernel file, Solaris x86.miniroot file and Solaris installation files.

    Downloading Solaris

    Go to opensolaris downloads page and choose Step 3b to download Solaris Express:Community Release from javashoplm.sun.com website. Download is free of charge but unfortunately you have to register (and it can be painful). Download 4 CD iso images or choose to download DVD image and later discover that this DVD iso image is also divided into 4 parts ;-) We will use iso images to obtains some important files (like Solaris kernel and x86.miniroot) mounting it with loopback on our Linux machine.

    Lets asume that our Linux machine has 192.168.1.1 address. It sould not be important what type of distribution you are running. Our DHCP server will be ISC DHCP (standard DHCP on probably every linux distribution)

    DHCP configuration

    
    option grubmenu code 150 = text;
    
    host solaris
    {
           hardware ethernet 00:02:a5:08:73:4d;
    	
            option grubmenu "solaris/menu.lst";
    	fixed-address 192.168.1.2;
    	filename "solaris/pxegrub";
    }
    

    Unfortunately you can expect problems with pxegrub and dhcp communication - for me it didn't work for the first time. Pxegrub couldn't get IP number and you could see error message in a dhcpd log file saying:

    dhcpd: parse_option_buffer: option dhcp-parameter-request-list (6) larger than buffer.
    
    I forced dhcpd to work by changing 147 line in common/options.c line 147 changed to if (offset + len + 1 > length) (can break other things?) but probably it was some misconfiguration rather that bug becase later on my pxegrub worked without any problems with standard isc-dhcp.

    NFS

    We will create and export "/export/solaris" directory that holds JumpStart configuration file and Solaris distribution installation files.
    Create /export/solaris directory and one subdirectory: jumpstart

     /export/solaris  (Solaris installation files copied from CD/DVD iso)
    	     |
    	      --jumpstart/ (JumpStart configuration files)
     
    Export /export/solaris/ directory (change /etc/exports, exportfs -a) and run NFS server. We will create JumpStart configuration file inside jumpstart directory later - see below. Loopback mount CD iso, one by one, and copy all of its contents to /export/solaris/. If you've got one DVD iso it's simple because you do one mount on loop device and then just copy all dvd content.

    TFTP

    TFTP is used twice during installation: first on the very early stage when PC performs PXE booting and downloads Network Bootstrap Program and second when NBP (that is GRUB) downloads all the files it needs (for example menu description with booting options as well as solaris kernel and x86.miniroot file).

    Let's assume that root directory of our TFTP server will be /tftp.
    Download atftp (try debian download link because mamalinux.com seems to be dead) server or any other TFTP server for Linux but please remember about some TFTP/PXE issues that can prevent your system from PXE booting.

    Update: Some people expericence problems with TFTP server (I think they are using default TFTP server, that I guess isn't atftp on most linux distributions). Usually after first, initial PXE stage they cannot see pxegrub menu or they see a message that binary is in a wrong format and cannot be executed (i.e "Invalid or unsupported executable format"). Please download atftp server and try it first, you can save yourself lots of time.

    Basically you can have problems with some PXE clients that are not able to negotiate block size option. In such case your PC won't be able to PXE boot. That's why the version of tftp you run can be very important. I've been using atftpd and it's known to work good with PXE.

    Create /tftp directory and run atftpd:

    /usr/sbin/atftpd --daemon --no-multicast --group nobody --tftpd-timeout 0 -m 1000 /tftp

    Create /tftp/solaris directory. We will put there all the files needed for booting.
    Mount first Solaris CD iso (or DVD iso) on loopback and copy multiboot and x86.miniroot (compressed image of UFS filesystem) from boot/ directory to /tftp/solaris. Also copy pxegrub from grub/ directory

    mount /root/download/opensolaris/sol-nv-b18-x86-v1.iso /mnt/solaris -o loop
    cd /mnt/solaris/boot ; cp multiboot x86.miniroot /tftp/solaris
    cp grub/pxegrub /tftp/solaris

    Since we have iso mounted copy content of the Solaris_11/Misc/jumpstart_sample to some place. These files can be handy when writing JumpStart configuration files.

    Create /tftp/solaris/menu.lst that contains grub options. Content of this file should look like this:

    timeout=15
    title Solaris
      kernel /solaris/multiboot kernel/unix -v -m verbose install \
         nfs://192.168.1.1/export/solaris/jumpstart/config.tar -B install_media=192.168.1.1:/export/solaris/
      module /solaris/x86.miniroot
    
    Please take a note that grub cannot handle '\' and 'kernel' line should not be splitted. These options are very important for proper network installation.

    Basically it tells GRUB to run multiboot file (that load Solaris kernel) with verbose option (you can replace 'verbose' with 'debug' word to see many informations on the screen), to look for config.tar file on NFS mount (JumpStart configuration files inside this archive) and to get Solaris installation files from our NFS server that are located inside /export/solaris/ directory. During Solaris kernel booting or JumpStart installation you can use Ctrl-s to pause messages on the screen and Ctrl-q to resume. It is very useful especially if you use 'debug' option instead of 'verbose'.

    JumpStart configuration files

    4 configuration files are needed, some examples can be found in jumpstart_sample directory we have copied earlier:

  • rules file describes what profiles should be used to particular systems. Our rules file is the simplest one and matches any system. This is the content of the file:
    # The following rule matches any system:
    # rule_keyword   rule_value   begin     profile    finish
    any              -               -      any_machine  -
    
  • rules.ok is created from rules by running Solaris check script. We don't have to (and can't) use the script under Linux. Manually we can strip all the unecessary lines (like comments etc.) from rules and add one final line that contains checksum. Our rules.ok contains only two lines then:
    any - - any_machine -
    # version=2 checksum=num
    

    Actually 'num' should be computed by Solaris 'sum' program but it seems that this is irrelevant.

  • any_machine file that defines our profile for every machine - this is 'real' JumpStart configuration, that describes how to partition your system what to install etc. Once again, please take a look at contents of jumpstart_sample directory. After all your DHCP/TFTP configuration works fine then everything that needs to be changed to customize your Solaris installation will be this profile file. For now we stick to this:

    install_type    initial_install
    system_type     server
    #system_type    standalone
    partitioning    default
    cluster         SUNWCuser
    cluster         SUNWCxgl delete
    package         SUNWaudmo add
    filesys         any 40 swap
    filesys         any 50 /op
    
  • sysidcfg file hold information about timezone, ipv6, language and so one. If JumpStart can't find this file then it will ask you questions during install, so it won't be unattended installation. Also if some of the needed options are missing then you will end with a interactive installation. So remember: if your JumpStart is still interactive then it is very likely that you sysidcfg file doesn't contain all necessary information. Minimal sysidcfg looks like this (please look at the network interface name - it should corespond to your network device):
    system_locale=en_US
    timezone=US/Pacific
    terminal=sun-cmd
    timeserver=localhost
    network_interface=iprb0 {protocol_ipv6=no}
    name_service=NONE
    security_policy=NONE
    

    Solaris JumpStart program needs all this files to be packed all together - this is important. Take a look at grub configuration above. You can see nfs://192.168.1.1/export/solaris/jumpstart/config.tar line, telling JumpStart where to find all configuration it needs packed into one tar file.

    tar c rules rules.ok any_machine sysidcfg > /export/solaris/jumpstart/config.tar.

    Please remember that names are important. JumpStart looks for 'rules', 'rules.ok', 'sysidcfg' and profile file inside tar archive.

    That's it. Enable network booting in some ready-to-install machine BIOS and boot it. If you have all these configuration and TFTP, DHCP and NFS server running and you're lucky you will be able to install Solaris without any intervention. Now you can change any_machine to customize your Solaris installation and look at postinstall scripts to set up root password and other things.

    Links:

  • x86: Performing a Custom JumpStart Installation
  • Installing With Custom JumpStart (Examples)
  • Solaris Tips & Tricks