Articles‎ > ‎

Automating ASR Image creation

posted Dec 19, 2008, 5:39 AM by Philip Rinehart   [ updated Dec 19, 2008, 5:40 AM by Greg Neagle ]
Written by Richard Glaser   
Tuesday, 10 April 2007
In our environment, we manage our Mac OS X client file systems with radmind, but sometimes we run into issues that can only be resolved by zero-out data erase of the hard disk and re-imaging. To speed up the process we have created a script that updates a volume via radmind, creates a ASR image, and uploads it to a server that we use when re-imaging the clients. This article outlines the process, includes the script and launchd plist file.

Background


In our environment, we primarily manage around 500 Mac clients with radmind, but sometimes due to lower level hard disk issues we need to zero-erase & re-image the clients hard disk.The process of erasing a disk, partition, or volume by writing zeros to every sector is called zeroing. Zeroing finds bad sectors and maps them out of service, also known as sparing. When an attempt to write zeros to bad sectors fails, the bad sectors are both marked as occupied in the directory and added to the bad blocks file of the file system. Once the bad sectors have been spared, no attempt will ever be made to read-from or write-to them again.

The image “http://macenterprise.org/images/disk_utility-zero_out_data.jpg” cannot be displayed, because it contains errors.  

For details, see Apple Developer Connection Technical Note TN1150, "HFS Plus Volume Format."

From our experience, zero-out erasing hard disks has resolved issues with hard disks like I/O errors, kernel panics, etc. that wasn't resolved using a standard erase. So, as a standard procedure we zero-erase a hard disk when we re-imaging. Currently, we use NetRestore with pre/post scripts and read our radmind command files to automate image selection based on the client's radmind command file. If there is interest, we can post details in a future article.

To get the client up and working ASAP we have created Apple Software Restore [ASR] images based on our primary configurations:
 
  • Kiosk - PowerPC
  • Lab - Intel
  • Lab - PowerPC
  • Media Editing - PowerPC
 
We have many more configurations, like server, staff, etc. but based on distribution sizes & software we currently have these primary configurations.

The kiosk is a smallest image, roughly 1.5 GB; lab image, 36 GB and our media editing, 55 GB compressed.

For sheer speed, we decided to use the our primary configurations, instead of one base configuration and then use radmind to bring the client completely up-to-date. We still use radmind, but since radmind uses a file copy vs ASR's block copy, we try to have the ASR images distribute as much of the clients file system as possible, since block copy is much faster than file copy.

To keep the images up-to-date, we have setup imaging stations that use each of these primary configurations. Each imaging station has three volumes. First, the startup disk; second, the image source; and last the images volume where the images are saved.
 
  The image “http://macenterprise.org/images/imaging_station.jpg” cannot be displayed, because it contains errors.
 
Then weekly we update our source volume with radmind, and then create ASR image from the source volume, save it to the images volume, then upload it to our server we use with NetRestore to re-image clients. We use the same name for the images, so the NetRestore configuration remains the same.

Wow, you update your ASR images, weekly, that sounds a little obsessive. Yes, you are probably right, but we update our clients radmind file systems fairly often and many times each week and since its automated, it doesn't hurt us updating the ASR image each weekend, just in case we did update the client file system during that particular week.
 

ASR Creator Script


Below is the script, ASR Creator, that we use to update a volume with radmind, create the ASR image, and then upload it to a server we use to re-image clients.
 

#!/bin/sh
#
# ASR Creator V2
#
# Copyright (c) 2007 University of Utah Student Computing Labs.
# All Rights Reserved.
#
# Permission to use, copy, modify, and distribute this software and
# its documentation for any purpose and without fee is hereby granted,
# provided that the above copyright notice appears in all copies and
# that both that copyright notice and this permission notice appear
# in supporting documentation, and that the name of The University
# of Utah not be used in advertising or publicity pertaining to
# distribution of the software without specific, written prior
# permission. This software is supplied as is without expressed or
# implied warranties of any kind.

##################################
#Global Editable Variables

#Name of the volume that stores the radmind refresh
RADMIND_REFRESH="Refreshed"
#Location of the ASR Images
ASR_IMAGES_LOCATION="Images"
#Radmind server to use
RADMIND_SERVER="your.radmind.server.edu"
#Transcript Location
TRANS_LOCATION="/var/radmind/client"
#Who to email status updates too
MAIL_TO=" email@your.mailserver.edu"
#Where the log file is saved
PATH_TO_LOG_FILE="/var/log/asr_creator_log" 
 
##################################
#Global Non Editable Variables:

#BEFORE_TIME
#AFTER_TIME
#TIME_STAMP
#SET_DATE

##################################
#Functions:

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

set_box_type_name_f()
{
    BOX_TYPE=`grep '#ASR_Image_Name:' ${TRANS_LOCATION}/command.K | egrep -o "(lab_intel|lab_ppc|kiosk_ppc|video_ppc)"`
    if [ "$BOX_TYPE" = "" ];then
        my_logger_f "BoxType naming problem!"
        exit
    fi
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

my_logger_f()
{
    #Interdependencies:
    #before_time_f via global var BEFORE_TIME
    #after_time_f via global var AFTER_TIME
    
    #Global vars:
    #PATH_TO_LOG_FILE
    
    if [ "$1" == "-i" ];then
        echo "*****************************************" > "${PATH_TO_LOG_FILE}"
        echo "$2" >> "${PATH_TO_LOG_FILE}"
        echo "*****************************************" >> "${PATH_TO_LOG_FILE}"
        echo >> "${PATH_TO_LOG_FILE}"
        return
    fi
    
    if [ "$1" == "-n" ];then
        echo -n "$2" >> "${PATH_TO_LOG_FILE}"
        return
    fi
    
    if [ "$1" == "-b" ];then
        echo >> "${PATH_TO_LOG_FILE}"
        return
    fi
    
    if [ "$1" == "-t" ];then
        local TIME_TAKEN
        local DIV_SIXTY
        let "TIME_TAKEN = $AFTER_TIME - $BEFORE_TIME"
        let "DIV_SIXTY = TIME_TAKEN / 60"
        echo "${2}: $DIV_SIXTY minutes" >> "$PATH_TO_LOG_FILE"
        echo >> "${PATH_TO_LOG_FILE}"
        return
    fi
    
    echo "$1" >> "${PATH_TO_LOG_FILE}"
    echo >> "${PATH_TO_LOG_FILE}"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

#These two functions should be rewritten to be one function with different parameters

before_time_f()
{
    #Gloabl vars:
    #BEFORE_TIME
    
    let "BEFORE_TIME = `date +%s`"
}

after_time_f()
{
    #Global vars:
    #AFTER_TIME
    
    let "AFTER_TIME = `date +%s`"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

ktcheck_f()
{
    local KTCHECK_ERROR
    
    /usr/local/bin/ktcheck -c sha1 -h "$RADMIND_SERVER"
    
    KTCHECK_ERROR="$?"
    
    if [ "$KTCHECK_ERROR" -gt '1' ];then
        my_logger_f "ktcheck has an error, exiting script"
        exit 1
    fi
    
    return "$KTCHECK_ERROR"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

radmind_refresh()
{
    if (test ! -d "/Volumes/${RADMIND_REFRESH}") then
        disk_identifier_refreshed=`diskutil list | sed -n 's/.*Refreshed.*GB[[:blank:]]*//p'`
        diskutil mount $disk_identifier_refreshed
        if (test ! -d "/Volumes/${RADMIND_REFRESH}") then
            my_logger_f "You need a volume named \"${RADMIND_REFRESH}\""
            exit
        fi
    fi    

    ulimit -n 1024
    
    ktcheck_f
    
    #Verify CPU type vs. command file type
    typeset CPU_TYPE=`system_profiler | egrep "CPU Type" | egrep -o "(PowerPC|Intel)"`
    typeset COMMAND_FILE_TYPE=`egrep .*macosx.* /var/radmind/client/command.K | egrep -v "(neg|#)" | egrep -o "(ppc|intel)"`
    if [[ "${CPU_TYPE}" == "PowerPC" && "${COMMAND_FILE_TYPE}" == "intel" ]] || [[ "${CPU_TYPE}" == "Intel" && "${COMMAND_FILE_TYPE}" == "ppc" ]]
    then
        my_logger_f "Command file type does not match CPU type!!!"
        my_logger_f "CPU Type: '${CPU_TYPE}'"
        my_logger_f "Command file type: '${COMMAND_FILE_TYPE}'"
        exit
    fi
    
    #Make the negative base_load a positive
    #DEPENDS ON OUR NAMING CONVENTION
    sed 's/n os_base_macosx/p os_base_macosx/' "${TRANS_LOCATION}"/command.K > "${TRANS_LOCATION}"/temp_neg_to_pos
    mv "${TRANS_LOCATION}"/temp_neg_to_pos "${TRANS_LOCATION}"/command.K
    
    #Ignores permissions on the volume
    vsdbutil -a "/Volumes/${RADMIND_REFRESH}"

    before_time_f
    
    cd "/Volumes/${RADMIND_REFRESH}"
    /usr/local/bin/fsdiff -c sha1 -A ./ | /usr/local/bin/lapply -c sha1 -F -h ${RADMIND_SERVER}    
    if [ "$?" -gt '0' ];then
        ktcheck_f
        if [ "$?" != "1" ];then
            my_logger_f "lapply errors and command file has not been updated.  Please troubleshoot and try again"
            my_logger_f "Exiting"
            exit 1
        fi
        /usr/local/bin/fsdiff -c sha1 -A ./ | /usr/local/bin/lapply -c sha1 -F -h ${RADMIND_SERVER}
        if [ "$?" -gt '0' ];then
            my_logger_f "lapply errors multiple times.  Please troubleshoot and try again."
            my_logger_f "Exiting"
            exit 1
        fi
    fi
    
    after_time_f
    
    my_logger_f -t "Time lapply spent running"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

create_asr()
{
    if (test ! -d "/Volumes/${ASR_IMAGES_LOCATION}") then
        disk_identifier_Images=`diskutil list | sed -n 's/.*Images.*GB[[:blank:]]*//p'`
        diskutil mount $disk_identifier_Images    
        if (test ! -d "/Volumes/${ASR_IMAGES_LOCATION}") then
            my_logger_f "You need a volume named \"${ASR_IMAGES_LOCATION}\""
            exit
        fi
    fi
    
    #Checking to see if dmg's already exhist if they do it deletes them
    if (test -e "/Volumes/${ASR_IMAGES_LOCATION}/sparse_image.sparseimage") then
        rm "/Volumes/${ASR_IMAGES_LOCATION}/sparse_image.sparseimage"
    fi
    
    if (test -e "/Volumes/${ASR_IMAGES_LOCATION}/compressed_image.dmg") then
        mv "/Volumes/${ASR_IMAGES_LOCATION}/compressed_image.dmg" "/Volumes/${ASR_IMAGES_LOCATION}/old_compressed_image.dmg"
    fi
    
    #Now these lines create the disk images
    
    before_time_f
    
    hdiutil create -srcfolder "/Volumes/${RADMIND_REFRESH}/" -format SPARSE "/Volumes/${ASR_IMAGES_LOCATION}/sparse_image"
    
    after_time_f
    
    my_logger_f -t "Time spent creating ASR sparse image"
    
    before_time_f
    
    hdiutil convert "/Volumes/${ASR_IMAGES_LOCATION}/sparse_image.sparseimage" -format UDZO -o "/Volumes/${ASR_IMAGES_LOCATION}/compressed_image"
    
    after_time_f
    
    my_logger_f -t "Time spent creating compressed ASR image"
    
    
    before_time_f
    
    asr -imagescan -nostream "/Volumes/${ASR_IMAGES_LOCATION}/compressed_image.dmg"

    after_time_f
    
    my_logger_f -t "Time spent creating scanning compressed ASR image"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

#Function does not work yet need to see output of disk_verify() several times!
zero_out_HD()
{
    diskutil zeroDisk "${Radmind Refreshed}"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

upload_asr()
{
    mkdir /tmp/mount_point
    mount -t afp "afp://username: password@your.server.edu/share_point" /tmp/mount_point
    chmod -R 755 /tmp/mount_point
    
    typeset TIME_STAMP=`date "+%m.%d.%Y_%H:%M"`
    
    typeset NUMBER_OF_PREVIOUS=`ls /tmp/mount_point | grep -c "${BOX_TYPE}"`

    typeset BOOL_COUNT="yes"
        
    if [ "${NUMBER_OF_PREVIOUS}" -gt '0' ];then
        for i in `ls -rt /tmp/mount_point/*${BOX_TYPE}*`;do
            if [ "$BOOL_COUNT" == "yes" ];then
                rm $i
            fi
            BOOL_COUNT="no"
        done
    fi
    
    before_time_f
    
    cp "/Volumes/${ASR_IMAGES_LOCATION}/compressed_image.dmg" "/tmp/mount_point/${BOX_TYPE}.dmg"
    
    after_time_f
    
    my_logger_f -t "Time to copy image up to server"
    
    umount /tmp/mount_point
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

mail_status()
{
    cat "$PATH_TO_LOG_FILE" | mail -s "ASR Image Update $SET_DATE" "$MAIL_TO"    
}

disk_verify()
{
    if (test ! -d "/Volumes/${ASR_IMAGES_LOCATION}") then
        disk_identifier_Images=`diskutil list | sed -n 's/.*Images.*GB[[:blank:]]*//p'`
        diskutil mount $disk_identifier_Images    
        if (test ! -d "/Volumes/${ASR_IMAGES_LOCATION}") then
            my_logger_f "You need a volume named \"${ASR_IMAGES_LOCATION}\""
            exit
        fi
    fi
    
    if (test ! -d "/Volumes/${RADMIND_REFRESH}") then
        disk_identifier_refreshed=`diskutil list | sed -n 's/.*Refreshed.*GB[[:blank:]]*//p'`
        diskutil mount $disk_identifier_refreshed
        if (test ! -d "/Volumes/${RADMIND_REFRESH}") then
            my_logger_f "You need a volume named \"${RADMIND_REFRESH}\""
            exit
        fi
    fi

    typeset VOLUME_STATUS_1=`diskutil verifyVolume "/Volumes/$RADMIND_REFRESH"`
    
    my_logger_f "The volume $RADMIND_REFRESH had this exit status from diskutil verifyVolume:"
    my_logger_f "$VOLUME_STATUS_1"
    
    typeset VOLUME_STATUS_2=`diskutil verifyVolume "/Volumes/$ASR_IMAGES_LOCATION"`
    
    my_logger_f "The volume $ASR_IMAGES_LOCATION had this exit status from diskutil verifyVolume:"
    my_logger_f "$VOLUME_STATUS_2"
}

#////////////////////////////////////////////////////////////////////////////////////////
#////////////////////////////////////////////////////////////////////////////////////////

##########################
#Main 

my_logger_f -i "ASR Creator Log"
my_logger_f -n "Script ran on: "
SET_DATE=`date`
my_logger_f "$SET_DATE"

disk_verify

my_logger_f "Finished disk_verify"

radmind_refresh

my_loffer_f "Finished radmind_refresh"

set_box_type_name_f

my_logger_f "Box Type: $BOX_TYPE"

create_asr

my_logger_f "Finished create_asr"

upload_asr

my_logger_f "Finished upload_asr"

mail_status

 

Scheduling


Under Mac OS X 10.4, the new launchd process invokes each script on a schedule specified in a script-specific property list (.plist file) stored in the /System/Library/LaunchDaemons directory.
 
We use the following the following plist file with launchd to run the above script every Saturday.

 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>UserName</key>
    <string>root</string>
    <key>Label</key>
    <string>edu.utah.scl.imagecreator</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/asr_creator_v1</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Weekday</key>
        <integer>6</integer>
        <key>Hour</key>
        <integer>0</integer>
        <key>Minute</key>
        <integer>45</integer>
    </dict>
</dict>
</plist>

Last Updated ( Sunday, 20 May 2007 )
Comments