Expect is an automation and testing tool used to automate a process that receives interactive commands. If you can connect to any machine using ssh, telnet, ftp, etc then you can automate the process with an expect script. This works even with local programs where you would have to interact with a script or program on the command line.
When I first starting using Expect I wanted a short but detailed description on how Expect performed its magic. I had to read the Expect O'Reilly book to gather all of the information I needed, but that was not very short. I'm going to give my shortened version of how it works, and answer the questions I had at the time I was learning it.
Expect is written as a Tcl extension and can be installed on almost any Unix type distribution. The package on most distributions should just be called "expect" so look for that when installing it. For example on Linux distros that use apt-get to install packages just issue "sudo apt-get install expect".
After Expect is installed find where it was installed to by issuing the command "which expect". This should give you the path to the binary. If not try becoming root and issuing the same command. We need to know the path of the binary as it is the first line of our script. Note this path as we will be needing it soon.
Expect scripts are written like many other scripts. Like Bash or Perl the binary is called at the top "#!/usr/bin/expect" and the scripts commands below it. Expect is an extention to the Tcl scripting language so it uses all of Tcl's syntax.
Most scripts begin by spawning the process they want to interact with. This is done with the "spawn" command. Like "spawn ssh user@host or spawn ftp host". Then you can begin checking for output with the command "expect". Expect uses regular expressions to find patterns in the output, and when it matches a pattern you send it command with the "send" command. In the simplest form that is how expect works. You start the process and look for patterns in the output. Then send commands based on the matches from the output.
Below I will list the commands (with descriptions) I use the most in Expect. They should get you through the simple scripts that most people need.
expect - The expect command will wait until one of the patterns given matches the output of a spawned process, a specified time period has passed, or an end-of-file is seen. Since you can give the expect command multiple things to match on you can have it do different things when it matches. The first sentence bares repeating, but I will try to expand on it. Expect will constantly loop through the multiple given patterns until a match is found. Expect will match the first pattern it finds in the order you specified them. When it finds a match it will execute any commands given and keep following any more nested commands until it hits the last command. It will never return to the calling block. It then moves onto the next part of the script.
If at any time there are no matches it will timeout. If the pattern keyword (match word) "timeout" is used you can have it perform an action when the timeout happens.
If an eof is returned it will exit the spawned process and move on. If you use "eof" as a pattern keyword then you can have it also perform an action if an eof happens.
You can also use the pattern keyword called "default" that can perform an action if either eof or timeout are reached. We will see how to use this to make great error messages later.
Before we get to actual Expect script examples let me lay out how I design my scripts and what I have learned works best for me.
The first example will be logging into a linux box with ssh to see if the given accounts password works. By doing this we are also checking other things like if your machine is up and if you can login to it and get a prompt back. So in actuality this script does a bunch of things.
#!/usr/bin/expect
set timeout 9
set username [lindex $argv 0]
set password [lindex $argv 1]
set hostname [lindex $argv 2]
log_user 0
if {[llength $argv] == 0} {
send_user "Usage: scriptname username \'password\' hostname\n"
exit 1
}
send_user "\n#####\n# $hostname\n#####\n"
spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname
expect {
timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
"*assword"
}
send "$password\r"
expect {
timeout { send_user "\nLogin failed. Password incorrect.\n"; exit 1}
"*\$ "
}
send_user "\nPassword is correct\n"
send "exit\r"
close
Let me describe what is going on in this script line by line.
Line 1 executes the start of the script with the path to the expect binary.
Line 2 sets a timeout for each expect statement to 9 seconds.
Line 3,4,5 set variables for username, password, and hostname which are taken from the command line for when the script is run.
Line 6 turns off the output to STDOUT (printing the output to the screen). If we remove this line you will see the whole login process.
Line 8 is the first if statement. It checks to see if any arguments have been given to the script and if not it will print out how to use the script. If you do this with every script you will always be able to remind yourself of how to use it.
Line 13 prints a banner with the hosts name.
Line 15 starts the ssh process and turns on quiet mode and turns off host key checking (only turn off key checking on a trusted network). It uses the variables we give it on the command line to ssh to the host.
Line 17 is our first expect statement. First we set our timeout message in case our expect statement can not find a match for what its looking for. According to our timeout we set this will occur in 9 seconds. This means it will loop through the ssh output for 9 seconds and if it can not find a match it will print this error an then exit as we have told it to do. We also set our EOF value here. If our ssh session does not connect or gets disconnected EOF will be returned and our error message will print. This will then exit like we tell it to do. The last and most important part of this statement is line 20 the pattern to match which is "*assword". This is a regular expression that looks for 0 or more characters with the letters assword after it. Many password login prompts for ssh show up as "Password: " so this should match that. If it does match then it moves to the next statement. If it does not match we will hit the timeout, then our error message will print and the script will exit.
Line 23 will actually send the password we specified on the command line to ssh. We know we can send the password now because we verified we had a matching password prompt.
Line 25 starts our next expect statement. We set the timeout error message to say that the login must be incorrect. Why? Because the matching loop has started again looking for output. This time we are looking for a command prompt. If we can not match a command prompt in the output then that means we never got one. That means our password must have failed. Line 27 after the timeout line is the matching expression "*\$ ". The prompt I was matching for looked like this "user@hostname:~$ ". So the match says 0 or more of any character (*) and then a dollar sign and then a space. Notice the dollar sign has a \ before it. We have to escape this because it is a special character for regular expressions which means end of line match. If we don't put the \ before the $ it will still work, but the match is to broad and matches anything. We want it to be a specific match to be sure it is a correct prompt so we want to look for any character with a $ and a space. Your prompts are likely to vary so change your matching expression to meet your needs.
Line 30 sends output to the screen to tell the person the we got a correct prompt back. We can send this because we know we successfully got this far and our prompt matched from the previous expect statement.
Line 31 and 32. Line 31 sends the exit command to the linux machine. Line 32 then closes the connection to the ssh process. That's it!
This script uses command line arguments for a reason. If you can build your scripts like this then you can use it to check many different accounts across many different hosts. It does not lock you into one host and account per script. If you have a list of hosts you can use it in a bash loop and check many machines at once. Like: for x in server1 server2; do ./script.exp opt1 opt2 $x; done
Notice we have to put the password on the command line with this script. First off make sure you put your passwords in single quotes. That removes any issue with special characters being interpreted by your shell. Second, is a bad idea to use password prompts on the command line because many shells (like bash) keep a history of your commands (like ~/.bash_history). That means your password is sitting in a file in plaintext in your home dir. That's bad. To help mitigate this with bash you can put the following line in your ~/.bashrc file or ~/.profile. "export HISTCONTROL=ignorespace" . This will then execute on login and after it is run will allow you to put a space before any command and it will not be kept in your history. The password will still be on your screen after you have run it but usually this is scrolled off soon after the script is run. Typing "clear" will scroll it off the current screen, but just keep it in mind.
This is a more complicated example that has to deal with different situations depending on the device type. The following script will SSH into 2 different types of Cisco routers and switches and change the SNMP password. The reason this is more complicated is that the 2 different types of devices uses 2 different operating systems with some similar and some different commands.
#!/usr/bin/expect
set timeout 15
set hostname [lindex $argv 0]
set snmpshapass [lindex $argv 1]
set snmpaespass [lindex $argv 2]
set username [lindex $argv 3]
set password [lindex $argv 4]
set enable [lindex $argv 5]
set send_slow {10 .001}
log_user 0
if {[llength $argv] == 0} {
send_user "Usage: scriptname hostname \'snmpshapass\' \'snmpaespass\' username \'userpassword\' \'enablepassword\'\n"
send_user "For Cisco Nexus devices just give hostname snmpshapass and snmpaespass if you have ssh keys installed\n"
exit 1
}
send_user "\n#####\n# $hostname\n#####\n"
if { [info exists $username] } {
spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname
} else {
spawn ssh -q -o StrictHostKeyChecking=no $hostname
}
expect {
timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
"*#" {}
"*assword:" {
send "$password\r"
}
}
send "\r"
expect {
default { send_user "\nCould not get into enabled mode. Password problem?\n"; exit 1 }
"*#" {}
"*>" {
send "enable\r"
expect "*assword"
send "$enable\r"
expect "*#"
}
}
send "show ver | inc Cisco\r"
expect {
default { send_user "\nFailed to determine OS or get back correct prompt while changing pass.\n"; exit 1 }
"Nexus" {
send "config t\r"
expect "*(config)#"
send snmp-server user snmpUser network-operator auth sha $snmpshapass priv AES-128 $snmpaespass\r"
expect "*(config)#"
send "exit\r"
expect "*#"
send "copy run start\r"
expect "100%"
expect "*#"
}
"IOS" {
send "config t\r"
expect "*(config)#"
send snmp-server user snmpUser snmpGroup v3 auth sha $snmpshapass priv AES 128 $snmpaespass\r"
expect "*(config)#"
send "exit\r"
expect "*#"
send "write mem\r"
expect "*#"
}
}
send "exit\r"
send_user "\nSuccessfully changed SNMP password on $hostname\n"
close
I will not go through this script line by line like it did the first one but I will describe what it is doing at some key parts. You notice this script checks for a specific command line option on line 19. If the username variable is set then it will execute the ssh line with a username. If no username is given then it assumes your using ssh keys and uses the username you logged in with. Nexus devices allow ssh keys per user, IOS does not.
After starting the ssh session we have to make sure we are logged and in enabled mode. Cisco and Nexus devices show your in enabled mode with the prompt matching "*#". When we login to a Nexus device with ssh keys we are already at that prompt, this is not true for IOS devices or Nexus devices that do not use keys. Line 25 is our first expect statement and first checks to see if have the correct prompt "*#". If so then nothing is executed and the expect statement is short circuited with {}. If we don't match that prompt expect moves onto the second match line "*assword". That will match an IOS device or a Nexus device that is not configured with ssh keys. It will then give the correct password. If that fails we get the error message.
On line 36 we use expect again to see if we in the correct mode by checking for the prompt matching "*#". For a Nexus device with ssh keys setup we are already at that prompt so we short circuit again and move on to the next area. If there is no match we move to looking for the correct IOS prompt "*>". If we find it then we issue an enable command and give the password to finally get to the prompt we have been looking for that matches "*#". We check for that pattern one last time. if it matches then we move onto the next block.
On line 47 we send the command "show ver | inc Cisco\r". This command works for both devices and will give us a line that we can match on that shows us which operating system we are using. It will either say Nexus in the line or IOS in the line. We will use this output in our next expect statement
On line 49 we start our expect statement and check for one of 2 OS types. Either IOS or Nexus. When one matches we enter configuration mode and run the command to change the SNMP password for that device type. After the command is run we write our configuration to memory and make sure we get back to prompt successfully. The save command takes a few seconds to run. By making sure we get back to a prompt after we issue the save command it assures we have given the save command the right amount of time to run successfully. It is possible there could be an error in the save, but right now we are not checking for that.
Then we finally exit out of the router or switch. If we got this far and did not hit any exit timeouts we set then it is very likely this worked, so we send our success message. Then close our process down.
