This is Joe on Data.

Amazon ElasticMapReduce and the Screen Command

How many times have you SSH’d into a remote GNU/Linux server, ran a long-running command, only to come back to a stuck or disconnected SSH session? GNU Screen to the rescue! GNU Screen is perhaps one of the most subtle, useful, and difficult to understand commands in the GNU/Linux command arsenal. The important thing it does for this discussion is create a terminal session from which you can detach, allowing you to log out from the server while your session persists and runs. Then when you log back in at a later time or from another client, you can re-attach the previous screen session exactly as if you never left. Screen acts like a container for your session, keeping it’s program(s) running, command history, and other aspects of the session.

The difficult part of learning to use GNU Screen is the key commands it uses. Screen commands all begin with control-a key sequence (abbreviated “C-a” from now on), followed by the screen command. The most important combination is “C-a d“. This means hold the control key while pressing and releasing the a key. You then can release the control key and press the d key. The “C-a d” command detaches the current screen session, bringing you back to where you ran the screen command. You can then attach the screen session again by running the command “screen -r“. There are many more screen commands for which you will have to consult the GNU Screen documentation (ex: man page).

The importance of GNU Screen for Amazon ElasticMapReduce is it lets you use the Hive interactive command line client, run a long-running query, and not lose the hive interactive session if you logout or the connection is interrupted. I’m surprised the Amazon AMIs for EMR don’t have this installed by default. So the first thing I have to do when running a Hive interactive job is install screen via:

$ sudo apt-get install screen

I should just script the EMR cluster creation, adding a step to install Screen. But I’ll leave that as an exercise for myself and the reader.

GNU Screen has many other useful features such as being able to multiplex multiple terminal windows, scrollback and copy/paste. Checkout the GNU Screen website or man page for more details.

Post a Comment

Your email is kept private. Required fields are marked *