Setup Prep

Setting up our programming toolbox and environment.

Due: Wednesday, January 7
Points: 10

Get acquainted with the Unix shell on Stanford’s FarmShare and set up accounts on Github and Twitter so that we have a (remote) place to work and a usable data source.

Deliverables

  • Create a Github account

    Github gives us a convenient place to store, backup, and publish our work. Sign up for the free plan. Apply for the Github Education Pack.

  • Create a compciv repo on Github

    This is where you will submit homework assignments for the class. If you have already been given the Github Education Pack, make the compciv repo private.

    If your Github username is hello, the URL for your repo will be github.com/hello/compciv. Also, if you can figure it out, give my Github account (dannguyen) read access to your compciv repo. (if you can’t figure it out, we’ll go over it in class).

  • Set up SSH keys on corn.stanford.edu

    Set up password-less access to Github from corn.stanford.edu via a SSH key-pair.

  • Create a Twitter account

    Register at twitter.com; you can use an existing personal Twitter account if you have one. Then follow at least 50 other users.

  • A command-line delivered email

    From the corn.stanford.edu command-line, send me an email to dun@stanford.edu with the following:

    • A subject of “Setup prep for {your SUnet id, e.g. ‘dun’}”
    • A body containing:
      • The URL to your Twitter account
      • The URL to your Github account
      • Your public key, i.e. ~/.ssh/compciv_id_rsa.pub
  • Background

    If you've never done this before, this will probably be the most seemingly arbitrary and confusing set of steps that you'll have to take for this course. But they are necessary for setting up your the working environment for the rest of this course, and also serves as an introduction to remote/cloud computing.

    But there's nothing here meant to trick you. Just follow each step one at a time and please contact me or post on Piazza when you run into problems.

    Walkthrough: Setting up Github and SSH

    Creating a Twitter and Github account are straightforward enough.

    However, working from corn.stanford.edu and setting up the SSH keys may be new to you, so follow these steps.

    I've listed every command here: if you're completely new, read the commands and re-type them yourself (don't copy and paste). Just the physical act of typing helps you get a feel of how things work, even if you don't have any clue as to what the commands are doing. That's OK, this is the arbitrary and confusing part of the course.

    1. Log into corn.stanford.edu

    If you are on Windows

    You have a less direct path to logging in. Follow these instructions by Stanford's IT department on Downloading and Configuring SecureCRT 7. At the end of the instructions, you should be able to log into corn.stanford.edu

    You should see a prompt (the same as for the OS X users) that looks like this:

    dun@corn01:~$ 
    
    If you are on OS X

    In your Finder, open the folder at Applications > Disk Utilities and double-click the Terminal app. Then type (using your own SUnet ID):

      ssh YOUR_SUNET_ID@corn.stanford.edu
    

    You'll receive a response like this:

    The authenticity of host 'corn01.stanford.edu (171.67.216.66)' can't be established.
    RSA key fingerprint is 0b:e7:b4:95:03:c1:1e:07:df:04:ca:a2:3d:8e:e3:37.
    
    Are you sure you want to continue connecting (yes/no)? 
    

    Just type in yes (for yes) and hit Enter. You will then be asked for your password. Type it in (it'll seem like nothing's happening, but that's because the Terminal is obscuring your password) and hit Enter.

    At this point, you should see a prompt that looks like this:

    dun@corn01:~$ 
    

    2. Generating a SSH key

    Note: We may not need to actually do this SSH key thing (it's cumbersome on corn.stanford.edu). Read the guide on /recipes/devops/git-and-github-setup for more details, particularly the section about the "access token"

    At this point, the instructions are similar to what you'll find on Github…but not exactly. So keep following along here.

    Changing into your .ssh key directory

    At the prompt, type the following commands in and hit Enter after each one:

    mkdir -p ~/.ssh
    cd ~/.ssh
    

    Note: If it didn't seem like anything happened, i.e. a confirmation or error message, then you're good. This is a consequence of the Unix Philosophy tenet, "Rule of Silence":

    Rule of Silence: When a program has nothing surprising to say, it should say nothing.

    Generating your key-pair

    Type the following, substitute your SUnet ID (i.e your email), and hit Enter:

    ssh-keygen -t rsa -C "your_sunet_id@stanford.edu"
    

    You'll see this output:

    Generating public/private rsa key pair.
    

    And then you will be given a prompt that looks like this…don't hit Enter just yet:

    Enter file in which to save the key (/afs/.ir/users/d/u/dun/.ssh/id_rsa): 
    

    At the prompt here, type in:

    compciv_id_rsa
    

    Then hit Enter. This compciv_id_rsa string is just the filename for the key.

    You'll then be asked for a "passphrase" (i.e. a password). This is optional but I recommend specifying a passphrase.

    You'll then see this (and a few other lines of code you can ignore)

    Your identification has been saved in compciv_id_rsa.
    Your public key has been saved in compciv_id_rsa.pub.
    

    Your public key

    The file with the .pub extension – that is your public key. You can generally share this with the world, and in fact, you are going to share it with Github in the next step.

    Let's print it out to screen:

    cat compciv_id_rsa.pub
    

    The output will look something like this:

    ssh-rsa AAAAB3NzaC1yc2EAAAAXbZuyQIh/g9+LjVb39hjRm+HNgcurEUiRbPdcdMTB2YhReiPTDAQABAAABAQDSmc8DCOkHaloYGOounfjEHv0jk2+XEBfR99Ynj9nq21F7H8qipCjB9h5aGJTeMjxOqPEoIy3wUw43q76CZcJ0eLyn14YS7b+pMV0mFIMGfumYtuiybpg3E0RHlQtd6BVsGaF dun@stanford.edu
    

    Highlight all of that line (and just that line, down to your email address) and copy it to your clipboard (via Ctrl-C, or what have you).

    By the way, that other file – compciv_id_rsa? That's your private key. You never just want to give that out to anyone else. You can cat it just to see what's in it though. As you can see, it's much longer than your public key. Do not copy the private key to your clipboard

    Adding the public key to Github

    Assuming you have copied your public key to your clipboard, you can follow the rest of the Github instructions at Step 3: Add your SSH key to your [Github] account

    When you get to the step of typing in ssh -T git@github.com, you will probably receive an error like this:

      Permission denied (publickey).
    

    Read on to the next step.

    Setting up the ssh-agent

    So try this command:

    ssh -T git@github.com
    

    I'm assuming you get this error:

    Permission denied (publickey).
    

    The Github help page, Error: Agent admitted failure to sign, has the information.

    Execute this command:

    eval "$(ssh-agent -s)"
    

    You should see this:

    Agent pid 45765
    

    Then run this command and enter your passphrase if you set it up:

    ssh-add ~/.ssh/compciv_id_rsa
    

    Now you should be good to go:

    ssh -T git@github.com
    
    # Response
    Hi dannguyen! You've successfully authenticated, but GitHub does not provide shell access.
    

    Cleaning up with Github

    At this point, you're good to go. If you already know how to set up Github repos, you can try doing so from the command line, but we'll do it in class on Wednesday.

    As stated in the deliverables, one of the things you need to send me, via email, is that public key, which we'll do in the next step.

    Walkthrough: Sending me an email via command-line

    As before, I list out the steps needed to finish this part of the homework. Re-type them manually to get a feel for how things work. If you want a little more insight, read the Software Carpentry guide to the Shell

    Log into corn.stanford.edu again

    If you aren't still logged in, log back into corn.stanford.edu.

    The Stanford corn server allows you to send mail, via the command-line, from your Stanford email address. However, it only allows you to send emails to other @stanford addresses.

    Send yourself an email via CLI

    Type in the following at the prompt:

      mail your_sunet_id@stanford.edu
    

    You should then be prompted for a Subject, and when you hit Enter, your prompt will move to a blank line. At this point, you can type whatever you want.

    At the end of it, hit Control-D on your keyboard to exit out.

    You will then be prompted for a Cc: (hit enter to leave it blank). Then, the mail program sends the email (again, Unix will not tell you when things go as expected.

    Your screen will probably look like this:

    corn01:~$ mail your_sunet@stanford.edu
    Subject: heysadf
    This is an email blah 
    blah blaha
    
    BLAH
    
    Cc: 
    @corn01:~$  # back to the prompt
    

    Send yourself an email, but with options

    Most command-line programs have options (also known as flags) that can be set. There's no consistent rhyme or reason to them across different programs (i.e. -s for the mail program is not the same as it is for a program like ls)

    @corn01:~$ mail your_sunet@stanford.edu -s "This is my subject"
    Hello again
    I now know how to use options
    Cc: 
    @corn01:~$  # back to the prompt
    

    Send an email using a pipe

    Now let's send an email the more Unixy way (i.e faster, with less typing).

    The Software Carpentry's guide to Unix pipes is relevant here. But jumping straight into the action:

    The echo command simply takes a string of text and prints it to screen. The usage looks like this (note that what's printed is in single quotation marks):

      echo 'hello world'
      # outputs:
      hello world
    

    Using a pipe, denoted by the | character, we can send the text from echo to any other program that can be connected to a pipe (which is most Unix programs). So let's send it to mail:

      echo 'Pipes are pretty fun' | mail dun@stanford.edu -s 'PIPES WHOOOooo'
    

    You should see that message in your inbox straightaway. If you try it without the -s flag, you send it again, except without a subject. Oftentimes, the use of a pipe to connect programs will let you run them with the minimal amount of options and fuss.

    Finishing the assignment

    Once you're done testing mail, now you can complete the assignment which involves sending me (my SUnet is dun):

    1. The URL for your Github repo
    2. The URL for your Twitter account
    3. The contents of your public key at ~/.ssh/compciv_id_rsa.pub

    It doesn't matter what order they're sent in, just that they are separated by a newline or a space.

    Let's follow the same process as before, but to make things a little conceptually easier, we'll create a temporary text file that we'll pipe into the mail command.

    First, change directory to your home directory:

    cd ~
    

    Using echo and the > operator, we can send text into a new file:

    echo 'Hey dan this is my homework' >> stuff.txt
    

    Now add your Twitter and Github URLs in the same fashion, but use the >> (double brackets) so that the text is appended to stuff.txt (as opposed to overwriting stuff.txt):

    echo 'https://github.com/dannguyen' >> stuff.txt
    echo 'https://twitter.com/dancow' >> stuff.txt
    

    Go ahead and cat stuff.txt to see that your text is in there. The cat program is short for concatenate, and so every argument (in this case, filename) passed into cat will be added together. Try this:

    cat stuff.txt ~/.ssh/compciv_id_rsa.pub 
    

    That's pretty much your homework! Try using pipe to mail me in one line (note: the | and > and >> seem to do the same thing, but they don't. The pipe character facilitates the passing of text between programs. Whereas the right-angle-brackets sends text into files)

    After you're done, go ahead and delete the stuff.txt (not that there's anything private there):

    rm stuff.txt
    

    This concludes the "mail-in" part of the homework.