COS-461 Assignments: Router

Assignment 2: Building your own Internet Router

  • The assignment source code for those running the VM is here.
  • The assignment source code for those using the labpc is here.

    Note : All the instructions below should work in both the cases except for the wget instruction (see below) which is slightly different for the labpc users


    In this lab assignment you will be writing a simple router with a static routing table. Your router will receive raw Ethernet frames. It will process the packets just like a real router, then forward them to the correct outgoing interface. We’ll make sure you receive the Ethernet frames; your job is to create the forwarding logic so packets go to the correct interface.

    Your router will route real packets from a emulated host (client) to two emulated application servers (http server 1/2) sitting behind your router. The application servers are each running an HTTP server. When you have finished the forwarding path of your router, you should be able to access these servers using regular client software. In addition, you should be able to ping and traceroute to and through a functioning Internet router. A sample routing topology is shown below:

    alt text

    If the router is functioning correctly, all of the following operations should work:

    Additional requirements are laid out in the 'Requirements' section.


    This assignment runs on top of Mininet which was built at Stanford. Mininet allows you to emulate a topology on a single machine. It provides the needed isolation between the emulated nodes so that your router node can process and forward real Ethernet frames between the hosts like a real router. You don't have to know how Mininet works to complete this assignment, but more information about Mininet (if you're curious) is available here.

    Get Started

    Note : Please run the following commands as root on the VM.

    Environment Setup

    cd ~
    tar -xzvf ass2.tar.gz
    cd ass2/

    Configuration Files

    There are two configuration files.

    The default IP_CONFIG and rtable should look like the following:

    > cat ~/ass2/IP_CONFIG
    > cat ~/ass2/rtable eth3 eth1 eth2

    Test Connectivity of Your Emulated Topology

    Start Mininet emulation by using the following command

    > cd ~/ass2/
    > export PYTHONPATH=$PYTHONPATH:/usr/lib/python2.7/site-packages/:/usr/lib/python2.6/site-packages/:~/ass2/pox_module/
    > ./
    You should be able to see some output like the following:
    *** Shutting down stale SimpleHTTPServers  
    *** Shutting down stale webservers  
    *** Successfully loaded ip settings for hosts
     {'server1': '', 'sw0-eth3': '', 'sw0-eth1': '', 'sw0-eth2': '', 'client': '', 'server2': ''}
    *** Creating network
    *** Creating network
    *** Adding controller
    *** Adding hosts:
    client server1 server2 
    *** Adding switches:
    *** Adding links:
    (client, sw0) (server1, sw0) (server2, sw0) 
    *** Configuring hosts
    client server1 server2 
    *** Starting controller
    *** Starting 1 switches
    *** setting default gateway of host server1
    *** setting default gateway of host server2
    *** setting default gateway of host client
    *** Starting SimpleHTTPServer on host server1 
    *** Starting SimpleHTTPServer on host server2 
    *** Starting CLI:
    Keep this terminal open, as you will need the mininet command line for debugging. Now, use another terminal to continue to the next step. (Do not do ctrl-z)

    Mininet requires a controller, which we implemented in POX. To run the controller, use the following command:

    > cd ~/ass2/
    > ./
    You should be able to see some output like the following:
    POX 0.0.0 / Copyright 2011 James McCauley
    DEBUG:.home.ubuntu.ass2.pox_module.cos461.ofhandler:*** ofhandler: Successfully loaded ip settings for hosts
     {'server1': '', 'sw0-eth3': '', 'sw0-eth1': '', 'sw0-eth2': '', 'client': '', 'server2': ''}
    INFO:.home.ubuntu.ass2.pox_module.cos461.srhandler:created server
    DEBUG:.home.ubuntu.ass2.pox_module.cos461.srhandler:SRServerListener listening on 8888
    DEBUG:core:POX 0.0.0 going up...
    DEBUG:core:Running on CPython (2.7.3/Aug 1 2012 05:14:39)
    INFO:core:POX 0.0.0 is up.
    This program comes with ABSOLUTELY NO WARRANTY.  This program is free software,
    and you are welcome to redistribute it under certain conditions.
    Type 'help(pox.license)' for details.
    DEBUG:openflow.of_01:Listening for connections on
    Please note that you have to wait till Mininet is able to connect to the POX controller before you continue to the next step. Once Mininet is connected, you will see the following output:
    INFO:openflow.of_01:[Con 1/249473472573510] Connected to e2-e5-11-b6-b0-46
    DEBUG:.home.ubuntu.ass2.pox_module.cos461.ofhandler:Connection [Con 1/249473472573510]
    DEBUG:.home.ubuntu.ass2.pox_module.cos461.srhandler:SRServerListener catch RouterInfo even, info={'eth3': ('', '86:05:70:7e:eb:56', '10Gbps', 3), 'eth2': ('', 'b2:9e:54:d8:9d:cd', '10Gbps', 2), 'eth1': ('', '36:61:7c:4f:b6:7b', '10Gbps', 1)}, rtable=[]
    Keep POX running. Now, open yet another terminal to continue to the next step. (Dont’ do ctrl-z)

    Now you are ready to test out the connectivity of the environment setup. To do so, run the binary file of the solution “sr_solution”

    > cd ~/ass2/
    > ./sr_solution
    You should be able to see some output like the following:
    Loading routing table from server, clear local routing table.
    Loading routing table
    Destination     Gateway         Mask    Iface     eth3    eth1    eth2
    Client ubuntu connecting to Server localhost:8888
    Requesting topology 0
    successfully authenticated as ubuntu
    Loading routing table from server, clear local routing table.
    Loading routing table
    Destination     Gateway         Mask    Iface     eth3    eth1    eth2
    Router interfaces:
    eth3    HWaddr86:05:70:7e:eb:56
            inet addr
    eth2    HWaddrb2:9e:54:d8:9d:cd
            inet addr
    eth1    HWaddr36:61:7c:4f:b6:7b
            inet addr
     <-- Ready to process packets --> 

    In this particular setup, is the IP for server1, and is the IP for server2. You can find the IP addresses in your IP_CONFIG file.

    Now, back to the terminal where Mininet is running. To issue an command on the emulated host, type the host name followed by the command in the Mininet console. For example, the following command issues 3 pings from the client to the server1.

    mininet> client ping -c 3
    You should be able to see the following output.
    PING ( 56(84) bytes of data.
    64 bytes from icmp_req=1 ttl=63 time=66.9 ms
    64 bytes from icmp_req=2 ttl=63 time=49.9 ms
    64 bytes from icmp_req=3 ttl=63 time=68.8 ms

    You can also use traceroute to see the route between client to server1.

    mininet> client traceroute -n
    You should be able to see the following output.
    traceroute to (, 30 hops max, 60 byte packets
     1  146.069 ms  143.739 ms  143.523 ms
     2  226.260 ms  226.070 ms  225.868 ms

    Finally, to test the web server is properly working at the server1 and server2, issue an HTTP request by using wget or curl.

    mininet> client wget

    Note : For those running this on the labpc, you have to use this command :
    mininet> client wget -O /tmp/index.html
    You should be able to see the following output.
    --2012-12-17 06:52:23--
    Connecting to connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 161 [text/html]
    Saving to: `index.html'
         0K                                                       100% 17.2M=0s
    2012-12-17 06:52:24 (17.2 MB/s) - `index.html' saved [161/161]
    If you stop the "./sr_solution", you will find that ping/traceroute/wget won't work anymore. In this assignment, you will replicate the functionality of sr_solution. To help you get started, we provide some starter code described in the following section..

    Starter Code

    You should now have all the pieces needed to build and run the router:

    A routing table file that corresponds to the routing table for the router node in that topology, which we'll call rtable.

    The starter source code

    You can build and run the starter code as follows:

    > cd ~/ass2/router/
    > make

    Logging Packets

    You can log the packets receives and generated by your SR program by using the "-l" parameter with your SR program. The file will be in pcap format, i.e., you can use wireshark or tcpdump to read it.

    > ./sr -l logname.pcap

    Besides SR, you can also use mininet to monitor the traffic goes in and out of the emulated nodes, i.e., router, server1 and server2. Mininet provides direct access to each emulated node. Take server1 as an example, to see the packets in and out of it, go to mininet CLI:

    mininet> server1 sudo tcpdump -n -i server1-eth0

    or you can bring up a terminal inside server1 by

    mininet> xterm server1

    then inside the newly popped xterm,

    > sudo tcpdump -n -i server1-eth0

    General Forwarding Logic

    To get you started, an outline of the forwarding logic for a router follows, although it does not contain all the details. There are two main parts to this assignment: Handling ARP and IP forwarding

    IP Forwarding

    Given a raw Ethernet frame, if the frame contains an IP packet that is not destined towards one of our interfaces:

    Protocols to Understand


    You are given a raw Ethernet frame and have to send raw Ethernet frames. You should understand source and destination MAC addresses and the idea that we forward a packet one hop by changing the destination MAC address of the forwarded packet to the MAC address of the next hop's incoming interface.

    Internet Protocol

    Before operating on an IP packet, you should verify its checksum and make sure it meets the minimum length of an IP packet. You should understand how to find the longest prefix match of a destination IP address in the routing table described in the "Getting Started" section. If you determine that a datagram should be forwarded, you should correctly decrement the TTL field of the header and recompute the checksum over the changed header before forwarding it to the next hop.

    Internet Control Message Protocol

    ICMP is a simple protocol that can send control information to a host. In this assignment, your router will use ICMP to send messages back to a sending host. You will need to properly generate the following ICMP messages (including the ICMP header checksum) in response to the sending host under the following conditions:

    Address Resolution Protocol

    ARP is needed to determine the next-hop MAC address that corresponds to the next-hop IP address stored in the routing table. Without the ability to generate an ARP request and process ARP replies, your router would not be able to fill out the destination MAC address field of the raw Ethernet frame you are sending over the outgoing interface. Analogously, without the ability to process ARP requests and generate ARP replies, no other router could send your router Ethernet frames. Therefore, your router must generate and process ARP requests and replies. To lessen the number of ARP requests sent out, you are required to cache ARP replies. Cache entries should time out after 15 seconds to minimize staleness. The provided ARP cache class already times the entries out for you. When forwarding a packet to a next-hop IP address, the router should first check the ARP cache for the corresponding MAC address before sending an ARP request. In the case of a cache miss, an ARP request should be sent to a target IP address about once every second until a reply comes in. If the ARP request is sent five times with no reply, an ICMP destination host unreachable is sent back to the source IP as stated above. The provided ARP request queue will help you manage the request queue. In the case of an ARP request, you should only send an ARP reply if the target IP address is one of your router's IP addresses. In the case of an ARP reply, you should only cache the entry if the target IP address is one of your router's IP addresses. Note that ARP requests are sent to the broadcast MAC address (ff-ff-ff-ff-ff-ff). ARP replies are sent directly to the requester's MAC address.

    IP Packet Destinations

    An incoming IP packet may be destined for one of your router's IP addresses, or it may be destined elsewhere. If it is sent to one of your router's IP addresses, you should take the following actions, consistent with the section on protocols above:

    Code Overview

    Basic Functions

    Your router receives a raw Ethernet frame and sends raw Ethernet frames when sending a reply to the sending host or forwarding the frame to the next hop. The basic functions to handle these functions are:

    void sr_handlepacket(struct sr_instance* sr, uint8_t * packet, unsigned int len, char* interface)
    This method, located in sr_router.c, is called by the router each time a packet is received. The "packet" argument points to the packet buffer which contains the full packet including the ethernet header. The name of the receiving interface is passed into the method as well.
    int sr_send_packet(struct sr_instance* sr, uint8_t* buf, unsigned int len, const char* iface)
    This method, located in sr_vns_comm.c, will send an arbitrary packet of length, len, to the network out of the interface specified by iface.

    You should not free the buffer given to you in sr_handlepacket (this is why the buffer is labeled as being "lent" to you in the comments). You are responsible for doing correct memory management on the buffers that sr_send_packet borrows from you (that is, sr_send_packet will not call free on the buffers that you pass it).

    void sr_arpcache_sweepreqs(struct sr_instance *sr)
    The assignment requires you to send an ARP request about once a second until a reply comes back or we have sent five requests. This function is defined in sr_arpcache.c and called every second, and you should add code that iterates through the ARP request queue and re-sends any outstanding ARP requests that haven't been sent in the past second. If an ARP request has been sent 5 times with no response, a destination host unreachable should go back to all the sender of packets that were waiting on a reply to this ARP request.

    Data Structures

    The Router (sr_router.h):

    The full context of the router is housed in the struct sr_instance (sr_router.h). sr_instance contains information about topology the router is routing for as well as the routing table and the list of interfaces.

    Interfaces (sr_if.c/h):

    After connecting, the server will send the client the hardware information for that host. The stub code uses this to create a linked list of interfaces in the router instance at member if_list. Utility methods for handling the interface list can be found at sr_if.c/h.

    The Routing Table (sr_rt.c/h):

    The routing table in the stub code is read on from a file (default filename "rtable", can be set with command line option -r ) and stored in a linked list of routing entries in the current routing instance (member routing_table).

    The ARP Cache and ARP Request Queue (sr_arpcache.c/h):

    You will need to add ARP requests and packets waiting on responses to those ARP requests to the ARP request queue. When an ARP response arrives, you will have to remove the ARP request from the queue and place it onto the ARP cache, forwarding any packets that were waiting on that ARP request. Pseudocode for these operations is provided in sr_arpcache.h. The base code already creates a thread that times out ARP cache entries 15 seconds after they are added for you. You must fill out the sr_arpcache_sweepreqs function in sr_arpcache.c that gets called every second to iterate through the ARP request queue and re-send ARP requests if necessary. Psuedocode for this is provided in sr_arpcache.h.

    Protocol Headers (sr_protocol.h)

    Within the router framework you will be dealing directly with raw Ethernet packets. The stub code itself provides some data structures in sr_protocols.h which you may use to manipulate headers easily. There are a number of resources which describe the protocol headers in detail. Network Sorcery's RFC Sourcebook provides a condensed reference to the packet formats you'll be dealing with:

    Debugging Functions

    We have provided you with some basic debugging functions in sr_utils.h, sr_utils.c. Feel free to use them to print out network header information from your packets. Below are some functions you may find useful:

    Length of Assignment

    This assignment is considerably harder than the first 2 labs, so get started early.

    In our reference solution, we added 520 lines of C, including whitespace and comments:

    > wc -l ~/ass2/router/*.c | tail -1
     1585 total
    > wc -l ~/cos461/router/solution/*.c | tail -1
     2104 total
    Of course, your solution may need fewer or more lines of code, but this gives you a rough idea of the size of the assignment to a first approximation.

    Reference Binary

    To help you debug your topologies and understand the required behavior we provide a reference binary and you can find it at ~/ass2/sr_solution in your directory:

    We have talked about how to use it in the "Test Connectivity of Your Emulated Topology" section, please refer to the instructions there.

    Required Functionality

    Also, don't forget to fill out your readme! (It is worth 1 point)

    Extra Credit (2 points):

    Supporting Outbound Two-way TCP Traffic (1.0/2.0)

    Note that the extra credit part for this assignment is "on your own" meaning that you can only ask clarification questions and not get other help from the TAs or fellow students.

    The extra credit is basically a simple firewall. If you implement the extra credit, it should be able to be turned on/off. So we can test your required functionality.

    Note that simply allowing packets from the internal hosts to go through the router is not enough to establish a working connection to an external service, because most (if not all) TCP/IP services entail two-way communications. Therefore, packets that belong to a flow initiated by an internal end-host that arrive to the external interface must be allowed through the secure router. To support this feature the router maintains a "flow table" that contains all the active (and allowed) flows that traverse it. In this context, a flow is defined as a 5-tuple <srcIP, dstIP, IPprotocol, src-port, dst-port>.

    When the first "internal" packet arrives at the router, two entries are added to the flow table, one for each direction of communication. The entry for the external-to-internal flow can be generated by inverting the order of source and destination IP addresses and ports. When a packet arrives to the external interface, the router checks if it matches one of the entries in the flow table. If it does then the packet is not dropped and it is forwarded to the internal interface.

    Entries remain on the flow table as long as packets that match these entries go through the firewall. To support this feature each entry has a time-to-live (TTL). Each time a packet matching the flow entry is received, the entry's TTL is set to X seconds. The router periodically scans the flow table and removes all entries whose TTL has expired. (Note: You should update the entries associated with both directions of a flow when a packet is received.)

    The flow table can hold up to Y entries at each time. You can make this parameter Y a constant. If a new entry needs to be added when the flow table is full, first a scan is initiated to determine if one or more stale entries exist in the flow table. If all entries are valid, then an ICMP response is returned (Destination Unreachable - Port Unreachable) to the originator and a log entry is generated.

    A proper firewall supports adding explicit rules to allow/disallow flows to traverse the firewall in a "rule table". You don't need to worry about adding exceptions or creating a "rule table" i.e., you reject all external to internal traffic by default unless it matches an entry in the flow table. You don't need to worry about responding properly to pings or traceroutes in SE mode. Also you don't need to worry about packet fragments i.e., if a packet fragment arrives with the transport level header missing then the fragment (as well all other subsequent fragments) are dropped.

    Your log of dropped packets should have the following format:

    <srcIP, dstIP, protocol, src-port, dst-port, drop-code>

    The drop-code for 'flow not allowed' is 2. The drop-code for 'flow table full' is 3.

    Here's an example:

    <,, UDP, 54321, 23, 2>
    <,, TCP, 12345, 80, 3>


    You should submit your completed router by the due date to CS Dropbox. You may add files to those provided with the stub code; be sure to update your Makefile if you do. You should answer the questions in the README file, describe design decisions that you made, and any trade offs that you encountered in the design.

    Last updated: Wed Jul 24 09:25:41 -0400 2013