[ This content is protected and may not be shared, uploaded, or distributed. ]

(Please also check out PA4 FAQ.)

Lab 11 is probably the most difficult lab so far, you should start this lab as early as possible. Lab 11 has two parts:

Part A (lab11a) - three nodes (useful for PA4):

In this lab, we will impmlement part of layer 3 (network layer) functionalities in 353NET. We will continue with Part A of your Lab 10 code and add a new message, LSUPDATE, to implement a link state routing algorithm for the 353NET (although we will not do the actual "routing" until Lab 12). Your Lab 10 code needs to be pretty solid before you should attempt this lab.

In Lab 10, a node in the 353NET knows its own link state (i.e., when you type the "neighbors" console command, you are printing the link state of your node). The goal of this lab is for that node to obtain the link state information from all the nodes in the 353NET. You must combine the node's link state and the link states of all the other nodes in the network into an adjacency list data structure and now you have the topology of the partition of the 353NET that your node is in and that's the goal of this lab.

PA4 spec talks about the difference between using a soft state approach vs. a hard state approach to maintaining link state information of all nodes in the network. For this lab (and for PA4), you are required to implement the hard state approach. Every node must send its link state information to the entire network via flooding and they must run the same distributed algorithm so that in the end, all nodes in the same network partition will see the same network topology.

With the hard state approach, the link state information from another node is considered correct unless you get an update from that node asking you to replace the old state with a new state. This approach is easier to implement and debug since you will not have to deal with a lot of asynchronous timers going off at random times! The problem with the hard state approach is that you can get race conditions. Also, you need to deal with the case where you have lost contact with a node. Since that node may be down, how can you get an update from that node?

New 353NET Message

The basic idea is as follows. Every time a node's link state information has changed (such as establishing a new active connection with a neighbor or losing an active connection), you must flood an LSUPDATE (link state update) message to the entire 353NET. An LSUPDATE message looks like the following:
    353NET/1.0 LSUPDATE\r\n
    TTL: NUMBER\r\n
    Flood: 1; reason=REASON\r\n
    MessageID: MSGID\r\n
    From: ORIGIN_NODEID\r\n
    OriginStartTime: ORIGIN_START_TIME\r\n
    Content-Length: LENGTH\r\n
    \r\n
    NODEID_1,NODEID_2,...,NODEID_K
where NUMBER is an integer, REASON is an integer, ≥ 1 and ≤ 4, as mentioned below, to indicate why this LSUPDATE message was initiated, MSGID is a network-wide unique identifier (can be created by calling the GetObjID() funtion), ORIGIN_NODEID is the NodeID of the node that initiated this message, ORIGIN_START_TIME is the time ORIGIN_NODEID got started, and LENGTH is the number of bytes in the message body, which immediately follows the blank line at the end of the message header (in this case, is the string length of the last line shown above). The message body is simply a list of NodeIDs separated by "," characters between any two consecutive NodeIDs. These are the list of active neighbors of the node that initiated this message, which is the link state of that node. There must be no "\0", "\t", "\r", "\n", or space characters in the message body.

When you flood an LSUPDATE message, the node that initiated the flood is referred to as the origin node. When the origin node generates an LSUPDATE message, it needs specify the values for NUMBER, MSGID and ORIGIN_START_TIME. The origin node must set NUMBER to be the value of the "max_ttl" key in the [params] section of the CONFIGFILE. It also must call the GetObjID() funtion with the 2nd argument being a C-string constant "msg" and use the string returned in hexstring_of_unique_obj_id as MSGID and origin_start_time as ORIGIN_START_TIME.

In the above example, the node whose NodeID is ORIGIN_NODEID has K active neighbors with NodeIDs NODEID_1, NODEID_2, ..., and NODEID_K. Clearly, K must be ≥ 1. ORIGIN_START_TIME is the time ORIGIN_NODEID got started. When ORIGIN_NODDID initiated this message, MSGID must be a freshly created unique object ID, obtained around the time this message was generated.

Please note that by using the GetObjID() function in PA4, MSGID would have the format of "ORIGIN_NODEID_#_SHORTTIMESTAMP" where "#" is a sequence number and SHORTTIMESTAMP is the time the ORIGIN_NODEID initiated this LSUPDATE message. The format of this SHORTTIMESTAMP is "tv_sec.tv_usec" where "tv_sec" is the number of seconds since 1/1/1970 and "tv_usec" is the number of microseconds in the current second. This way, when a node receives two distinct LSUPDATE messages initiated by the same node, it can parse and compare the timestamps and keep the more recent; and therefore, more correct link state information about the origin node.

Distributed Algorithm

The basic rule for hard state is that at any time the link state of a node has changed, that node must flood its new link state information to the entire network. That sounds simple enough. What are the complications?

When a new node, say node X, comes up and forms active connection with its neighbors. As each of its neighbors gets a new connection from node X, it will send a LSUPDATE message to node X to tell node X who its neighbors are. But how can node X get the link state information from other nodes in the 353NET (i.e., nodes that are two hops or more away)?

The solution is as follows. Let's say that node Z is two hops or more away from node X (i.e., not a neighbor of X), it maintains an adjancy list data structure to represent the network topology. The adjacency list data structure is simply a collection of link states that it has received from all nodes in the network. (In your implementation, you can think of a link state inside the adjacency list is simply a key/value pair where the key is the ORIGIN_NODEID from an LSUPDATE message and value is the corresponding message body.) When node Z receives an LSUPDATE message initiated by node X, it must check to see if it has a link state from node X already. If it doesn't, it should add the link state of node X into its adjacency list data structure. It also KNOWS that node X only has link state information from its immediate active neighbors! Therefore, node Z must send its link state information to node X (by generating an LSUPDATE message and flood it through the network to reach node X.

How does node Z knows that X is a new node? When node Z tries to update the link state of node X in its adjacency list, it cannot find the previous link state of node X and it needs to add a new row in the adjacency list. Therefore, it knows that node X is new. (Although if X is already an active neighbor of node Z, there is no need for node Z to flood its link state to the entire network since X already has that information. Therefore, you only need to initiate link state flooding if a new node is a "remote node", i.e., at least 2 hops away from you.)

What if node Z has X in its adjacency list, but then node X went down and came up at a later time? Since we are taking the hard state approach, node X's link state is still in node Z's adjacency list. This is where we use the ORIGIN_START_TIME in the LSUPDATE message. If we see that the ORIGIN_START_TIME of an LSUPDATE message is newer than the ORIGIN_START_TIME of node X's link state in the adjacency list, then we know that this node X is new and we can delete the link state information of the old node X from the adjacency list and replace it with the link state information of the new instance of node X. Since X is now a new remote node, node Z must initiate link state flooding. Please note that this is the main reason why we must have ORIGIN_START_TIME in an LSUPDATE message!

Finally, what do you do with nodes that went down? With the hard state approach, we will never know that it has gone down! The only way we can find out that a node, say node Y, has gone down is when we run BFS (which is preferred over the Dijkstra's algorithm since we don't have edge costs/weights) to construct the forwarding table, we found that node Y is not reachable! At that time, we can delete node Y from the adjacency list. We don't want to run BFS all the time. We should run them on-demand, i.e., wait till we absolutely have to have the most up-to-date network graph or forwarding table. One case that you should run BFS is when you received an LSUPDATE message and see that REASON is 2 (meaning that the node that "initiated the LSUPDATE flood" did it because it has just lost an active neighbor). In that case, the network may have been partitioned and you should run BFS to delete all the unreachable notes from your adjacency list data structure (after you have updated your adjacency list data structure).

IMPORTANT NOTE: Please understand that there are many ways this can work. Also, due to the asynchronous nature of multithreading, it is possible that different theads within a node may not agree on the link state of the node! Of course, it goes without saying that when you want to look at or modify the link state of your node, you must only do so when you have the main mutex locked. But even with that, different theads within a node still may not agree on what they are seeing. So, it may be the case that sometimes your node may send more LSUPDATE messages and sometimes it may send less. If you send extra LSUPDATE messages, it may not be incorrect, as long as eventually all nodes stop generating new LSUPDATE messages and all nodes within the same network partition agree on the network topology of that partition, which achieves the goal of this lab!

Console

For this lab, you will need to add a new console command called "netgraph" to display the network graph/topology. When the user types "netgraph" in the console, you must run BFS to find out which nodes in the adjacency list are reachable from your node and delete all the unreachable nodes from the adjacency list. Then you print the adjacency list to the console. If the node, whose console you are typing into, does not have any active neighbors, you must also print an appropriate message.

Messsage Cache

You also have to implement a message cache to implement flooding correctly. To avoid implementing timers in this lab, please do not timeout objects in the message cache. It is possible that your message cache can grow to a very large size, but since this is just a lab, we will not worry about that.

Logging

We will use the log message format in PA4. When a node receives an LSUPDATE message, you must print the following log message to cout (since the value of the LSUPDATE key in the [logging] section of the CONFIGFILE is 0):
    [TIMESTAMP] r LSUPDATE NEIGHBOR TTL F CONTENT_LENGTH MSGID ORIGIN_START_TIME ORIGIN_NODEID (NodeIDs)\n
where TIMESTAMP is the current time in the same format as a PA2 timestamp, r means that the message was received, NEIGHBOR is the NodeID of the neighbor from which you got this message, TTL is the value of the TTL key in the message header (i.e., before you decrement it), F means that the value of the Flood key in the message header is 1, CONTENT_LENGTH is the value of the Content-Length key in the message header, MSGID, ORIGIN_START_TIME, and ORIGIN_NODEID are the corresponding values in the message header, and NodeIDs is message body. Please note that you must surround TIMESTAMP with a pair of square brackets and NodeIDs with a pair of parentheses so they stand out.

When a node initiates an LSUPDATE message, you must print the following log message to cout:

    [TIMESTAMP] i LSUPDATE NEIGHBOR TTL F CONTENT_LENGTH MSGID ORIGIN_START_TIME ORIGIN_NODEID (NodeIDs)\n
where i means that the message was initiated. Please note that since this is an initiated message, TTL above must be equal to max_ttl value in the configuration file

When a node sends an LSUPDATE message due to flooding, you must print the following log message to cout:

    [TIMESTAMP] d LSUPDATE NEIGHBOR TTL F CONTENT_LENGTH MSGID ORIGIN_START_TIME ORIGIN_NODEID (NodeIDs)\n
where d means that the message was flooded. Please note that since this is a flooded message, TTL above must be less than the max_ttl value in the configuration file

Testing

We will use a set of slightly different configuration files from what's used in Lab 10. Please do the following:
  • Create an empty directory (call it "lab11") and change directory into it.
  • Download lab11data.tar.gz into that directory and type:
        tar xvf lab11data.tar.gz
    This should create a subdirectory called "lab11data" with several .ini files in it which we will use as configuration files for our nodes.
The usage information (i.e., commandline syntax) for "lab11a" is as follows:
    lab11a CONFIGFILE
where CONFIGFILE is similar to the ones that was used in Lab 10. The main difference is directory name and the values for the SAYHELLO key in the [logging] section. For this lab, you must log SAYHELLO messages into specified log file (and log LSUPDATE messages to cout). The "max_ttl" key specifies the value you must use for the TTL key in the LSUPDATE message header when a node initiates an LSUPDATE message.

For this lab, the main job is for nodes in the same partition to agree on the network graph by running the algorithm mentioned above. Please note that you need to have a solid Lab 10 before you start this lab (i.e., a reliable network layer can only be built on top of a reliable link layer).

When you are done with implementing lab11a, please do the following:

  • Change directory into the "lab11" directory mentioned above.
  • Open three Terminals and change directory into the same directory.
  • In the first Terminal window, type "script lab11a-12000.script" to start a transcript.
  • In the 2nd Terminal window, type "script lab11a-12002.script" to start another transcript.
  • In the 3rd Terminal window, type "script lab11a-12004.script" to start another transcript.
  • In the first Terminal window, type:
     
        uname -a
        cat /etc/os-release
        make clean
        make lab11a
        ./lab11a lab11data/lab11-12000.ini
    When you get the command prompt (which should look like ":12000> "), type "neighbors" then "netgraph". In both cases, you should see: You should see:
        :12000 has no active neighbors
  • In the 2nd Terminal window, type:
        ./lab11a lab11data/lab11-12002.ini
    In both the 1st and 2nd windows, you should see that an LSUPDATE message has been sent/initiated and an LSUPDATE message has been received.

  • When you get the command prompt in the 2nd window (which should look like ":12002> "), wait for a couple of seconds and type "netgraph" in both the 1st and 2nd windows. You should see the following in both windows (order does not matter) since both nodes should agree on the same network topology:
        :12000: :12002
        :12002: :12000
  • In the 3rd Terminal window, type:
        ./lab11a lab11data/lab11-12004.ini
    You should see 4 LSUPDATE messages in the 1st window (3 received and 1 initiated). You should see 8 LSUPDATE messages in the 2nd window (3 received, 2 initiated, and 3 flooded). You should see 4 LSUPDATE messages in the 3rd window (2 received and 2 initiated). If those are not the exact number of LSUPDATE messages you see, it may be due to timing differences. As long as all 3 nodes agree on the same network topology, it's find if you get a few extra messages exchanged. (If you see way too many messages getting exchanged, you should fix your code.) Let's see if all 3 nodes agree on the network topology.

  • When you get the command prompt in the 3rd window (which should look like ":12004> "), wait for a couple of seconds and type "netgraph" in all three windows and you should see something like the following (order does not matter):
        :12000: :12002
        :12002: :12000,:12004
        :12004: :12002
    Please check the above printout against a network graph that looks like the following to see that they represent the same thing:
        +-------+   +-------+     +-------+
        | 12000 +---+ 12002 +-----+ 12004 |
        +-------+   +-------+     +-------+
    As mentioned in the note above, the number of LSUPDATE messages you see in these windows are not too important, as long as all nodes agree on the network topology. Below is one possible scenario.

    In the 1st window, you may see that it has initiated one LSUPDATE message and received LSUPDATE messages from nodes :12002 and :12004. In the 2nd window, you may see that it has initiated one LSUPDATE message to :12000 and one LSUPDATE message to :12004, received LSUPDATE messages from nodes :12000 and :12004, and flooded these LSUPDATE messages to :12004 and :12000, respectively with a decremented TTL value. In the 3rd window, you may see that it has initiated two LSUPDATE messages and received LSUPDATE messages from nodes :12000 and :12002.

  • Type "quit" in the 2nd window to cause a network partition, then type "neighbors" and "netgraph" in the 1st and 3rd window and make sure that the corresponding nodes have removed the link state for node :12002 from their adjacency lists. The nodes in the 1st and 3rd window should have no neighbors.
  • In the 2nd Terminal window, type:
        ./lab11a lab11data/lab11-12002.ini
    to start node :12002 again. You should see a bunch of LSUPDATE packets flying by in all 3 windows. (In the 1st window, you may see that it has initiated two LSUPDATE messages and received four LSUPDATE messages. In the 2nd window, you may see that it has initiated four LSUPDATE messages and received four LSUPDATE messages, and flooded four LSUPDATE messages. In the 3rd window, you may see that it has initiated two LSUPDATE messages and received four LSUPDATE messages.)

  • Type "neighbors" followed by "netgraph" in all three windows to make sure that all three nodes see the same network graph with the configuration depicted above.
  • Type "quit" in the first window and type "netgraph" in the 2nd and 3rd window and verify that both of them sees:
        :12002: :12004
        :12004: :12002
  • Type "quit" in the 2nd and 3rd window and make sure that they all terminated gracefully.
  • In the 3rd Terminal window, type the follow to display all the log files (to see that SAYHELLO messages are properly logged):
     
        more lab11data/*.log
    The "more" command will pause when it shows a screen full of data or finishes displaying the current file. Just keep pressing the spacebar on your keyboard to see the next page of data or the next file.

  • Type "exit" in all three windows to close the transcripts.
To save typing all some of the commands above, a tmux script, "tmux-lab11a.txt" is provided in the "lab11data" directory created above and you can run it by typing:
    lab11data/tmux-lab11a.txt
Plesae note that it's provided for your convenience (i.e., to save typing) and it may not be exactly the same as the above sequence. Please see PA2 FAQ regarding how to use tmux in general.

Part B (lab11b) - four nodes (useful for PA4):

There is no new code to write. Just want to make sure that your code works correctly when more nodes are activated.
  • Change directory into the "lab11" directory mentioned above.
  • Open three more Terminal windows and change directory into the same directory.
  • In the first Terminal window, type "script lab11b-12000.script" to start a transcript.
  • In the 2nd Terminal window, type "script lab11b-12002.script" to start another transcript.
  • In the 3rd Terminal window, type "script lab11b-12004.script" to start another transcript.
  • In the 4th Terminal window, type "script lab11b-12012.script" to start another transcript.
  • In the first Terminal window, type:
     
        ./lab11a lab11data/lab11-12000.ini
  • In the 2nd Terminal window, type:
     
        ./lab11a lab11data/lab11-12002.ini
  • In the 3rd Terminal window, type:
     
        ./lab11a lab11data/lab11-12004.ini
  • In the 4th Terminal window, type:
     
        ./lab11a lab11data/lab11-12012.ini
  • Once these 4 nodes are connected to each other, the network should look like the following (from the configuration files):
        +-------+   +-------+     +-------+
        | 12000 +---+ 12002 +-----+ 12004 |
        +-------+   +---+---+     +---+---+
                        |             |
                        |  +-------+  |
                        \--+ 12012 +--/
                           +-------+
  • Type "neighbors" followed by "netgraph" in all 4 windows and make sure everything looks right. The "netgraph" printout should look like the following (order does not matter):
        :12000: :12002
        :12002: :12000,:12004,:12012
        :12004: :12002,:12012
        :12012: :12002,:12004
  • Now type "quit" in the 2nd Terminal window to bring down node :12002. This should cause network partition. In all other windows, you should see that they have disconnected from :12002. Wait a couple of seconds and type "neighbors" followed by "netgraph" in windows 1, 3, and 4 and you shold see that node :12000 should have no neighbors and nodes :12004 and :120012 should have each other as neighbors. The "netgraph" printout in the 3rd and 4th windows should look like the following (order does not matter):
        :12012: :12004
        :12004: :12012
  • In the 2nd Terminal window, restart node :12002 by typing:
     
        ./lab11a lab11data/lab11-12002.ini
    You should see some LSUPDATE messages flying by. After things settle, type "netgraph" in all windows and the printout should look like the following (order does not matter):
        :12000: :12002
        :12002: :12000,:12004,:12012
        :12004: :12002,:12012
        :12012: :12002,:12004
  • Type "quit" in the first window to bring down node :12000. Wait a couple of seconds then type "netgraph" in all remaining windows and the printout should look like the following (order does not matter):
        :12002: :12004,:12012
        :12004: :12002,:12012
        :12012: :12002,:12004
  • Type "quit" in all remaining windows.
  • In the 4th Terminal window, type the follow to display all the log files (to see that SAYHELLO messages are properly logged):
     
        more lab11data/*.log
  • Type "exit" in all four windows to close the transcripts.
To save typing all some of the commands above, a tmux script, "tmux-lab11b.txt" is provided in the "lab11data" directory created above and you can run it by typing:
    lab11data/tmux-lab11b.txt
Plesae note that it's provided for your convenience (i.e., to save typing) and it may not be exactly the same as the above sequence. Please see PA2 FAQ regarding how to use tmux in general.

All pseudo-code is incomplete and error checking is often left out in pseudo-code. Since some details are left out, depending on you how write your code, you may create race conditions and you may need to fix your code so that your program won't freeze or crash. Feel free to send your questions (and not your code) to the instructor.

Pseudo-code

Code in red are critical section codes. Please note that the pseudo-code is not necessarily complete.

Socket-reading Thread

    ...
    if connection is not a duplicate then
        initiate LSUPDATE
        do forever
            m = read message from c.socket
            lock mutex
            if listening socket is (-1) or m is NULL then
                unlock mutex
                break;
            end-if
            if (msg_cache.lookup(m) == not found) then
                msg_cache.add(m)
                if (m.flood) then
                    decrement m.ttl
                    foreach active connection d ≠ c in connection list do
                        d.add_work(m)
                    end-for
                end-if
            end-if
            unlock mutex
            /* process LSUPDATE message m, including updating adjacency list */
        end-do
    end-if
    ...
    initiate LSUPDATE before c.write_thread_ptr.join();
    ...
Please note that we are using a lock hierarchy in the above code! A 2nd level mutex is involved in d.add_work(m) and we only attempt to lock it when we are holding the first level mutex (i.e., the main mutex). You must follow the rule of using a lock hierarchy, i.e., in your code for d.add_work(m), you must not attempt to lock the main mutex (or you may end deadlocking your program)!

When to "initiate an LSUPDATE flood"?

There are 4 cases when a node must "initiate an LSUPDATE flood" (or "initiate LSUPDATE", the keywords here is "initiate") and they are:
  1. When it gained an active neighbor.
  2. When it lost an active neighbor.
  3. When it got an LSUPDATE message originated from a node that's at least 2 hops away and this node is not currently in the adjacency list.
  4. When it got an LSUPDATE message originated from a node that's at least 2 hops away and this node is currently in the adjacency list but the ORIGIN_START_TIME in the message is newer than the ORIGIN_START_TIME kept in the adjacency list for that node.
I would recommend that you write a function and call it something like initiate_LSUPDATE_flood() and just call it when any of the above 4 situations is encountered.

Please note that "initiate an LSUPDATE flood" is different from "flooding". When you are doing "flooding", you are helping other nodes to flood their LSUPDATE messages to the entire 353NET. When you "initiate an LSUPDATE flood", you create a new LSUPDATE message and you send a copy of it into each of your active connections.

Below is the grading breakdown:
  • (1 pt) submitted a valid lab11.tar.gz file with all the required files using the submission procedure below
  • (1 pt) content in "lab11a-12000.script", "lab11a-12002.script", and "lab11a-12004.script" are correct
  • (1 pt) content in "lab11b-12000.script", "lab11b-12002.script", "lab11b-12004.script", and "lab11b-12012.script" are correct
  • (1 pt) "Makefile" works for "make lab11a"
  • (1 pt) source code of your node looks right
Minimum deduction is 0.5 pt for anything that's incorrect. Please note that for the "Makefile" item, you can only get credit for it if your "source code" is relevant to this lab; therefore, you can only get as many points as the "source code" item in the best case.

Please keep in mind that even though lab grading is "light", it doesn't mean that you can just put anything into your submission! It's still your responsibility to make sure that the files in your submission contains information that's relevant to the tests you were supposed to run. Use the "more" command to view your script/log files to make sure that they contain the right information. If a file has the wrong stuff in it, you should delete it and create the file again and verify. If most of the stuff in your script/log files are wrong and you did not notice it, we will most likely have to take points off.

To submit your work, you must first tar all the files you want to submit into a tarball and gzip it to create a gzipped tarfile named "lab11.tar.gz". Then you upload "lab11.tar.gz" to our Bistro submission server.

Change into the "lab11" directory you have created above and enter the following command to create your submission file "lab11.tar.gz" (if you don't have any ".h" files, don't include "*.h*" at the end):

    tar cvzf lab11.tar.gz lab11*.script Makefile *.c* *.h*
    ls -l lab11.tar.gz
The last command shows you how big the created "lab11.tar.gz" file is. If "lab11.tar.gz" is larger than 1MB in size, the submission server will not accept it.

If you use an IDE, the IDE may put your source code in subdirectories. In that case, you need to modify the commands above so that you include ALL the necessary source files and subdirectories (and don't include any binary files) ane make sure that your code can be compiled without the IDE since the grader is not allowed to use an IDE to compile your code.

You should read the output of the above commands carefully to make sure that "lab11.tar.gz" is created properly. If you don't understand the output of the above commands, you need to learn how to read it! It's your responsibility to ensure that "lab11.tar.gz" is created properly.

To check the content of "lab11.tar.gz", you can use the following command:

    tar tvf lab11.tar.gz
Please read the output of the above command carefully to see what files were included in "lab11.tar.gz" and what are their file sizes and make sure that they make sense.

Please enter your USC e-mail address and your submission PIN below. Then click on the Browse button and locate and select your submission file (i.e., "lab11.tar.gz"). Then click on the Upload button to submit your "lab11.tar.gz". (Be careful what you click! Do NOT submit the wrong file!) If you see an error message, please read the dialogbox carefully and fix what needs to be fixed and repeat the procedure. If you don't know your submission PIN, please visit this web site to have your PIN e-mailed to your USC e-mail address.

When this web page was last loaded, the time at the submission server at merlot.usc.edu was 27Nov2025-18:59:18. Reload this web page to see the current time on merlot.usc.edu.

USC E-mail:    @usc.edu
Submission PIN:  
Event ID (read-only):  
Submission File Full Path:  
   
 
If the command is executed successfully and if everything checks out, a ticket will be issued to you to let you know "what" and "when" your submission made it to the Bistro server. The next web page you see would display such a ticket and the ticket should look like the sample shown in the submission web page (of course, the actual text would be different, but the format should be similar). Make sure you follow the Verify Your Ticket instructions to verify the SHA1 hash of your submission to make sure what you did not accidentally submit the wrong file. Also, an e-mail (showing the ticket) will be sent to your USC e-mail address. Please read the ticket carefully to know exactly "what" and "when" your submission made it to the Bistro server. If there are problems, please contact the instructor.

It is extreme important that you also verify your submission after you have submitted "lab11.tar.gz" electronically to make sure that every you have submitted is everything you wanted us to grade. If you don't verify your submission and you ended up submit the wrong files, please understand that due to our fairness policy, there's absolutely nothing we can do.

Finally, please be familiar with the Electronic Submission Guidelines and information on the bsubmit web page.