[ This content is protected and may not be shared, uploaded, or distributed. ]
(Please also check out PA3 FAQ.)
Lab 8 has two parts:
Part A (lab8a) - closing a connection from the console (useful for PA3):
This lab is web server-only.
Please continue to use "lab4data" as the root directory of your server for the rest of this lab.
In this part of the lab, you need to add a "close" command to your console thread to close a connection.
Please first do the following:
- Create an empty directory (call it "lab8") and change directory into it.
- Copy your code for Part C of Lab 7 into the current directory, create a Makefile
so that when you type "make lab8a", an executable called lab8a will be created.
- We will use the test data from Lab 4.
Please download lab4data.tar.gz into that directory and type:
tar xvf lab4data.tar.gz
This should create a subdirectory called "lab4data" with a bunch of files in it.
The commandline syntax for lab8a is: "lab8a PORT LOGFILE".
The problem with Part C of Lab 7 is that you have to wait for all downloads to complete
before you can quit your program. For this part of the lab, you need to add a "close" console command
and tweek the behavior of other console commands as follows:
| close # |
Close connection #.
If # is not an active connection number, you must print the following to cout:
Invalid or inactive connection number: #\n
Otherwise, you must locate the corresponding Connection object,
[BC: fixed 3/1/2023] shutdown the corresponding socket
(please note that for
Part B, you should not close the socket here since closing the socket is the job of the reaper thread),
change the socket in the Connection object to be (-2) to mean that a request has been made to close the connection,
and print the following to cout:
[#]\tClose connection requested.\n
|
| status |
Print status information about all connections (in any order).
For each connection, you must print the following to cout:
[#]\tClient: IP:PORT, Socket: SOCKET_FD, KB sent: NUMBER\n
where IP:PORT is where the client is connecting from,
SOCKET_FD is the value of the socket file descriptor in the Connection object
and NUMBER is the number of times you have called better_write() to send 1KB (or less for the last chuck of data)
of response body to the corresponding client.
If there are no active connections, you must print:
No active connections.\n
|
| quit |
Do the following in your console thread:
- close all active connections
- shutdown and close the listening socket and set the listening socket to (-1)
- console-thread must then self-terminate
Your main thread's accept() should then return with an error condition,
and you would break out of the infinite loop
- once you are out of the infinite loop, your main thread must join with the console thread
- finally, your main thread must join with all the connection-handling threads and self-terminate
(it's probably not a good idea to join with a thread if you have the mutex locked unless you are 100%
sure that the target thread will not need to lock the mutex)
|
| help |
You must print the following to cout:
Available commands are:\n
\tclose #\n
\thelp\n
\tstatus\n
\tquit\n
|
The above would require you to modify the Connection object a little bit. Your new Connection object should look like
(please note that this is slightly different from what's in the Lab 8 video and slides):
class Connection {
public: /* I know this is bad form, but this make things easier (and shorter) */
int conn_number; /* -1 means that the connection is not initialized properly */
int socket_fd; /* -1 means closed by connection-handling thread, -2 means close by console thread and connection may still be active */
int orig_socket_fd; /* copy of the original socket_fd */
shared_ptr<thread> thread_ptr; /* shared pointer to a thread object */
int kb_sent; /* number of KB (including partial KB at the end) of response body written into the socket */
Connection() : conn_number(-1), socket_fd(-1), thread_ptr(NULL), kb_sent(0) { }
Connection(int c, int s, shared_ptr<thread> p) { conn_number = c; socket_fd = orig_socket_fd = s; thread_ptr = p; kb_sent = 0; }
};
The main difference between a Lab 7 Connection object and the Connection object here
is that when a connection is closed by the console thread (i.e., shutdown() but not close() the socket_fd) in this lab,
you need to set socket_fd to (-2) afterwards.
This way, the connection-handling thread would know that it should self-terminate as soon as possible.
As in Lab 7, when a connection-handling thread is about to self-terminate, it must set socket_fd to (-1).
Please understand that socket_fd can be accessed by both the console thread and the connection-handling thread simultaneously.
Therefore, you must only access socket_fd when you have the mutex locked.
Please note that when you shutdown a client socket, the corresponding connection-handling thread can
be either sleeping or writing into the client socket (i.e., socket_fd). If it's writing into the client socket,
better_write() should return immediately with an error code. If it's sleeping (say, for 1 second),
it can take up to a second for it to wake up. Then when it writes to the socket the next time, it will get an error immediately.
If one second feels too long, you can actually have the connection-handling thread sleeps less. For example, if you want
the connection-handling thread to wake up within 250 millisecond,
you can simply modify the code in the Lab 6 pseudo-code and change:
usleep(usec_to_sleep)
to:
if (usec_to_sleep > 250000) then
usleep(250000)
else
usleep(usec_to_sleep)
This way, when the thread wakes up from usleep() 0.25 seconds later, you should check to see if socket_fd is (-2).
If it is, the thread should self-terminate soon and not call usleep() again.
When you are done with implementing lab8a, please do the following:
- Change directory into the "lab8" directory mentioned above.
- Type "script lab8a.script" to start a transcript. Then type:
uname -a
cat /etc/os-release
make clean
make lab8a
./lab8a 12345 lab8a.logfile
- You should see the command prompt. Type the following commands:
help
status
- Run two more Terminal windows and cd into the same "lab8" directory.
- In the second Terminal window, type:
wget -O x http://localhost:12345/textbooks-1-small.jpg
- In the third Terminal window, type:
wget -O y http://localhost:12345/viterbi-seal-rev-770x360.png
- Make sure wget is seeing 1KB/sec (you should try to get within 10% of 1KB/sec, at least at the beginning of the downloads).
- In the first window, type "status" and make sure that it sees two active connections (connections 1 and 2).
- In the first window, type "close 1". The corresponding wget programs should reconnect using a different connection.
(On viterbi-scf1.usc.edu and viterbi-scf2.usc.edu, auto-reconnect does not appear to work and
wget would simply die with an error message saying that it cannot write to a file. In this case, you just have to
run the wget program again manually.)
- In the first window, type "status" and make sure that it sees two active connections (connections 2 and 3).
- In the first window, type "close 2". The other wget program should reconnect using a different connection.
(On viterbi-scf1.usc.edu and viterbi-scf2.usc.edu, auto-reconnect does not appear to work and
wget would simply die with an error message saying that it cannot write to a file. In this case, you just have to
run the wget program again manually.)
- In the first window, type "status" and make sure that it sees two active connections (connections 3 and 4).
- In the first window, type "quit" and make sure that the server has gracefully shutdown.
- Repeat. Start your server in the first window and start the same downloads in the 2nd and 3rd window. Wait 5 seconds and type:
status
quit
- Type:
cat lab8a.logfile
and make sure that you can see the following
within a run of the server (i.e., between the "Server started" and the "Server stopped" log entries):
- there is exactly one "Connection closed" log entry for each connection number
- Type "exit" to close the transcript.
Alternatively, you can also do everything inside one Terminal and run tmux.
You can split the screen vertically into 3 panes.
Part B (lab8b) - joining dead threads with a reaper thread (useful for PA3):
One problem with Part A of this lab is that our connection list keeps getting
longer and longer because we only clean it up when the main thread is about to self-terminate.
In this part of the lab, we will add a reaper thread to clean up a dead connection soon after
the connection-handling thread has died.
Please make a copy of your "lab8a.cpp" and call it "lab8b.cpp" and add another target in the Makefile
so that when you type "make lab8b", an executable called lab8b will be created.
The commandline syntax for lab8b is: "lab8b PORT LOGFILE".
The reaper thread runs an infinite loop. At the top of the loop, it will sleep for 0.25 seconds When it wakes up,
it will lock the mutex, iterate through the connection list to see if the connection has its socket_fd equal to (-1). If it is, then it
means that the corresponding thread is either dead already or about to die. Therefore, it can join with that thread
and expects join() to return quickly. When join() returned, you must print the following line into the log file:
[TIMESTAMP] [#]\tReaper has joined with connection thread\n
where # is the corresponding connection number.
You must then remove that Connection object from the connection list and proceed to examine the next connection.
After all connections have been examined, it should unlock the mutex and go back to the top of the infinite loop.
Please be extra careful when you are iterating through a list while removing/erasing elements from that list.
If you are using a vector, since vector is implemented as a dynamic array, when you erase an element
of an array, you need to shuffle all the elements after the element you are erasing up one position inside the array.
If you break out of the loop immediately after you call erase() or if you are not iterating through the list, then there is no problem.
But if you continue to iterate to the next element of the vector using an array index, you may have skipped an element!
Therefore, I would recommend that you do the following if you want to iterate through a vector v while erasing elements from v
(this is the first approach mentioned on this web page).
In the code below, I'm assuming that you have a vector of shared pointers to Connection objects.
for (vector<shared_ptr<Connection>>::iterator itr = v.begin(); itr != v.end(); ) {
shared_ptr<Connection> connection_ptr = (*itr);
if (should_erase_this_connection(connection_ptr)) {
itr = v.erase(itr);
} else {
itr++;
}
}
We also need to figure out a way to ask the reaper thread to self-terminate after the "quit" command has been entered into
the console. Since the console thread will set the listening socket to a value of (-1) after the "quit" console command has been entered,
the reaper thread should also check the value of the listening socket after it locked the mutex. If the value of the listening socket is (-1)
and if the connection list is empty, it should unlock the mutex immediately and break out of the infinite loop and self-terminate.
If the connection list is not empty, even if the listening socket is (-1), it should go back to sleep for 0.25 second continue to stay inside the infinite loop to wait for
all connection-handling threads to die.
When you are done with implementing lab8b, please do the following:
- Change directory into the "lab8" directory mentioned above.
- Type "script lab8b.script" to start a transcript. Then type:
make clean
make lab8b
./lab8b 12345 lab8b.logfile
- You should see the command prompt. Type the following commands:
help
status
- Run two more Terminal windows and cd into the same "lab8" directory.
- In the second Terminal window, type:
wget -O x http://localhost:12345/usc-seal-1597x360.png
- In the third Terminal window, type:
wget -O y http://localhost:12345/viterbi-seal-rev-770x360.png
- Make sure wget is seeing 1KB/sec (you should try to get within 10% of 1KB/sec, at least at the beginning of the downloads).
- In the first window, type "status" and make sure that it sees two active connections (connections 1 and 2).
- In the first window, type "close 1". The corresponding wget programs should reconnect using a different connection.
(On viterbi-scf1.usc.edu and viterbi-scf2.usc.edu, auto-reconnect does not appear to work and
wget would simply die with an error message saying that it cannot write to a file. In this case, you just have to
run the wget program again manually.)
- In the first window, type "status" and make sure that it sees two active connections (connections 2 and 3).
- In the first window, type "close 2". The other wget program should reconnect using a different connection.
(On viterbi-scf1.usc.edu and viterbi-scf2.usc.edu, auto-reconnect does not appear to work and
wget would simply die with an error message saying that it cannot write to a file. In this case, you just have to
run the wget program again manually.)
- In the first window, type "status" and make sure that it sees two active connections (connections 3 and 4).
- In the first window, type "quit" and make sure that the server has gracefully shutdown.
- Repeat. Start your server in the first window and start the same downloads in the 2nd and 3rd window. Wait 10 seconds and type:
status
quit
- Type:
cat lab8b.logfile
and make sure that you can see the following
within a run of the server (i.e., between the "Server started" and the "Server stopped" log entries):
- there is exactly one "Connection closed" log entry for each connection number
- there is exactly one "Reaper has joined with connection thread" log entry for each connection number
- Type "exit" to close the transcript.
Alternatively, you can also do everything inside one Terminal and run tmux.
You can split the screen vertically into 3 panes.
All pseudo-code is incomplete and error checking is often left out in pseudo-code.
Since some details are left out, depending on you how write your code, you may create race conditions
and you may need to fix your code so that your program won't freeze or crash.
Feel free to send your questions (and not your code) to the instructor.
Pseudo-code for lab8b:
Code in red are critical section codes.
Console Thread
do forever
write prompt
cmd = read command from user
if cmd is "status" then
lock mutex
iterate through connection list and print status
unlock mutex
else if cmd is "close #" command then
lock mutex
c = find connection object of connection #
if c != null and c.socket ≥ 0 then
shutdown c.socket
c.socket = (-2)
print required message
else
print required message
end-if
unlock mutex
else if cmd is "quit" then
break out of infinite loop
else if cmd is "help" or unrecognized then
print required message
end-if
end-do
lock mutex
shutdown and close listening socket
set listening socket to (-1)
for each connection c in connection list
if c.socket ≥ 0 then
shutdown c.socket
c.socket = (-2)
end-if
end-for
unlock mutex
Reaper Thread
do forever
sleep for 0.25 seconds
lock mutex
if listening socket is (-1) and connection list is empty
unlock mutex
break out of infinite loop
else
for each connection c in connection list
if c.socket == (-1) then
join with thread c.thread
log required message
remove c from connection list
end-if
end-for
end-if
unlock mutex
end-do
Main Thread
log required message
create console thread
create reaper thread
do forever
client socket = my_accept()
if client socket is (-1) then
break out of infinite loop
end-if
lock mutex
if listening socket is (-1) then
shutdown and close client socket
unlock mutex
break out of infinite loop
end-if
create connection object c
create connection-handling thread t
set c.thread to t
add c to connection list
unlock mutex
end-do
join with console thread
join with reaper thread
log required message
Connection-handling Thread
c = connection object in argument of first procedure
lock mutex
unlock mutex
not_closed = true
while not_closed
read HTTP request
if disconnected then
break out of while-loop
else if should respond with error message then
send error response
else
send HTTP header and blank line
while there is more data to send
read 1KB from file
send 1KB into c.socket
lock mutex
c.KB_sent++
if c.socket is (-2) or listening socket is (-1) then
unlock mutex
not_closed = false
break out of while-loop
end-if
unlock mutex
sleep
lock mutex
if c.socket is (-2) or listening socket is (-1) then
unlock mutex
not_closed = false
break out of while-loop
end-if
unlock mutex
end-while
end-if
end-while
lock mutex
if c.socket ≥ 0 then
shutdown c.socket
end-if
set c.socket to (-1)
close c.orig_socket
log required message
unlock mutex
To make your server more responsive, instead of just "sleep" above,
you should modify the pseudo-code there in a loop as suggested.
Please note that the pseudo-code for the connection-handling thread is for
part A of this lab. For part B, don't close the socket and leave that to the reaper thread.
(Actually, closing the socket doesn't have to be done in the reaper thread.
You can close the socket in the connection-handling thread if you are sure that the thread won't write into the socket.
Therefore, you can close the socket at the same place as in the pseudo-code.)
Below is the grading breakdown:
- (1 pt) submitted a valid lab8.tar.gz file with all the required files using the submission procedure below
- (1 pt) content in "lab8a.script" and "lab8a.logfile" are correct
- (1 pt) content in "lab8b.script" and "lab8b.logfile" are correct
- (1 pt) "Makefile" works for "make lab8a" and "make lab8b"
- (1 pt) source code of your server programs look right
Minimum deduction is 0.5 pt for anything that's incorrect.
Please note that for the " Makefile" item, you can only get credit for it if your "source code" is relevant to this lab; therefore, you can only get as many points as the "source code" item
in the best case.
Please keep in mind that even though lab grading is "light", it doesn't mean that you can just put anything
into your submission! It's still your responsibility to make sure that the files in your submission contains
information that's relevant to the tests you were supposed to run.
Use the "more" command to view your script/log files to make sure that they contain the right information.
If a file has the wrong stuff in it, you should delete it and create the file again and verify.
If most of the stuff in your script/log files are wrong and you did not notice it, we will most likely have to take points off.
To submit your work, you must first tar all the files you want to submit into a tarball and
gzip it to create a gzipped tarfile named " lab8.tar.gz".
Then you upload " lab8.tar.gz" to our Bistro submission server.
Change into the "lab8" directory you have created above and enter the following command
to create your submission file "lab8.tar.gz" (if you don't have any ".h" files, don't include "*.h*" at the end):
tar cvzf lab8.tar.gz lab8*.script lab8*.logfile Mak* *.c* *.h*
ls -l lab8.tar.gz
The last command shows you how big the created " lab8.tar.gz" file is.
If " lab8.tar.gz" is larger than 1MB in size, the submission server will not accept it.
If you use an IDE, the IDE may put your source code in subdirectories. In that case,
you need to modify the commands above so that you include ALL
the necessary source files and subdirectories (and don't include any binary files)
ane make sure that your code can be compiled without the IDE since the grader is not allowed to use an IDE to compile your code.
You should read the output of the above commands carefully to make sure that "lab8.tar.gz" is created properly.
If you don't understand the output of the above commands, you need to learn how to read it!
It's your responsibility to ensure that "lab8.tar.gz" is created properly.
To check the content of "lab8.tar.gz", you can use the following command:
tar tvf lab8.tar.gz
Please read the output of the above command carefully to see what files were included in " lab8.tar.gz"
and what are their file sizes and make sure that they make sense.
Please enter your USC e-mail address and your submission PIN below. Then click on the Browse button
and locate and select your submission file (i.e., "lab8.tar.gz").
Then click on the Upload button to submit your "lab8.tar.gz".
(Be careful what you click! Do NOT submit the wrong file!)
If you see an error message, please read the dialogbox carefully and fix what needs to be fixed and repeat the procedure.
If you don't know your submission PIN, please visit this web site to have your PIN e-mailed to your USC e-mail address.
When this web page was last loaded, the time at the submission server at merlot.usc.edu was
27Nov2025-18:59:11.
Reload this web page to see the current time on merlot.usc.edu.
If the command is executed successfully and if everything checks out,
a ticket will be issued to you to let you know "what" and "when"
your submission made it to the Bistro server. The next web page you
see would display such a ticket and the ticket should look like
the sample shown in the submission web page
(of course, the actual text would be different, but the format should be similar).
Make sure you follow the Verify Your Ticket instructions
to verify the SHA1 hash of your submission to make sure what you did not accidentally submit the wrong file.
Also, an e-mail (showing the ticket) will be sent to your USC e-mail address.
Please read the ticket carefully to know exactly "what" and "when"
your submission made it to the Bistro server.
If there are problems, please contact the instructor.
It is extreme important that you also verify your submission
after you have submitted "lab8.tar.gz" electronically to make
sure that every you have submitted is everything you wanted us to grade.
If you don't verify your submission and
you ended up submit the wrong files, please understand that due to our fairness policy,
there's absolutely nothing we can do.
Finally, please be familiar with the Electronic Submission Guidelines
and information on the bsubmit web page.
|
