TCP states are well-known but how they are handled on different clients, and or servers are not!....
TCP automatically attempts to retry up to a certain amount of time before it times-out, sends the proper responses and cleans up resources. This is only on one side whereby the stack has control. It's up to the other side to determine is a particular socket (session) should be closed.
There are things you can test knowing this information...
One side starts a session and does nothing... (When will the other side send the RST or FIN).
One side starts and does something but doesn't finish (When will the other side close?)
One side starts and does something wrong, maybe sends a response that wasn't expected (again a test to see if the other side recovers)...
Knowing whether or not a particular application is in-session depends on the application under test. For Example: FTP always has two sockets one for control and the other for data transfer. FTP Stays read for additional data transactions as long as the control session is not closed.
There are two ways to detect if an application (on the network) is up and running. One is called Port Pinging. All this does is attempt to open a socket to the IP/Port (Socket). If it opens then the application is listening! The other is to open and then send data to see it the application is able to receive content. From there you have any number of permutations you can dream up to "really test" this thing.
So to test what you asked for automatically. You have to think like a tester...
Start first with both sides up and running (Sunny day test).
After first test passes run 2nd test which only opens connection and does nothing else... How long until other side times out?
Now open connection send data for 3rd test but don't complete it properly, just stop sending data without closing? What happens?
Now try to open a connection when the other side is not listening...
Now try to bombard the other side with too many clients and data....
Fun indeed but it takes a lot of programming effort.