The TCP three-way handshake (or three step handshake) is a fundamental process in establishing a reliable connection between two devices over a TCP/IP network. It's essential for ensuring that both ends are ready to exchange data and are in sync with each other.

This is a widely used transport layer protocol in computer networks. It ensures reliable, ordered, and error-checked delivery of data between devices. TCP is responsible for breaking down larger data into smaller packets, managing data flow, and guaranteeing that data arrives intact and in the correct order.

Before two devices can start exchanging data, they need to establish a connection. This connection setup involves negotiating various parameters, such as initial sequence numbers and window sizes.

Three Steps of the Handshake:

• Step 1: SYN (Synchronise): Explain that the client initiates the handshake by sending a TCP segment with the SYN flag (synchronise) set to the server. The client's initial sequence number (ISN) is included in this segment.
• Step 2: SYN-ACK (Synchronise-Acknowledge): Describe how the server responds with a TCP segment that has both the SYN and ACK flags set. The server also generates its own ISN and acknowledges the client's ISN.
• Step 3: ACK (Acknowledge): Detail how the client sends an ACK segment to the server in response. The ACK flag is set, and the client acknowledges the server's ISN.

Flow Control and Sequence Numbers:

• Sequence Numbers: Explain the role of sequence numbers in tracking the order of data segments and detecting any missing or out-of-order segments.
• Acknowledgment Numbers: Describe how acknowledgment numbers are used to acknowledge received segments and indicate the next expected sequence number.

Error Handling and Timeouts:

• Timeouts: Discuss the importance of timeouts in the handshake process. If a party doesn't receive an expected response within a certain time, it assumes a problem and might retransmit the initial SYN.
• Error Handling: Briefly touch on error scenarios like duplicate SYN segments, out-of-order responses, and how they're managed to ensure a successful handshake.

Connection Establishment and Termination: Once the three-way handshake is complete, the connection is established, and data transfer can begin. Similarly, when either party wants to terminate the connection, a four-way handshake is used to ensure all remaining data is transmitted before the connection is closed.

These are examples of how the TCP three-way handshake is used in the real-world:

Web Browsing: When you open a web browser and enter a URL to visit a website, your browser initiates a TCP connection to the web server hosting that website. The three-way handshake ensures that your browser and the server are in sync before any data transfer occurs. Here's how it works:

Step 1: SYN - Your browser sends a SYN segment to the web server, indicating its intention to establish a connection.

Step 2: SYN-ACK - The web server responds with a SYN-ACK segment, acknowledging your browser's request and indicating its readiness to establish a connection.

Step 3: ACK - Your browser sends an ACK segment back to the server, confirming the connection establishment. Now your browser can request and receive the website's content.

Email Communication: When you send an email, your email client connects to an email server using the SMTP (Simple Mail Transfer Protocol). The TCP three-way handshake ensures that your email client and the server are synchronised before transmitting the email. Here's how it happens:

Step 1: SYN - Your email client sends a SYN segment to the email server to initiate a connection.

Step 2: SYN-ACK - The email server responds with a SYN-ACK segment, acknowledging the connection request and indicating its readiness.

Step 3: ACK - Your email client sends an ACK segment back to the server, confirming the connection. Now you can start sending the email's content.

File Transfer: File transfer protocols like FTP (File Transfer Protocol) rely on the TCP three-way handshake to establish connections for secure and reliable data transfer. Here's how it's used:

Step 1: SYN - The client sends a SYN segment to the server to initiate a connection for file transfer.

Step 2: SYN-ACK - The server responds with a SYN-ACK segment, acknowledging the client's request and indicating readiness for the transfer.

Step 3: ACK - The client sends an ACK segment back to the server, confirming the connection establishment. Now the actual file transfer can take place.

In all these examples, the TCP three-way handshake ensures that both communicating parties are prepared for data exchange. This process minimises the risk of data loss, prevents out-of-order data, and helps maintain the reliability of the communication.

Security Considerations:

TCP three-way handshake-related attacks are a concern in networking, and there are measures to mitigate them. Here are a few potential attacks and how to prevent them:

SYN Flood Attack:

In a SYN flood attack, an attacker sends a large number of fake SYN segments to a target server, overwhelming the server's resources and preventing it from accepting legitimate connections.

Prevention Measures:

SYN Cookies: Implement SYN cookies in the server's TCP stack. SYN cookies encode some of the necessary connection information into the initial SYN-ACK response. This allows the server to validate connection requests without maintaining a large number of half-open connections.

Rate Limiting: Monitor incoming connection requests and apply rate limiting to prevent excessive SYN requests from a single source within a short time. Really helpful against bots.

Firewalls and Intrusion Detection Systems (IDS): Utilise firewalls and IDS systems to detect and filter out unusual traffic patterns associated with SYN flood attacks.

Man-in-the-Middle (MitM) Attack during Handshake:

An attacker intercepts the initial SYN segment from the client and pretends to be the server, sending a fake SYN-ACK to the client. The client then responds with an ACK, establishing a connection with the attacker instead of the intended server.

Prevention Measures:

Secure Communication Channels: Use encryption protocols like TLS (Transport Layer Security) to ensure that the connection establishment and subsequent communication are secured against eavesdropping and tampering.

Certificate Verification: When using encryption, ensure the server's identity by validating its digital certificate during the handshake.

Spoofed Source Address Attack:

An attacker sends SYN segments with spoofed source IP addresses to a victim server. The server responds with SYN-ACK segments to the spoofed addresses, consuming resources and potentially causing the victim server to become unresponsive.

Prevention Measures:

Ingress Filtering: Implement ingress filtering at network boundaries to prevent packets with spoofed source IP addresses from entering the network.

Reverse Path Forwarding (RPF): Employ RPF checks to verify that incoming packets have valid source IP addresses based on routing information.

Connection Hijacking:

Attack Description: An attacker intercepts a legitimate TCP connection in progress, guesses or learns the sequence numbers, and injects or manipulates data in the connection.

Prevention Measures:

Sequence Number Randomization: Ensure that sequence numbers are randomised and not predictable. This makes it significantly harder for attackers to guess or manipulate the sequence numbers.

Encryption: Use encryption to secure the data being transmitted, making it difficult for attackers to understand and manipulate the data.

Denial of Service (DoS) Attack during Handshake:

Attack Description: An attacker overwhelms a server by initiating numerous legitimate connection requests, consuming server resources and preventing it from handling other connections.

Prevention Measures:

Rate Limiting: Implement rate limiting to restrict the number of connection requests from a single IP address within a given time frame.

Connection Tracking: Use connection tracking mechanisms to identify and drop connection requests that exceed reasonable thresholds.

Remember that a multi-layered approach to network security is essential. While these prevention measures can help mitigate risks, no solution is entirely foolproof. Regular security assessments, updates, and staying informed about emerging threats are crucial to maintaining a secure network environment.

Sources and further reading

Harris, Shon & Maymí, Fernando. CISSP EXAM GUIDE Seventh Edition. New York McGraw Hill Education, 2016.

Kerry A. McKay David A. Cooper, NIST Special Publication 800-52 Revision 2, Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations, August 2019

GeeksforGeeks. (2021). TCP 3 way Handshake process. GeeksforGeeks. https://www.geeksforgeeks.org/tcp-3-way-handshake-process/