Different ways of handling real-time data (Long Polling, Short Polling, WebSockets)

Before we even discuss the various ways of handling real-time data. Let's first understand how data is transferred in the normal circumstances aka. HTTP requests.

As shown below, HTTP requests work pretty straightforward, client sends some request then the server sends some response.

Now as you can imagine, if we were to use HTTP as a way to send and recieve data in real-time, it would not only be heavy on the server but would also be inefficient and resource-intensive. Why?

Everytime an HTTP request is made, a new connection is established to the server. So, having to establish numerous connections can be really resource-intensive on the server.

Consecutively, this would lead to high latency as well. (Imagine a server dies because of the 1000+ connections you made on a free plan so now not only is your server dead but the person at the other end is also waiting for your reply 💀)

There's also encoding and decoding with HTTP; it's a text-based protocol, so every single time you need to send something, you'll need to first encode, then decode on the server. Just writing this makes me hate HTTP for real-time communication!

Alright Zaid, we get it- HTTP is a nightmare for real-time communication, 
so now what?

Well, to that I say- "Hold your thoughts a bit! There is a workaround 😉"

Long Polling

What if, instead of making a new request every time we need to get data, we keep the TCP connection open and wait for the server to respond when new data is available? This technique is called Long Polling.

Here's how it works:

1. The client sends an initial request to the server.

2. The server holds the TCP connection open, waiting for new data to become available.

3. When new data is available, the server responds to the pending request with the new data.

4. The client processes the response and immediately sends a new request to the server, repeating the cycle.

Wait, if you understand closely, you'll get to know one of its drawbacks- ONLY THE SERVER RESPONDS!!

This is not ideal, a real-time connection should be like a freeway for both the client and server to send the data! (am i foreshadowing.. ? 😁)

So this brings to our first drawback of Long Polling-

1. It is a unidirectional communication method, meaning the server can only send data to the client in response to a client request.

There are other drawbacks as well, I won't bore you with those so here they are-

2. It can be inefficient if there are long periods of inactivity, as the client and server are still maintaining an open connection.

3. It can be challenging to implement correctly, especially when dealing with timeouts and connection management.

Okay okay, well Zaid, dumb question but if you have "Long" Polling, 
do we also have "Short" Polling?

Yes! We actually do 😆

Short Polling

The only difference between Long Polling and Short Polling is that Short Polling is- well, short.

What i mean by that is even if the server does not have data available it will send whatever response it has at that moment.

Now, because the server sends whatever it has, the client needs to make multiple requests to the server until the data becomes available.

I'm sure you can guess the drawbacks of this, well here they are-

1. Inefficient: Short polling generates a lot of unnecessary network traffic and server load, as the client is constantly sending requests, even when there is no new data available.

2. High Latency: There is an inherent delay between when new data becomes available on the server and when the client receives it, as the client has to wait for the next polling interval before sending a new request.

3. Scalability Issues: As the number of clients increases, the server has to handle a large number of requests, which can lead to performance issues and increased resource consumption.

Phew... okay now let's finally talk about Websockets!

WebSockets

Both long polling and short polling are workarounds for the limitations of HTTP. When it comes to real-time communication, however, there is a better solution: WebSockets.

Okay, now think of websockets like a wire with two plugs at the end which are connected to the two sockets at the either end creating a path for the current to flow with ease.

Now in that analogy, replace current with data and the two ends you connected with client and server respectively. There you go, that's a websocket connection.

Now of course, it's not that straightforward, "the pathway" is really established by first doing a "handshake" between the server and the client.

After the initial handshake, the client and server can exchange data in both directions, making it a truly bi-directional communication channel. This makes WebSockets ideal for real-time applications like chat applications, online games, and collaborative tools.

Here's how it looks like-

WebSockets also (most definitely) have several advantages over traditional HTTP-based communication:

• Efficient: WebSockets use a lightweight protocol that reduces overhead and improves performance.

• Low Latency: Data is transmitted as soon as it becomes available, without the need for polling or waiting for a response.

• Scalable: WebSockets can handle a large number of concurrent connections with minimal resource consumption.

• Full-duplex: Both the client and server can initiate communication and send data simultaneously.

---

Well that's all folks, i hope you enjoyed reading my blog and I hope you understood the different ways of handling real-time data.

(some shamless plugs)

Follow me on Twitter (or, X)- @zaidahmad__dev

Connect with me on LinkedIn- @zaidahmad25

(okay i'll stop now)

--
Have a great day (or, night) !

btw, there is an even better way of handling real-time data than websockets, let me know if you want to read about that as well below 👇