Sockets, message rate limiter Java

Imagine I have a server that can produce messages at a rate of 10,000 messages per second. But my client can only receive up to a maximum of 1000 messages per second.

System 1
If my system sends 1000 messages in the 1st milisecond and then does nothing for the remaining 999ms.

System 2
My system sends 1 message per milisecond, hence in 1000ms (1second) it will send 1000 messages.

Q1) Which system is better given that the client can handle a maximum of 500 messages per second?

Q2) What will be the impact of system 1 on the client? Will it overwhelm the client?


Wil it overwhelm the client: It depends of the size of your messages, and the socket buffer size. The messages the sender sends are buffered. If the client cannot consume because the buffer is full, the output stream the sender is using will block. When the client has consumed some messages, the sender can continue writing as his OutputStream gets unblocked.

A typical buffer size on a windows system used to be 8192 bytes, but size can differ by the OS and settings in the OS.

So System 1 will not overwhelm the client, it will block at a certain moment.

What is the best approach merely depends on the design of your application.

For example: I had a similar issue while writing to an Arduino via USB (not socket-client, but otherwise the same problem). In my problem, buffered messages where a problem because it were positions of a face tracking camera. Buffered positions were no longer relevant when the Arduino read them, but it MUST process them because such a buffer is a queue, and you can only get the most recent if you read out the old one’s. The Arduino could never keep up with the messages being produced, because by the time a new position reached the Arduino code, it was outdated. So that was an “overwhelm”.

I resolved this by using bi-directional communication. The Arduino would send a message to the producer saying: READY (to receive a message). Then the producer would send one (up-to-date) face tracking position. Then the Arduino repositioned the camera and requested a new message. This way, there was a kind of flow control, that prevented the producer to overflow the Arduino.

  1. Neither is better. TCP will alter the actual flow whatever you do yourself.
  2. Neither will overwhelm the client. If the client isn’t keeping up, its socket receive buffer will fill up, and so wil your socket send buffer, and eventually you will block in send, or get EAGAIN/EWOULDBLOCK if you’re in non-blocking mode.