Its a flaw in the way Shoutbox actually works. It needs to be issued an update signal before it calls for unique postnumbers it doesn't already have, to store locally. In conventional chat, the update signal (affectionately called ping/pong (pong?ping!) based on call and response) is updated every few miliseconds. To conserve bandwidth and for massively multiuser chats, Shoutbox reduces the pong count significantly to a hard call ratio. It hurts the latency hugely but its an enormous saving in bandwidth and allows the chat to scale to exponentially larger sizes, with some shoutboxes (famously the forks used by Twitch.tv) scaling massively to thousands and tens of thousands of users properly, with the pong being updated and distributed on a peer-basis so other users pick up the slack.
In a nutshell, the more users we get, the faster it goes. The minimum dropoff is around 3 users. Around 7 or so, the number of users means if two or more are having bandwidth problems, sometimes the handshake (the exchange from OTHER users who already have the data, NOT the server) can fail and as a result the clientend throws a wobbly.