You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, UserALE bundles logs into a buffer and flushes the buffer on a regular interval or in response to certain event triggers before POSTing them to the REST API url. This works well in most circumstances, but adds unnecessary network overhead. Also if requests are dropped we have no back up mechanism.
Solution
Add support for websocket connections if the backend REST API supports them. Upon first connection to the back-end, we would check to see if web sockets are supported, if so, we would upgrade the connection.
This achieves two things:
Minimizes network overhead
Adds redundancy
We would still store logs in a client-side memory buffer and flush them over the socket. However, the socket reduces network overhead. And if the connection ever gets dropped before the buffer has been flushed, then we can concurrently work to re-establish the connect while continuing to store event logs in the buffer. Upon reconnect, you flush all logs to relieve back-pressue. In this way, no logs are ever dropped. As an escape hatch, if the buffer exceeds a certain memory threshold, then you can treat it like a FIFO queue and start dumping old logs to avoid a memory leak.
The text was updated successfully, but these errors were encountered:
EandrewJones
changed the title
Features(streaming): support streaming logs with webscoket
Feature(streaming): support streaming logs with webscoket
May 7, 2024
EandrewJones
changed the title
Feature(streaming): support streaming logs with webscoket
Feature(streaming): support streaming logs with websocket
May 7, 2024
Problem
Currently, UserALE bundles logs into a buffer and flushes the buffer on a regular interval or in response to certain event triggers before POSTing them to the REST API url. This works well in most circumstances, but adds unnecessary network overhead. Also if requests are dropped we have no back up mechanism.
Solution
Add support for websocket connections if the backend REST API supports them. Upon first connection to the back-end, we would check to see if web sockets are supported, if so, we would upgrade the connection.
This achieves two things:
We would still store logs in a client-side memory buffer and flush them over the socket. However, the socket reduces network overhead. And if the connection ever gets dropped before the buffer has been flushed, then we can concurrently work to re-establish the connect while continuing to store event logs in the buffer. Upon reconnect, you flush all logs to relieve back-pressue. In this way, no logs are ever dropped. As an escape hatch, if the buffer exceeds a certain memory threshold, then you can treat it like a FIFO queue and start dumping old logs to avoid a memory leak.
The text was updated successfully, but these errors were encountered: