WebSocket
The WebSocket API is a technology for two-way communication between a browser and a server. It can be used for very high performance data transfer. It's robust, flexible and well supported!
The easiest way to send data for visualization using WebSocket is to pack messages into stringified JSON. Here's how that would look in code:
// Server
// newDataPoints: Array<{ timestamp: number, measurement: number }>
ws.send(JSON.stringify(newDataPoints))
// Client
ws.onmessage = (e) => {
const newDataPoints = JSON.parse(e.data)
lineSeries.appendJSON(newDataPoints, { x: 'timestamp', y: 'measurement' })
}
This is super easy, and really not too bad performance-wise. However, it is not the most performant way to transfer data. Long decimals and property names add a lot of extra bandwidth that can be removed by using non-string formats, like binary:
// Server
// newMeasurements: Float32Array
ws.send(newMeasurements)
// Client
ws.onmessage = (e) => {
const newMeasurements = e.data
lineSeries.appendSamples({ yValues: newMeasurements })
}
The above example sends only Y measurements (and automatically increases X by 1 for every sample). Here's an example of sending both X and Y measurements.
// Server
// xValues: Float32Array
// yValues: Float32Array
// Combine x and y values into 1 array
const xyValues = new Float32Array(xValues.length * 2);
xyValues.set(xValues, 0);
xyValues.set(yValues, xValues.length);
ws.send(xyValues)
// Client
ws.onmessage = (e) => {
const data = e.data
const sampleCount = data.length / 2;
const xValues = data.subarray(0, sampleCount);
const yValues = data.subarray(sampleCount);
lineSeries.appendSamples({ xValues, yValues })
}
Often, LightningChart JS applications involve several channels which all receive real-time data. In this kind of setting, it is recommended to buffer data streams so that 1 data package is sent every 10-20 milliseconds. For best performance, this data package should include all new data points for all the channels, rather than sending a separate message for every channel.
This is easy to achieve with the stringified JSON approach mentioned at the start, but in intensive use cases with large amounts of streaming data, it is recommended to use the binary formats, even if it requires a bit of extra work compared to the simple cases showcased above.
Data transfer library
We have prepared a small library dedicated to high performance data transfer between a server and application. It includes a few methods for different use cases (like having several charts, no timestamps, shared timestamps, etc.) and covers both encoding (backend) and decoding (frontend) messages. Here's a sample:
// Server | 5 channels, 1 data point every 16 milliseconds
setInterval(() => {
const dataAllChannels = new Array(5).fill(0).map((_, ch) => {
const xValues = [performance.now()];
const yValues = [Math.random() + ch];
return [xValues, yValues];
});
const msg = encodeMultiChannelArrXY(dataAllChannels);
ws.send(msg);
}, 1000 / 60);
// Client
ws.onmessage = async (e) => {
const raw = await e.data.arrayBuffer();
const dataAllChannels = decodeMultiChannelArrXY(raw);
channels.forEach((series, i) => {
series.appendSamples({ xValues: dataAllChannels[i][0], yValues: dataAllChannels[i][1] })
});
};
Under the hood, this does all the binary magic and uses ~10x less bandwidth (*) compared to transferring data as JSON. These methods are intended to be directly utilized by our customers. The source code of the methods is also available, so you can suit them to your exact needs. Please contact us if you want access to this separate data transfer library.
(*) JSON bandwidth use fluctuates heavily based on uncertainties, like property names, decimal point counts, etc.
Other topics that are closely related to handling real-time data:
- Scrolling axes and
- Data cleaning. For documentation, refer to specific feature. For example, Line Series