DIY video monitoring system – Part IV Video streaming using MJPEG
Now that we know how to serve single images from our Raspberry Pi Camera, it is time to do some video streaming. To make this happen, we’re going to use the MJPEG video format.
If you haven’t seen my previous posts on DIY video monitoring system, I strongly encourage you read them first as some bits of the code below are coming from them.
MJPEG
Before we start modifying our code, let’s quickly talk about MJPEG. M-JPEG, or Motion JPEG, is a video compression format, where each video frame is compressed as a JPEG image. It is a widely supported format, often used by video capturing devices like webcams or IP cameras. What interests us mostly, is the fact that it’s natively supported by modern web browser (Chrome, Firefox, Safari, Edge) so no additional plugins are required to play it back in one of them.
When streaming MJPEG over HTTP, we start with a GET request from the client browser and in response, we get a bunch of headers that tell the browser how to handle it. Most notable response headers are:
Content-Type: multipart/x-mixed-replace;boundary="<boundary_name>"
It’s a special mime-type content type header that tells the client to expect the response to consist of multiple parts, delimited by the boundary equal to <boundary_name>.Connection: keep-alive
This one tells the client to keep the connection open as we’re expecting more data (subsequent frames) to come from the serverCache-Control: no-cache, no-store, max-age=0, must-revalidateExpires: Thu, Jan 01 1970 00:00:00 GMTPragma: no-cache
The last three headers are responsible for cache control – in short, we want do disallow browser caching so that the resource loaded with our initial GET request always gets fetched from the server
Once the connection is established, the server starts providing subsequent frames:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
--<boundary_name> Content-Type: image/jpeg Content-Length: <frame_1_size> <frame_1_data> --<boundary_name> Content-Type: image/jpeg Content-Length: <frame_2_size> <frame_2_data> ... --<boundary_name> Content-Type: image/jpeg Content-Length: <frame_n_size> <frame_n_data> |
Each chunk (frame) starts with the boundary marker, Content-Type header, Content-Length header and the content itself. Chunks can also be extended with some additional headers, like frame dimensions or timestamps.
The streaming will continue until one of the sides closes the connection.
Something, that may have caught your eye is the fact that we don’t see any information about the video frame rate. Strange, isn’t it?
In fact it is one of MJPEG’s advantages – as the server keeps feeding the viewers with frames as soon as there are new ones, it can adapt the frame rate to e.g. low light conditions, where taking each frame may take more time.
Implementation
Let’s head back to our refactored script from the previous post. This time we only need to modify the server part as our camera code is smart enough to handle concurrent capture calls thanks to the Promises.
In the server script – capture.js – we need to define the boundary to be used in MJPEG stream:
|
1 2 |
// MJPEG frame boundar const BOUNDARY = 'MYBOUNDARY'; |
Now, we need to update the callback passed to http.createServer.
We’ll start with replacing static.jpg with live.jpg in our HTML template, so that the main page will serve our live stream:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
if (req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/html;charset=utf-8' }); res.write(`<!doctype html> <html> <head> <title>Raspberry Pi Camera</title> <meta charset="utf-8"> </head> <body> <img src="live.jpg"> </body> </html>`); res.end(); } |
Next, we’ll create a new if statement that will handle requests to /live.jpg. Let’s have a look at the code first:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
if (req.url === '/live.jpg') { res.writeHead(200, { 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Connection': 'keep-alive', // here we define the boundary marker for browser to identify subsequent frames 'Content-Type': `multipart/x-mixed-replace;boundary="${BOUNDARY}"`, 'Expires': 'Thu, Jan 01 1970 00:00:00 GMT', 'Pragma': 'no-cache' }); // connection closed - finish the response res.on('close', () => res.end()); let loop = () => { return camera.capture() .then((frame) => { if (res.finished) { return; } // serve a single frame res.write(`--${BOUNDARY}\r\n`); res.write('Content-Type: image/jpeg\r\n'); res.write(`Content-Length: ${frame.length}\r\n`); res.write('\r\n'); res.write(Buffer(frame), 'binary'); res.write('\r\n'); // attempt to cache another frame after 50ms = ~20fps setTimeout(loop, 50); }) .catch((e) => { res.writeHead(500, 'Server Error'); res.write(e.message); res.end(); }); }; loop(); } |
We start with writing all the headers mentioned in the previous chapter:
|
1 2 3 4 5 6 7 8 |
res.writeHead(200, { 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Connection': 'keep-alive', // here we define the boundary marker for browser to identify subsequent frames 'Content-Type': `multipart/x-mixed-replace;boundary="${BOUNDARY}"`, 'Expires': 'Thu, Jan 01 1970 00:00:00 GMT', 'Pragma': 'no-cache' }); |
Notice we’re using an ES2015 template string to pass the boundary constant value to the content-type header.
Next, we bind a listener to response’s close event. Once the connection closes, e.g. the client disconnects, we need to finish the response using the res.end() method.
Then we define a loop() function that captures the frame using our camera instance and responds with a frame once the capture promise fulfills. Before we do that, though, we check if the response hasn’t finished in the meantime. If so, then there is no point in proceeding further with the script. Finally, after 50 milliseconds, we execute the loop() function once more to serve another frame.
In case when our promise gets rejected, we respond with 500 Server error and finish the response.
Finally, we execute the loop() function for the first time to kicking off the streaming.
And here’s how it looks in the browser:
Summary
In this post we’ve learned:
- what MJPEG format is and how it works
- how to stream MJPEG video using Node.js
The complete code used in this post can be found in my Github repository.
In the next post we’re going to do something different – we’ll use Node.js to control a servo via Raspberry Pi’s GPIOs.
See you next time!
