CW>

Projects / Lego Rover

Remote Control

After I got the rover driving around using the keyboard, I made some improvements so it could be controlled remotely from a web browser using touch controls and show a live camera feed from the rover. Here are a video and pictures of the rover being controlled from the web browser with the camera view from the rover.

Remote control from browser
Rover controlled from browser on iPad
Screen shot of browser interface

The browser remote control needs quite a few extra bits and changes to the control program to make it all work, which I will explain in the sections below. You can download a zip file containing the code here.

Camera

Camera attached to Pi Zero on front of rover
I am using a ZeroCam camera (£15 from PiHut). You could use any camera that works with the Raspberry Pi. The Pi Zero uses a smaller camera connector than than full size Raspberry Pis, so you have to get a camera with the correct connector for whatever Pi version you have or use an adapter cable. As the ZeroCam has a very short cable on it, I've just taped it to the bottom of my Pi Zero case and mount it so the camera is pointing in the right direction.

Once you have a suitable camera attached to you Raspberry Pi, you can take pictures using the raspistill command. Use raspistill -o photo.jpeg to save to a file. You then need to download the photo to view it, unless you have a screen and GUI running on the Pi.

The raspistill command can also be setup to keep taking pictures at a regular interval. The -tl (timelapse) parameter tells it to take a picture every specified number of milliseconds and keep doing it for the time given by -t. E.g. raspistill -o photo-%02d.jpeg -tl 1000 -t 10000 makes it take a series of photos every second for 10 seconds, with a frame number (01 to 10) added to each file name.

In my script to start the rover (start.sh), I am using raspistill -w 640 -h 480 -o photo.jpeg -tl 100 -t 9999999 to take a 640x480 pixel photo every 100ms for a long time and overwrite the same photo.jpeg file every time. I then display this photo in the webpage used to control the rover (see below). I decided to use this approach, rather than getting into proper video streaming, as it is easy to setup and doesn't need extra software installing on the Raspberry Pi.

Web Server

I am using Node.js for the web server that will run on the Raspberry Pi and serve up the web page that will be used to view the camera feed and control the rover from a web browser on another device (like a phone or tablet). I like Node.js because it allows you to write the code for the web server in JavaScript, which is the same programming language used in web pages and similar to the C++ code used to program the rover. Type node -v at the terminal to check it is installed and what version you have. You can use apt-get to install and update it if needed.

In the code zip file there is a folder called "server" containing two files:

These two bits of code work together to communicate between the browser on your control device (phone/tablet) and the Rasberry Pi on the rover. This is basically how all web pages on the internet work, sending requests from your browser to a server on the internet (or in this case the Raspberry Pi on your local Wifi network), which responds with a HTML page, JPEG photo or some other data to display in your browser. The JavaScript code in the "server.js" file is shown below, followed by an explanation:

var http = require('http');
var url = require('url');
var fs = require('fs');

// Setup the web server (on standard port 80)
http.createServer(function (req, res) {
  
  // Check the requested URL
  var query = url.parse(req.url).search;
  if (query && query.indexOf("?control=") == 0)
  {
  	// Control input sent back from page
  	var control = query.substr(query.indexOf("=") + 1, 1).toUpperCase() + "\n";
  	
  	// Update the control.txt file with the direction (to tell the rover what to do)
  	res.writeHead(200, {'Content-Type': 'text/plain'});
  	fs.writeFile("control.txt", control, function () {});
  	
  	res.end(control);
  }
  else if (req.url.indexOf("/photo.jpeg") == 0)
  {
  	// Request for photo
  	res.writeHead(200, {'Content-Type': 'image/jpeg'});
  	res.end(fs.readFileSync('photo.jpeg'));
  }
  else
  {
  	// Treat anything else as request for the web page
  	res.writeHead(200, {'Content-Type': 'text/html'});
  	res.end(fs.readFileSync('index.html'));
  }
}).listen(80);

The first thing the code above does, after including some modules it needs, is create a HTTP web server listening on port 80 (the standard port for web pages). All the other code inside the curly braces ({}) is the code for our web server, which decides how to respond to requests from the browser. This code checks the request URL (the address sent by the browser) and responds in one of three different ways depending on the URL.

Web Page

The "index.html" page is served by our web server on the Rasberry Pi and displayed in the browser on our control device. This file contains the HTML code for the web page. HTML is the markup language used for the content of web pages. I won't explain HTML in detail here, if you're new to it have a look at w3schools.com for a quick introduction. These are the main parts of the HTML code in the page:

Below is the JavaScript code from the page, followed by explanation:

// Set interval to update photo			
setInterval(function () {
	let photo = document.getElementById("photo");
	
	// Add the date and time to the end to make the photo update
	photo.src = "photo.jpeg?v=" + (new Date());
}, 250);

// Setup the controls to respond to touch
document.querySelectorAll(".controls div").forEach(control => {

	control.addEventListener("touchstart", (ev) => {
		// Touch started, change the direction
		directionChange(ev.target.className.substr(0, 1).toUpperCase());
	}, false);
	
	control.addEventListener("touchend", (ev) => {
		// Touch ended, clear the direction
		directionChange("");
	}, false);
});

// Change direction when the controls are touched
function directionChange (direction) {
	
	// Show the selected direction in the center of the controls
	document.getElementById("control").innerText = direction;
	
	// Send the control direction back to the server (Raspberry Pi)
	var http = new XMLHttpRequest();
	http.open("GET", "/?control=" + direction, true);
	http.send();
}

There are three main parts to the JavaScript code in the page:

Control Program

The control program is mostly the same as the original explained in the Programming section. The updated code is in the zip file download and the process for compiling and running it is the same as before. The updated "main.cpp" file is shown below. The main change is that the program now runs in the background and doesn't respond to keyboard input like before. It checks for control signals so you can end it with Ctrl + C from the terminal. Rather than checking for key presses with getch, it now reads the "server/control.txt" file that contains the control command code (U,D,L,R) sent back from the webpage. It reads this control code into the "control" variable and then turns on and off the motors depending on the code. U (Up) = both motors forwards, D (Down) = both motors backwards, L = turn left, R = turn right.

#include <iostream>
#include <fstream>
#include <string>
#include "screen.h"
#include "output.h"

using namespace std;

// Main program
int main() {

	// Catch any errors while the program is running
	try {
		// Setup in background mode (no terminal output)
		Screen io = Screen(true);
		
		// Setup the motor outputs
		Output leftForward = Output(17);
		Output leftReverse = Output(27);
		Output rightForward = Output(22);
		Output rightReverse = Output(10);
		
		// Loop until program is stopped by terminate signal
		while (true) {
			
			// Get the control input from the file
			string control = "";
			ifstream file("server/control.txt");
			if (file.is_open()) {
				getline(file, control);
				file.close();
			}
			else {
				// Failed to open file
				throw runtime_error("Failed to open control.txt");
			}
			
			// Check which key is pressed and turn on the correct motors
			if (control == "U") {
				leftForward.on();
				leftReverse.off();
				rightForward.on();
				rightReverse.off();
			}
			else if (control == "D") {
				leftForward.off();
				leftReverse.on();
				rightForward.off();
				rightReverse.on();
			}
			else if (control == "L") {
				leftForward.off();
				leftReverse.on();
				rightForward.on();
				rightReverse.off();;
			}
			else if (control == "R") {
				leftForward.on();
				leftReverse.off();
				rightForward.off();
				rightReverse.on();
			}
			else {
				// Stop, turn all the motors off
				leftForward.off();
				leftReverse.off();
				rightForward.off();
				rightReverse.off();
			}
			
			// Check for signal to end program
			io.checkSignal();
			
			// Sleep for a short time before the next update
			io.sleep(0.1);
		}
	}
	catch (const exception& e) {
		// Error or program aborted
		cerr << e.what() << "\n";
	}
	
	return 0;
}

Start/Stop Scripts

In the zip file with the code there are also two script files: "start.sh" and "stop.sh". The "start.sh" script does the following:

These processes are all started in the background, rather than in the current terminal session, by adding "&" after each command. This means that you can close the terminal after starting everything and control the rover from the web page in the browser. Run sudo sh start.sh to start the rover processes. You can see the running processes on the Raspberry Pi by typing ps at the terminal. Once they have all started up (which can take a few seconds), open a browser on your control device (I use my Android phone or iPad) and enter the local IP address of your Raspberry Pi (e.g. mine is 192.168.0.11) in the address bar. You can find the IP address from the Pi by entering hostname -I at the terminal. You should then see the web page with the image from the camera feed updating and the controls should feed back to the rover and move it. Once you've finished driving, you can run sudo sh stop.sh to stop all the processes running on the Raspberry Pi.