We try our first biomedical project and create the world's first ElectroOculogram-controlled console game. Using the principles of the eye, we can track the movement using some sticky sensors and some custom circuitry. Watch us blink and nod our way through classic Mario. You can create your own Eye Gaming System using LabVIEW, a LEGO MINDSTORMS NXT, and a biomedical sensor.
In this episode we show off our Eye Mario system that allows you to play any NES video game just using your eye movements. In this video we give you a basic overview of how this system works.
The goal of this project is to control a Nintendo Entertainment System using solely eye movement. This involves reading in eye position using a technique known as electrooculography (EOG) as well as interfacing with the NES standard protocols to simulate a game controller all using an NI sbRIO and a custom PCB Daughter card.
Electrodes placed on the face around the eyes feed a small electrical signal into isolation and analog input circuitry that was custom built on an NI Singleboard RIO daughter card. All processing is done on the NI Singleboard RIO and information is passed back out digitally to the daughter card which communicates with an original Nintendo Entertainment System.
The Waterloo Labs Eye Mario Team
photo credit: Jay Janner - Austin Statesman
The human eye is polarized due to the concentration of electrodes in your retina on the back of the eye. This create a dipole potential. When your eye moves around in your head the electric field aroudn your eye rotates with it. We used electrodes ( Ambu Blue Sensor N wet electrodes) to detect very small changes in voltage that would indicate which way you are looking .
These electrodes are placed around the eye in a diamond pattern to detect its current position using electrooculography. The voltage differences between the horizontal and vertical pairs are taken to determine eye position. Additionally a ground reference electrode is applied to the neck. This reference is used to filter out any base biological electrical activity not related to eye position. By using a wet electrode a conductive material is used to drop the electrical resistance of the skin and allows for better electrical conduction.
diagram of the polarized human eye
The use of wet electrodes, as previously stated, drops the natural electrical resistance of the skin for improved conductivity. Unfortunately this does not only make for improved measurements but also weakens the body’s defenses against external shock, for instance from a power surge in the measurement circuitry. To protect the test subject from a possibly deadly shock isolation circuitry is used.
The EyeMario EOG Filter & Amplification Circuit
After the information has been digitized by the isolation and analog input circuitry it is passed down to the Singleboard RIO for analysis. It is initially acquired on the FPGA of the RIO and then passed through DMA transfer to the built in Real Time Controller for processing.
Both horizontal and vertical eye position signals are compared to preset thresholds. If the measured signal surpasses the threshold a controller command is triggered and sent back out to the daughter card for communication to the Nintendo. For example, if the test subject looks far enough to the left a digital line corresponding to a left button being pressed is asserted and handed out to the daughter card to be passed off the gaming system.
The layout schmatic for the daughter card
A 3D render of the daughter card
The Nintendo controller is essentially a shift register that serializes 8 digital lines, each corresponding to a button on the controller. All of this is done based on signals coming in from the Nintendo such as a latch and clock signal. By recreating this on the daughter card and replacing the buttons from controller with digital lines the team could successfully communicate with the NES.
The Daughter Card Display
Huge thanks to the engineers that made this project possible:
"The Compound" Staff