Play NES with your Eyes - Episode 04

Eyes + Electrodes + Filter/Amp + sbRIO + NES = Awesome

We try our first biomedical project and create the world's first ElectroOculogram-controlled console game. Using the principles of the eye, we can track the movement using some sticky sensors and some custom circuitry. Watch us blink and nod our way through classic Mario. You can create your own Eye Gaming System using LabVIEW, a LEGO MINDSTORMS NXT, and a biomedical sensor. 

In this episode we show off our Eye Mario system that allows you to play any NES video game just using your eye movements. In this video we give you a basic overview of how this system works.

System Overview

The goal of this project is to control a Nintendo Entertainment System using solely eye movement. This involves reading in eye position using a technique known as electrooculography (EOG) as well as interfacing with the NES standard protocols to simulate a game controller all using an NI sbRIO and a custom PCB Daughter card.

 

System Set Up

Electrodes placed on the face around the eyes feed a small electrical signal into isolation and analog input circuitry that was custom built on an NI Singleboard RIO daughter card.  All processing is done on the NI Singleboard RIO and information is passed back out digitally to the daughter card which communicates with an original Nintendo Entertainment System.

 

 

The Waterloo Labs Eye Mario Team

photo credit: Jay Janner - Austin Statesman

Electrooculogram (EOG)

The human eye is polarized due to the concentration of electrodes in your retina on the back of the eye. This create a dipole potential. When your eye moves around in your head the electric field aroudn your eye rotates with it. We used electrodes ( Ambu Blue Sensor N wet electrodes) to detect very small changes in voltage that would indicate which way you are looking . 

 

These electrodes are placed around the eye in a diamond pattern to detect its current position using electrooculography.  The voltage differences between the horizontal and vertical pairs are taken to determine eye position.  Additionally a ground reference electrode is applied to the neck.  This reference is used to filter out any base biological electrical activity not related to eye position. By using a wet electrode a conductive material is used to drop the electrical resistance of the skin and allows for better electrical conduction.

 

Eye Diagram.png

diagram of the polarized human eye

Eye Mario Warning.png

Isolation Circuitry

The use of wet electrodes, as previously stated, drops the natural electrical resistance of the skin for improved conductivity.  Unfortunately this does not only make for improved measurements but also weakens the body’s defenses against external shock, for instance from a power surge in the measurement circuitry.  To protect the test subject from a possibly deadly shock isolation circuitry is used. 

 

 
 
 
 
 
 
 
 

 

How to Make a Custom Daughter Card Circuit Board

The EyeMario EOG Filter & Amplification Circuit

NI Singleboard RIO

After the information has been digitized by the isolation and analog input circuitry it is passed down to the Singleboard RIO for analysis.  It is initially acquired on the FPGA of the RIO and then passed through DMA transfer to the built in Real Time Controller for processing. 

 

Both horizontal and vertical eye position signals are compared to preset thresholds.  If the measured signal surpasses the threshold a controller command is triggered and sent back out to the daughter card for communication to the Nintendo.  For example, if the test subject looks far enough to the left a digital line corresponding to a left button being pressed is asserted and handed out to the daughter card to be passed off the gaming system.

How to Make a Custom Daughter Card Circuit Board

The layout schmatic for the daughter card

A 3D render of the daughter card

Nintendo

The Nintendo controller is essentially a shift register that serializes 8 digital lines, each corresponding to a button on the controller.  All of this is done based on signals coming in from the Nintendo such as a latch and clock signal.  By recreating this on the daughter card and replacing the buttons from controller with digital lines the team could successfully communicate with the NES.

The Daughter Card Display

Software Overview
The EyeMario software has 3 compontnets: The FPGA code, Real-Time code, and the Windows host.  The FPGA code is used to read the signal from the daughter card using a Synch 3 filter to extract the data from the Data Sigma ADC. The Real-Time code takes the processed data and compares it to range limits to determine if it is a blink or a look. The Windows host shows the signal on a plot that allows the user to set the calibration limits. The Real-Time code then decides which controller output to send to the NES.

Download EyeMario Code

EOG Data

The Team

Huge thanks to the engineers that made this project possible:
Doug Farrell

Will Schoettler

Stephen Meserve

Hunter Smith

Chris Culver

Barron Stone

Elizabeth Jameson

"The Compound" Staff

Why are you even looking down ​here​?