Eye Tracking Based Virtual Mouse Using Webcam and Computer Vision

Abstract

Most people still interact with a computer through a mouse, a keyboard, or a touchpad, none of which are usable by everyone. This project explores whether an ordinary webcam, without any specialised eye-tracking hardware, can be turned into a working substitute for a mouse. The prototype is built in Python around OpenCV for video capture, MediaPipe for facial and IRI landmark detection, NumPy for the underlying geometry, PyAutoGUI for using mouse commands on the operating system, and Tkinter for a small control panel. A single RGB camera watches the user's face; the position of the iris within each eye is estimated frame by frame, and a nine-point calibration routine learns how that position maps onto the screen. To keep the cursor from jittering, its motion is smoothed rather than updated instantaneously. Left and right clicks are triggered by closing one eye at a time, while a longer, deliberate closure of both eyes switches the system into a text-selection mode; looking toward the top or bottom of the frame scrolls the page. The system should be read as a working proof of concept rather than a finished product — it has not yet been through a formal user study, so this paper reports what was built and how and leaves the accuracy number and completion-time figure for the evaluation described in Section VIII.