back to projects
2024 · Robotics

Auto-Targeting Turret System

A computer vision turret that detects and locks onto a red target in real time using a webcam and servo motor. Python and OpenCV handle the vision and PD control, while an ESP32 drives the hardware over serial - smoothly and without overshoot.

PythonOpenCVESP32ArduinoSerialPD Control
LOCK · RED
HSV · dual mask
PD · active
30Hz · serial
x:50 y:62
30Hz
Serial rate
1:32
Gear ratio
2merged
HSV masks
0Pi
Compute

Demo · live footage

Pipeline · frame → servo

01
Capture
USB webcam → Python
02
HSV mask
Dual red range
03
Largest blob
Contour detection
04
PD loop
Error + derivative
05
Serial @30Hz
Laptop → ESP32
06
Servo
Slew-limited rotation

Overview

A real-time auto-targeting turret system that uses a webcam to detect a red object and physically rotate a servo motor to track it. Vision and control logic run entirely in Python, with an ESP32 handling hardware-level servo movement over serial communication.

How It Works

Every frame is converted to HSV. Red sits on both ends of the hue range, so two masks are merged and filtered with morphology. The largest red blob becomes the target. A PD controller measures error and rate of change, then ships a motor angle over serial to the Arduino, which rotates the turret.

Auto-targeting turret hardware

Control System · PD loop

Tracking uses a Proportional-Derivative controller instead of a simple follow. P pushes the motor toward the target; D reads the rate of change of error and brakes before the center, preventing overshoot. A hardware slew limiter on the Arduino caps degrees-per-tick for a second layer of smoothness.

P · Proportional

Pushes the motor toward the target.

D · Derivative

Brakes as error shrinks - no overshoot.

Slew limiter · ESP32

Caps degrees-per-tick in firmware for mechanical smoothness.

live sim · target vs responselive
Target
P onlyovershoots
PDlocks on

Key Features

  • Real-time red object detection using dual HSV masking
  • Morphological noise filtering before contour detection
  • PD controller with tunable gain values for smooth lock-on
  • Serial rate-capped at 30Hz to prevent buffer flooding
  • Hardware slew limiter on ESP32 for mechanical smoothness
  • Exponential decay when target is lost - no sudden snap to center
  • Minimum area filter ignores small noise blobs

Challenges & Learnings

Overshoot

Control

Early on the turret would chase the target and just blow right past it, back and forth, never really settling. The controller had no way to slow itself down as it got close. Once I added a derivative term that brakes based on how fast the error is changing, it stopped oscillating and started locking on smoothly.

fix · Added a derivative term

Serial flooding

Comms

I was sending data every single frame which was way too fast for the Arduino to keep up with. Commands were piling up in the buffer and the turret would lag behind or act on stale data. Capped the serial writes to 30 times per second and it cleared up immediately.

fix · Capped writes to 30Hz

Heap fragmentation

Firmware

The Arduino kept crashing after running for a while and I couldn't figure out why at first. Turned out the String class constantly allocates and frees memory, and over time that causes instability. Switched to a plain char buffer and the crashes stopped.

fix · Swapped String for char buffer

Red detection

Vision

Red is weird in HSV because it wraps around both ends of the hue range, so one mask wasn't catching everything. Some shades of red were just being ignored. Once I added a second mask for the other end of the range and merged them together, detection became much more consistent.

fix · Dual HSV mask, merged

Hardware

The entire chassis is 3D printed, including base and motor mounts. The base uses an N20 gear motor with a 1:32 ratio for precise rotation. Two Li-ion cells power it. A custom board and the Arduino both connect to the laptop alongside the USB webcam - no Raspberry Pi, everything runs from the laptop.

N20 gear motor
1:32 ratio
Power
2× Li-ion
Chassis
3D printed
Controller
ESP32 + custom board

Software

Vision and control run entirely in Python. OpenCV handles the live feed, colour detection, and on-screen drawing. NumPy powers mask operations and math. PySerial moves commands between the laptop and the Arduino, which drives the motor. Everything talks over serial in real time.

OpenCVvideo, HSV, contours, draw
NumPymask ops, vector math
PySeriallaptop ↔ Arduino link
Arduino / ESP32servo + slew limiter