Highlights
- Designed the end-to-end perception-to-control loop: camera capture, YOLO-based ball detection, pixel-to-world mapping, motion buffering, quadratic trajectory prediction, and MQTT servo commands.
- Fine-tuned a YOLO-family detector on more than 1,000 custom labeled ping pong ball images and reported 92% mAP with approximately 2.7 ms inference per frame.
- Built a lightweight 3D position estimate from 2D bounding-box centers and apparent ball radius, then fit a quadratic model to predict the interception point on the goalie plane.
- Integrated Raspberry Pi 5, Pi Camera, and a servo-driven blocker into a functional real-time hardware demo, with servo alignment typically within 5-10 degrees of a good blocking angle.
- Analyzed system-level failure cases caused by noisy detections, single-camera depth uncertainty, bounce timing, limited field of view, and fixed-frame-rate assumptions.
Key metrics
Media
Tech stack
Problem and objective
This project reduced the table-tennis robot problem into a compact automatic goalie: observe a ping pong ball in a small arena, predict where it will land, and rotate a blocker before impact.
The key challenge was not only training a detector. The system had to close the loop across perception, calibration, prediction, communication, and actuation under real timing constraints on lightweight hardware.
Hardware and sensing setup
The prototype used a Raspberry Pi 5, Pi Camera, and servo motor inside an approximately 40 cm by 23 cm workspace. The camera observed the ball from above while the servo rotated a simple goalie arm across the interception region.
The design intentionally used affordable hardware so the main engineering challenge was software integration: reliable detection, approximate 3D position estimation, fast trajectory prediction, and responsive servo control.
- Pi Camera stream at 60 FPS for fast visual feedback.
- Servo-driven blocker mounted at the front of the arena.
- MQTT interface separating prediction logic from motor command publishing.
Perception and calibration
The perception stack used a custom YOLO-family detector trained on more than 1,000 labeled images collected from the project setup. The detector localized the orange ping pong ball in each frame and produced bounding boxes used for downstream position estimation.
After detection, the system converted the bounding-box center into approximate workspace x-y coordinates. The apparent radius or diameter of the ball was used as a depth cue for z, which kept the setup simple but introduced sensitivity to small detection-size errors.
- Reported detector performance: 92% mAP.
- Average processing latency: approximately 2.7 ms per frame.
- Calibration mapped image coordinates and ball size into approximate real-world position.
Trajectory prediction and servo control
The system stored a short buffer of recent ball positions, smoothed noisy measurements, and fit a quadratic trajectory model. The prediction target was the landing or interception position where the ball would cross the goalie plane.
The predicted x-y point was mapped to a servo angle and published through MQTT. This kept the software modular while allowing the servo controller to rotate the blocker toward the predicted landing region.
- Used quadratic regression as a lightweight physics-inspired predictor suitable for Raspberry Pi deployment.
- Mapped predicted landing points to servo commands for reactive blocking.
- Logged throw outputs so errors could be traced to detection, calibration, timing, or actuation.
Results and representative predictions
The prototype produced strong results on several test throws. Representative targets at (0, 20) cm and (40, 10) cm were predicted exactly as (0.0, 20.0) cm and (40.0, 10.0) cm, while a mid-field target at (20, 18) cm was predicted as (20.5, 20.0) cm.
The system also exposed clear failure modes. Low-y targets were often overestimated, and aggressive extrapolation sometimes pushed the prediction toward the workspace boundary. These errors were usually linked to noisy paths, bounce ambiguity, or mismatch between real timing and the fixed 60 FPS assumption.
- Exact predictions on selected corner and side targets in representative tests.
- Servo usually moved within about 5-10 degrees of a good blocking angle.
- Demonstrated a real closed-loop robot behavior rather than only offline ball detection.
My contribution
I led the design of the full pipeline and hardware setup. My work covered detector training, camera calibration, trajectory prediction logic, and integration of the MQTT servo-control loop with the rest of the system.
This made the project a complete robotics integration exercise: data collection, model training, calibration, real-time inference, prediction, motor command publishing, and hardware testing all had to work together.
Lessons learned and next steps
The project showed that robotics performance depends on the full sensing-to-action chain. A detector can work well on individual frames but still miss the block if calibration, timing, or servo response is off.
A practical next version would add a Kalman filter or similar state estimator, dynamic frame-rate handling, better bounce detection, more robust lighting tests, and improved depth estimation through stereo vision or stronger calibration.
- Improve trajectory tracking with a Kalman filter instead of a simple moving average.
- Handle variable frame timing rather than assuming fixed 60 FPS.
- Upgrade depth estimation using stereo vision or a more reliable calibration model.
- Test under varied lighting and backgrounds for robustness.
Related projects
Vision-Guided Differentiable Physics for Robotic Manipulation
Robotics · 2025
A presentation-style robotics case study that connects RGB observations, robot state, temporal Transformer prediction, differentiable rollout losses, and Isaac Sim Franka data to learn multi-step manipulation behavior from visual context.
Autonomous Drone Navigation System
Robotics · 2025
A vision-based landing system where a Parrot Mambo drone tracks a moving line-follower robot, stays aligned using image feedback, and executes a timed descent onto the platform.
S&P 500 Deep Learning Forecasting System
Financial ML · 2025
A research-grade forecasting system that evaluates Temporal Fusion Transformers against LSTM and ARIMAX baselines for S&P 500 return prediction using mixed-frequency market and macroeconomic data, then extends TFT with regime-aware attention and interpretability diagnostics.
Event-Based Star Tracking for Spacecraft Attitude Estimation
Space Autonomy · 2026
A Speed-Aware EBS-EKF research prototype for event-camera star tracking that improves low-light spacecraft attitude estimation by making centroid correction depend on both brightness and image-plane speed.