Magic hand and Magic cube

The “Magic Hand” and “Magic Cube” were our creations for the MakeNTU contest, a 24-hour hackathon combining software programming and hardware implementation.

Motivation

The idea for these inventions stemmed from our experiences as senior students, attending numerous meetings, such as individual, group, and project meetings. We noticed inefficiencies in some of these meetings due to factors like large attendance or poor presenter-attendee interaction. To address this issue, we envisioned the “Magic Hand” and “Magic Cube.”

Design

The “Magic Hand” is essentially a depth camera embedded with a projector. Using openCV, the camera captures the presenter’s hand movements, converting it into a controller. Hand gestures allow switching between tools like pointer, highlighter, and zoomer. The presenter can point to the projection board, and the cursor will track the gesture in real-time. This feature frees the presenter’s hand from holding a presentation controller or pen, enabling two-handed activities like prototype demonstrations. Additionally, the tracking system supports multiple people and arms, facilitating smoother interactions during presentations and discussions, saving time and effort.

• Pointer

The “Magic Cube” is a small but powerful device equipped with an ESP32 MCU, gyro, LED, vibration motor, and built-in LiPo battery. Its compact size (15mm x 15mm x 35mm) allows each attendee to have a cube in hand. The cube serves multiple purposes: the presenter can perform roll call using the cube’s ID, and the vibration and LED will notify attendees. Furthermore, the cube enables anonymous or named voting through different orientations. The gyro senses the orientation and communicates the vote to the main controller via MQTT. This fosters better interaction and higher-quality feedback compared to using cellphones, which often become distractions during meetings.

• Voting

• Roll call

Slides

Conclusion

Our creations garnered recognition in the contest, earning us the Best Application Award and 3rd place in the Enterprise Award category. We aspire for these applications to be adopted commercially in the future. Throughout the competition, we honed our skills in programming with openCV software, achieving real-time acceleration up to 10-15 fps, and gained valuable hardware implementation experience.