Posters         About
Mark

Learn Semaphore 


Teaching flag semaphore with pose recognition and gamification

Try it out ︎





Project Overview

Learn Semaphore is a game designed and developed at HackUMBC 2019. 

The game introduces flag semaphore in a fun and competitive way. The player is prompted with a letter and a corresponding pose that they must recreate. As the player completes more rounds, they develop muscle memory and think more quickly about each pose, eventually learning at least some form of flag semaphore. Who knows when they will need it?

Contribution: Web-based machine learning programming, visual design, front-end development

Collaborator Yana Gevorgyan





Design Challenge

How can we teach semaphore in a fun and intuitive way?





Flag semaphore is used by the navy for distance communication.




The game is developed with PoseNet, a web-friendly pose recognition model.






In addition to coding, I also worked on the visual design.
Here are a few iterations.








How the game works


To start the game, the player waves his/her hands up and down, similar to how real semaphore communication is initiated. When the game starts, the player is given 10 seconds to complete as many semaphores as possible.

On the screen, an alphabet letter is presented alongside an icon of the corresponding semaphore hand position. The webcam capture's the player's movement, and compare it to that of the correct semaphore position with pose recognition. The player must correctly pose in the position the receive the next letter.

As the player completes more rounds, he/she will develop muscle memory and eventually learn flag semaphore. Who knows when they will need it?

The pose is built with PoseNet using the ml5.js library that allows for performance-efficient machine learning done through JavaScript on the web. Animations were created with Adobe After Effects.



Learnings


Aside from strengthening my machine learning application skills, I’ve learned a thing or two about flag semaphore!





Say Hi!


I would love to talk about anything and everything. Reach out!

︎︎ ayeh01@mica.edu ︎