UI – TSL Gen 0.0 (Target selecting system)
Brief
TSL, short for Target Selecting System.
Which is an imagination about the future interaction mode between human and drone, robot, and security system….
The biggest feature is the interaction between gaze to zooming and target selecting. Compared to the common way of selecting target with the mouse, using the gaze of the eye provides a most natural, intuitive and fast interaction way for human to select the target.
This demo achieves using eye gazing to control the laser beam and adjust zooming through the gaze point.
The ultimate goal is to use AR/VR to build a distributed system to let humans connect to different types of terminals (drones, UAVs, aircraft, robots) to enhance humans’ abilities just like iron man.
(ex. Fighter pilots lock on to enemy aircraft with their eyes; The tank gunner controls the field of view through VR by turning his head and controlling the weapon system with his eyes )
Story
The way that LDMS(Laser mosquito defence system) selects the desired object is to use the mouse to click and drag. Is very slow and requires the object to stay motionless during selection. Which makes the interaction inefficient and inhuman.
Or in my words, not cool. Also, I am a big fan of iron man, which made me think of Jarvis and using gaze points to interactive.
My previous plan is to using Microsoft HoloLens(an AR with eye tracking). But it’s too expensive and not familiar with the tool chain of developing UI for that. So I choose to use eye tracker to build a prototype first.
And for selecting object, it requires using deep learning image recognize to pick object from background first. Which another big field I haven’t involve. So I decide to wait until I get university and group working.
For this UI, I don’t want it to just been used on this laser system, but used on any robot and any system need seek and tracking object like rescuer and anti-terrorist.
And my plan is to make this system a distributed platform which people only need to carry a terminal and able to connect to any device, vehicle, robot, drone, laser system…… just like iron man.
