Hands free VR: a smile, frown or clench will suffice

Photo of author
Written By Chris

Sweet Gibson* says handheld digital actuality controllers might be on their method out.


Totally different facial expressions can now manipulate objects in a VR setting.

Our face can unlock a smartphone, present entry to a safe constructing and velocity up passport management at airports, verifying our id for quite a few functions.

A global staff of researchers from Australia, New Zealand and India has now taken facial recognition expertise to the subsequent degree, utilizing an individual’s expression to govern objects in a digital actuality setting with out using a handheld controller or touchpad.

In a world first examine led by College of Queensland researcher Dr Arindam Dey, human laptop interplay consultants used neural processing methods to seize an individual’s smile, frown and clenched jaw and used every expression to set off particular actions in digital actuality environments.

One of many researchers concerned within the experiment, College of South Australia’s Professor Mark Billinghurst, says the system has been designed to recognise completely different facial expressions by way of an EEG headset.

“A smile was used to set off the ‘transfer’ command; a frown for the ‘cease’ command and a clench for the ‘motion’ command, instead of a handheld controller performing these actions,” says Prof Billinghurst.

“Basically we’re capturing frequent facial expressions comparable to anger, happiness and shock and implementing them in a digital actuality atmosphere.”

The researchers designed three digital environments – blissful, impartial and scary – and measured every particular person’s cognitive and physiological state whereas they have been immersed in every situation.

By reproducing three common facial expressions – a smile, frown and a clench – they explored whether or not modifications within the atmosphere triggered one of many three expressions, based mostly on emotional and physiological responses.

For instance, within the blissful atmosphere, customers have been tasked with transferring by means of a park to catch butterflies with a web.

The consumer moved after they smiled and stopped after they frowned.

Within the impartial atmosphere, members have been tasked with navigating a workshop to choose up objects strewn all through.

The clenched jaw triggered an motion – on this case choosing up every object – whereas the beginning and cease motion instructions have been initiated with a smile and frown.

The identical facial expressions have been employed within the scary atmosphere, the place members navigated an underground base to shoot zombies.

“General, we anticipated the hand-held controllers to carry out higher as they’re a extra intuitive technique than facial expressions, nevertheless individuals reported feeling extra immersed within the VR experiences managed by facial expressions.”

Prof Billinghurst says counting on facial expressions in a VR setting is tough work for the mind however offers customers a extra reasonable expertise.

“Hopefully with some extra analysis we will make it extra consumer pleasant,” he says.

Along with offering a novel method to make use of VR, the approach will even enable individuals with disabilities – together with amputees and people with motor neurone illness – to work together fingers free in VR, now not needing to make use of controllers designed for absolutely abled individuals.

Researchers say the expertise can also be used to enhance handheld controllers the place facial expressions are a extra pure type of interplay.

The examine findings have been printed within the Worldwide Journal of Human-Laptop Research.


*Sweet Gibson is Media Liaison Coordinator on the College of South Australia.

This text first appeared at unisa.edu.au.

Leave a Comment