Handheld Video Game Controller Can Sense Players' Emotions
Researchers in the lab of Gregory Kovacs, an electrical engineering professor at Stanford University, typically develop practical ways of measuring physiological signals to determine how a person's bodily systems are functioning. Now they have developed a handheld game controller that can sense players' engagement and make gameplay more exciting. Doctoral candidate and project leader Corey McCall replaced the back panel of an Xbox 360 controller with a 3D printed plastic module packed with sensors. Small metal pads on the controller's surface measure the user's heart rate, blood flow, and both the rate of breath and how deeply the user is breathing. Another light-operated sensor gives a second heart rate measurement, and accelerometers measure how frantically the person is shaking the controller. Meanwhile, custom-built software gauges the intensity of the game – a simple but fast-paced racing game in which the player must drive over colored tiles in a particular sequence. McCall can then compare all this data to generate an overall picture of the player's level of mental engagement.
Transcript
00:00:00 [MUSIC] The autonomic system is the part of the brain that changes when you get excited, or when you are happy, or when you are sad We want to measure the autonomic activity in the context of video game play. You can measure this directly from the brain, or you can do what we are doing which is sensing peripheral signals. So the autonomic system also affects your heart rate, your skin temperature, your respiration rate. So we want to tell if the person playing the video game is excited, are they enjoying it, are they engaged in the game, are they happy.
00:00:37 This is a modified Xbox controller. We have taken the bottom part out of a regular xbox controller and put in varius physiological sensors. So what we have here is we have a photoplethysmogram measurement that can get the pulse from the optical measurement, as well as we have four electric plates on the side. And from that we can calculate the electrical beat of the heart as well as the the respiration rate. So what you see in the screen here is the biosignals coming directly out of the video game controller.
00:01:05 So on the top here we have the photo based pulse measurement, and on the second line it is the respiration rate. So that shows how much air is going into the player's lungs or out of the lungs. The third line is the accelerometer. So we can tell how much the player's hands are moving. On the fourth line what we try to do is correlate this with the game activity. So this is a measurement we have of the video in the game. Eventually we want to figure out a way to sense how the player is feeling about the game,
00:01:35 and then control the game. One example would be if the player wants to be maximally engaged, like they want to play a really exciting game. Then what we can do is sense when they get bored. So say their heart rate goes down, the respiration rate goes down. Then we can say send in more zombies in the game. Or a lot of parents are concerned that their children are getting too involved in the game, or too excited. In the same way we can sense how they are feeling and turn it down for them.
00:01:57 [MUSIC] For more please visit us at stanford.edu