By: Brendan Duncan, Kevin Ramirez, and Heather Shen
Hardware Setup:
The IMU sensor and microcontroller board are rigidly mounted above another board using spacers. The spacers provide the user a simple way to grasp the board while holding it horizontally in his/her hand. This setup also minimizes unintentional shaking that could affect pitch or acceleration readings.
Software Setup:
There are three main parts to our software setup: first, we send the data from the board tracking system, to the Arduino program, to a volume control program. Second, we process the IMU data within the volume control program so that its numerical values can be readily used. Third, we interface with the computer’s master volume control to alter the volume based on our data.
In the first step, it is relatively easy to interface the board system with the Arduino program. Building off of previous labs, we can easily receive the quaternion and accelerometer data. Within the Arduino program, we process the quaternion data to compute the rotation of the sensor (calculating the yaw, pitch, and roll). As we describe below, we decided that pitch is the most useful measurement for our design. Thus, our Arduino program contained all our necessary data: pitch and x, y, and z acceleration. Our next challenge was finding a method to transfer the data from the Arduino to a client program we were building.
Knowing that interfacing between Arduino and C++ is probably a common issue, we looked at various blogs to see what the precedent was. In general, many people use serial communication between Arduino and C++ on a Windows based system, so we decided to use a method suggested by the Arduino education blog: http://playground.arduino.cc/Interfacing/CPPWindows.
Once the data is received by our personal program, we parse the data to separate the pitch from the x, y, and z accelerations. Because the data is sent in as serial communication, the client program receives it as an array of characters. By tokenizing the string, we can isolate what corresponds to the x acceleration, y acceleration, z acceleration, and pitch, and assign the correct numbers to these variables.
The third part to our software stack is using Windows IAudioEndpoint library to control the master volume of the computer. Special thanks to Larry Osterman for his useful blog on how to control the master volume on a Windows system (see below for link). Using this audio library, we could easily interface between our program and the system’s volume control, determining and setting the master volume. During our experimental setup, we determined the maximum and minimum values for pitch, and using these values, we created three thresholds. If the pitch is under the first threshold, the board data does not affect the volume, allowing the user flexibility for unintentional movement. Once the pitch crosses each subsequent threshold, the volume increment increases (i.e. when pitch is greater, the volume increases faster). During our experimental setup, we also identified thresholds for the x, y, and z acceleration to differentiate between gestures, such as swiping the board left, pulling back, or swinging up.
When the program detects that the board is moved left, it mutes the volume. Pulling the board back returns the volume to its original, unmuted volume, and swinging the board up sets the master volume to a predetermined value (we set it to 75% of the maximum volume).
Software Data-Processing Chain:
For the data-processing, we built upon the code from previous labs in the class and decided to connect directly to a PC via a USB to avoid multiple second latency required for WiFi transmission. In our design, we used both the quaternion and acceleration values from the IMU sensor. From the quaternions, we calculated (in real time) the pitch of the board. We used the value of this pitch to determine how much to increase or decrease the volume of the computer. After experimental testing, we isolated the maximum and minimum values for pitch when the board was tilted in a natural way (i.e. if it were in someone’s hand how much would they be able to turn it).
From our acceleration values, we were able to determine if someone was swiping to the left, swiping up, or pulling the board back towards themselves (the three gestures we wanted to recognize).
As we sent the data from our Arduino to the computer (via the USB and Serial.print() function) we read it in via Visual Studios using C++. Our program (on both sides) continually loops, while sending or receiving data. Our computer takes this data in as an array of characters, parses it to isolate numerical values, and then determines the corresponding values for the x, y, and z acceleration as well as the pitch. From the pitch, as stated above, we are able to determine how much to increase or decrease the volume. The acceleration enables special functions, as described earlier.
Experimental Methodology:
We first experimented to identify the different directions of pitch, roll, and yaw and to calculate the maximum and minimum values for the corresponding direction using the board and the Arduino Serial Monitor. We discovered that when holding the board in your palm like a remote, pitch was the most natural variable to change, and thus, we decided that pitch was the value we wanted to utilize in our design.
As we were building the code, we began testing values for volume increments per cycle based on the pitch’s value. Because we didn’t want the volume to increase or decrease too drastically, but for the volume change to be a noticeable, we spent a good amount of time qualitatively listening to the volume change in response to the board’s pitch. Ultimately, we discovered how fast was too fast (+/-0.2% of max volume per tick) and how much was too little (+/-0.001% max volume per tick). We finally settled on what we feel are good variables.
One surprising thing that we discovered when using both the acceleration and quaternion data is that the IMU switches direction for pitch values (negative instead of positive and vise versa) after a large change of acceleration in the x-direction. This meant that after we muted the volume (swiping the board left), we would try to turn the volume up, but the volume would decrease, and when we went to turn it down, it would increase the volume. However, after this initial switch, the pitch values would not revert back. Because of this, one initialization step for our remote is that we must mute the volume a couple of times to stabilize the pitch values. After that, the remote works exactly as planned without any pitch related issues.
Results:
Here is a link to the demo video on Youtube. We couldn’t compress it enough to send via email/Piazza.
https://www.youtube.com/watch?v=G-4PHgrecT8&feature=youtu.be
In the end our project worked fairly well with only one minor problem. Due to some unknown interaction in the sensor, our quaternion data changed fairly drastically when a motion with high acceleration towards the negative x-axis was performed. To account for this we perform the mute gesture as an initialization. Other than this we found no other major bugs in performance. All gestures worked with minimal latency, and the reaction time from the volume up and down gestures was also reasonably good.
Some things to think about if ever attempting to improve or expand on this project are perhaps investing in a better sensor for more precise measurements or possibly investigating possible causes for the current sensor’s buggy readings, working to reduce size of the board, adding a wireless (bluetooth?) component, adding a casing over the board, and possibly adding code to check for false readings.
ee107blog 