New Smartphone App to Interpret Activities, People

A pair of summer interns at Microsoft have collaborated on a project to create an Android application capable of interpreting photographs -- complete with the identities and activities of the individuals involved -- via a smartphone camera without any human input whatsoever.

'TagSense', as it's called, is the brainchild of Microsoft Research interns Chaun Qin and Xuan Bao.

The software works by utilizing Android's accelerometers to determine the exact physical activity the person in the photograph is involved in. This is supported by built-in microphones that can determine whether the individual is talking, laughing, crying or silent. (Source: fastcompany.com)

TagSense is even able to record the precise weather conditions as per the location data extracted from an internal GPS system.

Smartphone Interaction Leads to "Collaborative Tagging"

While the above features are impressive enough, what separates TagSense from the other advanced smartphone applications on the market is its ability to tap into information stored onto other, nearby smartphones, which is where the correlations made to a spy machine come into play.

In other words, once a picture has been taken via a smartphone camera running TagSense, the application will interact with the data stored on their phone, enabling collaborative tagging of the individual's identity, along with the other aforementioned information.

Collaborative tagging, however, will be a strict opt-in requirement for those looking to use that particular feature of the app.

App Promotes Photo-Specific Searches

For all of its potential uses, TagSense's creators hope their app will ultimately be viewed as a means to automate the process of tagging photographs via sensor-assistance.

For example, a user can go back and search their personal photo collection for "Pictures of Sara", in addition to being more specific in searching for "Pictures of Sara Laughing" or "Pictures of Sara in the Snow". (Source: eurekalert.org)

As it stands, TagSense is still in prototype mode, but Qin and Bao believe that a commercial product based on the app could be available in a few years' time.