Called M2, the system operates across heterogeneous systems, including Android and iOS, combining the functionality of multiple mobile systems into a more powerful one that gives users a seamless experience across various systems.
M2 has been developed to address the growing demand for multi-mobile computing, users can instead dynamically switch their Netflix or Spotify streams from their smartphones to a collection of other nearby systems for a larger display or better audio. Instead of using smartphones and tablets in isolation, users will be able to combine their system's functionalities since they now can all work together. Users can even combine photos taken from different cameras and from different angles into a single, detailed 3D image.
"Given the many popular and familiar apps out there, we can combine and mix systems to do cool things with these existing unmodified apps without forcing developers to adopt new set of APIs and tools," explained Naser AlDuaij, the study's lead author and a PhD student. "We wanted to use M2 to target all apps without adding any overhead to app development. Users can even use M2 to run Android apps from their iPhones."
The challenge for the team was that mobile systems are not only highly heterogeneous, but that heterogeneous device sharing is also difficult to support. Beyond hardware heterogeneity, there are also many diverse platforms and OS versions, with a wide range of incompatible device interfaces that dictate how software applications communicate with hardware.
While different mobile systems have different APIs and low-level devices are vendor-specific, the high-level device data provided to apps is generally in a standard format. So AlDuaij took a high-level device data approach and designed M2 to import and export device data in a common format to and from systems, avoiding the need to bridge incompatible mobile systems and device APIs. This method enables M2 not only to share devices, but also to mix and combine devices of different types of data since it can aggregate or manipulate device data in a known format.
"With M2, we are introducing device transformation, a framework that enables different devices across disparate systems to be substituted and combined with one another to support multi-mobile heterogeneity, functionality, and transparency," said AlDuaij.. "We can easily manipulate or convert device data because it's in a standard format. For example, we can easily scale and aggregate touchscreen input. We can also convert display frames to camera frames or vice versa. M2 enables us to reinterpret or represent different devices in different ways."
Among M2's device "transformations" are fusing device data from multiple devices to provide a multi-headed display scenario for a better "big screen" viewing or gaming experience. By converting accelerometer sensor data to input touches, M2 can transform a smartphone into a Nintendo Wii-like remote to control a game on another system. Eye movements can also be turned into touchscreen input, a useful accessibility feature for disabled users who cannot use their hands.
Using M2 is intended to be simple, all a user would have to do is to download the M2 app from Google Play or Apple's App Store. No other software is needed. One mobile system runs the unmodified app; the input and output from all systems is combined and shared to the app.
The Columbia team has started discussions with mobile OS vendors and phone manufacturers to incorporate M2 technologies into the next releases of their products.
