From left, Microsoft researchers Robert Schapire, John Langford, Alekh Agarwal, Siddhartha Sen and Jennifer Wortman Vaughan. Credit: John Brecher When Robert Schapire started studying theoretical machine learning in graduate school three decades ago, the field was so obscure that what is today a major international conference was just a tiny workshop, so small that even graduate students were routinely excluded.
Machine learning still isn't exactly a topic of discussion at most family dinner tables. But it has become one of the hottest fields in computer science, turning once-obscure academic gatherings like the upcoming Annual Conference on Neural Information Processing Systems in Barcelona, Spain, into a sold-out affair attended by thousands of computer scientists from top corporations and academic institutions.
"It's been really something to see this field develop, and to see things that seemed impossible become possible in my lifetime," said Schapire, a principal researcher in Microsoft's New York City research lab whose machine learning research is widely used in the field.
The NIPS conference, which starts Monday, is so popular because machine learning has quickly become an indispensable tool for developing technology that consumers and businesses want, need and love. Machine learning is the basis for technology that can translate speech in real time, help doctors read radiology scans and even recognize emotions on people's faces. Machine learning also helps you sort the spam out of your inbox and remember your day's tasks.
It's a far cry from Schapire's early days in the field, when he said some of the hard problems were things like getting a computer to accurately read handwritten digits.
"Bit by bit, we've really been building this field from the bottom up, starting with basic problems," Schapire said. "Machine learning has become applicable to such a huge array of problems. It's really amazing."
Along the way, researchers say the field has benefited from people who dreamed about big breakthroughs with real-world benefits, such as the ability to create technology that can recognize words in a conversation as well as a person.
"Somehow the field of machine learning has been very fortunate in that we've had brilliant theorists who had a very practical outlook on things," said Alekh Agarwal, a researcher in Microsoft's New York lab.
Democratizing machine learning
Schapire, Agarwal and their colleagues at Microsoft and elsewhere say this is just the beginning. With the work they are presenting at NIPS and beyond, they are investigating ways to make machine learning even more useful for – and accessible to – a broader array of people.
The Microsoft researchers say they are at the forefront of efforts to democratize machine learning by making it easier for developers and engineers without a machine learning background to take advantage of these breakthroughs. That puts them on the cutting edge of finding ways to share the benefits of these systems widely with the rest of us.
"Machine learning has traditionally been a field where if you didn't have a Ph.D. you'd be at a loss – and if you did have a Ph.D. you might still be at a loss," said John Langford, a principal researcher in Microsoft's New York lab. "We're trying to make these things useful to someone who's a programmer without a lot of machine learning expertise."
Machine learning is useful in part because it can help people make predictions about anything from how many servers they'll need to deploy for a certain task to what news article a person might want to read. One of Langford's recent projects is looking at ways to make multiple predictions less burdensome, by creating systems that systematically eliminate common data errors with applications that use reinforcement learning and structured learning.
With reinforcement learning, researchers aim to get systems to use trial and error to figure out how to achieve a task. For example, a program could learn how to win at backgammon by playing against itself over and over again, picking up on what worked and what didn't over the course of those many games. The system is given very little outside guidance to make those decisions. Instead, decisions it makes early in the process can then affect how it succeeds later on.
Reinforcement learning is a counterpart to supervised learning, in which systems get better at doing things as they are fed more relevant data. For example, a supervised machine learning tool may learn to recognize faces in pictures after being shown a training set containing a huge array of faces.
The video will load shortly Credit: Microsoft Helping with decision making
In the more recent reinforcement approach Langford has been working on, the system also gets partial credit for choosing actions that are partially correct, making it easier to winnow down to the right answer.
Microsoft researchers say the decision service is such an exciting breakthrough because it can help systems make decisions using context.
"When you make a decision, you usually have some idea of how good it was," said Siddhartha Sen, a researcher in the New York lab. "Here's an opportunity to use machine learning to optimize those decisions."
The researchers say the cloud-based system, which is available in preview, is groundbreaking in part because it can be applied to so many different situations.
For example, it could be used by a news service that wanted to personalize content recommendations, a mobile health app that could personalize fitness activities or a cloud provider looking to optimize server resources.
Sen said one key goal for the testing service is to make it easy and accessible for people who may not be able to build these kinds of machine learning techniques on their own.
"The way it's democratizing machine learning is by making it very easy to interface with the system," said Sen, who will help run a workshop on the intersection of machine learning and systems design at NIPS. "We tried to hide all the difficult steps."
Microsoft has been developing the building blocks for a system like the decision service for years. But the system's current abilities would not have been possible even a few years ago, said Sarah Bird, who began working on it as a postdoctoral researcher in Microsoft's New York lab.
Bird, who is now a technical advisor in Microsoft's Azure division, said systems like these are improving rapidly because all the elements needed for machine learning – the computing power of the cloud, the algorithms and the data – are improving quickly, and at the same time.
"It's really amazing to watch all the pieces we need mature in parallel," she said. "It's a fun time for consumers and developers and researchers."
Fast pace of change
Many researchers say reinforcement learning holds a lot of promise because it could be used to create artificial intelligence systems that would be able to make the type of independent and complex decisions that could truly augment and complement human abilities.
Researchers caution that they are still in the early stages of finding success with reinforcement learning, but they say what they are seeing so far is promising.
"The sense of what's achievable is constantly changing, and that's what makes it so exciting to me," said Katja Hofmann, a researcher in Microsoft's Cambridge, UK, research lab. Hofmann has led development of Project Malmo, which uses Minecraft as the testing ground for reinforcement learning, and which will be demonstrated at NIPS.
Together with her colleagues, Hofmann has most recently been looking at ways that artificial intelligence agents can learn to do several tasks, rather than just one, and can apply the experience of how they completed one task to another. For example, an artificial intelligence navigating one Minecraft space may learn to recognize lava, and then use that knowledge to avoid lava in another place. Some of this research is being presented at the European Workshop on Reinforcement Learning, which is co-located with NIPS.
Explore further: How quantum effects could improve artificial intelligence
