MIT system outperforms human intution with algorithms

The Massachusetts Institute of Technology is testing a new computer system aimed at finding patterns in data sets faster than human beings.

Researchers at MIT designed a Data Science Machine that searches for patterns in data sets, such as a database of promotional sale dates and weekly profits.

While computers can do many things faster than humans, human input is still required to choose what to look for in a large data set -- to find meaning in patterns, not just the patterns themselves. MIT researchers hope to automate that, too.

In three competitions, the Data Science Machine competed against 906 human teams and outperformed 615. The teams worked on their predictive algorithms for months while the Machine was able to compute its predictions in two to 12 hours.

To conduct analyses, the Machine looks at correlations between data tables using numerical identifiers. It then continually updates these identifiers as it continues to import data. As the identifiers add up, the Machine carries out various mathematical operations such as averages and sums and attempts to find trends in the data.

Max Kanter, an MIT student whose thesis served as the foundation for the Machine, says the device could be "a natural complement to human intelligence" and expedite the process of analyzing data. Kanter worked with his advisor Kalyan Veeramachaneni to prep the thesis for presentation next week at the IEEE International Conference on Data Science and Advanced Analytics.

Veeramachaneni said the machine could be a crucial asset in finding what components of a data set should be analyzed in order to draw conclusions.

For example, although MIT records student performance on online courses, it does not record statistics that could predict a student's likelihood to drop out. The Machine could identify variables such as how long it takes a student to get started on an assignment as well as how much time the student is active in the course and thereby infer the likelihood of course dropout.

Harvard University computer science professor Margo Seltzer said the project is "one of those unbelievable projects" seeking to solve real-world problems through a new approach. She further said the technology will "become the standard quickly -- very quickly."