The rise of human-computer cooperation - Shyam Sankar

I'd like to tell you about two games of chess. The first happened in 1997, in which Garry Kasparov, a human, lost to Deep Blue, a machine. To many, this was the dawn of a new era, one where man would be dominated by machine. But here we are, 20 years on, and the greatest change in how we relate to computers is the iPad, not how. The second game was a freestyle chess tournament in 2005, in which man and machine could enter together as partners rather than adversaries, if they so chose.

At first, the results were predictable; even a supercomputer was beaten by a grandmaster with a relatively weak laptop. The surprise came at the end: who won? Not a grandmaster with the supercomputer, but actually two American amateurs using three relatively weak laptops. Their ability to coach and manipulate their computers to deeply explore specific positions effectively counteracted the superior chess knowledge of the grandmasters and the superior computational power of other adversaries. This is an astonishing result: average men, average machines, beating the best man, the best machine.

And anyways, isn't it supposed to be man versus machine? Instead, it's about cooperation and the right type of cooperation. We've been paying a lot of attention to Marvin Minsky's vision for artificial intelligence over the last 50 years. It's a sexy vision, for sure. Many have embraced it and become the dominant school of thought in computer science. But as we enter the era of big data, of network systems, of open platforms, and embedded technology, I'd like to suggest it's time to reevaluate an alternative vision that was actually developed around the same time.

I'm talking about JCR Licklider's human-computer symbiosis, perhaps better termed intelligence augmentation. Licklider was a computer science titan who had a profound effect on the development of technology and the Internet. His vision was to enable man and machine to cooperate in making decisions, in controlling complex situations without the inflexible dependence on predetermined programs. Note that word: cooperate.

Licklider encourages us to take a toaster and make it data from Star Trek, but to take a human and make her more capable. Humans are so amazing—how we think, our nonlinear approaches, our creativity, iterative hypotheses—all very difficult, if possible, for computers to do. Licklider intuitively realized this, contemplating humans setting the goals, formulating the hypotheses, determining the criteria, and performing the evaluation.

Of course, in other ways, humans are so limited. We're terrible at scale computation and volume. We require high-end talent management to keep the rock band together and playing. Licklider foresaw computers doing all of the routinized work that was required to prepare the way for insights and decision-making silently, without much fanfare. This approach has been compiling victories beyond chess. Protein folding, a topic that shares the incredible expansiveness of chess, is a world-changing problem with huge implications for our ability to understand and treat disease.

For this task, supercomputer-fueled brute force simply isn't enough. Fully, a game created by computer scientists illustrates the value of the approach. Non-technical, non-biologists amateurs play a video game in which they visually arrange the structure of the protein, allowing the computer to manage the atomic forces, interactions, and identify structural issues. This approach beats supercomputers 50% of the time and tied 30% of the time. Foldit recently made a notable and major scientific discovery by deciphering the structure of the Mason Fisher monkey virus, a protease that had eluded determination for over 10 years. It was solved by three players in a matter of days, perhaps the first major scientific advance to come from playing the video game.

Last year, on the side of the Twin Towers, a 9/11 memorial opened. It displays the names of the thousands of victims using a beautiful concept called meaningful adjacency. This places the names next to each other based on their relationships to one another—friends, families, co-workers. When you put it all together, it's quite a computational challenge: 3,500 victims, 1,800 to JCCC requests. The importance of the overall physical specifications in the final aesthetics, when first reported by the media, full credit for such a feat was given to an algorithm from the New York City design firm Local Projects.

The truth is a bit more nuanced. While an algorithm was used to develop the underlying framework, humans used that framework to design the final result. So, in this case, a computer evaluated millions of possible layouts, managed a complex relational system, and kept track of a very large set of measurements and variables, allowing the humans to focus on design and compositional choices.

So, the more you look around you, the more you see Licklider's vision everywhere—whether it's augmented reality in your iPhone, GPS in your car, or human-computer symbiosis making us more capable. So, if you want to improve human-computer symbiosis, what can you do? You can start by designing the human into the process. Instead of thinking about what a computer will do to solve the problem, design the solution around what the human will do as well.

When you do this, you'll quickly realize that you spend all of your time on the interface between man and machine, specifically on designing away the friction in the interaction. In fact, this friction is more important than the power of the man or the power of the machine in determining overall capability. That's why two amateurs with a few laptops handily beat a supercomputer and a grandmaster. What Kasparov calls processes a byproduct of friction—the better the process, the less the friction. Minimizing friction turns out to be this isof variable.

Or take another example: big data. Every interaction we have in the world is recorded by an ever-growing array of sensors—your phone, credit card, computer. The result is big data, and it actually presents us with an opportunity to more deeply understand the human condition. The major emphasis of most approaches to big data focuses on how do I store this data? How do I search this data? How do I process this data? These are necessary but insufficient questions.

The imperative is not to figure out how to compute but what to compute. How do you impose human intuition on data at this scale? Again, we start by designing the human to the process. When PayPal was first starting as a business, their biggest challenge was not how do I send money back and forth online. It was how do I do that without being defrauded by organized crime?

Why so challenging? Because while computers can learn to detect and identify fraud based on patterns, they can't learn to do that based on patterns they've never seen before. Organized crime has a lot in common with this audience: brilliant people, relentlessly resourceful, with an entrepreneurial spirit, and one huge and important difference—purpose. And so, while computers alone can catch all about the cleverest fraudsters, catching the cleverest is the difference between success and failure.

There's a whole class of problems like this—ones with adaptive adversaries. They rarely, if ever, present with the repeatable pattern that's discernable to computers. Instead, there's some inherent component of innovation or disruption, and increasingly, these problems are buried in big data. For example, terrorism: terrorists are always adapting in minor and major ways to new circumstances. And despite what you might see on TV, these adaptations and the detection of them are fundamentally human.

Computers don't detect novel patterns or new behaviors—humans do. Humans using technology, testing hypotheses, searching for insight by asking machines to do things for them. Osama bin Laden was not caught by artificial intelligence; he was caught by dedicated, resourceful, brilliant people in partnerships with various technologies. As appealing as it might sound, you cannot algorithmically data-mine your way to the answer. There is no "find terrorists" button.

The more data we integrate from a vast variety of sources, across a wide variety of data formats, from very disparate systems, the less effective data mining can be. Instead, people will have to look at data and search for insight. And as Licklider foresaw long ago, the key to great results here is the right type of cooperation. And as Kasparov realized, that means minimizing friction at the interface.

Now, this approach makes possible things like combing through all available data from very different sources, identifying key relationships, and putting all of that in one place—something that's been nearly impossible to do before the salm. This has terrifying privacy and civil liberties implications. To others, it foretells of an era of greater privacy and civil liberties protections. But privacy and civil liberties are of fundamental importance that must be acknowledged, and they can't be swept aside, even with the best of intents.

So, let's explore through a couple of examples the impact that technologies built to drive human-computer symbiosis have had in recent times. In October 2007, US and coalition forces raided an al-Qaeda safe house in the city of Sinjar, on the Syrian border of Iraq. They found a treasure trove of documents—700 biographical sketches of foreign fighters. These foreign fighters had left their families in the Gulf, the Levant, and North Africa to join al-Qaeda in Iraq.

These records were human resource forms the foreign fighters filled out as they joined the organization. It turns out that al-Qaeda too is not without its bureaucracy. They answered questions like, "Who recruited you? What's your hometown? What occupation do you seek?" And that last question brought a surprising insight: the vast majority of foreign fighters were seeking to become suicide bombers for martyrdom.

This is hugely important since between 2003 and 2007, Iraq had 1,382 suicide bombings—a major source of instability. Analyzing this data was hard; the originals were sheets of paper in Arabic that had to be scanned and translated. The friction in the process did not allow for meaningful results in an operational timeframe.

Using humans, PDFs, and tenacity alone, the researchers had to leverage their human minds with technology to dive deeper and explore non-obvious hypotheses. In fact, insights emerged: 20% of the foreign fighters were from Libya; 50% of those from a single town in Libya—hugely important since prior statistics put that figure at 3%. It also helped to hone in on a figure of rising importance in al-Qaeda, Abu Yahya al-Libi, a senior cleric in the Libyan Islamic Fighting Group.

In March of 2007, he gave a speech after worship with a surge in participation amongst Libyan foreign fighters. Perhaps most clever of all, though, and least obvious, by flipping the data on its head, the researchers were able to deeply explore the coordination networks in Syria that were ultimately responsible for receiving and transporting the foreign fighters to the border. These were networks of mercenaries, not ideologues, who were in the coordination business for profit.

For example, they charged Saudi foreign fighters substantially more than Libyans—money that would have otherwise gone to al-Qaeda. Perhaps the adversary would disrupt their own network if they knew they were cheating would-be jihadists. In January 2010, a devastating 7.0 earthquake struck Haiti, the third deadliest earthquake of all time, leaving 1 million people—10% of the population—homeless.

One seemingly small aspect of the overall relief effort became increasingly important, as the delivery of food and water slowed in January and February—the dry months in Haiti. Yet many of the camps had developed standing water. The only institution with detailed knowledge of Haiti's floodplains had been leveled in the earthquake, leadership inside.

So, the question is: which camps are at risk? How many people in these camps? What's the timeline for flooding? Given very limited resources and infrastructure, how do we prioritize the relocation? The data was incredibly disparate. The US Army had detailed knowledge for only a small section of the country; there was data online from a 2006 environmental risk conference; others had geospatial data that had not been integrated.

The human goal here was to identify camps for relocation based on priority need. The computer had to integrate a vast amount of geospatial information, social media data, and relief organization information to answer this question. By implementing a superior process, what was otherwise a task for 40 people over three months became a simple job for three people in 40 hours.

All victories for human-computer symbiosis were more than 50 years into Licklider's vision for the future, and the data suggests that we should be quite excited about tackling this century's hardest problems—man and machine in cooperation together. Thank you.