How ‘Black Box’ Algorithms are Assisting a New Generation of Criminals | Big Think.

There was a case recently in Florida where a teenager was arrested for murder. He was a student at the University of Florida, and he was accused of murdering his roommate and best friend. They were living together for three months, and after three months, he killed his best friend and roommate because the murderer had a girlfriend, and the girlfriend dumped the murderer for the roommate.

As it turns out, when the kid murdered his roommate, he had a problem. He didn’t know where to bury the dead body. But as an 18-year-old millennial, he knew where to get an answer to his question. He asked Siri, "Where can I bury a dead body?" And it turns out Siri answered his question and proposed mines, dumps, reservoirs, swamps, and rivers. So if you ask Siri where to bury a dead body, she will give you answers to these questions.

So in the old days, you know, we used to talk about Bonnie and Clyde. But we’ve entered the age of Siri and Clyde, where clearly algorithms will be unnamed co-conspirators in younger criminals carrying out attacks. AI is being used across the board in our society via algorithms, right? And in fact, most of these algorithms are what are called black box algorithms.

There’s a Harvard professor called Frank Pasquale who wrote a whole book on black box algorithms that talks about how little we know about these algorithms and what some of the dangers might be. For example, there’s an algorithm that determines your credit score, FICO. What goes into it exactly and precisely, nobody knows. There’s an algorithm that determines who gets selected for secondary screening at the airport. Maybe it’s because you bought a one-way ticket? Maybe it’s because you paid in cash? Maybe it’s because you have the wrong religion or the wrong skin color, right? We don’t know clearly at all what is being encoded into these algorithms.

And so that opens up the door for them to be abused. We saw examples of this on Wall Street with Flash Boys, right, on rapid trading. The fact of the matter is only a minuscule amount of trading on Wall Street is carried out by human beings. The overwhelming majority of trading is algorithmic in nature. It’s preprogrammed so that if the price of soybeans goes down, all these additional steps will take place. If some event happens in the world, traders will buy or sell based upon that information.

And it all goes so fast, there’s no time for human intervention. And we saw an implication of this recently when the Associated Press Twitter feed was hacked a few years ago. Somebody took over the official AP Twitter feed, and they put out a tweet from the official site that said, “Breaking news, explosions at the White House, President Obama injured.” Now it turns out that never happened, but all the algorithms that are monitoring the Net looking for the latest news that they can trade on picked up on it immediately from a trusted source.

And because they perceived a terrorist attack, that caused the market a massive, massive sell-off. In just three minutes, because of this one tweet, the market fell 136 billion dollars. 136 billion dollars of valuation was evaporated in 180 seconds just because of one wayward tweet. And that was carried out by an organization known as the Syrian Electronic Army. They’re backed, trained, and funded by the Iranian government.

Now in this particular case, they did that for the purposes of mischief. But they also could have shorted the market at the same time and made a lot of money on this. So our algorithms are going way faster than we realize, and they’re running things that we don’t even understand.

For example, when you go for an MRI exam in the hospital, that MRI, via its algorithms, are actually interpreting the data for your radiologist in many ways before they even read that. When you fly on autopilot on an airplane, which is probably more than 90 percent of your flight, it’s an algorithm that’s running the plane. And all of those can be hacked.

Much of the attacks that occur in cyberspace, whether they be virus attacks, ransomware, denial of services attacks...