What the NSA can do with “big data”
Courtesy Sean Gallagher, ArsTechnica
One organization’s data centers hold the contents of much of the visible Internet—and much of it that isn’t visible just by clicking your way around. It has satellite imagery of much of the world and ground-level photography of homes and businesses and government installations tied into a geospatial database that is cross-indexed to petabytes of information about individuals and organizations. And its analytics systems process the Web search requests, e-mail messages, and other electronic activities of hundreds of millions of people.
No one at this organization actually "knows" everything about what individuals are doing on the Web, though there is certainly the potential for abuse. By policy, all of the "knowing" happens in software, while the organization’s analysts generally handle exceptions (like violations of the law) picked from the flotsam of the seas of data that their systems process.
I’m talking, of course, about Google. Most of us are okay with what Google does with its vast supply of "big data," because we largely benefit from it—though Google does manage to make a good deal of money off of us in the process. But if I were to backspace over Google’s name and replace it with "National Security Agency," that would leave a bit of a different taste in many people’s mouths.
Yet the NSA’s PRISM program and the capture of phone carriers’ call metadata are essentially about the same sort of business: taking massive volumes of data and finding relationships within it without having to manually sort through it, and surfacing "exceptions" that analysts are specifically looking for. The main difference is that with the NSA, finding these exceptions can result in Foreign Intelligence Surveillance Act (FISA) warrants to dig deeper—and FBI agents knocking at your door.
So what is it, exactly, that the NSA has in its pile of "big data," and what can they do with it?.