“Big data” technologies have the potential to revolutionize policing, but they also raise new questions about privacy, accuracy, and when trade-secret protection must give way to much-needed public accountability.
A case in point is the Fresno, California police department’s controversial, year-long test use of “Beware.” Produced by the private software company Intrado, Beware analyzes publicly available data to calculate individual “threat scores.”
Here’s how the software works: When officers respond to a house call, Beware scans the names associated with that address against billions of data points, relying on property records, criminal records, vehicle registrations, and other public data. It has the capacity to comb social media and assign a color-coded “threat score” of red, green, or yellow for each resident. The Fresno police department touted the software for providing extra information to officers to help them respond to calls safely.
But early on, community members expressed concerns about the system’s accuracy and accountability. When Fresno City Council Member Clint Olivier asked to be checked against the system, his address registered a threat level of “yellow.” Olivier learned that he may have “earned” this threat level thanks to a previous resident of his home. “So, even though it’s not me who is the yellow guy, your officers are going to treat whoever comes out of that house in his boxer shorts as the yellow guy,” he said at a city council hearing. “And that not may be fair to me because I’m the green guy.”
The software’s lack of transparency did not help its case. Fresno council members said the community needed more information about the algorithms and variables used before it would approve the use of Beware. Yet Intrado considers this information a trade secret, making public oversight challenging, if not impossible.
After months of controversy, this March the city council voted 5-0 to reject a proposal by the police department to enter a five-year contract with Intrado. The council did leave the door open to reconsidering a partnership with Intrado after soliciting public input around the software.
Similar questions regarding police use of big data are likely to arise in jurisdictions beyond Fresno. Beware is just one of many data-aggregating technologies that municipal, state, and federal police agencies are implementing around the country. These technologies analyze very large data sets for critical patterns that enable strategic policing decisions. For instance, LexisNexis Risk Solutions’ Social Media Monitor collects information from profiles and posts on various social media sites and allows users to conduct targeted searches for suspicious social network activity. On a larger scale, the New York City Police Department’s Domain Awareness System (DAS) integrates the output from police databases, radiation detectors, an estimated 9,000 surveillance cameras and a newly expanded, national license plate reader database into a single interface.
These systems have the potential to greatly increase the efficiency and effectiveness of police agencies. Predictive policing systems can focus limited police resources on the neighborhoods and blocks most vulnerable to crime. Pattern-spotting abilities enabled by data analytics are credited with making police more effective in solving more cases, more quickly. Fresno Police Chief Jerry Dyer emphasized that the Beware system enhances officer safety during 911 calls by giving them more information about largely unknown situations.
Still, along with the tremendous benefits of big data analytics come concerns about privacy, accuracy, and discrimination. To legitimize this powerful technology, affected citizens should have a say in the policies and rules governing the use of potentially invasive surveillance technologies. While Fresno appears to be headed in the right direction, soliciting public input before agreeing to a trial run with Beware would have done much to allay community concerns by putting safeguards and oversight mechanisms in place in advance.
In addition, policing agencies trying out new technologies can bolster public accountability by collecting data that measures the impact of such technologies on their citizens. Fresno’s eighteen-month trial period using Beware would have been an excellent opportunity to track 911 call outcomes broken down by race, which could have permitted the department to directly address worries about Beware’s racially disparate impact.
Private companies like Intrado may argue that public access to information about their proprietary programs is incompatible with trade-secret protection. However, this supposed tension may be a bit of a red herring. Permitting public scrutiny need not be an all-or-nothing proposition. It may be enough simply to reveal the types and sources of information the program uses in order to allow meaningful public comment, without necessarily revealing the proprietary algorithm used to do something like calculate a “threat score.”
Ultimately, the Fresno city council made the right call when it agreed to solicit public input into the adoption of technology like Beware. Transparency and public input are always essential when police agencies are considering the adoption of potentially intrusive new technologies.