A Closer Look at NIBIN: A Study In Evaluating Worth

This post was written by Policing Project externs Leah Rosenberg, Miranda Murillo and Michael Rebuck.

Policing technology has seen incredible advances in recent years. Innovations such as drones, predictive-analytics, and automated license plate readers are just some of the technologies increasingly found in police departments of all sizes — often with high price tags.

However, as new policing technologies become more common, their effectiveness isn’t always comprehensively evaluated. One such new technology is NIBIN, the National Integrated Ballistic Information Network, a system that aims to harness the power of ballistic imaging technology to help police officers solve—and potentially prevent—gun crimes, but not without significant upfront costs in time and money.

What is NIBIN?

NIBIN is designed to help law enforcement agencies solve firearm-related crime by comparing ballistic evidence—specifically, shell casings—and generating matches. NIBIN stores approximately 16 million high-resolution images of over 3 million shell casings and compares any new image with the entire database.

Proponents argue that much like a fingerprint, the unique markings on the casings create a signature that may be traced back to the gun that fired the bullet. Storing images of these casings in a system that can identify matches enables law enforcement to connect crimes across cities and states. After a match is generated, specialists conduct manual inspections to confirm the findings.

The Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) has been working with local law enforcement since the 1990s to integrate NIBIN across jurisdictions. However, for a local department, implementing the technology can be a complicated process that typically requires purchasing equipment, training officers on its use, and uploading all ballistic evidence pertaining to any crime as soon as feasible. This includes uploading casings from reported gun crimes—both with victims and without—and test firing any firearm recovered in an arrest.

Even if the implementation barriers are overcome, it’s not entirely clear how worthwhile the system actually is. While NIBIN’s matching capabilities have been touted as an effective way to both solve and prevent crime, there is very little data that measures the technology’s success on either metric.

How effective is NIBIN?

ATF does publicize NIBIN’s hit rate — meaning the number of confirmed matches between shell casings. But, as The Marshall Project reported, what the agency does not track is how many of these hits actually lead to arrest or conviction, or what effect— if any — NIBIN has had on violent crime rates in jurisdictions that utilize the technology.

Another complication for evaluating NIBIN’s success is what The Marshall Project referred to as the “scattershot” way in which departments adopt the technology: Officers often fail to enter every casing recovered into the system and inconsistently follow up on generated leads. As the Police Executive Research Forum noted in a recent study, “Despite the apparent utility of NIBIN . . . the program has significant untapped potential.” Speaking with The Marshall Project, John Risenhoover, a former coordinator of NIBIN for the ATF, noted the inconsistent use of NIBIN has made the program a “huge waste of cash.”

Police departments deciding whether or not to adopt this technology, and legislatures thinking about mandating its use therefore face a dilemma: purchase the technology and integrate it into daily policing operations without any certainty that it fills its intended purpose, or forgo the potential benefits entirely.

But how can a department measure how much the technology is worth absent an evaluation of its success rate over the last 25 years?

Weighing cost and benefits when the data is unclear

At the Policing Project we encourage departments to measure and weigh potential benefits and costs prior to implementing new policing technologies — even in the absence of easily quantifiable data like monetary amounts and clearance rates. Departments should first identify the problem they are trying to solve and the importance of that problem. In looking at the potential benefits, they should consider the likelihood that these benefits will be achieved through the considered technology.

In the case of NIBIN, jurisdictions have reported several benefits from the system, including solving “dead end” firearm cases, linking one weapon across multiple fatal shootings, and coordinating with departments in other states to apprehend suspects in violent crimes. On the other hand, the technology is not without downsides. There are budgetary and resource constraints, the possibility of false positives, a potential “blindspot” to any weapon that is not a semi-automatic, and a necessary reallocation of officer time from other crucial activities, both for NIBIN training and to quickly upload images to the system ahead of other investigative steps.

Costs, however, can often be mitigated by implementing certain procedures and policies. Departments can train their officers to follow the control chart created by the National Institute of Standards and Technology, which establishes “traceability and quality assurance” by identifying best practices. Another cost mitigation tactic is sharing a single NIBIN machine between multiple departments.

The California Legislature recently considered and rejected a proposal to mandate use of NIBIN across the state. The bill never made it out of committee due to stiff resistance from the Attorney General’s office and other local law enforcement agencies. Arguments against adopting NIBIN included the cost, the availability of competing technology already on the market, and a more general resistance from state and local law enforcement to policies governing how they run their crime labs.

California estimated its upfront NIBIN costs could exceed $12 million and that annual costs would be approximately $10 million. New Jersey and Delaware, two states that have recently passed laws mandating NIBIN use, managed to overcome some of these cost hurdles. Assistance from the ATF in purchasing the equipment and re-allocating personnel to avoid exorbitant hiring costs made widespread adoption feasible.

Lacking an identifiable rate of success, departments considering the technology and state legislatures— like California—interested in mandating its use, should begin by identifying what they hope to achieve. If the goal is fewer unsolved gun crimes, they should ask, “How many such crimes are currently unsolved?” and “What other options exist for addressing this problem?”

If the goal is deterring future crime, another question could be, “How effective has this technology been at reducing crime rates in places like New Jersey and Delaware?” On the costs side, a proper analysis would consider, “How many personnel will have to be hired? How many machines will be required? What other law enforcement functions might be minimized in order to make room for NIBIN?”

While costs and benefits cannot always be defined to a dollar amount upfront, committing to new policing technology should still require jurisdictions to follow a cost-benefit approach. This means clearly identifying the goals, considering whether the proposed technology will achieve those goals, enumerating all potential costs, and creating procedures to effectuate the goals while mitigating the costs.

Cost-benefit analysis can help departments make informed decisions about whether something like NIBIN is worth their time and money.