Health Department Had Opioid-Tracking AI Forced On Them By Utah Lawmakers, Emails Show

Last year, Motherboard reported how an artificial intelligence surveillance firm called Banjo had ingratiated itself with state officials in Utah and had gotten the state to give it a $20 million contract to create predictive policing systems within the state. Our reporting showed that, among other things, Banjo promised that its artificial intelligence could be used to combat the opioid epidemic, and the firm was given $250,000 to build an "opioid dashboard" that state health officials believed would be better spent on life-saving drugs like nalaxone.

Now, newly released internal emails obtained from the Utah Department of Health using a public records request shows that top officials within the state's health department believed Banjo's technology to be incapable of doing what it promised, a privacy nightmare, a waste of money, and not worth engaging with. The emails show that Department of Health officials believe they were forced to work with the company by the Utah state legislature, and that the tools that Banjo was paid to build already existed in-house at the agency.

"My continued concern with the credibility of this company is they are providing misinformation which is simply not true to the press," George McEwan, director of information technology at the Utah Department of Health, wrote in one email, referring to claims the company made about how it could fight the opioid epidemic. He said that he believed the company would use good press to "attempt to leverage into other agencies based on a falsehood."

Banjo said its "opioid dashboard" would be able to help state officials predict where overdoses might happen by making an overdose "heat map." It plugged it as part of its "live time intelligence" product which the company said would monitor social media, surveillance cameras, state data, and other inputs to predict crime and track down criminals. A state audit, launched after investigations by Motherboard and OneZero, found that Banjo essentially didn't actually have any artificial intelligence capabilities.

The newly obtained emails, though, show that even before Banjo launched in the state health officials were skeptical of its capabilities and utility. McEwan said he did not believe the department should work with Banjo for a variety of reasons, including their privacy practices and the fact that their technology was unproven. He also felt that they were "dealing with us via the legislature to gain access [to the agency]."

Parts of his email were cut off due to email formatting errors, but the general thrust of his argument was clear: "As far as I know they haven't been successful in using their technology for any real [cut off] mirrors and as data stewards of some of the most private information in the State, I think we should continue to pursue a policy of nonengagement with Banjo. At some point the emperor has [cut off] away from supporting this group," McEwan continued.

A series of Department of Health officials expressed skepticism about legislation that would give Banjo yet more access to state health records. One employee stated that “we currently have real time data for ER and do not require Banjo technological support,” and “I am not aware of indicating that we were interested in contracting with Banjo.” After a series of emails, four different DOH employees said they had serious misgivings about a partnership with Banjo.

Joseph Miner, the state health director, said "there are too many concerns about violations of privacy," in another email.

Another document obtained by Motherboard showed potential use cases for Banjo, all of which required the state to send the company large amounts of data.

As Motherboard showed last year, Utah awarded Banjo with a contract to build a surveillance system that would take feeds from all sorts of data sources, including CCTV spread around the state, and produce alerts for authorities. Motherboard also found that Banjo used a secret company called Pink Unicorn Labs to make fake apps and surreptitiously harvest data from social media sites.

OneZero reported that Banjo's creator, Damien Patton, once helped a KKK leader shoot up a synagogue. After that report, Patton left the company and Banjo rebranded as safeXai.

The Department of Health did not respond to a request for comment. Neither did Justin Lindsey, the CEO of safeXai.

Subscribe to our cybersecurity podcast, CYBER.