When he landed in Moscow in early March from South Korea, Vladimir Bykovsky said he was ordered into self-quarantine for two weeks. Under the rules, he was not permitted to leave the house. But after a few days—and showing no symptoms of novel coronavirus-- he said he stepped out of his building and threw out his trash. Then he went back inside.
“Thirty minutes after that, the police were ringing at my door,” Bykovsky, a 32-year-old worker at a patent firm, told ABC News.
Bykovsky said officers told him he had been spotted on a camera set up on the front door of his apartment building and identified through a facial recognition system, which had then alerted them.
“Honestly, I always thought it was something to do with our intercom,” Bykovsky said of the camera. “But it turns out it’s connected to the surveillance system that’s working across the whole of Moscow.”
Tune into ABC at 1 p.m. ET and ABC News Live at 4 p.m. ET every weekday for special coverage of the novel coronavirus with the full ABC News team, including the latest news, context and analysis.
Russia’s capital now has one of the largest facial recognition systems in the world and it is using it to police its quarantine restrictions imposed in response to the coronavirus pandemic.
Since January, the system, developed over the last five years, has moved from its test phase into large-scale use, according to authorities. It is now operating across 105,000 cameras around Moscow, the head of video surveillance at the city's information technology department, Mikhail Golovin, said in February.
It's an example of how countries like Russia and China have turned to authoritarian technologies to fight their epidemics. In Moscow, authorities also appear to view it as an opportunity to showcase their surveillance capabilities. And while some experts question how effective the facial recognition system is yet in reality, civil rights advocates worry how the system and its capabilities for pursuing political enemies will outlast the health crisis.
The system allows police to search in real-time across those cameras for a person using just a single image, according to its developers.
Russia tries again to win UN approval for virus resolutionAs in other countries, Moscow’s technology works through training a neural network (machine learning modeled loosely on the human brain) to recognize faces by feeding it millions of images. The system can then construct a unique profile for each face it analyzes. In Moscow, the AI software scans and creates a temporary profile for every person passing in front of the tens of thousands of cameras and compares them to those of people on a watchlist, its developers said.
If the system detects a match, it sends an alert to a police command center.
Originally designed to catch criminals, authorities have found the system can be used to track people during lockdown. Since late March, Moscow residents have been ordered to stay in their homes, except for visiting grocery stores, pharmacies or seeking urgent medical care. Those who don't obey face fines.
For now, the facial recognition technology is being used only to monitor a narrower category: those who have tested positive for the virus or deemed high risk of having it, including those returning from abroad, like Bykovsky. He described how passengers landing in Moscow were stopped by medics and photographed three times while being tested for the virus. They were then entered into a quarantine watch-list.
The system can also be used for contact tracing, allowing police to follow an infected person’s movements over previous days, said Artyom Kukharenko, the head of research and development at NtechLab, the primary developer for the system.
Kukharenko said the system can complete a search in under 10 seconds and that it largely works even when people are wearing face masks by identifying them by their eye line.
The company soon hopes to roll out a function analyzing silhouettes, he said, meaning it could spot large gatherings and also automatically detect when people are standing too close together.
Since March, Moscow’s authorities said the system has allowed them to catch several hundred people violating quarantine. Other Russian cities are also introducing similar systems.
Russia is not the only country to turn to expansive surveillance technologies. Facial recognition was already in wide-use in China before the pandemic and many countries are now already using or considering using mobile phone location data as well as downloadable surveillance apps to track people. In the United States, the federal government, through the Centers for Disease Control and Prevention, is using location data from millions of cellphones to understand how people are moving during the pandemic.
Outside observers though cautioned about the Russian claims for the facial recognition system, noting its application is likely more limited than officials suggest.
“I wouldn’t discount what they’re saying, but I would caveat it significantly in terms of their spread and abilities to do what they’re claiming,” said Magarita Konaev, a research fellow at Georgetown University’s Center for Security and Emerging Technology, who is focused on the Russian military’s use of artificial intelligence.
She noted that the technology was still emerging and applying it in real-world conditions on a large-scale would usually mean a fall off in accuracy. “I think right now that reporting it the way that they’re selling it is doing marketing for them,” she said.
Some police departments in the United States have also experimented with facial recognition before the virus, but outside China, there is perhaps nowhere where authorities claim to have implemented it as widely as in Moscow.
Aside from anecdotal evidence, it is hard to verify the Moscow system’s effectiveness. A study in March commissioned by the U.S. Department of Commerce included some of the algorithms it uses, including those from NTechLab and another company, Vision Labs. It found both algorithms could perform a correct identification in a fraction of a second using a single basic computer processor.
Patrick Grother, a scientist at the Commerce Department's National Institute of Standards and Technology and an author of the study, said whether the technology could maintain similar speeds in the real world was primarily a question of how much computing power a city was willing to purchase.
The contract to supply the equipment for the system, published on Russia’s government procurement site, shows the city ordered 258 servers as well as 16 of the U.S. company Nvida’s latest graphics processing units, the Tesla T4. The Russian newspaper, Kommersant, calculated that the system’s planned total storage capacity is 9 petabytes. For comparison, 1.5 petabytes is equivalent to 60 billion photos stored by Facebook
Several companies provided algorithms for Moscow. The lead developer, NtechLab, was previously known for an app, FindFace, which allowed users to find people’s social media profiles with a photo.
The Russian system’s developers said the Moscow system had been designed from the beginning to work across massive numbers of cameras without demanding too much power or storage capacity. Importantly they said it could be applied to ordinary CCTV cameras already in-use.
“We built a super power-efficient system, so we consume very few resources,” said Alexander Khanin, founder of Vision Labs, which supplied one of the algorithms.
Three developers said it was not difficult to turn the systems to monitoring quarantine and cities with the system already installed would have sufficient capacity for it.
“If the system is set up properly from the beginning, then it needs 2-3 days to switch it,” said Andrey Khrulev, business development manager at Speech Technology Center, which has installed the system in other Russian cities, including St. Petersburg and Nizhnii Novogorod. “We just adjust the algorithm a little.”
In coronavirus pandemic, authoritarians around the world see opportunity to crack downCritics of the surveillance also tend to believe it is rapidly becoming a reality. The anti-Kremlin opposition worry it provides a powerful new tool for repression and some activists have warned it will further deter people from joining protest rallies.
Behind the authorities’ sleek representation, its critics said there is also a seamier side to the system.
A freelance journalist, Andrey Kaganskikh, in December reported it was possible to buy access to the system. Dealers on Russian internet forums offer to perform searches for people that, Kaganskikh said, are carried out by law enforcement officers.
For a report for the independent Russian news outlet, MBKhMedia, Kaganskikh said he paid one dealer to run a search for him using his photograph. The dealer sent him back a dossier containing around 200 screengrabs from cameras where the system thought it had spotted him. But although many of the people in the images looked like him, Kaganskikh said none of them was actually him.
He said the system reported it had found him with 67% certainty, suggesting it may not be so accurate. But, Kaganskikh said, at the time only 3,000 cameras had been connected to the facial recognition software. With over 100,000 now linked, he said, he thought it would not be a problem to find him.
Security for the system is also minimal. It can be logged into from the open internet by anyone with a username and password. Kaganskikh said the price for buying a search, as he did, is low—around $40 - $70.
Kaganskikh said while he believes there are legitimate uses for the surveillance system, it was being implemented with virtually no discussion and few legal safeguards, making it ripe for abuse. The pandemic, he said, was accelerating the system’s expansion.
“They were going to implement it anyway but now they have the perfect time to test it,” he said. “This pandemic can become like a big advertising campaign for this system.”