Walmart just won a patent for audio surveillance technology that measures workers’ performance, and could even listen to their conversations with customers at checkout.
The “listening to the frontend” technology, as its called, is one of many futuristic ideas Walmart has sought to patent in recent years as it competes with Amazon for domination of the retail industry.
While there’s no guarantee that Walmart will ever build this technology, the patent shows the company is thinking about using tech not just to facilitate deliveries or make its warehouses more efficient, but also to manage its workforce, which is the largest in the United States.
Walmart declined to comment on whether it plans to use audio sensors to measure the productivity of its staff in the near future, but said in a statement, “We’re always thinking about new concepts and ways that will help us further enhance how we serve customers, but we don’t have any further details to share on these patents at this time.”
Based on the application, Walmart’s patented surveillance system would use a series of sensors in the cashier area to collect audio data — everything from “beeps” to “rustling noises” to “conversations between guests and an employee stationed at the terminal.”
It would then analyze this information and use it to calculate various “performance metric[s]” for the employee.
“Employee efficiency and performance can help decrease costs for a shopping facility as well as increase guest satisfaction,” the patent reads. “Tracking performance metrics for employees to ensure that the employees are performing their jobs efficiently and correctly can aid in achieving these costs savings and increases in guest satisfaction.”
But Ifeoma Ajunwa, an assistant professor at Cornell’s Industrial and Labor Relations School, told BuzzFeed News that surveilling employees can actually have the opposite effect. “Several studies have shown that there is a psychological impact of pervasive surveillance,” Ajunwa said. She cited the work of Harvard professor Ethan Bernstein, who found that workplace surveillance “can lead to this opposition feeling, where employees view the employer not as benevolent, but as dictators. And it can impact that attitude toward the higher-up and can lead to resistance.”
She also cited a study that found that excess surveillance can slow down workers, who are less creative and efficient when they’re nervous or feel they’re being watched.
One way the system could be used, according to the patent description, would be to measure the distance of the sound of customers’ voices from the cash register, in order to approximate the length of the line. This could help Walmart determine how quickly a given cashier is able to move through customers, or whether “more terminals should be opened to decrease line length.”
It could also be used to measure “bagging efficiency, such as a number of items per bag, a number of bags used per transaction, etc.,” by transmitting the sound of items being bagged.
Most invasively, the system could also be used to analyze voices of customers and guests to see if they’re interacting, and even to listen to what they’re saying. “If however the performance metric is based on the content of the conversation (e.g., was a specific greeting used or script followed), the system can process the audio detected by the sound sensors 102 (e.g., using speech recognition) to determine the performance metric,” the patent description explains.
Similar systems have long been in use in call centers, where calls are recorded and reviewed, and employees are rated based on what they say and what the outcome of the call was.
Walmart’s patent description says this audio surveillance technology could be used to monitor the performance of one or more employees at a time, and could measure one or more metrics simultaneously — e.g., “items per bag and line length, line length and guest greeting, or any combination of two or more performance metrics.”
In a world where Amazon and other e-commerce options are pushing big-box stores out of the picture, Walmart is under intense pressure to increase efficiency. Many of Walmart’s major costs — retail real estate and full-time employees — aren’t as costly for Amazon, which relies on robots in warehouses and a vast network of contracted delivery workers to cut costs. As a company, Amazon is notoriously data-obsessed; it measures employee productivity by the minute, uses cameras in its warehouses, and even wants to track their movements with a recently patented smart wristband. This “listening to the frontend” patent signals that Walmart is ready to compete.
But the technology, if Walmart ever builds it, could be a threat to both consumer and employee privacy. New data privacy laws in Europe require employers to tell employees what data they collect about them and how the company plans to use it. But in the United States, those rules don’t exist.
“There’s sometimes a misconception that the consent of employees is required for surveillance, but frankly, as long as the employer can make an argument for why the surveillance is necessary for a business purpose as opposed to a discriminatory purpose, there’s no law that says consent is required,” said Ajunwa. If Walmart did build its audio sensor system, it would be perfectly legal, she said, and the company wouldn’t even have to tell employees. A union could bargain for a contract that would require Walmart to “give notice and obtain consent,” Ajunwa said, but Walmart employees aren’t unionized.
Ajunwa said one of the most troubling aspects of this kind of tech is that the system’s purpose could change after it’s installed, a concept known as mission creep.
“The sound recording is helpful for determining if the line is too long, let’s automatically open a new cashier. But then there’s potential for mission creep where it’s more like, ‘as a cashier you’re too friendly, you’re talking too much, and therefore not moving people along, so let’s penalize you.’ Even though the technology is presented as interested in one thing, the fact that it has the potential for both things to be captured is of concern. There’s a lot of potential for misuse.”
In the past, Walmart workers’ concerns have concentrated on things like low pay, exploitative HR policies around sick days, and gender discrimination. Surveillance of Walmart workers, especially those interested in organizing a labor union within the retail giant, has also been a concern. In recent years, a major pay raise seems to have improved relations between management and staff.
But an audio surveillance system that measures an individual worker’s productivity and listens to their conversations could change that.
“Employers need to think long and hard about whether this will solve the problem of worker productivity, or if it’s going to make that problem worse,” Ajunwa said.