What Wal-Mart Knows About Customers' Habits
Susan Notes: Wal-Mart's devotion to data mining has more relevance to education policy than we'd like to believe, probably more than we can yet imagine.
HURRICANE FRANCES was on its way, barreling across the Caribbean, threatening a direct hit on Florida's Atlantic coast. Residents made for higher ground, but far away, in Bentonville, Ark., executives at Wal-Mart Stores decided that the situation offered a great opportunity for one of their newest data-driven weapons, something that the company calls predictive technology.
A week ahead of the storm's landfall, Linda M. Dillman, Wal-Mart's chief information officer, pressed her staff to come up with forecasts based on what had happened when Hurricane Charley struck several weeks earlier. Backed by the trillions of bytes' worth of shopper history that is stored in Wal-Mart's computer network, she felt that the company could "start predicting what's going to happen, instead of waiting for it to happen," as she put it.
The experts mined the data and found that the stores would indeed need certain products - and not just the usual flashlights. "We didn't know in the past that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane," Ms. Dillman said in a recent interview. "And the pre-hurricane top-selling item was beer."
Thanks to those insights, trucks filled with toaster pastries and six-packs were soon speeding down Interstate 95 toward Wal-Marts in the path of Frances. Most of the products that were stocked for the storm sold quickly, the company said.
Such knowledge, Wal-Mart has learned, is not only power. It is profit, too.
Plenty of retailers collect data about their stores and their shoppers, and many use the information to try to improve sales. Target Stores, for example, introduced a branded Visa card in 2001 and has used it, along with an arsenal of gadgetry, to gather data ever since. But Wal-Mart amasses more data about the products it sells and its shoppers' buying habits than anyone else, so much so that some privacy advocates worry about potential for abuse.
With 3,600 stores in the United States and roughly 100 million customers walking through the doors each week, Wal-Mart has access to information about a broad slice of America - from individual Social Security and driver's license numbers to geographic proclivities for Mallomars, or lipsticks, or jugs of antifreeze. The data are gathered item by item at the checkout aisle, then recorded, mapped and updated by store, by state, by region.
By its own count, Wal-Mart has 460 terabytes of data stored on Teradata mainframes, made by NCR, at its Bentonville headquarters. To put that in perspective, the Internet has less than half as much data, according to experts.
Information about products, and often about customers, is most often obtained at checkout scanners. Wireless hand-held units, operated by clerks and managers, gather more inventory data. In most cases, such detail is stored for indefinite lengths of time. Sometimes it is divided into categories or mapped across computer models, and it is increasingly being used to answer discount retailing's rabbinical questions, like how many cashiers are needed during certain hours at a particular store.
All of the data are precious to Wal-Mart. The information forms the basis of the sales meetings the company holds every Saturday, and it is shot across desktops throughout its headquarters and into the places where it does business around the world. Wal-Mart shares some information with its suppliers - a company like Kraft, for example, can tap into a private extranet, called Retail Link, to see how well its products are selling. But for the most part, Wal-Mart hoards its information obsessively.
It also takes pains to keep the information secret. Some of the systems it uses are custom-built and designed by its own employees, the better to keep competitors off the trail. Companies that sell equipment and software to Wal-Mart are bound by nondisclosure agreements. Three years ago, Wal-Mart summarily announced that it would no longer share its sales data with outside companies, like Information Resources Inc. and ACNielsen, which had paid Wal-Mart for the information and then sold it to other retailers.
"When you look at their behavior, you can tell that Wal-Mart considers data to be a top priority," said Christine Overby, a senior analyst for consumer markets at Forrester Research. Over the years, she said, Wal-Mart executives have spent handsomely for their systems, paying $4 billion in 1991 to create Retail Link and signing onto innovations like bar codes and electronic data interchange, a forerunner of the Internet, well ahead of the pack. Wal-Mart is also driving manufacturers to invest in radio frequency identification. By next October, the company will require its biggest suppliers to tag shipments to some of its distribution centers with tiny transmitters that would eventually let Wal-Mart track every item that it sells.
With so much data at Wal-Mart's corporate fingertips, what are the risks to consumers? Most have no clue that their habits are monitored to such an extent. There are no signs - like the ones for Wal-Mart's anti-shoplifting cameras - advising customers that information is being collected and stored. And there is no giveback: Wal-Mart doesn't use loyalty cards and rarely offers promotions based on past purchases.
NOT everyone agrees. "People don't know that Wal-Mart is capturing information about who they are and what they bought, but they are also capable of capturing a huge amount of outside information about them that has nothing to do with their grocery purchases," said Katherine Albright, the founder and director of Caspian, a consumer advocacy group concerned with privacy issues. "They can find out your mortgage amounts, your court dates, your driving record, your creditworthiness."
One source of information can be a credit card or a debit card, Ms. Albright said. Wal-Mart shoppers increasingly use the cards to pay for purchases, particularly in the better-heeled neighborhoods where the company has been building stores recently.
Some companies specialize in what is known as data enhancement, in which a customer's name and address, or a telephone number, can open the door to additional information. "If Wal-Mart had a customer database and wanted to start e-mailing their customers, we could append their e-mail addresses," said Sarah Stansberry, director of marketing for AccuData America, a company based in Fort Myers, Fla., that specializes in such services but does not use credit card records. With e-mail addresses, AccuData can track names and home addresses, she added. Other information follows: "We can access what they paid for their house, and their mortgage," though not driving records. The company has not done any work for Wal-Mart, she said.
Ms. Dillman said that she did not think Wal-Mart had ever tried to squeeze data from credit cards to learn more about customers' buying habits. Indeed, she said, it wouldn't be necessary. "We can do that without the credit card information," she said. "We can look at what's happening in the market, and look at what's happening in other markets that are similar."
WAL-MART uses its mountain of data to push for greater efficiency at all levels of its operations, from the front of the store, where products are stocked based on expected demand, to the back, where details about a manufacturer's punctuality, for example, are recorded for future use. The purpose is to protect Wal-Mart from a retailer's twin nightmares: too much inventory, or not enough.
"They recognize that technology is a critical tool for them to have an efficient supply chain," said Kathryn Cullen, a principal at Kurt Salmon Associates, a consulting firm, who said that she has not advised Wal-Mart. "They track the purchases and very quickly route that back to their suppliers so they can be replenished. They are very strict with their suppliers, but they give them the data that they need."
Armed with sales results from past weeks and months, Wal-Mart meets with each of its suppliers to establish sales goals for the coming year. Suppliers are actively encouraged, so to speak, not to miss those goals. A manufacturer that fails to meet its sales target - or has data-documented problems with orders, delivery, restocking or returns - can expect even tougher negotiations in the future from Wal-Mart, which is renowned for its steeliness in such situations.
Still, achieving sleeker operations is not the whole story. In many ways, data are used to forecast and drive Wal-Mart's business. "We use it in real estate decisions, understanding what the draw is like and what the customers will be like," Ms. Dillman said, referring to the company's planning for new stores, including the number of shoppers it expects to attract to each.
When it comes to Sam's Club, Wal-Mart's membership warehouse chain, "we know who every customer is," she added. So Wal-Mart does a kind of outreach, contacting nearby convenience store owners, for example, to let them know that "the items they buy, they could save money on by buying at Sam's."
At Wal-Mart, problems are referred to as "exceptions," and technology is essential for what Ms. Dillman calls "exception management." Within the company's empire, "we keep watching everything that just happened," she said. "We are pretty near real time. We can tell people that they need to go do something, and we are within hours, depending on the event."
The "event" may be a truck's failure to drop off or pick up something, or the delivery of a load of shoes missing their mates. It could be the absence of an important product in a store's backroom, or in the distribution center that serves that store. Or it could be an act of nature like the hurricanes that descended, one after another, on Florida and other parts of the Southeast this year.
Eventually, some experts say, Wal-Mart will use its technology to institute what is called scan-based trading, in which manufacturers own each product until it is sold.
"Wal-Mart will never take those products onto its books," said Bruce Hudson, a retail analyst at the Meta Group, an information technology consulting firm in Stamford, Conn. "If you think of the impact of shedding $50 billion of inventory, that is huge."
The impact will probably be felt by suppliers, he added, but none are likely to complain.
"You can see the pattern of Wal-Mart's mandates, and as Wal-Mart grows in power, it is getting more dictatorial," he said. "The suppliers shake their heads and say, 'I don't want to go this way, but they are so big.' Wal-Mart lives in a world of supply and command, instead of a world of supply and demand."
Consumers willingly turn over plenty of information. For example, cashing a payroll check at Wal-Mart requires a two-step process, said an assistant manager in a Wal-Mart in Saddle Brook, N.J., who asked to be identified only by her first name, Mary. "First you enter your Social Security number into the system, twice," she said, pointing to the number pad hooked up to a register in the checkout lane. "The cashier can enter it, but some people don't like to share that information." Next a customer must enter his or her driver's license number, the assistant manager said. If payroll checks are cashed regularly at Wal-Mart, there is no need to keep punching in the Social Security number, only the driver's license number: "The system will recognize you the next time."
All of that information winds up at the company's office in Bentonville, the assistant manager added.
Ms. Dillman said it was "separated out, along with any personal identifiable information," and warehoused in a way that requires special permission to gain access. For check approval - when a customer writes a personal check to pay for something at a Wal-Mart, for example - "we don't keep it any longer than we need it for that transaction," she said. "All it's linked to is the checking account number, when we scan your check," she added. "We don't mine that data. We don't use it for anything other than the transaction."
Historically, Wal-Mart's focus has been on the products it sells, not to whom it sells them. One of the most difficult pieces of information to harvest is which customer bought what. Such information is expensive, too.
"When you are in the everyday-low-price market, you tend not to gather a lot of information about customers directly because you don't spend a lot of time with them gathering name, address, telephone numbers through a loyalty card," said Gene Alvarez, a vice president at the Meta Group. "That is the proper focus, because when you want to get customer-intimate, you have to offer a loyalty program, and there's the cost of that loyalty program."
Wal-Mart has discovered the potential of its own Web site in learning more about customers. Ms. Dillman said the site was beginning to allow users to buy a product online and have it delivered to a store near them, an option that Sears, Roebuck and other retailers have had for years. Naturally, some personal information would have to be submitted as part of the transaction. "You can do some association there, what products are of what interest," Mr. Alvarez said.
But Wal-Mart executives tend to care more about how products sell as part of a larger basket. "Me knowing what you specifically buy is not necessarily going to help me get the right merchandise into the store," Ms. Dillman said. "Knowing collectively what goes into one shopping cart together tells us a lot more."
Analyzing what ends up together in that cart drives Wal-Mart's pricing, other experts said. Shoppers might buy cold medicine along with chicken soup and orange juice during flu season, but not all of those products need to be priced at rock-bottom, said Ms. Overby, the Forrester analyst. "They might say, 'If we get really good at pricing the cold medicine and promoting it and letting people know that, hey, we have that product in stock and also at the best prices,' then they get people into the store," she said. "The other items in the basket might not be the lowest price in town, but the entire basket will be 10 to 20 percent less."
STILL, as Wal-Mart recently discovered, there can be such a thing as too much information. Six women brought a sex-discrimination lawsuit against the company in 2001 that was broadened this year to a class of about 1.6 million current and former female employees. Lawyers for the women have said that Wal-Mart has the ability to use its human-resources database to calculate back pay for the plaintiffs as well as to determine whether women were fairly promoted and paid. The judge hearing the case, which is pending in a federal court in San Francisco, has agreed.
The database is unusually detail-rich, said Joseph Sellers, a lawyer for the plaintiffs. "They've put into their work force database the information that bears on virtually every facet of compensation," he said. "They have performance reviews, along with seniority, the time spent with the company, which store they worked in. So you can compare people working in the same store, to measure whether men and women are paid differently."
If that comes to pass, it will be a rare moment indeed, with Wal-Mart's carefully assembled data being channeled for a purpose Wal-Mart did not desire.
Constance L. Hays
New York Times
INDEX OF RESEARCH THAT COUNTS