Good Enough Accuracy

What makes In-Store Customer Tracking a WINNER?

If you are deploying or selling a People Tracking solutions and are STILL focused on the technology…

…You will miss the TRUE value of In-Store Optimization if your DATA audits do not have Good Enough Accuracy.

Here are topics you should care about TODAY:

• Why the Accuracy formula is a function of Detection, Recognition, and Tracking
• How to define the Good Enough Accuracy for YOUR customer tracking
• What are the scientific open standards of precision tracking
• How to run an Audit (A Template based on Levels of Difficulty)
• Why Queue Management is a special case for people tracking technologies
• Why knowing the differences between Facial Recognition and Demographics is a must
• A Word about the New Legal Trends on In-Store Customer Tracking & Privacy

Often a conversion with a senior retailer starts with “We have great Heat Maps”.

You should know…

…experts can easily transform the conversion from “A great feature” to “Now I understand why we don’t use the tracking data in store operations”.

The focus on the features & technicalities is an easy trap.

A simplified Accuracy Rate (i.e. 98% of counts) is misleading at best, and seriously distorts compensation plans and operational productivity at worse.

(Pro Tip) Simplicity does not mean a simplified Accuracy Rate. Bad traffic and sales conversion data can lead to staff revolt and eventually to bankruptcy. It happened.

---

Here’s the first thing you should know –

Regardless of the fast-moving trends in technologies, solutions, and you need a consistent framework to define and audit how you track staff and shoppers in physical retail.

“Good Enough Accuracy” is Data Validation as a function of Goals, Needs & Audits

Accuracy in tracking technologies has three core components:

#1 Object Detection

Object detection refers to the process of identifying real-world objects in a digital format, including images, videos, and devices.

In the context of tracking people, the solution defines:

• What object are we searching for
• Where is the area the object should be

For example,

People Counting refers to counting the number of people detected as they pass a line. The object is the head of the person. And the area of detection is passing the line.

Proximity Traffic and Number of Store Visitors are common examples of performance metrics that depend on Detection Accuracy.

Another example of In-Store Customer Tracking is detecting if the specific location is active or not, and the number of objects (people) in the Evaluation Zone.

In the tracking framework, the term Detection also refers to detecting a wireless signal such as WiFi or BLE Beacons.

(Pro Tip) Accuracy Rate reflects the percentage of detection of all objects in view, while the Detection Rate refers to the rate of tracked devices out of sum of people.

#2 Object Recognition

Object Recognition is the process of determining whether the object is a person, device, or something else. We can identify the object as a person or as a specific individual.

In Imaging, the recognition process is done:

• Compare to an Image: the captured image is compared to a set of images in a database, for example, in airport surveillance.
• Compare to an Attribute: the captured image is compared to a set of attributes, such as identification marks of gender in facial recognition.

Image Recognition, for example, plays an important role in online commerce and other industries such as driverless cars. Facebook, for example, has an advanced Machine Learning AI that can capture emotional attributes such as introvert or extrovert. Every time you “tag” yourself, and your family and friends, you contribute to Facebook Imaging.

In device-based personal tracking the objective is to identify the device’s identity with the Media Access Control (MAC) Address.

(Pro Tip) In 2019, there is a strong trend to use Vision AI technology for Product Recognition due to the relative ease of comparing static images and well defined business cases in Out-of-Stock and Stocking Compliance.

#3 Object Tracking

In essence, tracking means the “prediction of objects in motion”.

In sensor-based technologies, we track objects across multiple locations within the Field of View, and across multiple sensors. For example, we can track shoppers (objects identified as people and excluding staff) in various locations of the store.

In wireless technologies, we track the signals from the personal device such a smartphone. The underlying assumption is that the person carries a single device.

Therefore, tracking has two core steps:

• Predictive: the technology predicts where the object is moving next
• Precision: the technology assess how precise was that prediction

(PRO Tip) The people tracking solutions are defined by HOW they capture location (position) and time (how long) of the object (person). Details matter.

Next we will dive into the 3 steps framework of Good Enough Accuracy

---

#1 Define “Good Enough Accuracy” for Your In-Store Customer Tracking Technology

Accuracy requirements vary across technologies, features, and performance metrics.

For example, the performance of Proximity Traffic is different than Queue Management. And InStore Customer Engagement is different if the retailer sells groceries, apparel or electronics. Also, the accuracy of the data depends if the store is small in size or a big box layout.

Here are the core questions in defining the business objective for “Good Enough Accuracy”:

• Hardware or Software: Solution providers can also be manufacturer of the hardware, but most are value added resellers that combine technologies, analytics, and services into a full solution.
• Sensors or Device Tracking: In sensors, we test for accuracy of all objects within the field of view. In device-based tracking, the test is contained only to active devices.
• Counts or Attributes: Most common are the accuracy tests for whole objects such as a person or product, but keeping data as attributes is a trend. For example, buying groups.

Once you identified the components of the solution and decided what to test for accuracy, the next step is to define the test objectives in terms of location and time.

For example,

In this Sample Heat Map, we test for:

• People Counts: How many people are counted in each  virtual zone
• Location Accuracy: How precise are the pixels to the actual location of the person
• Stay Time Accuracy: How accurate are the Stay Time per person, per location

(Pro Tip) The process of first defining the business objectives is the #1 mistake that many techies make. Goals matter. Details matter.

#2 Follow the Science of Precision Tracking

Good Enough Accuracy is a framework based on scientific open standards. While this method is based on imaging technology, it can be adapted to 2D face recognition, 3D people in motion, and even device-based personal trackers.

Precision and accuracy are often used interchangeably, but in science they are different. Accuracy defines how value is close to a known value, while Precision measures how the distance between values. In-Store Customer Tracking requires both Accuracy and Precision.

Open standards for “Good Enough Accuracy”:

• % Recall: The ratio of missed detection (under-counts)
• % Precision: The rate of relevant results (over-counts)
• Data Consistency: The trend in detection errors.
• Detection Threshold: The trade-off for recall/precision in counting objects.
• #ID Switching: In object tracking (over a series of frames), the number of mismatched IDs
• % Precision Rate: How well the tracking technology captured the locations of multiple objects.
• % Tracking Accuracy: Total errors in tracking. It includes misses, false positives, mismatched IDs, failure to recover tracks, etc.

(Pro Tip) Since Detection Accuracy is the threshold between % Recall and % Precision, the Accuracy Rate requires a judgment call

#3 Define the Audit (Technical) Template

Not all audits are the same.

On the contrary, the Audit is the differentiator between solution providers.

Best Practices in Audits are:

1. Ground Truth: define a set of data that is checked manually. This will serve as the foundations for testing the ability of the software.
2. Difficulty Levels: define the difficulty levels according to YOUR business objectives (see below)
3. Repeated Feedback: Audits should be a regular and on-going process. Audits help optimize the performance of the tracking solution and serve as a warning indicator for errors.

Levels of Difficulty in Audits are:

• Evaluation Zone: In sensor solutions, accuracy can be tested in the full frame or a limited zone. In device-based scenarios, we determine the size of the Evaluation Zone.
• Audit Granularity: The length of the benchmark Ground Truth. In detection, we aim for shorter time segments (such as 15 minutes for door counts or 5 minutes for queues). In tracking objects, we want longer time segments. The recommendation is for 1 hour.
• Traffic Volume: A valid test requires sufficient traffic volumes. A low-traffic audit (less than 100 people) should be repeated in different time periods.
• Feeds Frequency: In video solutions, the Frame Rate (i.e. 25 frames per second) has a significant impact on the precision of tracking. In wireless solutions, we face the same challenges with Signal Frequency

(Pro Tip) Audits should be an integral part of how you monitor the people tracking solution. Audit templates create trust between retailer and provider

---

The Special Case of Queue Management

Queues are a special case in Good Enough Accuracy.

In detection, we count the number of objects. In tracking, we follow the object in motion.

In queues, people stand and move while waiting for service. This Stop & Move behavior presents challenges in the definition of accuracy.

For example,

In counting-based models such as “One in Front”, data validation include:

• Number of People Waiting: to measure the impact of crowds on customer behaviors
• Waiting Time in Seconds: to capture how fast the line is moving
• The Direction of Motion: to identify customer abandon behaviors
• Service Zone Behaviors: to boost forecasting with Active/Not Active indicators
• Multiple Zones Behaviors: to evaluate shopping groups behaviors

(Pro Tip) Queues serve as First Touch or Last Touch with Customers; therefore, managing queues is a priority in airports, retail banking, and in-store optimization.

The Special Case of Facial Recognition

Facial Recognition is no longer a question of technology.

With deep learning vision technology, biometrics software works as well as the human brain.

 Technically, in audits here are questions to ask:

1. Parameters: what are the features of the objects, for example, facial demographics such as gender or facial emotions such as sad or happy.
2. Prediction: what is the process of prediction, specifically does the prediction process work against an image or training process (vision)

(PRO Tip) The nuances of privacy and consent are muddled by HOW the data is stored and processed. Pay attention to facial attributes.

A Word about New Legal Trends on In-Store Consumer Privacy

In-Store Customer Tracking is the next step in the combination of Location Intelligence, indoor positioning systems, and deep learning vision technologies.

There are significant differences between continents, countries, and communities in attitudes to privacy and regulation. The European Union created GDPR. In China, the government actively supports the development of facial recognition technology.

In the United States, the retailers and brands are conflicted between the obvious business benefits from Facial Recognition and concerns about a backlash due to inaccuracy and privacy. The legal trends are evident in the work of the Future of Privacy Forum.  

Technically, you should clarify these 3 topics:

1. Are your People Tracking solution Interactive or Anonymous?
2. If customer Interactive, where are the opt-in & opt-out options?
3. If anonymous tracking, where is the data stored and processed?

(Pro Tip) You should do well just by clarifying your process…

Data Attribution Challenge of Anonymous & Interactive In-Store Customer Tracking

#1 source of confusion of In-Store Customer Tracking is the differences between Anonymous & Interactive (require opt-in) solutions.

That decision has many implications on marketing, operations, and information technology processes and procedures.

Most of the People Tracking Technologies can be adapted to either form. The key to such decision comes from the business objectives.

Good Enough Accuracy starts with the business definition of the solution, proceeds with open sources definitions of accuracy and precision, and ends with transparent audits.

Regardless of your sector, technology, and company, you should follow a Data Validation process like Good Enough Accuracy.