The RXA Case Study series seeks to illustrate the types of challenges our customers face, as well as the paths we take to meet their needs. Here’s an example of how we’re helping one company measure their impact. 

Overview 

Outsell is a Customer Data and Engagement platform for auto dealers and groups, based in Minneapolis. Their AI driven platform automates personalized customer communication to drive increased sales and service. Outsell sought to better understand their own value proposition and leveraged RXA’s analytical expertise. The end goal was to quantify the impact of Outsell’s communications on both sales and service. Using impenetrable statistics, Outsell could further stand out from the crowd and provide proof of efficacy to their current and future clients. 

Measuring Impact 

Directly measuring the impact of email communications on sales and service is typically straightforward. Sales and service rates would simply be compared between consumers that received communications (the treated group) versus those who did not (the control group). In a perfect world, this analysis would be over in 5 minutes. However, life is never that simple. 

Problems 

In Outsell’s case, there were many more members in the treated group than in the control group (an issue typically known as class imbalance). Furthermore, our exploratory data analysis identified characteristics in the treated group that made them more likely to appear in the treated group. In other words, the treated and control groups were not sufficiently randomized, and the treated group was fundamentally different from the control group (before communication was ever applied). These phenomena both introduced different types of bias, which confound and artificially inflate impact measurements. 

Solution 

We solved both issues with a single process: pair-wise propensity matching. We developed a model that predicted the likelihood of any sample being classified as treated rather than control. We then scored each sample and matched every control sample with a treated one having the most similar score. This process ensured that the data was completely balanced (1:1) and that group identity would not confound impact calculations. These computations were performed at the dealership level, providing Outsell’s clients with specific and accurate feedback on how communications were performing for them. 

Outcome 

This project was completed on-time and on-budget, allowing Outsell to present the findings at the Automotive Analytics & Attribution Summit (AAAS) conference.