The RXA Case Study series seeks to illustrate the types of challenges our customers face, as well as the paths we take to meet their needs. Here’s an example of how we’re helping one company measure their impact.
Outsell is a Customer Data and Engagement platform for auto dealers and groups, based in Minneapolis. Their AI driven platform automates personalized customer communication to drive increased sales and service. Outsell sought to better understand their own value proposition and leveraged RXA’s analytical expertise. The end goal was to quantify the impact of Outsell’s communications on both sales and service. Using impenetrable statistics, Outsell could further stand out from the crowd and provide proof of efficacy to their current and future clients.
Directly measuring the impact of email communications on sales and service is typically straightforward. Sales and service rates would simply be compared between consumers that received communications (the treated group) versus those who did not (the control group). In a perfect world, this analysis would be over in 5 minutes. However, life is never that simple.
In Outsell’s case, there were many more members in the treated group than in the control group (an issue typically known as class imbalance). Furthermore, our exploratory data analysis identified characteristics in the treated group that made them more likely to appear in the treated group. In other words, the treated and control groups were not sufficiently randomized, and the treated group was fundamentally different from the control group (before communication was ever applied). These phenomena both introduced different types of bias, which confound and artificially inflate impact measurements.
We solved both issues with a single process: pair-wise propensity matching. We developed a model that predicted the likelihood of any sample being classified as treated rather than control. We then scored each sample and matched every control sample with a treated one having the most similar score. This process ensured that the data was completely balanced (1:1) and that group identity would not confound impact calculations. These computations were performed at the dealership level, providing Outsell’s clients with specific and accurate feedback on how communications were performing for them.
This project was completed on-time and on-budget, allowing Outsell to present the findings at the Automotive Analytics & Attribution Summit (AAAS) conference.
RXA is a leading data science consulting company. RXA provides data engineers, data scientists, data strategists, business analysts, and project managers to help organizations at any stage of their data maturity. RXA accelerates analytics road maps, helping customers accomplish in months what would normally take years by providing project-based consulting, long term staff augmentation and direct hire placement staffing services. RXA’s customers also benefit from a suite of software solutions that have been developed in-house, which can be deployed immediately to further accelerate timelines. RXA is proud to be an award-winning partner with leading technology providers including Domo, DataRobot, Alteryx, Tableau and AWS.