Tag: A/B testing

Using stratified sampling technique

Complete code available here: https://github.com/bassalat/lookalikes

Lookalike modelling is searching for a set of observations (i.e. customers in the marketing domain) that have a similar profile to a target group. In essence, we are trying the mimic the features of the target control to form a control group. This is useful for a variety of applications that include A/B tests, post-campaign analyses and finding a specific set of customers.

To illustrate this method, we’ll use a Credit card customer profile data-set available on Kaggle and use R to explore it:

https://www.kaggle.com/arjunbhasin2013/ccdata

Let's say a marketing campaign was run on the first 500 customers (target group) in the data-set. Now we need to find look-alike customers from the remaining part of the data-set. Choosing which variables will form the basis for lookalikes is a business domain question. Let's assume for this exercise, that we want to match the distribution of BALANCE, PURCHASES AND CASH_ADVANCE.

As the first step, we are going to get the joint distribution of these variables for the target group. Continuous variables will be binned to convert them to categorical variables so that all variables are of categorical type. Thereafter, a joint categorical variable will be formed and frequency distribution against it will be used.

The following steps will be used to make joint distribution bins:

# Make bins for chosen variables
df <- df %>% mutate(
balance_bin = cut(BALANCE,c(-1,0,100,500,1000,2000,3000,            5000,7000,10000,20000)),
purchases_bin = cut(PURCHASES,c(-1,0,100,500,1000,2000,5000,10000,50000)),
cashadvance_bin = cut(CASH_ADVANCE,c(-1,0,500,1000,2000,5000,50000)))

# Make a combined bin
df <- df %>% mutate(combined_bin = paste(balance_bin,
                                         purchases_bin,
                                         cashadvance_bin))

# Combined distribution of target group:
target_profile <- df[1:500,] %>% group_by(combined_bin) %>%
  summarise(customers=n()) %>%
  mutate(bin_wt = customers/nrow(df[1:500,]))

The table above shows the final distribution across the three variables that we will replicate for the control group. The following piece of code samples out customers from the control group in the same frequency distribution as above:

#### Generate lookalikes

control_n = 500 # desired number of customers in the control group
j = 0 # counter
df_control = data.frame() # dataframe where control group will be formed
for (i in target_profile$combined_bin){
  j=j+1 
  print(j)
  df_control_bin <- df[(control_n+1):nrow(df),] %>% filter(combined_bin==i)
  df_control_bin_sample <- sample(1:nrow(df_control_bin),
                            size= as.integer(target_profile %>%
                                               filter(combined_bin==i) %>%
                                               select(bin_wt)*control_n), replace = TRUE)
  df_control_bin_sample <- df_control_bin[df_control_bin_sample,] # pick actual rows from sample index values
  df_control <- rbind(df_control,
                      df_control_bin_sample) # keep adding the main control group
                                              #data frame with each iteration
}

A point to note here is that we are sampling with replacement as there are some bins where the number of customers required for the control group are not sufficient to match the bin weight of the target group. Therefore, for some bins, customers are counted more than once (there are 478 distinct customers forming the 500 rows control group). For larger datasets, this problem would not arise and sampling without replacement should be used.

Finally, we can match the distribution across the three variables for both groups and verify that they match: