First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing BADM7020 Spring 2020

Business Understanding ___. A 95% lower confidence limit for the ad effect is ____
(Table 6).
(Use this word document as Template)
A/B testing Explained. What is it used for? How does it work?
Average revenue based on purchases (7): The average
revenue for ad A based on purchases is ______ and a 95%
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX confidence interval is (____-___). The average revenue for ad
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX B based on purchases is _____ and a 95% confidence interval
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX is (____-____. (Table 7)
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Average Difference in revenue based on purchases (8):
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX The average revenue based on purchases are (not) statistically
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX significant different at p=____. The average difference in the
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX average revenue is ___. A 95% lower confidence limit for the
ad effect is ____ (See Table 8).
Data Understanding:
Average revenue based on views (9): The average revenue
Describe the data in your own words. Pay attention to what is for ad A based on views is ______ and a 95% confidence
random (uncertain) and what is fixed. interval is (____-___). The average revenue for ad B based on
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX views is _____ and a 95% confidence interval is (____-____.
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX (Table 9)
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Average Difference in revenue based on views (10): The
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX average revenue based on views are (not) statistically
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX significant different at p=____. The average difference in the
average revenue is ___. A 95% lower confidence limit for the
Analysis
ad effect is ____ (See Table 10).
Provide an analysis of the problem at hand including
Cost per click per view (11): The average costs of clicks per
assessments of click rates, purchase rates and revenue.
view for ad A is ______ with a 95% confidence interval of
Click Rate (1): The click rate for ad A is 19.28% with a 95% (____,____). The average costs of clicks per view for ad B is
confidence interval for the true mean as (17.69%-20.96%) and ______ with a 95% confidence interval of (____,____).
the click rate for Ad B is 26.30% with a 95% confidence interval
Revenue per purchase with SH&H (12): The average revenue
of (24.51%-28.16%) (See Table 1).
per purchase with SH&H for Ad A is ___ and a confidence
Click Rate Difference (2) The click rates are statistically Interval is (____,___). The average revenue per purchase with
significant different at the p<0.0001. The average difference in SH&H for Ad B is ___ and a confidence Interval is (____,___).
the click rate is 7%. A 95% lower confidence limit for the ad
Revenue per view with discount (13): The average revenue
effect is 4.5% (See Table 2).
per view with discount for ad B is $______ with 95%
Purchase Rate Based on Views (3): The purchase rate for confidence interval ($___, $____).
ad A based on views is ______ and a 95% confidence interval Conclusion
is (____-___). The purchase rate for ad B based on views is The conclusion should recap what you found in your analysis.
_____ and a 95% confidence interval is (____-____. (see Table What are the pros and cons of the different metrics such as per
3) view and per click, or per purchase? Provide the results of the
Purchase Rate Difference Based on Views (4): The difference between ad A and B with LCL and explain the
purchase rates based on views are (not) statistically significant different conclusion drawn from:
different at p=. The average difference in the click rate is ___.
A 95% lower confidence limit for the ad effect is ____ (Table • Difference of click rate of the two ads: What does it
4). tell you about the ads?
• Difference of purchase per click: What does it tell you
Purchase Rate Based in Clicks (5): The purchase rate for ad about the ads?
A based on clicks is ______ and a 95% confidence interval is • Difference of revenue per purchase: What does it tell
(____-___). The purchase rate for ad B based on clicks is you about the ads?
_____ and a 95% confidence interval is (____-____). (Table 5) • Difference of revenue per view: Why is the
Purchase Rate Difference Based in Clicks (6): The confidence interval more reliable than that for
purchase rates based on clicks are (not) statistically significant revenue per purchase?
different at p=____. The average difference in the click rate is
References:
First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing BADM7020 Spring 2020

Table 1a: Click Percentages with Confidence Intervals Ad A Figure 6: T-Test for Diff. of Purchase % based on Clicks

Table 1a: Click Percentages with Confidence Intervals Ad B

Table 2: T-Test for Difference of Click Percentages Table 7: Average Revenues with 95% CI based on Purchases

Table 3: Purchase Percentages with 95% CI based on Views Figure 8: T-Test for Diff. of Average Revenues based on
Purchases.

Figure 4: T-Test for Diff. of Purchase % based on Views Table 9: Average Revenues with 95% CI based on views

Table 5: Purchase Percentage & 95% CI based on Clicks Figure 11: T-Test for Difference of average Revenues based on
Views
First & Last Name Assignment 3 – AB Testing applying Hypothesis Testing BADM7020 Spring 2020

Table 12: Summary Table

Ad A Ad B Difference p for Diff. >0

Diff>0 P<0.05 or
LCL>0
(Y/N)?
Line Type Mean LCL UCL Mean LCL UCL Diff LCL
1 Click/View 19.28% 17.69% 20.96% 26.30% 24.51% 28.16% 7% 4.5% 0.0001 Y
2 Purchase/view
3 Purchase/Click
4 Revenue/Purchase
5 Revenue/View
6 Revenue/view with NA
discount
7 Revenue/per view NA
with Sh&H
8 Cost of clicks/view NA

• You don’t have to compute the p for line 6-8, but you have to fill out the last column based on the LCL.
• Line 6 is compute from line 5.
• Line 7 is computed from line 6
• Line 8 is computed from line 1.