ANLY FPX 5510 Assessment 2 Evaluation of Advanced Analytics Project

Student Name

Capella University

ANLY-FPX 5510 Advanced Business Analytics

Prof. Name

Date

Business Problem

The executive team has made several business decisions based on the analytics group’s projects, yielding mixed results. While some decisions have been successful, others have failed, resulting in significant costs for the organization. Due to these inconsistent outcomes, the executives have placed the analytics group on a 90-day probationary period. At the end of this period, they will decide whether to retain or disband the group. The head of the marketing department has been assigned to monitor the group’s performance during this probationary period. All work will be reviewed by the head of marketing before submission to the executives.

To address the business problem, A/B testing was selected to compare the revenue per visitor generated by the same banner ad media run on two different websites: Yahoo and CNN. This test aimed to identify which site brings in more revenue from the company’s banner advertisement. The primary goal is to reduce the number of websites with which the company must manage advertising relationships without involving a third-party service provider. Two hypotheses were tested: (1) the null hypothesis posited that the average purchase price between the two companies is the same, and (2) the alternative hypothesis suggested that the purchase amounts differ. The results concluded a significant difference in mean purchase amounts between customers who clicked on the banner ads on Yahoo and CNN.

Testing Method

The alignment of the testing method with the business problem involves gathering data, evaluating it, and developing different hypotheses. A/B and hypothesis testing were used to examine the revenue generated from two websites in the earlier example, and the same method will be employed during the analytics group’s 90-day probationary period. The purpose is to determine the possible outcomes of the probation period and assess the analyst department’s decisions and their impact on revenue.

During the probationary period, an A/B test will be conducted, and all group activities will be closely monitored and reviewed before submission to the executives. The null hypothesis (Ho) posits that the analytics team will produce satisfactory results during the probation, and the reviewed work will be approved by the marketing manager. Conversely, the alternative hypothesis (Ha) states that the team’s performance will be unsatisfactory, leading to its disbandment.

Alignment of Method and Business Problem

A/B testing is a marketing experiment where the audience is divided to compare multiple campaign variations and determine which one performs better. This process involves showing version A of a marketing piece to one half of the audience and version B to the other. To run an A/B test, two versions of a single content piece are created, with changes to one variable. Both versions are shown to similar audience sizes, and performance is analyzed over a specific period (Kolowich, 2019).

Before conducting an A/B test, the first step is to research the current performance of the variables being tested. In the business problem, research would focus on the quality, accuracy, and reliability of the analytics group’s results, as well as the work previously submitted to the executives (Wingify, 2019). Next, a hypothesis is formulated, followed by testing to determine which hypothesis is more likely. Key parameters such as confidence levels, impact on broader goals, and ease of setup are also evaluated.

Conclusion

The analytics department will undergo a 90-day probation period to assess the quality, accuracy, and reliability of the work produced. Two hypotheses—either retaining or disbanding the department—will be evaluated based on the results gathered during this period. It is crucial that the head of the marketing department carefully review the work produced to ensure that it meets the company’s standards before being submitted to the executives.

References

Joseph, C. (n.d.). Problems With Performance Evaluations. Small Business – Chron.com. Retrieved from http://smallbusiness.chron.com/problems-performance-evaluations-1256.html

Kolowich, L. (2019). How to Do A/B Testing: A Checklist You’ll Want to Bookmark. Retrieved from https://blog.hubspot.com/marketing/how-to-do-a-b-testing

Saleh, K. (2019). AB Testing: 14 Sampling Issues That Can Ruin Your Test. Retrieved from https://www.invespcro.com/blog/ab-testing-14-sampling-issues-that-can-ruin-your-test

ANLY FPX 5510 Assessment 2 Evaluation of Advanced Analytics Project

Wingify (2019). What Is A/B Testing? Retrieved from https://vwo.com/ab-testing/