To assess the impact of the product advertisement on the conversation rate, we must establish clear definitions for the A and B groups:
The A group, also known as the test group, comprises users who viewed the advertisement.
The B group, known as the control group in our context, comprises users who did not encounter the advertisement but continued to view the public service announcement (PSA).
This distinction between the two groups is essential for our evaluation.
Total number of users involved in the experiment.
Total number of users in the Test group.
Total number of users in the Control group.
Total Users
For the experiment conducted on the company's community platform, a substantial user base was engaged, totaling 588,101 participants. Out of this sizable user pool, a relatively modest figure of 14,843 users achieved successful conversions, resulting in a conversion rate of 2.52%. This percentage highlights the portion of users who actively completed purchases during the experiment period, providing valuable insights into the effectiveness of the company's strategies in driving both conversions and revenue.
Test Group
Out of the total user base of 588,101 participants in the experiment, a substantial majority, totaling 564,577 users, were randomly selected and given the opportunity to view the advertisement. This group represents approximately 96% of the total participants and is considered the test group, forming the primary basis for comparison against the control group.
Within this test group, 14,423 users successfully completed conversions, resulting in an approximate conversion rate of 2.55%. This percentage corresponds to roughly 2.45% of the total users who viewed the advertisement.
Control Group
Among the participants, a smaller subset, consisting of 23,524 users were exposed to a Public Service Announcement (PSA). This group accounts for approximately 4% of the total user base and is designated as the Control group.
Within the Control group, 420 users successfully completed conversions, resulting in a conversion rate of approximately 1.79%.
The Control group plays a pivotal role in our experiment, serving as a benchmark against which we can assess changes and improvements. This role enhances our comprehension of the experiment's overall impact on user behavior and conversions.
| Most_Views_Day | Number_Users | Percentage | Total_Converted | Conversion_Rate | |
|---|---|---|---|---|---|
| 0 | Sunday | 82332 | 14.58 | 2027 | 2.462 |
| 1 | Monday | 83571 | 14.80 | 2778 | 3.324 |
| 2 | Tuesday | 74572 | 13.21 | 2270 | 3.044 |
| 3 | Wednesday | 77418 | 13.71 | 1963 | 2.536 |
| 4 | Thursday | 79077 | 14.01 | 1711 | 2.164 |
| 5 | Friday | 88805 | 15.73 | 1995 | 2.246 |
| 6 | Saturday | 78802 | 13.96 | 1679 | 2.131 |
| Most_Views_Day | Number_Users | Percentage | Total_Converted | Conversion_Rate | |
|---|---|---|---|---|---|
| 0 | Sunday | 3059 | 13.00 | 63 | 2.059 |
| 1 | Monday | 3502 | 14.89 | 79 | 2.256 |
| 2 | Tuesday | 2907 | 12.36 | 42 | 1.445 |
| 3 | Wednesday | 3490 | 14.84 | 55 | 1.576 |
| 4 | Thursday | 3905 | 16.60 | 79 | 2.023 |
| 5 | Friday | 3803 | 16.17 | 62 | 1.630 |
| 6 | Saturday | 2858 | 12.15 | 40 | 1.400 |
In the analysis of the highest number of ads and PSA views by day, several noteworthy patterns emerge. Friday emerges as the day with the highest number of users viewing the most ads in a single day, accounting for approximately 15.72% of users, closely followed by Monday and Sunday. However, despite its high ad views, Friday records one of the lowest conversion rates at 2.25%. On the contrary, Monday, with a substantial number of users who converted, achieves the highest conversion rate at 3.32%. Tuesday, despite low ad views, surprisingly secures the second-highest conversion rate at 3.04%.
In terms of PSA views, Thursday takes the lead, closely followed by Friday, each capturing around 16% of users. Monday and Wednesday also exhibit significant engagement. Notably, Saturday registers the lowest user activity for PSA views, with Tuesday trailing closely behind. The conversion rates for PSA views follow a slightly different trend, with Monday leading in conversions, followed by Thursday, and Saturday recording the lowest number of users who converted despite its view count.
These insights provide valuable information about user engagement patterns for each day of the week and how they relate to conversion rates.
| Most_Views_Hour | Number_Users | Percentage | Total_Converted | Conversion_Rate | |
|---|---|---|---|---|---|
| 0 | 0 | 5309 | 0.94 | 102 | 1.921 |
| 1 | 1 | 4615 | 0.82 | 62 | 1.343 |
| 2 | 2 | 5152 | 0.91 | 39 | 0.757 |
| 3 | 3 | 2590 | 0.46 | 27 | 1.042 |
| 4 | 4 | 694 | 0.12 | 11 | 1.585 |
| 5 | 5 | 742 | 0.13 | 16 | 2.156 |
| 6 | 6 | 1985 | 0.35 | 46 | 2.317 |
| 7 | 7 | 6168 | 1.09 | 114 | 1.848 |
| 8 | 8 | 16968 | 3.01 | 337 | 1.986 |
| 9 | 9 | 29802 | 5.28 | 582 | 1.953 |
| 10 | 10 | 37454 | 6.63 | 818 | 2.184 |
| 11 | 11 | 44149 | 7.82 | 992 | 2.247 |
| 12 | 12 | 45238 | 8.01 | 1092 | 2.414 |
| 13 | 13 | 45485 | 8.06 | 1140 | 2.506 |
| 14 | 14 | 43779 | 7.75 | 1251 | 2.858 |
| 15 | 15 | 42855 | 7.59 | 1279 | 2.984 |
| 16 | 16 | 35963 | 6.37 | 1111 | 3.089 |
| 17 | 17 | 33605 | 5.95 | 959 | 2.854 |
| 18 | 18 | 31052 | 5.50 | 853 | 2.747 |
| 19 | 19 | 29169 | 5.17 | 782 | 2.681 |
| 20 | 20 | 27846 | 4.93 | 843 | 3.027 |
| 21 | 21 | 28895 | 5.12 | 843 | 2.917 |
| 22 | 22 | 25515 | 4.52 | 675 | 2.646 |
| 23 | 23 | 19547 | 3.46 | 449 | 2.297 |
| Most_Views_Hour | Number_Users | Percentage | Total_Converted | Conversion_Rate | |
|---|---|---|---|---|---|
| 0 | 0 | 227 | 0.96 | 0 | 0.000 |
| 1 | 1 | 187 | 0.79 | 0 | 0.000 |
| 2 | 2 | 181 | 0.77 | 0 | 0.000 |
| 3 | 3 | 89 | 0.38 | 1 | 1.124 |
| 4 | 4 | 28 | 0.12 | 0 | 0.000 |
| 5 | 5 | 23 | 0.10 | 0 | 0.000 |
| 6 | 6 | 83 | 0.35 | 0 | 0.000 |
| 7 | 7 | 237 | 1.01 | 2 | 0.844 |
| 8 | 8 | 659 | 2.80 | 7 | 1.062 |
| 9 | 9 | 1202 | 5.11 | 13 | 1.082 |
| 10 | 10 | 1485 | 6.31 | 20 | 1.347 |
| 11 | 11 | 2061 | 8.76 | 30 | 1.456 |
| 12 | 12 | 2060 | 8.76 | 35 | 1.699 |
| 13 | 13 | 2170 | 9.22 | 36 | 1.659 |
| 14 | 14 | 1869 | 7.95 | 30 | 1.605 |
| 15 | 15 | 1828 | 7.77 | 46 | 2.516 |
| 16 | 16 | 1604 | 6.82 | 45 | 2.805 |
| 17 | 17 | 1383 | 5.88 | 28 | 2.025 |
| 18 | 18 | 1271 | 5.40 | 32 | 2.518 |
| 19 | 19 | 1183 | 5.03 | 29 | 2.451 |
| 20 | 20 | 1077 | 4.58 | 19 | 1.764 |
| 21 | 21 | 1081 | 4.60 | 24 | 2.220 |
| 22 | 22 | 917 | 3.90 | 15 | 1.636 |
| 23 | 23 | 619 | 2.63 | 8 | 1.292 |
In the examination of the highest number of ads views per hour, distinct patterns emerge throughout the day. User engagement spikes considerably starting from 8 am and peaks between 11 am and 3 pm, with an average of 44,301 users during these hours. Conversely, the period from midnight to 7 am sees the lowest number of users, averaging approximately 3,407 users, as expected. As for conversion rates during these hours, a compelling narrative unfolds. From 9 am onward, there's a consistent upward trend in conversion rates, reaching its zenith between 2 pm and 4 pm. Subsequently, conversion rates start to decline from 5 pm to 7 pm and again from 8 pm to 11 pm. These fluctuations in conversion rates align with the varying levels of user activity during these hours, revealing when users are most and least likely to convert.