I've gotten business from WF and I've learned a lot about SEO, and I don't have a lot to offer back since I'm new to Internet marketing. However, after doing some searching, I couldn't find much on here in terms of applying statistics to click-thru rates. Since I'm a math guy, I decided to put a little work into it so I could see what was going on for myself, and I'm going to share stuff here. I'm going to use an example of an AdSense site since that's what I'm working on myself, but the same principles apply to any binary event. By binary event, I mean either the user does what you want them to (click, buy, whatever).
The whole point of this is how to use a little math to decide if what you're seeing in your click-thru rates is just normal fluctuation or is a real cause for a concern.
There's one main formula to worry about here for what I'm going to be looking at. If you're looking at a sample of n impressions with an average CTR of c, then the chance of getting k clicks over the sample is:
(n choose k) * c^k * (1-c)^(n-k)
For people who don't know what combinations are go read here: Combination - Wikipedia, the free encyclopedia, and for people who don't care what combinations are, just know that it's an operation on numbers and you can do it in a spreadsheet or using Google if you're so inclined.
Let's start with a simple example. Suppose we have some page where we expect a CTR of 6%. If we get 50 impressions, what is the chance that we get no clicks?
In this example, n = 50, c = 0.06, and k = 0. So we have:
(n choose k) * c^k * (1-c)^(n-k)
(50 choose 0) * 0.06^0 * (1-0.06)^(50-0)
You can just copy and paste that into Google if you want and it will give you this:
(50 choose 0) * (0.06^0) * ((1 - 0.06)^(50 - 0)) = 0.0453307266
So there's about a 4.5% chance that you get no clicks at all. What that tells us is if you got 50 impressions each day, you would average having about 1.4 days each month where you had no clicks at all.
We can look at this scenario in more detail if we use a spreadsheet. In OpenOffice's spreadsheet, the function for doing combinations is =COMBIN(n;k). Note that it uses a semi-colon instead of a comma. Here's what a sample implementation could look like.
The actual values we would get from the formulas are as follows:
You should notice that the chance of getting 8 or more clicks is so small it comes out to less than 1% total, so it's not really worth adding to the chart here for our purposes (note that this implies that if you got a lot of clicks in one day, say 20, that you should be alarmed). With this chart that you've made inside of like 90 seconds using a spreadsheet, you can see that you're going to get one click or less about 19% of the time, which is much more often than I figure people will tend to expect. So with the baby example out of the way, let's look at one on a larger scale.
Suppose we have a page that typically has about a 3.4% CTR. We get 2500 impressions to this page, which means we should get 85 clicks on average. What is the chance we get 20 clicks or less?
Let's list our variables clearly first: n = 2500, c = 0.034, we're going to add up all of the probabilities for k = 0 up to k = 20. In our spreadsheet, it will look something like this and we'll just do a SUM(D2 : D22) to find the total probability of k = 0 to k = 20.
So our chance is 1.43260331764679E-018, which is not very fucking likely. Something is almost certainly up with our page, and we should check it out. Notice that the much larger sample size allows us to rely on our data with much more confidence. It will also make your earnings more steady. For example, over half of the time you will get between 79 and 91 clicks in this scenario, and over 80% of the time you will get between 73 and 97 clicks.
Hopefully this will help somebody who was searching for the same thing I was when I didn't find anything, and thanks WF.
The whole point of this is how to use a little math to decide if what you're seeing in your click-thru rates is just normal fluctuation or is a real cause for a concern.
There's one main formula to worry about here for what I'm going to be looking at. If you're looking at a sample of n impressions with an average CTR of c, then the chance of getting k clicks over the sample is:
(n choose k) * c^k * (1-c)^(n-k)
For people who don't know what combinations are go read here: Combination - Wikipedia, the free encyclopedia, and for people who don't care what combinations are, just know that it's an operation on numbers and you can do it in a spreadsheet or using Google if you're so inclined.
Let's start with a simple example. Suppose we have some page where we expect a CTR of 6%. If we get 50 impressions, what is the chance that we get no clicks?
In this example, n = 50, c = 0.06, and k = 0. So we have:
(n choose k) * c^k * (1-c)^(n-k)
(50 choose 0) * 0.06^0 * (1-0.06)^(50-0)
You can just copy and paste that into Google if you want and it will give you this:
(50 choose 0) * (0.06^0) * ((1 - 0.06)^(50 - 0)) = 0.0453307266
So there's about a 4.5% chance that you get no clicks at all. What that tells us is if you got 50 impressions each day, you would average having about 1.4 days each month where you had no clicks at all.
We can look at this scenario in more detail if we use a spreadsheet. In OpenOffice's spreadsheet, the function for doing combinations is =COMBIN(n;k). Note that it uses a semi-colon instead of a comma. Here's what a sample implementation could look like.
Code:
n c k probability
50 0.06 0 =COMBIN(A2;C2)*(B2^C2)*((1-B2)^(A2-C2))
50 0.06 1 =COMBIN(A3;C3)*(B3^C3)*((1-B3)^(A3-C3))
50 0.06 2 =COMBIN(A4;C4)*(B4^C4)*((1-B4)^(A4-C4))
50 0.06 3 =COMBIN(A5;C5)*(B5^C5)*((1-B5)^(A5-C5))The actual values we would get from the formulas are as follows:
Code:
n c k probability
50 0.06 0 0.04533
50 0.06 1 0.14467
50 0.06 2 0.22624
50 0.06 3 0.23105 <--- average clicks
50 0.06 4 0.17329
50 0.06 5 0.10176
50 0.06 6 0.04871
50 0.06 7 0.01954You should notice that the chance of getting 8 or more clicks is so small it comes out to less than 1% total, so it's not really worth adding to the chart here for our purposes (note that this implies that if you got a lot of clicks in one day, say 20, that you should be alarmed). With this chart that you've made inside of like 90 seconds using a spreadsheet, you can see that you're going to get one click or less about 19% of the time, which is much more often than I figure people will tend to expect. So with the baby example out of the way, let's look at one on a larger scale.
Suppose we have a page that typically has about a 3.4% CTR. We get 2500 impressions to this page, which means we should get 85 clicks on average. What is the chance we get 20 clicks or less?
Let's list our variables clearly first: n = 2500, c = 0.034, we're going to add up all of the probabilities for k = 0 up to k = 20. In our spreadsheet, it will look something like this and we'll just do a SUM(D2 : D22) to find the total probability of k = 0 to k = 20.
Code:
n c k probability
2500 0.035 0 2.08105404734837E-039
2500 0.035 1 1.88696610510863E-037
2500 0.035 2 8.55147618566456E-036
2500 0.035 3 2.58257534684854E-034
2500 0.035 4 5.84725835331161E-033
2500 0.035 5 1.05868702537887E-031
2500 0.035 6 1.59671579432141E-030
2500 0.035 7 2.06332082437181E-029
2500 0.035 8 2.33206034366013E-028
2500 0.035 9 2.34199543090428E-027
2500 0.035 10 2.11592613101958E-026
2500 0.035 11 1.73719229692281E-025
2500 0.035 12 1.30686966274983E-024
2500 0.035 13 9.07151934892432E-024
2500 0.035 14 5.84478461678103E-023
2500 0.035 15 3.51333132646713E-022
2500 0.035 16 1.97909321320906E-021
2500 0.035 17 1.04884098723801E-020
2500 0.035 18 5.24752596407136E-020
2500 0.035 19 2.48624259884853E-019
2500 0.035 20 1.11861593819177E-018So our chance is 1.43260331764679E-018, which is not very fucking likely. Something is almost certainly up with our page, and we should check it out. Notice that the much larger sample size allows us to rely on our data with much more confidence. It will also make your earnings more steady. For example, over half of the time you will get between 79 and 91 clicks in this scenario, and over 80% of the time you will get between 73 and 97 clicks.
Hopefully this will help somebody who was searching for the same thing I was when I didn't find anything, and thanks WF.
