I've been seeing these Clash of Clans ads on TV lately and was wondering how they would evaluate their effectiveness in relation to their usual PPC ads. Would you just look at the dates that the ads aired, and then see if you had any growth spikes in downloads on those dates?
But then if you do that, it doesn't account for other unobservable factors, for example maybe you aired an ad on a day that also happened to be a snow day, which caused kids to stay home from school spending all day downloading games. In which case your downloads would have increased anyway and you have no idea how much is related to the ad and how much is related to the snow day (assuming you even know about the snow day - if you don't know about it, you'll just think your ad was effective even though it may not have been).
And obviously there are many other factors that can't be directly observed and/or that you won't even know about.
Up until the past year or so I've mostly been involved in PPC from the affiliate/publisher side, but as my business has shifted I've often found myself testing new ideas but only being able to take away a broad, general analysis of their effectiveness due to the fact that you don't end up with super-specific figures that track everything down to the nanosecond like you can get with a typical PPC campaign.
To give one specific example, a friend hired me to do some advertising for his Minecraft server. It was pulling in about 3k monthly; the first 2 months I worked on it it did 11k, and the 3rd month it went back to 3k, 4th month was 2.5k, and 5th month was about 10k. I advertised it by buying banners, promoted YouTube videos, Facebook posts on popular pages, etc as well as tweaked some things on the server to make the checkout process faster and more user-friendly. Advertising methods were reasonably consistent throughout the 5 months. I also tracked every little change that was made on the server, for example if they added a new sign at the spawn location I took note of it, thinking I would be able to directly compare that event to a change in revenue to see if something as little as a sign placement affected revenue. In the end, it was extremely hard to measure/track anything due to the fact that you don't know which source the traffic/customers are coming from, and there are so many other factors such as school schedules, holidays, competitors adding/subtracting things from THEIR servers, random players posting YouTube videos of it that went viral (or likewise, videos that went viral and then suddenly stopped growing), and so on and so on. There was no way to account for all of these random factors and it became very annoying not knowing which events were the effects of which causes (if any). To this day I still only have a very general idea of what worked and what didn't and I could barely answer "why" any of those things worked, without just speculating.
That's just one example. I've noticed somewhat similar issues with other ventures I'm working on that don't have very specific tracking methods. And it seems like it would be true for basically any business that advertises or gains customers in ANY way other than direct PPC only. With PPC, you know that all of your traffic comes from your paid ads, you can see your CTR on each ad and the CTR for the jump to the lander to the offer completion, and you can very easily calculate the value of each ad as well as the exact value of any small change you make to the site via simply split testing it. With non-PPC, there are so many factors that aren't being tracked that it all becomes a jumble.
So I'm wondering if anyone has experienced this type of issue (revenue being affected by multiple non-measurable factors and so on) and if you have any ideas/suggestions for analyzing these types of reports.
But then if you do that, it doesn't account for other unobservable factors, for example maybe you aired an ad on a day that also happened to be a snow day, which caused kids to stay home from school spending all day downloading games. In which case your downloads would have increased anyway and you have no idea how much is related to the ad and how much is related to the snow day (assuming you even know about the snow day - if you don't know about it, you'll just think your ad was effective even though it may not have been).
And obviously there are many other factors that can't be directly observed and/or that you won't even know about.
Up until the past year or so I've mostly been involved in PPC from the affiliate/publisher side, but as my business has shifted I've often found myself testing new ideas but only being able to take away a broad, general analysis of their effectiveness due to the fact that you don't end up with super-specific figures that track everything down to the nanosecond like you can get with a typical PPC campaign.
To give one specific example, a friend hired me to do some advertising for his Minecraft server. It was pulling in about 3k monthly; the first 2 months I worked on it it did 11k, and the 3rd month it went back to 3k, 4th month was 2.5k, and 5th month was about 10k. I advertised it by buying banners, promoted YouTube videos, Facebook posts on popular pages, etc as well as tweaked some things on the server to make the checkout process faster and more user-friendly. Advertising methods were reasonably consistent throughout the 5 months. I also tracked every little change that was made on the server, for example if they added a new sign at the spawn location I took note of it, thinking I would be able to directly compare that event to a change in revenue to see if something as little as a sign placement affected revenue. In the end, it was extremely hard to measure/track anything due to the fact that you don't know which source the traffic/customers are coming from, and there are so many other factors such as school schedules, holidays, competitors adding/subtracting things from THEIR servers, random players posting YouTube videos of it that went viral (or likewise, videos that went viral and then suddenly stopped growing), and so on and so on. There was no way to account for all of these random factors and it became very annoying not knowing which events were the effects of which causes (if any). To this day I still only have a very general idea of what worked and what didn't and I could barely answer "why" any of those things worked, without just speculating.
That's just one example. I've noticed somewhat similar issues with other ventures I'm working on that don't have very specific tracking methods. And it seems like it would be true for basically any business that advertises or gains customers in ANY way other than direct PPC only. With PPC, you know that all of your traffic comes from your paid ads, you can see your CTR on each ad and the CTR for the jump to the lander to the offer completion, and you can very easily calculate the value of each ad as well as the exact value of any small change you make to the site via simply split testing it. With non-PPC, there are so many factors that aren't being tracked that it all becomes a jumble.
So I'm wondering if anyone has experienced this type of issue (revenue being affected by multiple non-measurable factors and so on) and if you have any ideas/suggestions for analyzing these types of reports.
