Many analysts, marketers, product managers, UX and CRO professionals nowadays rely on user counts provided by Google Analytics, Adobe Analytics, or similar tools, in order to perform various statistical analyses. Such analyses may involve the statistical hypothesis tests and estimations part of A/B testing, and may also include regressions and predictive models (LTV, churn, etc.). If you are one of them, brace yourself as this article will reveal some surprising ‘secrets’ about what these user counts really are.
I don’t say ‘secrets’ lightly. Even with many years of experience with Google Analytics I’ve only just recently become aware of what…
I’m fascinated with the insights one can get from Google Trends ever since I used it to predict the results of my country’s national elections in 2009 and the prediction was significantly more accurate than all respectable polls. With just one hour of work I published a prediction which matched the observed outcomes better than what took Gallup and others orders of magnitude more resources to produce.
I am now summoning the power of Google Trends one more time to get the lay of the land of data science by using search interest trends over the past ten years as…
Do you want to know how you can prove the Earth is flat using Occam’s Razor? Do you want to prove God is the reason for everything in the Universe? Do you want to refute Einstein’s theories without knowing anything other than high-school physics?
Well, I’m not going to tell you how to do those things, since they can’t be done unless you misquote Occam’s Razor — something which seems to be inevitable in almost any argument where it is invoked. I will further show that even when it is quoted correctly, the Occam’s Razor is non-operational in practice and…
Back in late January 2019 I wrote “The Gillette Advertising Fiasco: Paying for Competitor’s Brands Popularity?” going over the early data suggesting that the outrage sparked among Gillette’s core demographic by their “toxic masculinity” ad is likely to result in significant damages to the sales of the brand. I supplemented the fact that the YouTube video with the ad had over 750,000 downvotes at the time (currently 1.5M) and the general social media outrage with detailed search trends data for competitive brands. Based on all this I argued that P&G, Gillette’s owner, was practically paying for their competitor’s advertising.
I…
Extreme skeptics and epistemic nihilists often misquote the phrase “All Models Are Wrong” as supporting their claim that nothing can be learned from modelling our environment and to claim people are helpless in the face of the complexity of Nature. This article explains why they are wrong in doing so.
Prominent 20-th century statistician George Box is often quoted as saying “All Models Are Wrong” or sometimes, a bit more correctly: “All models are wrong, but some are useful.” usually in the context of epistemic nihilism: outright disbelief or extreme skepticism towards human ability to accumulate, process, and successfully use…
A/B tests (a.k.a. online controlled experiments) are widely used to estimate the effect of proposed changes to websites and mobile apps and as tools for managing the risk associated with such changes. A number of software vendors are competing in this field with custom-built testing rigs used by conversion rate optimizers, landing page optimization specialists, growth experts, product managers, and data analysts.
However, despite the relative maturity of the field, there are several widespread mistakes with using A/B testing statistics. Their effect ranges from making tests less efficient and thus more costly, to rendering the results of an A/B test…
If you have not heard by now, razor manufacturer Gillette recently found itself in hot waters after releasing a highly controversial commercial (see it on YouTube) which has generated a host of negative responses on various social media, news outlets and blogs, as well as calls for boycott. Despite being released only about 2 weeks ago (Jan 13, 2019) the ad dubbed “the toxic masculinity commercial” managed to gather the astounding 1.3 million downvotes versus 740k upvotes on YouTube amidst cries from users that their comments and downvotes are being deleted. …
What if I told you that irrelevant statistics are routinely used to estimate risks of tested treatments and pharmaceutical formulas in many clinical trials? That we fail time and time again to correctly identify good treatments or harmful effects of drugs due to a single practice that most researchers apply without realizing its inadequacy? What if I added that this poor practice continues unquestioned as it is enshrined in countless research papers, textbooks and courses on statistical methods, and to an extent perpetuated and encouraged in regulatory guidelines?
Here I will lay out the issue in as simple terms as…
What can you learn from 115 publicly available A/B tests? Usually, not much, since in most cases you would be looking at case studies with very basic data about what was tested and the outcome of the A/B test. Confidence intervals, p-values and other measurements of uncertainty will often be missing, and when present they are often poorly calculated and the statistical procedures are not shared and thus are hardly of any use. One source — GoodUI.org …
Under the European Union’s General Data Protection Regulation (GDPR) U.S. companies are projected to spend a total of $41.7 billion to achieve compliance or face a significant reduction of their market size (the EU is the second largest economy in the World). The new regulation, which just came into effect on May 25, 2018, also requires companies with over 250 employees and those handling the data of 10,000+ people to appoint a so-called “Data Protection Officer” (DPO). This person is to be responsible for overseeing the internal procedures and external communication related to data storage and protection.
European Union companies…
Applied statistician and optimizer by calling. Author of “Statistical Methods in Online A/B Testing”. Founder of Analytics-Toolkit.com and GIGAcalculator.com.