The folly of Google Analytics’ exit rates

Have the Web analytics folks designing automated metric reporting for Google Analytics missed a great opportunity to improve the content management of websites? Sure, they had to simplify things to make their automated reports broadly understandable. But simplification doesn’t fully justify their handling of the “Bounce Rate” and “% Exit” rate in the “Content Drilldown” section of Google Analytics.

The good news is that both of those two rates have the same concept of “exit”: an exit is a departure from a site. A visitor is measured as having “departed” from a site if the visitor has no activity on that site for some arbitrary period of time, usually defined as 30 minutes. So, a transit from page X to page Y on the same site is, for Google Analytics, not an “exit” from page X. Since the bounce rate and the %Exit rate refer only to site exits, that is how we’ll use the word “exit” here too.

The bad news is that, the percent exiting a site via a particular web page (the %Exit rate) includes (overlaps with) the percent bouncing from that same page (the Bounce Rate). As a result, the “% Exit” result can be so swamped by the volume of bounces (e.g., on those pages that receive a lot of traffic from external web pages) that it’s hard to evaluate the rate of non-bounce exits. In other words, Google Analytics leaves us unable to evaluate how much visits to that web page are leading to exits by people who arrived at the page from another page internal to the site. Suppose the exit rate of page X is 60%. And suppose the bounce rate of that same page is 70%. Then what is the exit rate for that page that isn’t coming from bounces? Google Analytics doesn’t directly reveal that.

How to solve this problem? If we’re only going to get two attrition rates in Google Analytics’ automated reporting, then the solution would seem to be to define the two metrics so that they don’t overlap. That solution would let us separately measure–and separately manage the budgets for— each of those two sources of attrition.

The first source of traffic attrition, correctly measured by the bounce rate, is immediate exits of visitors that were drawn to the page from an external web page or other external mechanism. Such externally-sourced visitor traffic is often driven by web advertising and SEO budgets. People generally bounce from a web page when its contents do not match the expectations set by the external page that led them to that page. Tracking bounce rates can help website managers align the content of their web pages to their promotion of those web pages.

The second source of traffic attrition is exits of visitors that got to the page via some other page within the same site. We’ll call this kind of exit an “internal-traffic exit.” Such internal traffic is often driven by different budgets, e.g., navigational and website-design budgets. People generally have an internal-traffic exit either when a transaction has been completed or when the content of the page did not match the expectations set by internal navigation. When internal-traffic exit rates are too high, navigation or internal links need to be reworked.

The trouble is that this second source of internally-driven attrition isn’t reported separately by Google Analytics. Google, in effect, rolls the first source of attrition (bounces) into the second source of attrition (internal-traffic exits) to get an overall measure of attrition, which it reports as the “exit rate.”

So how to separately measure the second source of attrition, the internal -traffic exits?

First, we need to understand the exact components of the bounce rate and the exit rate.

A web page’s bounce rate is a ratio. Every ratio has a numerator and a denominator. In the case of a bounce rate, the numerator is (a) the number of exits from a web page that was the first and only page of the visit to that domain. The denominator is (d) the number of views of that page’s web domain that were part of visits that started on that web page. The bounce rate is simply (a) divided by (d). If page X got 10 page views from visits that started on page X, and 6 of those page views were visits that ended before visiting any other page on the site, then the bounce rate was 60%.

A web page’s “exit rate,” as reported by Google Analytics, is also a ratio. Both its numerator and denominator are more broadly defined than is the case with the bounce rate. The numerator of the exit rate includes not only the exits (a) included in the bounce rate, but also two other types of exits from that page (which we’ll continue calling “page X”): (b) exits from visits that started on page X and ended on page X, but included a viewing of at least one other page on the domain; and (c) exits from page X of visits that started on a different page of the domain.

Similarly, the denominator of the exit rate includes not only page views from visits that started on that page (d), but also includes (e) page views of page X from visits that started on any other page of the domain. In other words, the denominator (d) of the “bounce rate” is page views where page X was the “landing page,” while the denominator of the “exit rate” (d+e) is “all views of page X.”

When we put the broadly defined numerator and denominator of the exit rate together, the “exit rate” equals (a+b+c)/(d+e). Suppose a website had 30 page views of page X. Suppose 10 of those 30 page views came from visits that started on page X. Of those 10 page views, suppose 6 came from visits that ended before going to another page, 1 came from a visit that ended after going to another page and then returning to page X to exit, and 3 came from visits that ended by exiting from another page on the site. Of the 20 other page views of the original 30, suppose 5 page views were immediately followed by exits from page X. Then Google Analytics would report the (overall) “exit rate” of page X to be (6+1+5)/(10+20) or 40%.

Now that we understand the two rates that Google Analytics is already reporting, we are ready to carefully extract the bounce rate from the exit rate, to arrive at the missing “internal-traffic exit rate” that navigational web-designers need.

The internal-traffic exit rate would be a ratio that removes the bounce-rate activity from the overall exit rate. How to do that? Well, any time that a view of page X doesn’t result in a bounce (a), the visit will potentially end on that page after visiting some other page on the site. In other words, all views of page X, except for page views that result in bounces (a), are candidates for an “internal-traffic exit.” So our denominator (or “base”) of potential internal-traffic exits equals (d+e-a).

The internal-traffic exit numerator should similarly exclude “a” (bounces) from the overall exit rate’s numerator. So the numerator would be (b+c).

So, overall, the internal-traffic exit rate would be (b+c)/(d+e-a). Using the numbers in the example above, the internal-traffic exit rate would be (1+5)/(30-6) or 25%, well below the overall exit rate of 40% and the bounce rate of 60%. And, in accord with our intuition, this internal-traffic exit rate will increase when the exits of page X within multi-page visits increase faster than total views of page X in those multi-page visits that include page X.

Last but not least, how does the bounce rate merge with the internal-traffic exit rate, in algebraic terms, to form the overall exit rate? The overall exit rate, as reported by Google Analytics, is, in effect, a muddy weighted average of the bounce rate and the internal-traffic rate: [a/d x (d/(d+e))] + [(b+c)/(d+e-a) x (d+e-a)/(d+e)]. Because some of the visits (visits d minus visits a) are included in the numerator of the weights of both component rates, the weights do not sum to 100%. As a consequence, it’s possible for the bounce rate and exit rate to both be equal (say 50%), even when the internal-exit rate is different (say 40%). But, since we have extracted an internal-exit rate that can be measured, managed, and budgeted for separately from the bounce rate, do we even care about the overall exit rate for each web page?

Share

Building your Theme Pool for Web Content

A swim in the pool is refreshing, but what if the pool is full of those critters called themes? Imagine a technology vendor that wants to win new customers by adding more “informational content” to its website. What themes should the new content emphasize? Or, to take one step back, which themes are candidates for emphasis?

The potential pool of candidate themes is large and deep, in part because the pool is fed by at least four major sources.

  • SEO themes: These are the themes or, more precisely, the keywords that the vendor believes its target market is most likely to search on, when gathering information related to the vendor’s products.
  • Editorial themes: those themes that the vendor believes will readers will most value. Ideas for these themes may come from an editorial staff, Web traffic analysts looking at what has worked well previously, or market research asking readers which topics and format would interest them.
  • Branding themes: those themes that the vendor believes will most drive readers to value the vendor’s products over the products of competitors. This list of themes will most likely come from the list used by the vendor’s marketers to construct its advertising and public-research communications. This list, in turn, may have come from a combination of market research and strategic-positioning decisions. Of course, if too much of the new informational content centers on these branding themes, the reader will see the content as mere marketing. But informational content still creates many opportunities for the vendor to convey branding themes.
  • Sales-conversion themes: those themes that the vendor believes will drive readers to take action, or be receptive to action from the vendor, that will in turn lead the readers toward becoming customers. Although some may think that “information content” is detached from the sales-conversion process, the right informational content can prepare the customer to seriously consider the vendor’s offerings.

Ready to jump out of the pool yet? Or maybe you have found a method to determine which themes to emphasize when.

Share

Some ad copy-testing lessons for content copy-testing

Copywriters of ads have long been wary of how well ad copy-testing works–and not just because, like the rest of us, they are sensitive to criticism of their work. Should the writers of informational digital-content copy also be wary of copy-testing? Do the problems with ad copy-testing apply to content copy-testing?

Even top researchers (e.g., Arthur Kover, Journal of Advertising Research, 1996, 36:2) have acknowledged that the ad copywriters had legitimate concerns about how well ad copy-editing can indicate which version of an ad will be more effective or even whether the copy needs improvement. Many of the copywriters’ concerns can be boiled down to two major concerns: the survey environment is (1) too distraction-free and (2) too rational compared to the environment where ads are consumed. Let’s look at the thinking behind those two concerns.

  • Copywriters have said that “the survey environment of copy-testing is too different from the distraction-filled environment in which the copy appears in real life.” This was unquestionably an issue when most of the copy being tested consisted of ads. In real life, ads often appear peripherally in cluttered settings or as undesired interruptions to the content people have opted to consume. So copywriters have to go to great lengths to make the interruption grab the attention of viewers. Early copy-testing methods rarely re-created these cluttered settings and, thus, underestimated the value of attention-getting ads. Later research methods partly solved this problem, by leading respondents to believe that they would be asked about the TV or magazine content they were to view (thus distracting respondents from the ads), but then asking them about the ads. Regardless, insufficient distraction is almost a non-issue when copy-testing digital content. In the real world, consumers opt to read digital content, usually by clicking a headline link or a link in an e-newsletter. This assures some level of attention will be given to the content. By the time the person has opted in, attention-getting has already been achieved (presumably by the content’s headline, which is less easy to test accurately via a survey). So survey pre-testing does not need to distract respondents away from the content, because respondents will generally not be distracted away from the digital content when consuming it in the real world.
  • And copywriters have said that “the research process of filling out pages full of checkboxes evokes excessively rational responses.” This complaint was especially a concern when the copy being tested was full-screen TV ads whose central thrust often hinged on visual- and music-driven emotional appeal. Much of Web content is different. Even though the long-lasting success of Web content also depends on its emotional appeal—its ability to tell a story that deeply resonates—much Web content seeks to appeal as much to reason as to emotion. Also, these days, pre-testing is different. A fair amount of digital-content copy pre-testing occurs not in mall intercepts or telephone interviews, but on the Web. Gone is the experience of having to take pen to paper or having to answer questions directly to an interviewer. Now, the click-and-progress process of completing an online survey is fairly similar to how consumers progress from one piece of content to the next on the Web. In other words, both Web content and the process of getting to that content are already putting users into a relatively rational mode that isn’t all that different from how users complete a survey online. So the excess rationality of surveys relative to digital content is much less than the excess rationality relative to TV ads. Nevertheless, the excess rationality of surveys remains an issue to be guarded against. Researchers are working on improving the online survey environment to counter this, while at the same time insisting on avoiding introducing unusual screen backgrounds and interactions that, independent of the content being tested, would create their own dynamics, skewing results.

copy-testing


In sum, at least one of copywriters’ major concerns with ad copy-testing does not seem to apply to digital copy-testing: the lack of distraction in the survey environment no longer seems a problem, because people opt to read digital content rather than being interrupted by it. The other concern—that the survey process is too rationalistic—seems less severe than with TV ads, because much of Web content hinges on rational appeals. But this latter concern persists. When the content being tested is clearly making emotional appeals, good researchers will know to rely less on merely quantitative results and look toward qualitative-research learnings, perhaps obtained earlier in the content-development process, or from open-ended or oblique emotion-detecting questions included in the formal copy-testing.

Share

Market segmentation for the Web

Market segmentation traditionally assumes that people can be segmented into groups that can be separately marketed to, based on their different product needs. But suppose, as often happens on the Web, the customer arrives in your midst—on your site—before or she has any inkling of his or her different product needs. And suppose what happens next—to that customer on your site—will strongly influence how that customer ends up perceiving his or her particular product needs. Now, what is your segmentation strategy for this Web stage of the game?

For that stage, I endorse the segmentation approach advocated by many theorists of Web design: segment your visitors by (a) what is unlikely to change as the result of their visit, e.g., lifestyle profile and prior level of experience with your product, and (b) their content preferences.

Some aspects of their content preferences will be unaffected by their visit (e.g., a general preference for graphics over statistics), but other aspects will be affected (e.g., if the buyer learns more about which features are important to a product category, their interest in which products possess those features will increase). In particular, segmenting your non-current-customer Web visitors primarily by their content preferences seems to make a lot of sense.

This way, your web designers and copywriters can generate Web pages and content that appeal to the persona, defined with heavy reference to their content preferences, that represent those key segments.

Share

Persuasive content and Web advertising

In the past, journalists generated content to serve one master—the public good—while advertisers generated content to serve another—the private good.

Journalists’ content provided evidence that it was serving a different master by criticizing the products produced by advertisers. That aspect of the past should continue.

The purpose of OpinionPath’s research methodology is not to change how journalists’ content is produced.

It is to help advertisers communicate better online.

In the past, the objective of advertising was to persuade. But the overt effort to persuade sets up conscious obstacles to being persuaded.

Suppose the most persuasive path is dialog, and the second most persuasive path is content that overtly delivers some value other than mere persuasion. Some advertisers recognized this long ago; recall their early sole sponsorship of TV shows dedicated to delivering entertainment and information values. Now advertisers are working to find comparable ways of working on the Web—ways that allow them to simultaneously deliver immediate value and longer-term persuasion. So persuasion still has its place. But, for many product categories and brands, the timing for the delivery of overt persuasion in the marketing process has shifted toward later stages. In the earlier stages, different forms of marketing-driven content are needed.

Share

Benefits of direct-to-consumer PR

The list of benefits that marketers are accruing from generating their own content continues to grow.

  • Speed. Offers faster dissemination paths than waiting on traditional media’s schedules.
  • High Volume. If the volume of a vendor’s content is large (e.g., a library of how-to videos), social media has the “space” that traditional media does not.
  • Offsets Media’s Shrinkage. With falling ad pages generally meaning shrinking content pages in traditional media, broadens the distribution pipelines.
  • Control. Compared to getting news covered by traditional media, allows more control over the message.
  • Audience Profile. Has the potential to reach a more diverse audience than readers of traditional media.
  • Engagement. People who choose to view it are more likely to have a strong interest in the product category than readers of general media.
  • Indirect Media Relations. Vendors’ postings are another way to get the attention of journalists.
  • Circumvents Journalists. Through co-branded partnerships with media companies, provides a way to get content onto their sites, without going through traditional filters.
  • Builds Direct Audiences. Provides content that can sustain the vendor’s efforts to build its own customer communities rather than depending on media.

    AdAge.com’s Oct-26-09 article “As Media Market Shrinks, PR Passes Up Reporters, Pitches Directly to Consumers” conveys how the experiences of companies such Procter & Gamble, Best Buy, MasterCard and Coldwell Banker have led to each of the above benefits, e.g., through the placement of their videos on YouTube. The article is careful to point out, though, that public relations professionals are still eager to earn placement of their news within traditional media, because success is inexpensive relative to advertising and traditional media still delivers considerable credibility. It’s just that the supply of traditional media content seems to be shrinking.

    New Media Words

    Share

Avoiding multicollinearity through conjoint analysis

The premier research companies have helped to make conjoint analysis and its sequel, discrete choice analysis, into popular survey-design research methods. These methods ask respondents to evaluate a series of products or, in special applications where the attributes center on positioning rather than product features, a set of brands.

Each product is generally presented as a bundle of listed attributes, e.g., a series of notebook computers, each with a different list of memory size, processor speed, screen size, price, and brand. In conjoint analysis, respondents are asked to rate or rank each of the products (not the attributes, but the overall products) on some desired outcome variable, such as likelihood to purchase. In discrete choice analysis, respondents are generally asked to choose (rather than rate) one product each from several sets of competing products.

The analysis of the survey’s results reveals how much the addition or subtraction of a particular attribute would affect the preference for the overall product. After calibration with historical results and inclusion of cost factors, researchers can then use these results to predict the changes in market share or profits that would occur as the result of changing the product’s attributes (or the brand’s messaging).

Research clients like such methods because they seem very “real world” compared to prior methods. Prior methods, still very much in use today, asked respondents to evaluate the importance of each attribute to their purchase decisions rather than asking respondents to choose or rate the overall product as they tend to when preparing “short lists” or shopping.

Nevertheless, from an analytical point of view, the greatest benefit of conjoint/discrete-based survey design is that they allow the researcher to avoid the problem of multicollinearity. If the objective of the research project is to figure out how important each factor (or product attribute) is to a market segment’s perception of a product or brand, it’s frustrating to learn “because the factors have multicollinearity, i.e., generally appear in the company of each other or change levels simultaneously (e.g., people who say quality is important are less likely to say that price is important), we can’t estimate the independent impact of each factor.” Conjoint and discrete analysis almost entirely avoid this problem by asking respondents to evaluate products that are as likely as unlikely to share the same pairs of product or brand attribute levels. For example, to figure out the independent impacts of price and quality, respondents can be shown products that have both high quality and high price, high quality but low price, and low quality but high price. It is this capacity of conjoint and discrete to isolate the impacts of each attribute that makes them such powerful aids to decision making–of the businesses marketing the products, not the respondents taking the survey.

Share

Reading on integrating experimental and observational research design

Because people stumble into the career of market research from so many other fields, many books on research and survey design need to be written at a basic level.

The books listed below, though, are different. Many leap to a level of mental gymnastics that some of us haven’t experienced since college. But the exercise can be good for you! Even if you initially only understand 10% of it, that 10% can set you free to rise above the mundane.

Unfortunately, since I haven’t reading these books myself, I can’t yet give you a full mapping yet of which fit together, which are too tough, and which might be skipped over.

  • Leslie Kish, Survey Sampling (1965) [Classic text for dealing with imperfect sampling (non-response, lack of coverage) and complex sampling (multi-stage and multi-level)]
  • Kerlinger and Lee, Foundations of Behavioral Research (1999) [Comprehensive text that reaches from the past almost into the present]
  • Shadish, Cook, and Campbell, Experimental and Quasi-Experimental Design for Generalized Causal Inference (2001) [The best book, and the one that new methods aspire to beat]
  • Paul R. Rosenbaum, Observational Studies (2002) [The new generation: propensity-scoring…]
  • Donald B. Rubin, Matched Sampling for Causal Effects (2006) [Key thinker of new generation]
  • Judea Pearl, Causality: Models, Reasoning and Interference (2000, 2009) [Coming out of the field of computer science, Pearl writes almost as a mathematical philosopher, challenging both the new generation and the old, and giving you a new understanding of structural equation modeling]
Share

Merging Surveys and the Experimental Method

Designing market research to learn what we want from participants is already hard enough. Why do we have compound the challenge by introducing “advanced methods” or “analytics”? Finish writing one report on a survey with the feeling “we still don’t really know which actions to take,” and you’ll sense the answer. Survey research does a terrific job of gathering descriptive information but, even with regression and correlation, it has a hard time confidently revealing what causes what.

Faced with this apparent defect of surveys, it’s tempting to retreat from the survey to a series of one-off “marketing experiments.” Let’s try this home page. No, let’s try that. How about this change? But even if one of these marketing experiments gets lucky and leads to a jump in some performance metric, how do you generalize the result to other pages? You are left with the nagging feeling that you still don’t know enough about the “why we succeeded” to apply the learnings elsewhere. Suddenly the survey doesn’t look so bad because, by randomly sampling a target population or set of market choices, at least its results are generalizable.

The solution, as others have argued, is neither the descriptive survey nor a series of simple experiments, but research design that permits surveys to behave like generalizable experiments.

Share

New Media and Content-Marketing Collide

An increasing share of content on technology-media sites is either produced by technology vendors (e.g., white papers) or produced at their behest (e.g., some webinars and mini-sites). 

Some of this “vendor content” (as distinct from user-generated content) is excellent.  Some is not. 

But the question is, why is so much of it appearing on media sites rather than just on the corporate sites of the vendors?  There are many possible explanations but, after some careful examination, most of those explanations seem peripheral to the deeper causes.

The link below points to my presentation on the topic last year.

http://www.slideshare.net/cstetson/new-blends-of-advertising-and-content-in-the-web-age

The deck closes with a discussion of how editors and journalists might develop strategies to respond to these changes.

Share