Analysis of these new social media channels has been hobbled by old world thinking, when it comes to marketing, from the world of Television and Magazines or, when it comes to measurement, from the world of traditional web analytics.

These new channels, twitter and facebook and youtube and tumblr and, yes, even blogs, are very distinct customer / participant experiences. Stale marketing or measurement thinking applied to them results in terribly sub optimal results for all involved.

So in this post my hope is to share with you what is unique about measuring one such channel, Twitter. The blog post is also sprinkled with my own words of folksy wisdom as to how you should use the channel for maximum impact.

My new book Web Analytics 2.0 also covers social media measurement, but I am going to cover something very different in this post.

First: An Ode to New Thinking:

One common thing between the all tools in this post is that they were built by “outsiders”.

One of the things I love and adore about Twitter (besides all that connection and conversation) is how it has completely lit a fierce fire of innovation when it comes to analytics. Anyone and their brother and ma-in-law can develop a tool, and they have! Much to the benefit of the rest of us.

Perhaps the most beneficial thing to me is how much out of the box innovation this has brought.

For example just look at traditional web analytics tools, there is absolutely no fresh thinking when it comes to Social Media Measurement. The in their thinking from all the constant “let’s figure out how to collect and report every more data and not bother with truly immersive understanding of these channels and what makes them unique”.

While there is some stale thinking in the new twitter tools, most of them have a lot of fresh thinking from people untainted by Omniture or CoreMetrics or WebTrends or, ok ok ok, Google Analytics.

I consider this massive proliferation of new thinking to be a gift from God.

To all of you developers who are toiling out there, you have my love and gratitude.

In this post four twitter analysis tools. Each while not yet all developed yet show sweet signs of

1. Truly understanding the medium and uniqueness of the channel and

2. Are not just reporting “hits”, rather coming up with clever metrics.

Quantitative Metrics / Analyses.

Most twitter analytics tools just do data puking. They find numbers that can be computed and then proceed to puke at you as many as they can find, with wonton disregard of value being provided or outcomes being measured.

Here is one of the mild ones:

You must pause and think: So what is this saying? What action can I take?

Always, always, always ask that question when faced with tools that simply puke data out at you (twitter or Google Analytics or whatever).

But as I mentioned at the start of the post one of the beauty of twitter’s open API is that there are a few pockets of truly innovative thinking.

Here are some that I humbly believe look promising. . . .

Klout. Twitter Analytics.

Klout is a wonderful little tool that measures Klout Score, a proxy for “influence”:

It is easy to understand the market demand to boil things down to one number, but this is perhaps the least useful thing in Klout.

While on the surface they might seem useful, I am always suspicious of compound metrics. They can be subjective, inapplicable to many and efficiently hide the insights you need to understand what actions to take. [See more here: Four Not Useful KPI Measurement Techniques]

Mercifully there is so much more to Klout than the that.

Klout measures a bunch of lovely metrics, specifically applicable to Twitter, that are grouped into four buckets: Reach, Demand, Engagement (!!) :), Velocity.

There are two lovely things about these computations.

1. Joe and team have wonderfully avoided the temptation make these compound metrics (as in Reach = Followers / Total Retweets * Friends + Pixie Dust). The factors used are laid out as individual metrics making it easy for you understand and the data and pick metrics that work for you.

2. (My favorite) The metric definitions are not “crap”. This seems like such a low bar to meet, sadly far too often metrics out there (not just for twitter) are just plain shoddy.

For example here are some clean definitions from Klout:

# Engagement

* How diverse is the group that @ messages you?
* Are you broadcasting or participating in conversation?

# Velocity

* How likely are you to be retweeted?
* Do a lot of people retweet you or is it always the same few followers?

# Reach

* Are your tweets interesting and informative enough to build an audience?
* How far has your content been spread across Twitter?
* Are people adding you to lists and are those lists being followed?

When I use Klout I simply pick the metrics that are most important to my own twitter strategy. I would suggest that this is very very very very important, pick what is right for you rather then a lemmings like strategy of “I am going to use metrics Y & Z that someone recommends”.

Here’s an example: I don’t care about Follower/Follow Ratio. I think it is disingenuous to follow everyone who follows you just for appearances sake when you have no intention of reading what they all say. Why be fake?

As you might have read in the new book I like “Message Amplification” in Social Media, and hence I do care a lot about Total Retweets.

[Sidebar: my favorite twitter metric is: # Of Retweets Per Thousand Followers, it's a measure of efficiency and value provided and people voting with their clicks, all rolled into one!]

I care a lot about Follower Retweet % (”Do a lot of people retweet you or is it always the same few followers?”) because I want to appeal to more people than my mom, dad, and best friend!

One of the biggest mistake companies and brands make about Twitter is that they think it is one more “shout channel” like TV and Radio and Magazine ads or Press Releases. Twitter is not that. Twitter is a “conversation channel”, a place where you can find the audience relevant to you (and your company and products and services and jihad) and engage in a conversation with them. It is not pitching, it is enriching the value of the ecosystem by participating.

Hence I like the metric Inbound Messages Per Outbound Message, as a primitive measure of the fact that you are participating in a conversation and not just yelling.

With Klout I can choose the metrics that best reflect my personal twitter strategy, I can easily find them and I can monitor my progress (using a handy dandy graph) and ensure my strategy is a success.

Your strategy might be different. Walk up to the buffet and pick the metrics that will help you best measure your own success.

[ Contest: Notice the metrics I have deliberately ignored: # of followers, # of retweets, @ mention count etc. Can you guess why? :) The person with the best guess gets a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]

At the bottom of the Stats tab Klout also includes a handy dandy Analysis table with trend indicators. . . .

As an Analyst it might be of some value to look at the trend pointers at the bottom (clearly I am doomed!), it might be cute to put this into a PowerPoint slide for the HiPPO’s who might like the Chinese fortune cookie messages for each metric group.

Ok, ok, I am just teasing the Klout team, I know it is very hard to “wordify” and programmatically make valuable recommendations. :)

GraphEdge. Twitter Analytics.

The reason I believe GraphEdge is interesting is that it has a set of really cute metrics that help bring a different perspective to measuring Twitter.

If want to contrast the difference in thinking applied compare some of the metrics below with, for example, what Omniture is touting with its Twitter “integration”. The difference between the old web analytics thinking and a new person’s could not be more clear.

[Allow me to rush and add that while Omniture has a hack to bring some twitter data into Site Catalyst to do something, Google Analytics has nothing. Not even something that is not useful. So perhaps GA stinks even more.]

Here are some, IMHO, differentiated metrics. . . .

Legitimate Followers:

If you have spent any time on Twitter you know that spam accounts are a problem so it is very nice that the first thing you see in GraphEdge is not a follower graph but rather their attempt at telling you how many legitimate followers you have (and the trend over time, cropped out in the image below). . .

To identify “legitimate” they use the following filters, direct quote:

Any of your followers who are following more than 2,000 people are considered not-Legitimate… they’re probably not really monitoring your feed, so we don’t count them as “Legitimate”.

Users who have been suspended by Twitter can’t read your tweets (and probably weren’t interested in the first place!). We don’t consider these Legitimate Followers.

It is ok to argue with their filters, but it’s a fine start and good enough.

Klout also measures something called “True Reach”, which is also their way of identify if you’ve got people or bots following you.

Churn Rate:

In my days at DirecTV one of the metrics that the company was obsessed with atleast then and rightly so, was Churn Rate. It reflected the value of not just going after new customers but doing all that was possible to take care and love the customers we already had. Makes sense?

So I have always had that obsession with tracking Churn, simply to try and understand, if possible, why people leave. The hope is if I can understand why then I can do something to fix the problem.

[By now I am sure you get the feeling that I am treating twitter analysis like I would business analysis. Twitter is my brand channel and I take this very seriously. It is perfectly ok to use Twitter to tell people where you are and what you are doing and not care about analysis.]

I have not really found any decent tool to track unfollows in twitter (yes I have tried the normal ones and they are either flaky or just outright stink). Hence I was happy to have this in GraphEdge. . . .

Yea!

Actually ouch! 291 un-follows!! So sad.

Atleast now I know.

It is nice to have the over all trends on the right, as well as for the period you choose smarter metrics like growth rate.

GraphEdge will also show a list of your new followers and un-follows (so you can send them bad vibes! Kidding, Kidding. :).

[Qwitter was one of the first tools I used to track unfollows, sadly it does not work any more, and it had a great feature: It would try to guess and report on which tweet resulted in the un-follows. Nice.]

And here’s what we were on the quest of. . .

I will admit to not being charmed by having three different lines, they clutter the left and the right and get in the way of understanding the data. But you I suppose you can learn to ignore it.

Here’s the definition from GraphEdge for Churn:

The number of removals (un-follows) over the average size of the existing base (followers) during the period measured:
Drops / (Current Followers – ((Adds – Drops) / 2))

As always look at the trends, the longer term the better. And remember that history is littered with companies that were growing just fine but they still died a painful death because of Churn Rate.

Loyalty:

Slightly along the same lines GraphEdge has a metric called Loyalty. At the moment I think it is too limited in what it actually measures, and it only starts measuring once you join GraphEdge. But there is kernel of promise in the metric, keep an eye on it.

Second Level Network Size:

Lastly… while most people overestimate their “twitter power” (I can bring you down with a single bad tweet Avinash!) I think a few people underestimate their reach, if they participate in twitter in the right way.

Looking at the second level report can give you a feel for your network size.

Followers’ Friends is an “incestuous” number, it shows all your followers and the people they are following. If you have ten followers and they all follow each other that’s 100 Followers’ Friends. Feel free to be proud of this number, but then promptly ignore it.

Unique Names is are the unique twitter account id’s in the network, less the “illegitimate” ones. Think of this as something close to, but not the same as, the Unique Visitor concept in web analytics.

This is a useful number.

Think of it this way: If you say something of incredibly profound :) . . .

. . .and a whole lot of other people who follow you think that and retweet it then you have a theoretical capability to reach 1.2 mil people (Unique Names in your Second Level network).

Now the reality is that that will rarely happen, if ever, but in our profoundly hyper connected world Unique Names is a good number to keep on your horizon.

Remember success in twitter comes from participating in the conversation and giving something of value, not by running “social media campaigns”. If you don’t internalize that be ready for a reality where both your Followers and Second Level Network sizes to be small potatoes.

Qualitative Metrics / Analyses.

Now let’s tackle the much much harder analysis to do in any filed, analyze the data from a qualitative perspective.

Linguistic Analysis:

I have given up on “Sentiment Analysis”. Well atleast for now. Everyone over-promises and massively under-delivers.

On paper it seems like such a great thing to want to have, this is a social / conversation medium after all. But most tools I have had the good fortune to try are simply either glorified versions of Google Alerts even if they promise you buzz metrics and the moon.

Now sentiment analysis is a very hard problem to solve. For example I just analyzed my account using a expensive “social media sentiment buzz analysis tools” and it marked this tweet from today as Negative:

avinashkaushik: There in nothing quite like AC power in your seat for a 10 hour flight. Oh and 20 hours of pending work to do.

Perhaps the tool does not find my dry wit as funny as I do, but it’s hardly “negative”!

With all that context I think TweetPsych holds a lot of promise.

Tweet Psych uses the Linguistic Inquiry and Word Count (LIWC) method and the Regressive Imagery Dictionary (RID) method to build a psychological profile of a person based on the content of their last 1,000 tweets.

Dan Zarrella, founder, says: “I think the possibilities of a system like this are enormous, from matching like-minded users to identifying users that exhibit certain useful or desirable traits.”

I am not sure I understand perfectly how it works (I need to send this to Joseph Carrabis!) but the analytical techniques looks very promising. . .

Hmm… interesting. I do like talking about “learning, thinking, knowing etc”! :)

As always rather then looking at my data in isolation I compare / contrast it with my friend who is a web analytics twitterer. . .

Now I understand a lot better how I am doing and how he is doing.

Remember there is nothing wrong or right here, we are both just very different people with different twitter strategy and what Tweet Psych’s linguistic analysis algorithms helps us understand if our psychological profiles are aligned with our twitter goals.

Tweet Psych also provides you with Primordial, Conceptual and Emotional Content analysis, here’s mine. . .

Nice.

Use this type of analysis to understand at a deep level what attributes are being associated with your brand, and if they are reflective of the goals that you set for yourself.

Content Visualization with Stream Graphs:

Stream graphs can be very good at visualizing data, content specifically. Twitter StreamGraphs is delightful for:

1. its visualization of highly associative words with the word you are querying and

2. viewing streams (tweets) for any associative word (hence sweet filtering, something so darn hard to do with twitter content)

Here’s today’s view of the data for my account (searching for @avinashkaushik). . .

Please click on the image for a higher resolution image.

You can choose the associative word stream that you are interested in most, click on it and at the bottom you can see the tweets. The size of the stream and shows you strength.

Once you choose the stream you can also click on the dates on the x-axis to filter down to the tweets for that particular stream for that particular date.

Sweet.

The stream graph for avinashkaushik would be different, as it looks for mentions. . . .

Please click on the image for a higher resolution image.

It is analyzing the last 1000 tweets and such a great way for me to understand the content, and filter down and review the relevant tweets easily.

Twitter StreamGraphs helps you visualize content in a very unique way and solves a very important problem to boot.

Parting Words of Wisdom.

I hope if there is one thing I have convinced you of then it is that you need to be a lot more critical when you think of analyzing these new media channels.

It is important to put aside stale (certainly current web analytics) thinking.

It is important to participate in these mediums so that you’ll truly appreciate what their real strengths are.

It is important to question metrics that have cute names, dig one step deep, just one single solitary step, to check if the metric definition passes the BS filter.

It is important to choose the metrics that help you measure your unique goals.

Finally it is important to realize there are no short cuts. Be willing to work hard. Be willing to put in the sweat equity. Be willing to try 45 things (tools / metrics / strategies) to find the 3 that work for you.

Good luck!

[Missed the contest? Go back and look for the red parenthesis, you'll win a copy of Web Analytics 2.0! Contest closed, thanks for the entries!]

Please share your feedback on this post via comments. Got any other tools that you love and adore? Please share them – with a quick comment on Why you love them. Got a piece of analysis that you think is profound? Please share that with all of us as well.

Thanks.

Social Media Analytics: Twitter: Quantitative & Qualitative Metrics is a post from: Occam's Razor by Avinash Kaushik

Even with a broken right hand I can’t help but write this post!

The waterfall of positive feeling stems from the fact that this book was very hard to write.

I only had one job, at Intuit, when I wrote my first web analytics book. I now have several full time jobs, plus this blog, plus speaking around the world, plus a family, plus… so much more.

It took weekends of writing and nights of editing and days of research combined with practicing the preaching by doing oodles of analysis and, more importantly, the support of the most understanding wife in the world.

At the end of it all it is rather gratifying to see one’s book at a bookstore, helps grasp the magnitude of the process. And there’s absolutely nothing quite like hearing your five year old yell in a busy Borders bookstore: “I FOUND DADDY’S BOOK!”

This blog post is in three parts: The pitch. Request for help. A lovely contest [Contest closed now, thanks for the entries!].

You don’t have to read the whole thing & skip ahead, but that would hurt my feelings. :)

Here we go. . .

The Pitch:

I invite you to consider buying my second web analytics book. It is not only the most current book on everything important and bleeding edge in Web Analytics, it is a labor of love that will help you transform your personal thinking and assist in revolutionizing your organization (big or small).

It is not a technical book, though it will make you technically dangerous. It is not just a business book, though every dna strand in this book is more about online marketing than online analytics. It is not a hard book to read, though it is brain food.

Here’s why I think you’ll love it:

Chapter 1 The Bold New World of Web Analytics 2.0

No dragging of the feet, the book starts with a bang by laying out the framework that will be the center of every company that will leverage data (qualitative, quantitative, competitive) on the web. It ends with a challenge to embrace Multiplicity – without this it’s goodbye greatness.

Chapter 2 The Optimal Strategy for Choosing Your Web Analytics Soul Mate

It will be hard for you to find a more compelling four step process to choose the right web analytics tool for your company. Soul searching, questions to torture vendors with, comparing vendors, running a pilot and negotiating a contract, it’s all in there. You be off to the races right.

Chapter 3 The Awesome World of Clickstream Analysis: Metrics

The thing I enjoyed about this chapter (I know I wrote it, but still. . .) was that the first half works really hard to evolve your critical thinking skills. I love that because we take too much for granted, now you’ll be skeptical. A good thing. The second half shows exactly how to pick the best metrics for your org and, my absolute favorite (Page 64), how to diagnose the root cause of a metrics performance.

Chapter 4 The Awesome World of Clickstream Analysis: Practical Solutions

When people think of web analytics everything they think about is chapter 4, and yet you’ll find so many yummy treats here. The best WA report, segmentation, site search, SEO & PPC analysis, email, rich media, cookies, data sampling. . . . I am out of breath!

Chapter 5 The Key to Glory: Measuring Success

If I have one jihad it is to massively convert every person who touches the web to focus on measuring Outcomes! It is the one reason we can’t achieve the greatness we so richly deserve. No more! Glory will be yours!! B2B. B2C. Small Biz. Large Biz. Non-Ecommerce. We make love to ‘em all! One thing you’ll read here that you’ll read no where else? Computing Economic Value, a concept that will liberate you.

Chapter 6 Solving the “Why” Puzzle: Leveraging Qualitative Data

Oh, oh, oh qualitative analysis!! I am a Mechanical Engineer with a MBA, a late covert to the power of understanding the super sexy “why” by leveraging lab usability studies, surveys, card sorts, online remote testing and more. You get a jump start. The thing you’ll adore: Pages 190 – 192.

Chapter 7 Failing Faster: Unleashing the Power of Testing and Experimentation

Sure you’ve heard of A/B and multivariate testing. But do you know how to truly win the game? There is no technical mumbo-jumbo here, just the real deal and how to get testing right. The thing you might not know / realize the power of: Controlled Experiments. I am convinced this is God’s gift to online humanity, you’ll agree with me by the time you reach Page 208.

Chapter 8 Competitive Intelligence Analysis

The most magnificent advantage the web possesses: everyone’s data is available for everyone else to use. If Hilton Hotels has the data for Choice Hotels why not use it to “crush” them (sorry Sarah!). This chapter shows you how. I think the thing you’ll be surprised by is at the start of the chapter (Data Sources, Types and Secrets).

Chapter 9 Emerging Analytics: Social, Mobile, and Video

The chapter I had the second most fun writing. Mobile, twitter, blogs, videos etc are just so darned hard to measure and so much changes every few hours that I had to really really work hard to find the essence of each and then make specific practical measurement recommendations that will stand the test of time. It was hard.

Chapter 10 Optimal Solutions for Hidden Web Analytics Traps

This is a collection of major reasons I think people fail at web analytics, and of course I boldly try to share how to avoid that fate. Behavior targeting, dashboards, accuracy, data mining, predictive analytics, and, the thing you’ll appreciate the most IMHO, five steps for intelligent analytics evolution!

Chapter 11 Guiding Principles for Becoming an Analysis Ninja

All my life learnings laid bare. . . this is where you, yes you, start to evolve from a Reporting Squirrel to an Analysis Ninja! No metrics, data pukes, guidance on creating every more reports. No, none of that. Rather… analytical techniques, tips and tricks to apply to your job, how to evolve your thinking to a higher level.

Chapter 12 Advanced Principles for Becoming an Analysis Ninja

The chapter I had most fun writing (and rewrote the most number of times). It deals with two of the hardest practical challenges we face in the field of measurement: multi-touch campaign attribution analysis and multi channel analytics. Both are very hard to get right, both have a ton of fud out there, it was fun to share my recommendations.

Chapter 13 The Web Analytics Career

The chapter I should have had in the first book. How to plan a career in web analytics (paths, salary, longevity), and how to then cultivate the right set of skills. If you are a leader then how to spot great talent, how to interview them and make the right choice.

Chapter 14 HiPPOs, Ninjas, and the Masses: Creating a Data-Driven Culture

Some might argue, rightly so, that the most elusive thing to accomplish is to truly bring data democracy to your organization. This chapter bravely hopes to help you do exactly that: excite people about data, remove organizational barriers, use data to change behavior, dealing with data quality, and creating data driven HiPPO’s.

Convinced?

Nothing, absolutely nothing, in life is easy. But if you have the will and access to knowledge then that just might help you choose an optimal path, a path where your hard work will yield above normal results. That’s my hope, and promise, with Web Analytics 2.0.

Jennie and I have decided to donate 100% of our proceeds from this book, just like for the first one, to two charities. This book benefits The Smile Train and Ekal Vidyalaya. We are very excited about that.

Request For Help:

As you all know my philosophy for this blog is eat like a bird, poop like an elephant. But if you are up for it I would love to ask you for a bit of help.

Recommend the book.
If you know someone who needs to turbocharge their online existence, please recommend Web Analytics 2.0 to them. Even in our hyper connected world, nothing works like a personal recommendation.

If you use a link please consider using: http://tr.im/akweb That link has an affiliate code, all proceeds of which go to the above mentioned charities.

Review the book.
If you have a blog, website, twitter account, any kind of platform, it would be great if you could write a review of the book and help spread the word.

If you purchased the book online then please, pretty please, review the book on the store’s website. Amazon. Borders. Target. Powells. Whatever you used.

Connect me.
I am very very bad at pimping. So if you know someone who is someone (or knows someone who knows someone) then please consider connecting us. Especially people outside our analytics / search circle. Authors. CEO’s. Journalists. Influencers. TV anchors (or weather man/woman). Oprah (I can dream, can’t I?).

Our world is separated by six degrees of separation, I am sure you know someone who just might consider helping me with my cause.

Share a picture.
I love getting to know my audience, and while your emails and tweets are pretty fun there is nothing like a picture.

I had a “Web Analytics: An Hour A Day Fan Mail” flickr group that has some incredible pictures from around the world, bringing my audience closer to me.

I would love to do the same again for my “Web Analytics 2.0: Fan Mail“. Be as creative as you want to be. Babies. Cats. Posters. Cars. Places. Or the best, you. All would be welcome.

I will only post the pictures with your permission. Please send them to blog at kaushik dot net. Thanks!

A Lovely Contest:

[The contest is closed now. Winning entry details.]

Steve Cunningham invited me to be a part of a little “contest” he is running. The prize is a delight, you get to win a pack of seven books on online marketing & social media: Six Pixels of Separation, The New Community Rules, The Whuffie Factor, Trust Agents, Crush It!, Duct Tape Marketing, and Web Analytics 2.0.

How to win you ask? Two ways.

1. Answer this question in comments below: If you were to measure the success of a company’s social media efforts how would you do it?

Pick any social media channel, or all. Only a short answer is required. The most innovative / interesting answer wins. No answer is too small or too simple.

[If you have my book already then my answers in the book to this question will win you major brownie points, but perhaps not the contest! :)]

2. You can get four more chances to win, if you want. Simply visit these blogs and answer a different question on each: Steve Cunningham, Beth Kanter, Tara Hunt, and John Jantsch.

Good luck!

A Word of Thanks:

This is from my book’s acknowledgment page…

I would like to express my deep appreciation to the readers of my blog, Occam’s Razor. In approximately three and a half years I have written 411,725 words in my 204 blog posts, and the readers of my blog have written 615,192 words in comments! Their engagement means the world to me and motivates me to make each blog post better than the last. It is impossible to thank each person, so on their behalf let me thank three: Ned Kumar, Rick Curtis, and Joe Teixeira.

A very solid case can be made for the fact that neither one of my books would exist without you and your engagement and encouragement.

Gracias. Arigato. Ngiyabonga. Xie xie. Obrigado. Shukriya.

Web Analytics 2.0 Book: In Stores Now!! is a post from: Occam's Razor by Avinash Kaushik

“You know what… web analytics tools like Site Catalyst, Yahoo! Web Analytics, WebTrends, and yes even Google Analytics, are mostly glorified data pukers. Each tries to outdo the other in trying to collect ever more data and regurgitating it. For all the math they do, it is astonishing how little intelligence they have, how little actual smarts are applied.”

Silence for a a few mins.

Awkward glances.

Then this: “What do you mean, and what can we do?”

Me: “I wish the tools would use an algorithmic approach to highlight the things an Analyst needs to know, give ‘em some starting points. Why make people dig for hours?”

You have to hand it to the team at Google, you “provoke” them and they respond. Google Engineers truly rock!!

Today Last week the Google Analytics team announced a raft of sweet features that take the current functionality in GA, wrap a liquid hydrogen fuel tank on it and shot it into a higher value orbit. Take some time to learn more about how you put more power behind your analysis punch: Google Analytics Now More Powerful, Flexible And Intelligent.

In this post I’ll want to share rest of the story, what came of the above provocation.

The first thing you’ll notice in Google Analytics is a new cool ability to better identify the “known unknowns”, i.e. we know what we want to know, but we don’t know if and when it is happening.

The feature is, rather cutely, known as Custom Alerts.

Here’s an example. Everyone tells me that Twitter is nothing but hype. But

[sidebar]
i started to write this post in preparation of the GA new features launch, unfortunately the next day i broke my right hand. that meant going to emetrics to do the announcement in a temporary cast, and of course no blog post. i had surgery this past thu. metal plate and some screws in, things will be normal in a few weeks.

i unfortunately still can’t type the thoughtful teachable post i had in mind, rather here are two videos that tell you about two features i am really proud of. hope you’ll love ‘em as well.
[/sidebar]

custom alerts: identifying the known unknowns

video: 8 mins:

intelligence: identifying the unknown unknowns!

video: 16 mins:

love it?

i hope you had fun learning a bit more about these two cool features. promise me you are going to set up two segmented custom alerts today!

let me answer one question that might be top of mind: the features are rolling out to all accounts starting last thu, it’ll get to yours any day.

it would be great to hear from you, please share your feedback, suggestions and critique via comments. thanks.

Analytics Becomes Intelligent. Hello Insights! is a post from: Occam's Razor by Avinash Kaushik

Measuring top line success of ecommerce websites is not very complicated, all the sweet revenue based outcomes are there (at the least).

Measuring non-profit websites is a bit complicated, but not really all that hard because we can, with a small amount of love, figure out outcomes to focus on (donations, # of sign-ups for the protest in DC, # of petitions signed, volunteer applications, etc).

Measuring government websites is a bit more complicated, if for no other reason than that it takes a pinch of effort with a dash of imagination to figure out what one is solving for. What are the desirable outcomes one can focus on to measure success?

The above question came to mind from a kind note I got from Ines Jans who is a part of the team that is responsible for www.belgium.be

Ines and team were just starting to think about analytics (because the love their customers!) and asked for some thoughts.

My first question to Ines was, (surprise!):

Q: Tell me a bit more about what your site does, like what are the real goals (or give me some ideas about it) and what challenges you face, what do you expect people to get out of it?

[Best Practice: Always, always, always start any measurement conversation with the above inquiry. The answer will be key to insights, and without it you'll simply be a glorified Reporting Squirrel.]

The answer, which might fit most government websites was:

A: The goal of our site is to be a portal to all the official information there is about Belgium and make information easy to find. Visitors should be able to figure out which Ministry is responsible for what tasks.

[Best Practice: Don't be surprised in your Analysis Ninja quest if you get answers that just start the conversation, rather then give you a prescription for what you need. Squirrels will despair here, but Ninjas will take clues from what they hear and visit the site and come up with a set of important measurable outcomes.]

Based on the answer above and some time spent on the English language site as well as those in other languages (Google Translate!), I came up with the following five questions I could ask data to measure success.

Let’s take each of these questions one at a time and figure out the best way to answer each using a true Web Analytics 2.0 strategy.

Q1. Are Visitors able to find the information they are looking for?

Given the singular purpose in life of this government website is to be the one stop shop for all the information one could possibly need, it should be pretty obvious that the very first, and magical, thing we would measure is if Visitors to the site are able to find what they might be looking for.

So would you use a web analytics tool?

Here’s the first surprise: No!

There are certainly tertiary ways in which you can answer this question using Omniture’s Site Catalyst or Google Analytics or other wonderful web analytics tools.

But the best way to answer this question?

Ask the Visitors!

Using a simple survey that pop-up on-exit (when Visitors leave the website) you can ask your customers to tell you if they were able to complete their task. No interpretation required.

4Q from iPerceptions, available in 18 languages, is a free on-exit survey you can use. If you don’t want to use a external survey build your own, ask four questions, analyze the data for:

“Were you able to complete the purpose of your visit today?“

The answer to this question becomes the #1 Key Performance Indicator (KPI). You are going to watch it like a hawk, you’ll post it on all your bulletin boards, you’ll set up custom alerts to ensure that your team gets a small electric shock every time this number drops below 65%!

The overall number is good enough, but the data that will be awesomely actionable will be, if you use 4Q: Primary Purpose by Task Completion Rate. . .

You see the second question in 4Q is “Which of the following best describes the primary purpose of your visit?” and a standard report in 4Q will paint the above picture.

Now you not only know if people find what they are looking for, but you also know which tasks are hard to complain.

You need to fix “Complain about the French” : ) because the Visitors are already upset and only 5% are able to complete their task, resulting in them becoming even more mad!

Remember: You don’t need to show the survey to everyone who comes to your site. You can sample just a small percent of your Visitors. You only need 300 responses in a month to get a statistically significant sample of data, and 1,200 if you want to do segmented analysis.

Q2. Are the Visitors satisfied with their experience?

HiPPO’s (the “highest paid person’s opinion”) in the organization, even in the government will love to have a more direct (than task completion rate) answer to the question: Are our Visitors happy with our website?

That’s were it is prudent to measure Customer Satisfaction.

4Q and other surveys of course measure that quite easily: Based on today’s visit, how would you rate your site experience overall?

Measure it. Trend it. Report it. Correlate the trend over time with changes you have made to the site and identify insights (any causal connection between site improvements / campaigns and customer satisfaction?).

An alternative, or additional, way to measure satisfaction is to count and analyze the Contact Us submissions. . . .

Start with the number of submissions. Trend over time.

Drill down into the type of complaints and do atleast rudimentary sentiment analysis (i.e. read & categorize) of actual messages to gauge customer satisfaction.

Remember: When you do surveys you don’t have to torture your Visitors with billions of questions! In researching this post I went to US government sites and I got a ugly 34 question on one single page looooong survey. 34 questions! Most were irrelevant. I would have answered a few, but this showed a fundamental disrespect for your customers. In the end your Visitors are upset and you suffer from a lack of data.

Only ask what you can action.

Q3. What is the most popular content on the site? What area can we prioritize higher than it currently is?

It is not unusual for content sites to produce content. It is even less unusual for them to produce content that they think potential visitors to the site might want.

What is rare is the analysis of what visitors to the site are actually consuming on the site.

Here’s a simple analysis I had learned from Tim Hart (who was with the J. Paul Getty Trust): Measure the distribution of content in each section of your website and the percentage of Visits to each section.

On the y-axis is each of the sections on the belgium.be website. In blue is the amount of content in each section. In red are the percentage of visits where that content was consumed.

Is it not awesome! Insights galore!!

If this were their data, and it is not, it would be pretty obvious that there is huge interest in content about Housing and Economy tiny fraction of the site’s content is about Housing and Economy.

The balance for Family is a lot less lop sided.

While the government might love Justice, Mobility and Health (and boy do they love Environment!), Visitors to the site are a lot less interested in those pieces of content.

Action? You know what people want, how about giving them more of that content?

When it comes time to prioritize the next set of web pages or videos or podcasts, how about giving higher priority to those big red lines?

Sweet right?

You can also do segmented version of this analysis, see what Visitors to English, Dutch, French and German sites prefer. Or within Family what group of content do people like. Etc etc.

Two more ideas to get into your Visitor’s head. . . .

Measure Downloads:

There are a ton of downloads (pdf’s mostly) on the belgium.be website. Forms, applications, useful guides (like how to marry a belgian or how to prepare for your first job) etc.

It is a trivial cost, analytically, to track these downloads using your web analytics tool. Do it. Measure what your Visitors are most interested in.

Yes, yes, yes I see my technical squirrel friends raising their hands and saying you can only track that someone clicked on the download link and not that the download was successful. I know.

For our analysis here just intent is fine.

In fact unless a vast majority of your Visitors are connecting using dial up it is safe to assume the download of small files went through. I know that does not make the squirrels happy. I am sorry. You keep squirreling while we make decisions about how to improve the site.

Outbound Link Tracking:

Another thing you’ll notice about the website (see why it helps to surf a site you are supposed to analyze?) is that there are a ton of links on the site that point to other government websites.

Track ‘em!

Of course the above is not their data :), it’s just an illustration of how absolutely easy it is to track this data.

From the report it is very easy to then figure out what links your Visitors click, which is a great, positive, indicator of the fact that they found what they wanted and also what they were interested in.

Remember: It is not very hard to do any of the above three types of analysis. All you need to get into your customer’s head is move away from “Top Pages Viewed” and “Page Views Per Visitors” and think a bit more creatively.

Q4. How long does it take for someone to find key information they want?

There are some pieces of content that are so darn important that they are heavily linked (say latest news in case of belgium.be) right from the home page, or that you really do want people to find them asap (in the Health section for example the pdf about how to deal with H1N1 virus in belgium).

For these important pieces of content measure Average Time To This Page.

That’s almost three minutes from the time that someone entered the website to the time they found this page (say the one about swine flu).

On average people give a page two and half seconds before they click/leave. Consider how long three minutes is, and how many people might have given up in the process of finding this key information.

Unfortunately not too many tools, Google Analytics and other Paid Solutions included, provide this as a standard metric. I use ClickTracks and that this delightful metric as a standard offering.

I wish others would have it.

Remember: You can use this data to ensure that your best information is found by Visitors to your site quickly. Fix your top / left / right / bottom / whatever navigation you have on the site. Consider creating a prominent “box” on the top right where you “merchandize” these important links. More things like that.

Q5. Does the right information actually exist on the website? What major things might we be missing on our website?

I have consistently advocated my love for internal site search analysis. It is simply da bomb!

Like many other sites belgium.be has a internal site search engine. Typically Visitors who have a harder time with normal navigation (or limited data on a page) will make liberal use of this site search box.

Why not use that data?

Again, this is not their data :).

I recommend looking at the top typed search terms by the Visitor but then also looking at the metric: % Search Exits.

That’s the bounce rate of your search results page. I.E. People come to belgium.be, search for the term hippo and the search results are so bad that 33.33% of the people exit from that page! They don’t even bother to do anything. Just bail. Bounce. Kaput!

Now you know both 1. what information they were looking for, 2. what search results stink and 3. likely because you don’t have the right or enough content about that keyword on your site.

Fix it!

I have one more idea to understand if you are missing information that your Visitors want on your government website.

Use Page Level surveys.

There are free page level surveys available or you can build your own (like the one above from a software vendor’s website).

These can be an excellent way to understand what content is missing from your website. You can of course also use the open text voice of customer (VOC) from surveys like 4Q, look for Visits with Task Completion = No.

Remember: These surveys don’t collect any personally identifiable information (PII) information, and that goes for your web analytics tools as well. Many government sites are extra concerned about privacy, as they should be. Do Please familiarize yourself with the privacy policies of the vendor.

Note: What was not tracked or emphasized. . .

Visits.

Unique Visitors.

Page Views.

Time on Site.

And so many more mundane and perhaps more “famous” web metrics.

I am sure most government or normal websites jump to that first. And why not, they are all staring you in the face when you crack open any analytics tool.

The problem is that these aggregate metrics barely contain any insight. If you focus on them, you’ll be left holding a empty bucket / cry a lot / get fired / not get your government pension / dread meetings with your boss.

I hope the above ideas inspire you to do more, go beyond the obvious and less than useful.

One last quick example. . .

You can use the same strategy for other sites. Though remember the job the site is trying to do and the desired outcomes will decide which key performance indicators you end up using.

For example for www.recovery.gov in addition to some of the metrics above I would probably also measure Visitor Loyalty and Recency. That’s because the government wants the data provided to be so sticky, and it is updated frequently, that it wants you to come and check it again and again.

In this case perhaps more than downloads I would also measure # of customized graphs created. When I measure content consumed (#3 above) I’ll probably focus on understanding which departments get looked at more on the site (are they the ones most spending money?).

You can also bet I am going to be totally on top of reporting how many complaints we have received on the site for Fraud, Waste & Abuse! Getting a ton of those would be a key performance indicator! : )

Makes sense?

Don’t despair just because you have a government site. Ignore the obvious. Focus on the site’s jobs. Identify key outcomes. Do productive analysis.

Good luck.

Ok now it’s your turn.

Are you responsible for a government website? What are your key performance indicators? What web metrics are important to you? Do you use any of the above strategies? If not, why not? Have you looked at www.belgium.be? What would you have recommend that I did not?

Please share your valuable advice / insights / feedback / critique.

Thanks.

PS: Like this post? Perhaps you’ll consider ordering my * new * book: Web Analytics 2.0.

PPS:
Couple other related posts you might find interesting:

• Brand Measurement: Analytics & Metrics for Brand Websites
• Measuring Success of Non Ecommerce Websites
• Measuring Web Analytics Success for Small Business Websites
• Outcomes, baby!! Measure Macro AND Micro Conversions
• Six Recommendations For Measuring Success of Your Blog

Web Analytics Success Measurement For Government Websites is a post from: Occam's Razor by Avinash Kaushik

Admit it, you’ve heard it.

I suspect you’ve even used it liberally!! : )

Before we go any further I must clarify that I love branding campaigns just as much as the next guy.

I love campaigns that Visa runs. I love watching the IBM ads (with the Linux kid perhaps the best of the lot). I loved the I’m a PC ads from Microsoft (and I am a proud PC!). I loved the Wario Land: Shake It ad from Nintendo on YouTube (now that’s creative!). I love a good billboard ad, Budweiser does good ones. My absolute favorite branding campaign of all time: Think Different.

I could keep going on.

The common theme through the above campaigns is that their primary purpose is “branding”. The hope is by connecting with you, or interrupting you, a lasting impressing, a feeling, might be left in you so when its time to get a credit card you think of Visa and not MasterCard, when it comes time for hiring consultants for a multi year project you’ll choose IBM and so on and so forth.

All well and good.

Here is the minor problem.

There is a very tenuous connection between these campaigns and outcomes, they are for the most part faith based initiatives. If supported by “data” then it tends to be of the most fragile kind (usually the the fact that the CEO saw it during the Super Bowl and felt happy suffices as actionable data).

None the less they persist.

Online it does not have to be that way.

It is criminal not to measure your direct response campaigns online. I think we have established that. I also believe that a massively under appreciated opportunity exists to truly measure impact of branding campaigns online. Paid Search or affiliates or email or display or YouTube or whatever channel you end up choosing.

[Oh and don't tell me that your "branding" campaign is to increase "engagement"! Remember: Engagement is not a metric, its an excuse.]

The Top Secret Hidden Never To Be Reveled Come Hell Or High Water Key To Measuring Branding Campaigns:

Answer this simple question: Why %&#$!^ are you doing the “branding campaign”?

Every campaign, and in turn website has a purpose. All you need to do is figure out what the purpose of your campaign is, no matter how outlandish (or childish) your goal.

The typical focus by companies, and the creative types in their employ, is to simply focus on figuring out what you are doing to do in the campaign.

I am recommending that they hold their horses / put their pants back on / slowly sit down in their over-stuffed chairs. You too!

Figuring our what you are going do do with your campaigns can come after you figure out why you are doing these campaigns. No, not just because you have money or because that is how things have always been and absolutely not because someone (a HiPPO!) asked you to.

Once you know the desired outcome you’ll be surprised to learn all the wonderful measurement possibilities that await you online, things that would be nearly impossible offline. [The web rocks!]

Measurement Recommendations for Desired Branding Outcomes.

In order to help you make the leap in the rest of this post I want to share the most common outcomes I have heard associated with branding campaigns, and my recommendations as to what they should measure for their brand campaigns online.

My hope is that this bushel of ideas will spark your own creativity when it comes to measuring your campaigns.

Outcome #1: To attract “prospects” / new customers.

This is perhaps the most common desired outcome: “I am doing branding campaigns to attract new prospects to our website. They will come, they will be wowed by our glory, they will immediately convert.”

It is not very hard to measure these campaigns.

Measure the change in the percentage of New Visitors to your website, its the orange line in the graph above.

Ideally you’ll measure the number prior to your branding campaign, say Feb 2009, and then you’ll measure it again during your campaign, March 2009. See if you were able to get more traffic to arrive at your site, and if they were Existing Visitors or New Visitors (hopefully measured with a first party cookie in your website analytics tool).

For good measure, just to be extra sure you’ll segment out the visitors who come by clicking on your campaigns (display/banner, YouTube, rich media, whatever), and see how many of them were truly new.

At this point there is no expectation that any other outcome was delivered, just a visit by someone who had never been to your site before. A fairly low bar.

Outcome #2: To share your business value proposition.

You are a news site like The New York Times or you are a non-profit like Idealist or you are the team running Google Maps.

The goal of your campaigns is to simple share your unique value proposition with everyone. They’ll be impressed enough to come visit your site and then do so repeatedly.

The ideal metrics for this desired outcome are Visitor Loyalty & Visitor Recency.

(Click on image for a higher resolution version.)

The data in the above report shows how frequently during a time period do the website’s visitors visit the website. In the Before version you can see that most people, 69.79%, visited the website just once. In the After version, when the branding campaigns were running, only 63.25% of the visitors visited just once. Which means atleast 7% of the visitors shifted to visiting more than once.

You can credit the branding campaigns with that shift (if that is all you were doing). Better still you can segment the traffic from the campaigns and validate that hypothesis.

If people were impressed enough with your value proposition and visited more often the the brand campaign was a success.

Another good idea is to measure segmented Visitor Recency.

In this case the analysis will try to judge if the traffic acquired by paid search branding campaigns is visiting my website more frequently in any time period, when compared with other segments of traffic (in this case I am comparing it to All Visits).

Outcome #3: To impress people about your greatness and buy more.

I wanted to put this as #3 because if it were a “conversion” campaign then it would not be a “branding” / feel good campaign.

But there are certainly campaigns that you run to prop up your brand that will entice people to buy more from you. If they were only going to buy underwear then now they’ll also buy a pair of shoes and headphones.

I recommend segmenting the traffic and measuring revenue lift but also measuring the average order size, if you did your job right then that latter number should be higher.

In this case our Yahoo! display campaigns did wonderfully in terms of conversion rate, but not in terms of the major goal of the campaign – sell more stuff.

Another thing people forget is to measure the overall impact, beyond simple conversions. Sure measure it as above but it is also good to marry up the qualitative data and measure Task Completion Rate using a onexit survey tool (use a free one like 4q from iPerceptions).

You can accomplish these goals:

~ Get an optimal understanding of what kind of people you ended up attracting to your website (look at primary purpose & distribution).

~ Were these people, even if all you wanted from them was to buy from you, able to complete their tasks.

In the above case that is clearly not true.

Perhaps your Average Order Size is not great because only 44% of the people who came to buy, as a result of your branding campaigns, were able to complete their task!

You would have fired your ad agency for a crappy campaign, turns out they did their job well but it was your website that stunk. Perhaps someone in your team needs to get fired? Perhaps you? (Just teasing!)

Note how marrying the Qualitative and Quantitative data can be helpful and identify true points of failure / success.

Outcome #4: To whisper sweet nothings to drive offline action!

Most commerce / love / stuff still happens in the real world and many many companies use various online marketing channels to drive people to take offline action (make purchases in stores or via their phone channel, show up for a woman’s rights rally, meetup at a concert etc).

You can measure the impact of these campaigns right on your website, using any onexit survey tool and by applying some delightful regressions on your data. You can compute two important metrics:

Likelihood to Recommend / Brand Lift

You can measure this at an aggregate level, or you can measure it just for your campaign traffic.

It helps you understand what was the brand lift, positive, as a result of the person’s complete experience (your campaign, plus your website).

Likelihood to make a Offline Purchase / Action

Simple right? Well it takes some planning but it is not that hard to measure (and if you get a decent sample size then you can also segment this data easily by source of traffic, brand campaigns, and show a causal relationship).

Phone Calls / Conversions Driven from Website

Another wonderful way to track offline impact of your campaigns is to use unique phone numbers with your campaigns (either on your display banner ads or on your website).

You can track the number of phone calls made to your call center, by campaign (or keyword or whatever) and if you have a integrated IVR then you can also track conversions / sign ups from those campaigns.

Companies like Fresh Egg in the UK or Mongoose Metrics and ClickPath in the US, amongst many others, provide these phone call tracking solutions.

[Bonus reading material: Multichannel Analytics: Tracking Offline Conversions. 7 Best Practices]

Outcome #5: To break through the noise / make an introduction to your business.

Very often when you run branding campaigns your goal is simply to introduce your business (like we are trying to do with Market Motive, our start up that provides certification courses in Web Analytics, SEO, PPC, PR etc etc).

A common mistake in this case is to simply focus on one outcome. If you are running a branding campaign then it is likely that you either have a very soft call to action or, more likely, you have a very general “our business is magnificent” message.

My recommendation is to quantify the online impact of these campaigns by measuring both the Macro & Micro Conversions.

For example if I were to measure impact of branding campaigns for this blog (remember it has no ecommerce of any sort) then this is how the report would look:

My macro conversion is to add to my current total of 27,300 RSS feed subscribers.

The above report shows the overall add to that number in this month but by segmenting my Yahoo! “Avinash is awesome” display campaigns I can see how many “macro conversions” occured.

But that’s just one part of the story.

I will also measure the “micro conversions” (goals 1, 3 & 4) to get a more complete picture (for example note the large percentage of “Loyalists” that ended up being from my brand campaigns!).

This methodology can be applied to any business.

For example if I were in-charge of campaigns for OfficeMax, I would measure ecommerce conversions but that’s just a sideshow for these types of campaigns.

For full impact analysis I would measure:

And so on and so forth.

In the case of Market Motive for our branding campaigns we will measure outcomes by focusing on # of sign-ups for the Master Certification program but also the # of people who sign up for the free webinars we do all the time, the Ask Us inquiries, the trial memberships, # of sample tutorial videos viewed etc etc etc.

When you are trying to break through the noise you’ll take any measure of success to detect a signal, use the macro & micro conversion mental model.

Outcome #6: To destroy your competition.

Very common goal for many marketing campaigns. Show how awesome your brand value is and directly or indirectly show your competition in poor light.

DirecTV does it well, though now Dish seems to be totally trashing them atleast during NFL games [Look at them go at it: DirecTV trashing Dish vs. Dish trashing DirecTV.]

Apple of course is a master at it (though sometimes they can be mean). CPG companies are perhaps a bit more subtle about it, but their goal is clear: get you to buy their brand of chicken bouillon or diaper or ageless blush.

There are a number of wonderful metrics you can use to measure online success of such marketing campaigns.

Share of Search.

The first thing you want to measure is how much “share of voice” you have “stolen” from your competitor. One great way to do this is to measure Share of Search.

If you have done a great job of branding then the number of people looking for you (searching for you) should go up. Oh and not in your Site Catalyst or WebTrends reports! Rather in the “ecosystem reports” you can get at a competitive intelligence tool…

www.diapers.com has a 3.64% “share of search” prior to the campaign. What was it after?

Did they make a dent in the universe?

I am using Compete for the above report. When you use it you’ll notice that Target is #10, Babycenter is at #15 (with a 0.77% share which seems looooow!). If you run branding campaigns for either company you now know how you’ll measure success.

You can index your performance for your campaigns, and against your competitors.

If you use broad-match like I did above you get a “category” view of your performance. If you use the exact match report. . .

. . .to get a better idea about about “brand” performance (and find new competitors – facebook anyone? :)).

You can do this for pantene, shampoos, styling treatments, pro-v, 2-in-1 shampoo + conditioner… the world is your oyster.

This analysis also helps you understand how well your offline branding campaigns are doing online.

For example I don’t know of airline companies that run more television campaigns than Southwest Airlines.

Yet currently for queries like cheap tickets, cheap airline tickets, cheap flights etc www.southwest.com does not show up in the top 15 in “share of search” reports, in some cases not even in the top 25. And that has not changed in the last few months (even with the barrage of new TV ads).

The TV ads are perhaps super productive in driving people to the phone or perhaps directly to the site, both desirable outcomes. But they are certainly not working in getting people who are looking for airline deals and searching for them to go to Southwest.com.

Traffic Differentials

If your campaigns are successful you’ll know it from your Site Analytics tools like Google Analytics or Yahoo! Web Analytics. But in the grand scheme of things did you have an impact?

Use tools like Google Trends for Websites (or Compete or HitWise) to analyze your performance…

In the report above if you (Hilton Hotels) ran your campaigns in March 2008 (purple arrow) then you managed to accomplish nothing. Notice the competitive trends?

If you ran your magnificent branding campaigns in Jan 2009 then buy yourself some beer and dance around because you can see, against competition, you clearly narrowed the gap (black arrow).

Of course you would not stop at the simple analysis above, that’s just a start. You can export data into excel, you can segment it by Geo and ensure the lift is where your campaigns were targeted, you can segment by demographic and psychographic visitor attributes to see if you got the right kind of people.

Using Share of Search or Traffic Differentials are just two of many ways in which you can measure if your branding campaigns are indeed crushing your competition. There are many other analyses you can do, there are many other tools you can use. Don’t give up, look.

Outcome #7: To emboss your brand into someone’s skull.

As Marketers we try and do this all the time as well.

You say Jeans, and people say Levis.

You say Jonas Brothers, and people say Pink by Victoria Secret (I kid you not, yes I was surprised, I have research to back this up!).

You say online search, and people say Bing! No, not so fast! Google!! Ok maybe it’s just a matter of time. :)

You catch my drift.

Branding campaigns are particularly effective at “embossing” brands into your psyche with the goal of improving unaided brand recall.

The challenge of course is how do you measure this elusive, but very desirable, outcome.

I have two suggestions.

Primary Market Research

In the online world we don’t make enough use of primary market research, and that’s a shame.

Field surveys, focus groups, interviews etc can be used very effectively to gain a indepth understanding of your customers and their influences (hopefully channels and methods you use to influence them show up in the answers!).

There is a ton of math and rigor involved in these studies that helps you get a great understanding of your audiences, even with small enough sample sets.

Plug into the Database of Intentions

I love that term: “database of intentions”.

For now atleast search is used by many people as they seek information online, and that allows for this data to reflect intent, rising and falling trends, preferences etc.

You can use Insights for Search for this type of analysis.

I am a Assistant Senior Vice President of Brand Marketing Campaigns at Orbitz. I have been spending my lovely marketing dollars on tons of TV campaigns (See: Orbitz Golfers). Oh and a smidgen on online campaigns as well.

So what was the impact, when people search for “hotels” do they think of Orbitz? Here’s the data you are looking for…

In the online database of intentions Orbitz is not on the horizon.

The data on the left is important because it tells you what people search for when they look for hotels.

The data on the right is killer. It shows which terms (hence brands, sites, properties) have risen the by the most statistically significant amounts. This is fantastic because it mines the data that is below the surface and brings the movers and shakers forward.

Some of Orbitz’s competitors show up there, their marketing dollars seems to be working well in improving the likelihood that when people are doing category searches, hotels here, that they would look for expedia, priceline, hotwire, etc etc.

But I am not going to give up, I just started running these massive tv campaigns a few months back. Let’s see the data for that!

Hmm…. the “winner” here seems to be Marriott from the hotels category in the last 90 days. Good for them, sad for me.

Let me hasten to add that it is quite possible that the desired result of these offline (and online) branding campaigns was to get people to go to the site directly or call Orbitz on the phone.

The first premise you can measure easily, how is our website traffic doing when we are running all these campaigns…

Two years of data. Expedia is green, Orbitz is blue, Priceline is orange (query on compete). It does not look too good for me (and if that was not enough Priceline just crossed me for the first time in my history, boo! boo!).

[It is important to point out I am simply doing outside-in analysis, a sport always fraught with risk. You on the other hand work at Orbitz and will have the tribal knowledge to make sense of this data better.]

For the last piece of analysis to measure unaided brand recall analysis, I’ll try is to correlate my brand marketing spend with the number of phone calls to 1-888-656-4546 (and keep my fingers crossed that I’ll see a massive spike in phone calls from the campaigns because, on the surface, it is hard to detect a impact directly on the site).

You can do this type of analysis for anything.

Let’s say Victoria’s Secret has indeed been heavily spending on branding campaigns for Pink by Victoria’s Secret in the last 90 days. I can look at this data to see if the brand Pink is amongst the fastest rising by those people who look for Victoria’s Secret “stuff”…

Not so much. Why?

Now if I were the Brand Manager for the Jessica White line then I would rejoice! While Jessica White is not in the “top related searches” category yet, it is rising very very fast as a result of my campaigns! Yes!

(Mental note to check later: What the heck is “victoria secret application”? Sounds dicey!! :)).

So there you go. A portfolio of seven strategies that you can use in the ultimate quest for any online marketer / analyst: measuring branding campaigns.

I hope they spark your creativity and lead you to finding even more innovative solutions to your unique challenges.

Buena Suerte!

Ok your turn now.

Are there other outcomes you can think of for your branding campaigns? Can you think of other ways to measure the seven outcomes mentioned above? If you have tried one of the above strategies did it work? If it did not, why not?

Please share your feedback / learnings / critique / kudos.

PS: Like this post? Perhaps you’ll consider ordering my * new * book: Web Analytics 2.0.

PPS:
Couple other related posts you might find interesting:

• Excellent Analytics Tip #15: Measure Latent Conversions & Visitor Behavior
• Multichannel Analytics: Tracking Online Impact Of Offline Campaigns
• “Dear Avinash”: Be Awesome At Comparing KPI Trends Over Time
• Excellent Analytics Tip #12: Unsuspected Correlations Are Sweet!
• Standard Metrics Revisited: #5 : Conversion / ROI Attribution

Brand Measurement: Analytics & Metrics for Branding Campaigns is a post from: Occam's Razor by Avinash Kaushik

It is with immense excitement that I am sharing the news that I have just finished writing my second book!

Web Analytics 2.0:
The Art of Online Accountability & The Science of Customer Centricity

It is a long title ain’t it? The good news is we are going to refer to it simply as Web Analytics 2.0.

In this post I wanted to share thoughts about the book, the process of writing it (and doing three rounds of edits!) and outcomes.

The Background

Since mid-2008 Willem Knibbe, my wonderful Acquisition Editor at Wiley, was very kindly encouraging me to update my (best selling!) first book, Web Analytics: An Hour a Day.

The “problem” was the book continued to sell at a nice rate and I was not sure what to update because 90% of the content was still current and relevant.

Still there was a lot of new stuff I had written, new models I had developed, new and more advanced techniques, new problems we were dealing with in the world and so on and so forth.

That lead to my proposal to Willem to write a new book that would use Web Analytics: An Hour a Day as a starting point. The second book would be an advanced book that would allow the first book’s readers to truly become Super Analysis Ninjas, and for those that had not read the first book to have the finest possible immersion in web analytics.

And that’s just what Web Analytics 2.0 is.

The 2.0 Book

The book’s core philosophy is based on the framework you have seen me talk about on this blog. . . the quest to answer four key questions: the What, How Much, Why, and What Else. . .

The awesome thing about writing a advanced book is that I can start with a bang! No history and what not. It starts with: Here is how your world should look like and this is why its important, now let’s get down to business.

That’s by page 9. : )

And then it just keeps kicking it up a notch. Bam! Bam! Bam!

Like the first book this is not a book about Omniture or Xiti or Google Analytics. It is not a “press this button in the tool and then press that one” book.

It hopes to be brain food.

Here is how you should think. Here are the traps to avoid when picking key performance indicators. Here are the core analytical techniques you should apply. Here are a bunch of reality checks. Here is how to embrace outcomes, regardless of the size of business you have. Here is how to achieve higher highs with testing and by listening to customers (literally). Here is how you leverage your competitor’s data. Here is how you becoming a true Analysis Ninja (step, by step, by step).

And none of that is even close to the coolest part of the book (see why I am so darn excited?).

There are so many topics I deal with each day that I have not had time to write about on the blog, all the things I practice all day long in the five jobs I hold.

The book gave me the impetus to write all that down.

So there are complete sections in the book that teach:

Why tracking the social web is such a massive problem.

How to measure success of blogs.

Meaningful non-crappy twitter analytics.

Mobile analytics! This was so much fun to write about.

Measuring rich applications whose primary usage happens with no internet connection.

And more such things.

But you might end up buying the book simply for Chapter 12, it covers two things that I think will rock your world:

1. Multi-touch campaign attribution analysis (dissected and presented in a way like you have not seen it any where, I think)

2. Multi-channel non-line analytics (practical tips, best practices, unique stories to inspire you)

Even after all that I was not completely satisfied. : ) There are two more new things to end the book. A complete chapter on how to start, nurture and advance a career in web analytics. The last chapter of the book is on how to overcome the hardest challenge of it all: creating a data driven organization!

The Writing Experience

This was a very hard book to write, in many ways harder than Web Analytics: An Hour a Day.

That’s partly because this time around I had my full time job, my work with my start-up Market Motive, my advisory roles in three companies, my world travel to support my professional speaking career, my blogging (the only thing that suffered), and of course my family.

It is difficult to find time and energy to write a book with all that (and impossible without a magnificent wife who takes on three times a normal human’s load to support you!). Especially to pull the writing and three rounds of edits in four months!

It was also hard because this is a much more advanced book with so many topics on the bleeding edge. It is hard to make sense of it all and understand it enough to apply a reality filter and then write something that people can apply today, and use for a very long time.

And yet it was a lot of fun to write this book.

I think that’s primarily because with the first book I had no real sense for what the book would become, who it would impact, how far it would go.

This time around I have a much better sense for all that.

So many of you have written to me about all the ways the book has touched your lives. As I wrote this book that was constantly at the back of my mind. It pushed me to work harder and do better because I realized all the places it would go, all the people who will crack it open, all the expectations it had to meet.

I had this visual of all the people who might buy this book and how in some way something I wrote could have an impact on them. That was pressure, but it was also fun.

The Second Little Book That Could

Some of you know that my wife Jennie and I had decided that we would donate all the proceeds from the first book to charity. We had chosen The Smile Train and Doctor’s Without Borders and split 50% of the proceeds between each.

My hope was that Web Analytics: An Hour a Day would sell enough for us to donate the $10,000 advanced we had received.

We have thus far received, and donated, 18 months worth of royalties from the book, approximately $70,000 (!!).

Not in my wildest dreams had I imagined that! And there is no way that we could have afforded to donate that much money.

In a very small way this blog and the book have helped other people in our lovely world. It has been an extremely gratifying experience for us.

With Web Analytics 2.0 we have decided to do the same again.

100% of my author proceeds from the book (and all the amazon affiliate sales) will be donated to The Smile Train and Ekal Vidyalaya.

Ekal Vidyalaya runs schools in remote locations that reach the poorest of the poor children in India. Without Ekal these children would have a very limited set of opportunities in life, if any.

When the going got really tough with this book the thing that kept me going was to produce a book that would have a big impact on people who buy it and a small impact on the charities Jennie and I choose.

The 411

The book can be pre ordered on amazon now, if you are so inclined.

It will be released mid-October 2009.

Wish me luck.

Web Analytics Books! is a post from: Occam's Razor by Avinash Kaushik

ROI is important for every function in business, but what if it’s measured too hastily?

I wrote about social media ROI last week and some of the comments in response to that post highlighted a tendency many have to oversimplify this calculation.

Simple is good; overly simple is not.

ROI Blah, Blah, Blah, More ROI

What if ROI is the thing so many people are distracted by while screwing up the metrics?

ROI is an economic equation that looks at costs and benefits, but how do you quantify what gets counted as a benefit? There are many steps on the road to ROI, but deciding what gets measured and how much weight to give it are the keys to giving an ROI calculation that actually makes sense.

For the record, let it be known that nobody is saying ROI calculation isn’t important — at least not me :) But, I don’t think hasty measurement or oversimplification is the way to go either.

This line from a recent post by Peter Kim got me thinking more about this:

“I think that too often, people jump from measurement to ROI too quickly. If we take a different approach that accounts for the interim steps required to get from start to finish, we’ll be able to sleep better knowing how our social business investments are performing.”

Yes, ROI is a simple economic equation that subtracts cost of investment from benefits of the investment and divides the result by the cost of the investment. Whatever.

Figuring out the costs has to do with the amount of money spent and work-hours devoted to the cause. That’s the easy part, right?

Slow Down

The tough part comes in when you try to put real economic justification on any social media endeavors. I’m of the belief that social media marketing works, but a lot of the numbers signifying failure in the industry are because too many measure the wrong things.

A significant disconnect between goals and success metrics was found in this year’s Tribalization of Business study by Beeline Labs and Deloitte.

Most goals of businesses surveyed centered around increasing word of mouth and brand awareness, but the top metrics positioned as indicators of success were things like number of active users and user engagement on the site.

I’m just thinking out loud here, but a better metric for word of mouth might be an increase in the number of branded keyword searches, and a better metric for an increase in brand awareness might be the number of incoming links and mentions on 3rd party sites including social networks.

My point is, the math involved in the ROI calculation is easy. Deciding how to measure and quantify what gets counted as a business gain could use some noodling.

Your Take

As we move into 2010 (time flies!), what are some of your goals and what do you plan to measure to ensure you’re meeting them? How does what you’re measuring eventually ladder up to ROI?

Does having a better sense of the role of measurement in ROI make you more confident in your efforts?

Photo by Vincent Ma