A few weeks ago I wrote about how we’ve changed the account expiration policies for Windows Live Mail Accounts. That was one example of an anti-user policy that we’ve worked to address in our new product. There are a few more we’ve been working on, and they all share a common thread… meeting the expectations and needs of the customer while at the same time being responsible about our business.
One thread that has remained constant since the first day that we started work on Windows Live Mail is the struggle we’ve had with the advertising experience. We knew that many aspects of WLM would be different from the current Hotmail. We added such features as the reading pane, something that we were not sure would make it into the final product. In an effort to remain consistent with our existing advertising footprint, we simply assumed that we would design our interface with all the same ad formats and number of ads. However, at the same time we generated this concept of flights. A flight is a variation on the default experience. For example, one flight might not have a reading pane at all, one flight might only have one ad etc. We did this so that we could objectively measure the satisfaction of the customers in those flights and make good decisions about such important issues as advertising.
Before I go into what we learned, lets talk about the ad formats. In WLM there are three ads. The first is a square ad and appears in the Today page and the Sent Mail Confirmation page. These ads you see infrequently. Then there is the super banner, which appears at the top of the window. Finally there is the skyscraper ad, which is the tall skinny ad that appears to the right of the reading pane.
Now, the ad that posed the largest challenge to the design is the skyscraper. Because we design for 1024 x 768, the sky scraper simply doesn’t fit. You see, the ad formats are standard in the industry and the sizes were chosen in a pre-ajax world; before you had applications living inside of a window. The skyscraper is great for a scrolling web page, but not for a fixed width and height product. As such we spent many many weeks designing and coding a mail product that tried to maintain an advertising footprint that had both the banner and the skyscraper in a 1024 x 768 window. Let’s just say it wasn’t easy.
Meanwhile, over a period of months we learned a lot from our users about what they thought about our design; they liked it but felt like the reading area was too small. They pointed to the skyscraper ad as the reason. The flights we ran allowed us to measure just how users felt about the different experience, and what percentage of them were opting out of the beta. Early on users complained about different problems, but as time went on and as we addressed the majority of feature and service complaints the space/advertising issue bubbled to the very top, and it was time we did something about it.
Now, let’s look at this from a different perspective; that of the business and the advertisers. In order to spend ad dollars you are going to want to go to a company that has ad inventory to offer you. You also want to go to a place that has a lot of eyeballs and engaged users. We are such a place, and as you might guess, our service is funded most significantly by advertising dollars. Any change in revenue needs to be explained internally (ultimately with sound data). While we were looking at the consumer feedback we were also looking at the advertiser feedback, particularly on the skyscraper. Was it a good ad offering? Was it effective? What would removing it do in their world? Would removing it make existing ad properties more valuable to them? Would removing the ad create a better customer experience which would in turn be more valuable to them? We believe the answer to this last question is “yes”.
Understanding the business of advertising, and the business of being an ad-funded communication service are things that I knew little about when I joined Hotmail 2 years ago. Our team has worked incredibly hard to measure and explain these things in the context of customer satisfaction (consumers AND advertisers) and our goals around growing our customer base as well as the number of minutes they spend in our service. We have gotten incredibly disciplined and downright good at knowing all about these things, while at the same time we’ve done the same with our Quality of Service and features relative to the competition.
At the end of the day why does this all matter? Because this is where all the competitive action is going to take place in the next few years. Google, Yahoo, AOL, and Microsoft are all going to be competing for one thing… advertising dollars. And how are they going to get the most advertising dollars? Better products, better advertising platform, better user experience, more engaged users and ultimately the right advertising inventory.
So, what’s the big deal, and why am I writing this. Well, one of my philosophies, and something I think our team shares is Don’t piss off the customer.
How do you piss off the customer?
- Give them a 2 MB inbox
- Don’t save their sent mail, or make it difficult to do so, and then delete their sent mail after 30 days.
- Make their inbox about advertising instead of about their email
- Have crappy Quality of Service.
Sound familiar? It sure does to me. All of these things are anti-customer. What’s the point of offering a service that’s anti-customer? I sure as heck have no intention of working on a service like that. I never would have taken the job that I did if I didn’t know and feel that everyone around me was driven and motivated to fix all of these things, and we have been working on all of these since day I started this job.
Starting next month we are reducing the number of advertising from two graphical ads to a single ad in the inbox. The skyscraper will be gone from Windows Live Mail! I hope people see this as an olive branch from us to the user, and the advertiser. The users will be happier and more engaged, and the advertisers will ultimately benefit in the end. This change and its impact is an investment that we believe is a smart one to make. Everyone in MSN has been supportive of this decision and we wouldn’t be making it if we didn’t feel that it was the right thing to do and better for all of us in the long term.
Like our decision to extend account expiration from 30 to 120 days, this decision was months of work from dozens of people in our organization. We have more work to do, but the signs are all looking good 
.