How the Index remains an unmatchable benchmark

Mobile websites and apps were added to the factors assessed in producing the latest edition of the corporate online world’s most respected ranking, David Bowen says.

Featured sites

The sixth annual Financial Times Bowen Craggs Index of web effectiveness will be published online next Wednesday, 25 April, by the Financial Times and on our website.

For those not familiar with it, the Index is an annual ranking of the online communications activities – websites and other channels – of the biggest companies in the world. It is the tip of an iceberg – the scores are generated from 800,000 words of analysis – but has come to be accepted as the most credible guide to best (and worst) practice. The people who think that are the people who know: online managers themselves.

To quote our own methodology statement, the Index ‘is the most detailed mass study of large websites produced, and has three aims. First, to help organisations know where to look for ideas. Second, to pick out trends in online communications. Finally, to allow the companies in the Index – broadly, the largest quoted companies in the world – to see how they are performing against their peers’.

Credibility in numbers

There is a simple reason why our numbers are so credible – they are the result of a huge amount of effort by analysts who understand both the internet and business. Eight of us have been working on the Index solidly it for the past three months, going through 81 giant web estates with a metaphorical toothcomb, producing a 10,000 word report on each. Why do we put so much work into it? Because the underlying database is the core of our business, not a device to generate interest in other activities.

Only subscribers get to see the reports themselves, but the scores will show who is doing well in which areas, as well as who is doing badly. The latter is particularly relevant this year – as we will show next week.

Evolution of methodology

We fine-tune the methodology each year to keep up with the shifting world we cover, though we are not great jumpers on bandwagons. The main changes this year are

An explicit analysis of corporate mobile websites and apps, built into the relevant metric. For example, if an app is aimed at investors and journalists, we will cover it in the ‘Serving investors’ and ‘Serving the media’ metrics. Together with our previous integration of YouTube and social media channels, this means the Index covers all areas of online communications. Purists may even wonder about our continuing use of the word ‘web’, given that apps are not web-based, but for the moment we will hang on to it.

Upgraded accessibility tests We are not technical testers, but feedback from corporate web managers told us that we should look specifically at accessibility. We did that in a light way last year, and have had a more rigorous methodology developed for us this year. This is an important but modest element of the Index: a site that does well on other construction criteria should be well prepared for accessibility, and our tests show this is true.
We have also increased slightly the number of companies covered, to 81 from 75. That is because we decided to keep the companies that performed well last year (placed in the top half of the Index), but no longer made the selection criteria see beow). That creates greater continuity and also allows us to hang on to a valuable body of good and interesting practice.

Ins and outs

Which leads on to the most common question we get: ‘Why are we not in the Index?’ In an ideal world we would examine a huge number of websites and rank them – we could then plausibly say ‘these are the best corporate sites’. The problem is that we take up to 20 hours to look at a website with its associated channels and apps from scratch, and not much less than that to review an existing analysis. So we need a way to define the sites to look at before we start.

Our association with the Financial Times gives an obvious starting point: the FT Global 500, published every summer. It ranks quoted companies by market capitalisation. To provide a useful regional spread, we take the top 25 from the US, and the same from western Europe and from the rest of the world (including Russia). The six extra companies retained this year (see above) are evenly split between the US and Europe.

Unmatchable guidance

Does this limit the value of the Index? We don’t think so. It provides a massive amount of analysis to identify the trends and good ideas that give it its value. Here’s what one of our clients – Ivette Johnson, Procter & Gamble’s digital brand manager – says: “The Bowen Craggs database is quite different from anything else in terms of having something so thorough and methodical that evaluates against multiple stakeholders, paying attention to a proper set of metrics. It’s something we can’t get anywhere else. Without it we’d be flying blind.”

First published on 18 April, 2012