FAQ & methodology

For over a decade the global Bowen Craggs Index of Online Excellence has used a tried and tested methodology to assess corporate online effectiveness

Our reviews cover the entire web 'estate' – the main corporate site, country and business sites, mobile, social media, apps and other online channels

Independent, detailed and constantly evolving

The global Bowen Craggs Index of Online Excellence includes reviews of the web estates of the top 200 companies in the world by market capitalisation. The published Index is the top 30 companies from among this pool of 200, revealing the companies that are best at online comms in the world.

The Bowen Craggs Index of Online Excellence holds the most rigorous and detailed analysis of large web estates and has three aims:

  • Help organizations to know where to look for ideas
  • Pick out trends in online communications
  • Allow companies in the Index to see how they are performing against their peers

The Index is unlike other rankings for three reasons. First it is far more detailed – the scores are the top level of a report that has taken at least 60 hours to produce, and lives in a 2m word constantly-updated database. Second, it takes a broad approach. Rather than concentrating on a particular element, it looks at the jobs a web presence is asked to do, and sees how well it does them – this is what we call effectiveness. Third, our approach is ‘expert’ rather than ‘check box’:  our analysts – experienced in both business and the web – make judgments at every stage.

Bowen Craggs & Co advises companies and other organizations on their online presence. It does not build sites, but helps improve effectiveness with strategy, measurement and continuous improvement. We have worked for a number of companies in the Index though we use different analysts for consulting and reviewing. These projects have not affected the scoring: we have no interest in marking companies either up or down. Our reviews are open for examination by subscribers to our database.

What is the Index of Online Excellence?

The Bowen Craggs Index of Online Excellence is a unique guide to the best in internet corporate communications, across main web sites, social media, country sites, apps and other online channels. It’s not really about communications, it’s about marketing - but marketing the whole enterprise, not just the things or services it sells. The Index is a ranking – in fact it is many rankings – but the real aim is not to stimulate praise, blame or panic, but to show what should be done (and what should not be done) to make a website as effective as possible.

How do the Index of Online Excellence and the Explain Yourself Index differ?

We are often asked what the difference is between the Index of Online Excellence, which has been running since 2007, and the newer Explain Yourself Index, first published in February 2019. The Index of Online Excellence is a multi-use tool designed to help communications professionals run their online channels better, while the Explain Yourself Index is a detailed but focused call to action to 'explain themselves' - that is to tell the outside world how they, as enterprises, are behaving. While there is overlap between the indices, they are quite different; indeed, it is possible to do very well in one and not at all well in the other. The Index of Online Excellence covers the nitty-gritty elements of running websites - construction and usability; overall message and reputational positioning; and the detailed service given to jobseekers, investors, journalists and CSR professionals. It is global. The lighter, more compact Explain Yourself Index focuses on US companies. Explain Yourself takes three metrics from 26 metrics in the main Index (on company information, CSR/ESG reporting and 'building a reputation for responsibility'); then divides them into more granular sub-metrics; and adds extra elements about political engagement. We look at usability and design, but give them less attention. In summary, the Index of Online Excellence is for digital professionals who already know the importance of corporate online communications and want to do better. The Explain Yourself Index is also a tool for practitioners, but it is especially powerful as a way of persuading time-starved bosses and your internal stakeholders about the importance of investing money and time in corporate digital channels.

What’s the timetable for reviews?

As the database is constantly reviewed, there is no longer a particular deadline. The Index, as published, is a snapshot of our current data as we are reviewing sites on a rolling cycle. The Interactive Results Table provides a live view of our latest ranking.

Are the scores strictly comparable from year to year?

Almost, but not quite. We adjust the questions to take account of changes in the world of the web. For example over time we have given more emphasis to the mobile web. Our guidelines are regularly revised, with training to ensure that analysts are always up to date.

Is this an objective exercise?

We always consider the profile and needs of the companies being examined: in that sense our benchmarking is not objective, nor should it be. For the same reason, it is not a ‘tickbox’ exercise: every metric is judged by its relevance and its utility to the client, rather than ‘Is it there or is it not?’. The key questions are much more likely to be ‘How appropriate is it?’ and ‘How well is it done?’.

How are the companies selected?

Companies included are taken from the FT Global 500, ranked by market capitalisation. We conduct reviews of the top 200 companies from this list. These ‘quick’ reviews follow a less in-depth methodology than our full reviews, but allow us to establish a list of companies worth looking at in-depth. From this shortlist, we conduct full reviews to finalise our Index ranking of the top 30 companies. The subscription database that underlies the published Index, contains best practice from all 200 companies we have reviewed, even if they have not made the top 30 overall best performers.

What is the methodology?

Our starting point is that the Index has to be credible – for us that means a checkbox approach cannot be used, because companies vary so much in what they can and should be doing online. Bowen Craggs has developed a methodology to benchmark corporate sites in great detail. The metrics are divided into two groups: overall and specific. Specific metrics concentrate on how well the site serves different groups. Within the overall metrics, Construction covers navigation and coherence, Message looks at the visual and content messages the site transmits, while Contact covers both ease of making contact and use of FAQs. Some of the questions we ask are broad (Is the look and feel appropriate? Does the site tackle relevant controversial issues?) – here analysts have to make a judgement based on their knowledge of the company, industry and the web. Others are quite specific (How well are CSR management and measuring systems explained? What format are provided for quarterly results?).

Does the Index cover social media?

Social media is a crucial component of our methodology. We assess whether the company is making appropriate use of social media; in particular, whether it is coordinating its social media activities with those on the global website estate. Questions addressed include:   * Are different social media channels effectively and consistently targeted at appropriate audience groups?   * Are these channels regularly updated with appropriate and engaging content?   * How effectively are critics handled via each online channel?   * Is output across different social channels coordinated or fragmented?   * Are different channels appropriately tailored to different geographies?

Who does the reviews?

We have seven consultants and analysts working on the Index, all with a background in the web and business, and many have been working on the Index since it started, in 2007. Our reviewers are all experts on the corporate web and are trained to ask first, what should be provided, and only then to judge how well it is provided. ‘Appropriate ‘is the key word. We look at the entire presence, covering non-English content where relevant.

Do your commercial relationships affect the Index ranking?

Bowen Craggs has or has had commercial relationships with approximately half the companies in the list. We are confident that this does not in any way affect our judgment and scoring:   * Most of the companies involved are subscribers to our best practice database - we have not provided consultancy.   * Where we have provided it, we have been careful to maintain Chinese walls between consultants and reviewers.   * Because we publish our detailed analysis in the database, it would be very difficult to 'adjust' the scores without that being noticed.