FAQ & methodology

The Index of Online Excellence is widely regarded as the gold standard for evaluating digital corporate communications

The Index measures the entire corporate digital presence: global websites, social media, country sites, brand and business sites; across desktop and mobile devices

Independent, detailed and constantly evolving

Our methodology is tailored to the needs of corporate digital channels. Unlike most brand or news sites, corporate online estates are complex, deep and multi-functional. We put great emphasis on usability, but we also look carefully at messaging, and put ourselves into the heads of different groups of visitors to see how well they are served.

The methodology was created in 2007 but constantly evolves based on relevant new trends, insights from extensive audience research and our own expertise.

Audience research that we draw on is unparalleled – more than half a million visitor surveys conducted since 2011 exclusively on corporate websites; in addition to insights from our Google Analytics benchmarking group.

It is immune to fashion. Our reviewers look for the hard business case behind the latest trends and ask whether they are appropriate for large organizations.

There are no perfect scores, and criteria for ‘best’ gets better every year. This reflects the fact that standards continue to rise and the leading companies are moving forward all the time. For example, a score of 11 out of 12 for visual effectiveness in 2015 is completely different from an 11 out of 12 in 2020. The same is true of all other metrics (see FAQ, below, for more detail about the metrics).

The methodology is rigorous but not automated or a ‘check box’ exercise. We offer detailed editorial judgments on the quality and appropriateness of text and video content. Every metric is judged by its relevance to the client, rather than ‘is it there or is it not?’ We are as much experts on how large organizations are run and managed as we are on digital channels.

 

What are your metrics?

There are eight main metrics and 26 sub-metrics in the Index. The eight main metrics are divided into two groups: three overall and five specific. Overall metrics: 1. Construction: overall usability on desktop and smartphone, navigation between global, country and brand sites and search. 2. Message: strength of the global home page, visual design and brand consistency, and company information. 3. Contact: how well contact information is signposted, how appropriate it is and the quality of FAQs.   The five specific metrics evaluate how well companies are serving specific audiences on digital channels - CSR professionals, investors, journalists, jobseekers and customers. To arrive at scores, we answer a number of questions about the company's digital output at the sub-metric level. Some of the questions we ask are broad (Is the look and feel appropriate? Does the site tackle relevant controversial issues?) – here analysts have to make a judgement based on their knowledge of the company, industry and the web. Others relate to detail (How well are CSR management and measuring systems explained? What formats are provided for quarterly results?).

How are the companies selected?

Our starting pool is the world's largest 200 companies by market capitalisation. Major companies outside this group are also eligible for evaluation by demonstrating best practice in digital corporate communications. Our team of consultants evaluates the digital output of these companies on a 12-month rolling schedule of reviews, to determine whether they are likely to meet the maximum score threshold for inclusion in the top 30 ranking. Companies that look likely to meet the criteria are evaluated with full reviews in the Bowen Craggs subscription database, and this becomes the top 30. Our database also includes best practice reviews of the companies that do not meet the maximum score threshold, but demonstrate best practice in any specific areas, such as home page, press release service, or jobseeker innovation. Top 30 scores are finalised after peer review, in which our senior consultants calibrate scores aross companies to ensure that they are compatible. The 'Top 30' ranking for 2020 has 32 companies due to tie scores in 30th position.

Does the Index cover social media?

Social media is a crucial component of our methodology. We assess whether the company is making appropriate use of social media; in particular, whether it is coordinating its social media activities with those on the global website estate. Questions addressed include:   * Are different social media channels effectively and consistently targeted at appropriate audience groups?   * Are these channels regularly updated with appropriate and engaging content?   * How effectively are critics handled via each online channel?   * Is output across different social channels coordinated or fragmented?   * Are different channels appropriately tailored to different geographies?

Who does the reviews?

We have seven consultants and analysts working on the Index, all with a background in the web and business. Our reviewers are all experts on the corporate web and are trained to ask first, what should be provided, and only then to judge how well it is provided. ‘Appropriate ‘is the key word. We look at the entire presence, covering non-English content where relevant.

What’s the timetable for reviews?

As the database is constantly reviewed, there is no longer a particular deadline. The Index, as published, is a snapshot of our current data as we are reviewing sites on a rolling cycle. The Interactive Results Table provides a live view of our latest ranking.

Is this an objective exercise?

We always consider the profile and needs of the companies being examined: in that sense our benchmarking is not objective, nor should it be. For the same reason, it is not a ‘tickbox’ exercise: every metric is judged by its relevance and its utility to the client, rather than ‘Is it there or is it not?’. The key questions are much more likely to be ‘How appropriate is it?’ and ‘How well is it done?’.

Do your commercial relationships affect the Index ranking?

Bowen Craggs has or has had commercial relationships with approximately two thirds of the companies in the Index. We are confident that this does not affect our judgment and scoring.  We are completely transparent in our subscriber database about the reasoning behind all scores, and the full methodology is made available to subscribers. We evaluate all companies, whether clients or not, on the same Index criteria. We conduct peer reviews to calibrate our consultants’ individual judgments about clients and non-client scores to ensure that the scoring is consistent and comparable across companies. We do not build websites or run social media channels, and have no vested interest. Our clients value our analysis for its unvarnished honesty about strengths, weaknesses and recommendations for improvement. The methodology is based on audience research and analytics as well as our consultants’ experience and expertise, so it captures real effectiveness. We are able to observe close correlations between the empirical satisfaction results shown by these external measurements and our own reviews.

How do the Index of Online Excellence and the Explain Yourself Index differ?

We are often asked what the difference is between the Index of Online Excellence, which has been running since 2007, and the newer Explain Yourself Index, first published in February 2019. The Index of Online Excellence is a multi-use tool designed to help communications professionals run their online channels better, while the Explain Yourself Index is a detailed but focused call to action to 'explain themselves' - that is to tell the outside world how they, as enterprises, are behaving. While there is overlap between the indices, they are quite different; indeed, it is possible to do very well in one and not at all well in the other. The Index of Online Excellence covers the nitty-gritty elements of running websites - construction and usability; overall message and reputational positioning; and the detailed service given to jobseekers, investors, journalists and CSR professionals. It is global. The lighter, more compact Explain Yourself Index focuses on US companies. Explain Yourself takes three metrics from 26 metrics in the main Index (on company information, CSR/ESG reporting and 'building a reputation for responsibility'); then divides them into more granular sub-metrics; and adds extra elements about political engagement. We look at usability and design, but give them less attention. In summary, the Index of Online Excellence is for digital professionals who already know the importance of corporate online communications and want to do better. The Explain Yourself Index is also a tool for practitioners, but it is especially powerful as a way of persuading time-starved bosses and your internal stakeholders about the importance of investing money and time in corporate digital channels.