FAQ & methodology

For over a decade the global Bowen Craggs Index of Online Excellence has used a tried and tested methodology to assess corporate online effectiveness

Our reviews cover the entire web 'estate' – the main corporate site, country and business sites, mobile, social media, apps and other online channels

Independent, detailed and constantly evolving

The global Bowen Craggs Index of Online Excellence includes reviews of the web estates of the top 200 companies in the world by market capitalisation. The published Index is the top 30 companies from among this pool of 200, revealing the companies that are best at online comms in the world.

The Bowen Craggs Index of Online Excellence holds the most rigorous and detailed analysis of large web estates and has three aims:

  • Help organizations to know where to look for ideas
  • Pick out trends in online communications
  • Allow companies in the Index to see how they are performing against their peers

The Index is unlike other rankings for three reasons. First it is far more detailed – the scores are the top level of a report that has taken at least 60 hours to produce, and lives in a 2m word constantly-updated database. Second, it takes a broad approach. Rather than concentrating on a particular element, it looks at the jobs a web presence is asked to do, and sees how well it does them – this is what we call effectiveness. Third, our approach is ‘expert’ rather than ‘check box’:  our analysts – experienced in both business and the web – make judgments at every stage.

Bowen Craggs & Co advises companies and other organizations on their online presence. It does not build sites, but helps improve effectiveness with strategy, measurement and continuous improvement. We have worked for a number of companies in the Index though we use different analysts for consulting and reviewing. These projects have not affected the scoring: we have no interest in marking companies either up or down. Our reviews are open for examination by subscribers to our database.

What is the Index?

The Bowen Craggs Index of Online Excellence is a unique guide to the best in internet corporate communications, across main web sites, social media, country sites, apps and other online channels. It’s not really about communications, it’s about marketing - but marketing the whole enterprise, not just the things or services it sells. The Index is a ranking – in fact it is many rankings – but the real aim is not to stimulate praise, blame or panic, but to show what should be done (and what should not be done) to make a website as effective as possible.

What’s the timetable for reviews?

As the database is constantly reviewed, there is no longer a particular deadline. The Index, as published, is a snapshot of our current data as we are reviewing sites on a rolling cycle. The Interactive Results Table provides a live view of our latest ranking.

Are the scores strictly comparable from year to year?

Almost, but not quite. We adjust the questions to take account of changes in the world of the web. For example over time we have given more emphasis to the mobile web. Our guidelines are regularly revised, with training to ensure that analysts are always up to date.

Is this an objective exercise?

We always consider the profile and needs of the companies being examined: in that sense our benchmarking is not objective, nor should it be. For the same reason, it is not a ‘tickbox’ exercise: every metric is judged by its relevance and its utility to the client, rather than ‘Is it there or is it not?’. The key questions are much more likely to be ‘How appropriate is it?’ and ‘How well is it done?’.

How are the companies selected?

Companies included are taken from the FT Global 500, ranked by market capitalisation. We conduct reviews of the top 200 companies from this list. These ‘quick’ reviews follow a less in-depth methodology than our full reviews, but allow us to establish a list of companies worth looking at in-depth. From this shortlist, we conduct full reviews to finalise our Index ranking of the top 30 companies. The subscription database that underlies the published Index, contains best practice from all 200 companies we have reviewed, even if they have not made the top 30 overall best performers.

What is the methodology?

Our starting point is that the Index has to be credible – for us that means a checkbox approach cannot be used, because companies vary so much in what they can and should be doing online. Bowen Craggs has developed a methodology to benchmark corporate sites in great detail. The metrics are divided into two groups: overall and specific. Specific metrics concentrate on how well the site serves different groups. Within the overall metrics, Construction covers navigation and coherence, Message looks at the visual and content messages the site transmits, while Contact covers both ease of making contact and use of FAQs. Some of the questions we ask are broad (Is the look and feel appropriate? Does the site tackle relevant controversial issues?) – here analysts have to make a judgement based on their knowledge of the company, industry and the web. Others are quite specific (How well are CSR management and measuring systems explained? What format are provided for quarterly results?).

Does the Index cover social media?

Social media is a crucial component of our methodology. We assess whether the company is making appropriate use of social media; in particular, whether it is coordinating its social media activities with those on the global website estate. Questions addressed include:   * Are different social media channels effectively and consistently targeted at appropriate audience groups?   * Are these channels regularly updated with appropriate and engaging content?   * How effectively are critics handled via each online channel?   * Is output across different social channels coordinated or fragmented?   * Are different channels appropriately tailored to different geographies?

Who does the reviews?

We have seven consultants and analysts working on the Index, all with a background in the web and business, and many have been working on the Index since it started, in 2007. Our reviewers are all experts on the corporate web and are trained to ask first, what should be provided, and only then to judge how well it is provided. ‘Appropriate ‘is the key word. We look at the entire presence, covering non-English content where relevant.

Do your commercial relationships affect the Index ranking?

Bowen Craggs has or has had commercial relationships with approximately half the companies in the list. We are confident that this does not in any way affect our judgment and scoring:   * Most of the companies involved are subscribers to our best practice database - we have not provided consultancy.   * Where we have provided it, we have been careful to maintain Chinese walls between consultants and reviewers.   * Because we publish our detailed analysis in the database, it would be very difficult to 'adjust' the scores without that being noticed.