How is Progress calculated?

Progress is a measure between 0 and 100 of how much of a given problem exists, where 0% means “the problem is everywhere” and 100% means “there is no problem”.

The way Progress is calculated depends on the specific Check.

Issue checks

Around 95% of Checks report a list of issues with individual pages, PDFs, sections etc. These use a 50:50 split of the following two calculations:

1. Which % of pages have the issue. For example on a 100-page website where 50 pages have a spelling error, the percentage will be 50%.
2. The number of open issues are there, divided by the number of pages, as a percentage of an expected ‘worst case’. For example, spelling might define the ‘worst case’ as 3 spelling errors per page. On a website where there are 200 spelling errors and 100 pages (or an average of 2 errors per page), and our defined ‘worst case’ is 3 errors per page, this would give us a total of 66% which we subtract from 100% to get a progress measure of 33%.

The ‘worst case’ varies per Check, for example a broken link is less common and more severe than a grammar issue.

Other Checks weigh specific types of issue differently, for example an invisible spelling error might count as 50% of a visible spelling error, or a misspelling inside a PDF might be weighted as less significant to Progress than a misspelling on a web page.

Scale Checks

The remaining Checks have custom progress calculations based on a sliding scale metric.

In these cases, we tend to measure progress against that sliding scale. For example, content readability has an ‘ideal’ reading age of 9 and a ‘worst case’ of 22. The Progress of readability is the average reading age, between these two extremes.

For example, if a website had an average reading age of 10, we would calculate progress as:

progress = 100 - ((average - lowest) / (highest - lowest) * 100) 

In this case:

progress = 100 - ((10 - 9) / (22 - 9) * 100) = 92.3

The result is subtracted from 100, because for reading age we’re aiming for the lowest age.

Other Checks would define a different range, depending on what they’re testing for.