Article Text

Download PDFPDF

16 A friendly accessible description of the ‘L-test’ – measuring (dis)information in incomplete incident reporting
  1. Keith Reid
  1. CNTW NHS Foundation Trust; Northumbria University


Objective Incomplete incident reporting is concerning. England’s Mental Health Units Use of Force Act 2018 (Seni’s Law), responding to deaths and incomplete reporting, will mandate central restraint reporting per-person including ethnicity. ‘L’ is a proposed test for disinformation, i.e. ‘false surprise’ regarding true reports. Information, or ‘surprise’, is measurable as H = -log(p) ‘bits’, as defined by Shannon (1948).

The author explains his conjectured ‘L-test’, in a friendly accessible way. It is generalisable from incomplete restraint reports to other incomplete centralised safety reports. L is increased if complete reports seem falsely surprising consequent to noise from incomplete reports.

Methods Incident registers and minimum data sets are ubiquitous. Each hospital reports diverse incidents alongside measures of size or need. Notionally then data may include a) restraints; b) detentions … m) bed days n) injuries.

L postulates that each hospitals’ report of {a, b, ... m, n}, implies signals of ratios (log a/log b), (log a/log m)… which each can be received from the set of reports and combined to estimate e.g. a typical ratio of safety events per-patient per-month. Omissions are noise.


  1. Split the ordered list of complete report estimates into alternate halves E ‘even’ and O ‘odd’.

  2. Derive a probability p(E~O) that E and O are similar using Mann-Whitney U test, approaching p(E~O) =1.0 for large similar E and O. The test tolerates non-normally distributed estimates.

  3. Calculate h(E~O) information as -log(p(E~O)), approaching zero as O and E seem unsurprisingly similar.

  4. Construct a noisy odd group ‘NO’ made of O mixed with estimates from incomplete reporters.

  5. Calculate h(E~NO) information, approaching high values as incomplete reporters make E seem falsely surprising.

L is the proportional increase in h(E~O) due to noise: h(E~NO) – h(E~O)

L = _____________________ h(E~O)

Results Estimate signals support funnel plotting, scatter plotting, and coefficients of determination (R^2) as a measure of correlation.

The author will show that omissions (allowing for size and Poisson distribution) can be obvious on visual inspection of funnel and scatter plots and aid categorisation.

Where the estimates follow a normal distribution among reasonably complete reporters, this can be used to plot a typical ratio and infer incidents, with confidence intervals, even in null reporters, from measures of size and need.

Funnel plots from safety reports may have interesting properties such as innate asymmetry; they may reflect institutional-social processes such as regulation and closure as much as academic processes such as purported ‘publication’ bias.

H varies with the effect of incomplete reports and has other desirable features such as being zero when there are no omissions.

Conclusion Omitted reports have a measurable effect upon the standing of complete reports.

The author responds to this observation quantitatively, showing the roots and reasoning behind their conjectured ‘L-test’, in a friendly accessible way, with reference to papers under submission, other public data, and toy data sets.

In summary, L can tell investigators which incomplete reports skew the overall picture most.

In a context of restricted resource, regulatory efforts could concentrate on the omissions which have the most distorting effect – the biggest L score.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.