Based on the test results in a Proficiency Test (PT), each participant receives an indication of its performance. I.I.S. uses the z-score as the performance indicator, which gives an indication of the laboratories competence. Prior to calculation of the labs z-scores, certain statistical tests have to be applied. 

The statistical procedure of the Institute is based on international accepted documents (e.g. ISO 5725, ISO 43), with a number of sequential steps: 

• Detection of obvious errors
• Check on normal distribution of the test results
• Detection and removal of statistical deviating and erroneous results
• Calculation of the summary parameters
• Calculation of the performance indicators (z-scores)
• Graphical presentations of all results

The notification of deviating results is done shortly after the closing date for sending in the results, normally within 2 days after deadline. 

Any corrected results replace the erroneous ones. In the final report the results sent in at first are mentioned as a remark. 

In case of anormality, the explicit warning to interpret the conclusions with caution is made in the final report. 

The detection of statistically deviating results is given thorough attention. The Institute uses 3 different numerical tests for the detection of outliers and stragglers: Cochrans’ test, Dixons test and Grubbs (single and paired) test. 

Erroneous and all  statistically deviating results are removed before summary parameters are calculated. 

If possible, the calculated repeatability and reproducibility of the group of participants are compared with the values given in the corresponding international accepted (ASTM, IP, ISO,DIN) standard test method. 

For each test the z-score of lab i is calculated as:

z_i = (x_i - X ) / s

in which 

x_i is the result of lab i for that specific test 

X is the assigned value, an estimate for the ‘true value’. I.I.S. tries to use in its Proficiency Tests real samples. This garantees a close resemblance between the PT-test items and the samples the participating laboratories normally analyse. The items do not have a known composition (e.g. concentrations or amounts). The mean of all valid lab results is used as the assigned value. 

s  is the target standard deviation (of the reproducibility). This value is derived from the corresponding, internationally accepted test method, e.g. ASTM, IP, ISO, DIN or another accepted standard in the industry.

Thus, the z-score calculation of I.I.S. results in a simple, straight forward comparison of a lab results with the reproducibility stated in the corresponding international accepted (ASTM, IP, ISO, DIN) test method.

The interpretation of the z-scores is easy: 
|z| < 2 :  the participants result differs from the ‘true value’ less than 2 times the reproducibility standard deviation of the corresponding internationally accepted test method. Such a result is mentioned: good or satisfactory (will occur with normally distributed results in about 95% of all cases) 
2< |z| < 3 : questionable (will occur with normally distributed results in about 5% of all cases) 
|z| > 3 : unsatisfactory (will occur with normally distributed results in about 0.3% of all cases) 

Single sample: a Gauss plot: 
On the Y-axis the sorted test results are plotted. The corresponding lab numbers are on the X-axis. Valid results are presented by a triangle, rejected results as an *. The mean and the reproducibility limits of the corresponding international accepted (ASTM, IP, ...) test method are presented by (dotted) lines. 

 

Two samples: a two-sample or Youden plot: 
On the X-axis the result for one sample is plotted and on y-axis the result for the other sample. Valid results are presented by a triangle, rejected results as an *. The means and repeatability and reproducibility limits are all presented by (dotted) lines.

A Youden plot visualises systematic errors as well as random errors. If the results from the laboratories vary entirely because of random errors, the results will fall randomly around the average; approximately equal number of points will be present in each of the four quadrants. If systematic errors are the main cause of the variation, a predominance of points will be visible in the top right and the lower left quadrants of the plot.