FIT - 007 Evaluation of Quantitative Faecal Immuncohemical Tests For Haemoglobin
FIT - 007 Evaluation of Quantitative Faecal Immuncohemical Tests For Haemoglobin
FIT - 007 Evaluation of Quantitative Faecal Immuncohemical Tests For Haemoglobin
QUANTITATIVE FAECAL
IMMUNOCHEMICAL TESTS FOR
HAEMOGLOBIN
Date of original publication: 20 November 2013
Date of revision: 8 December 2014
Affiliations:
Royal Surrey County Hospital, Egerton Road, Guildford, Surrey GU2 7XX
NHS Cancer Screening Programmes, Old Fulwood Road, Sheffield S10 3TH
Contact address:
PAGE
SUMMARY ...................................................................... 6
THE TEST.................................................................. 6
SCOPE OF USE ............................................................ 6
EVALUATION SYSTEMS .................................................. 6
EVALUATION OUTCOMES: OPERATIONAL PERFORMANCE/
CONSIDERATIONS ........................................................ 8
FIT UNITS .............................................................. 10
INTRODUCTION ............................................................. 11
PRODUCT CHOICE .......................................................... 12
PRODUCT DESCRIPTION COMPARISON TABLE ......................... 13
STUDIES UNDERTAKEN .................................................... 17
EVALUATION METHODS.................................................... 18
PREPARATION OF TEST SAMPLES ................................... 18
ANALYTICAL SENSITIVITY ............................................ 19
CARRYOVER............................................................. 19
IMPRECISION ........................................................... 19
PRECISION PROFILE ................................................... 20
LINEARITY .............................................................. 20
HOOK/PROZONE EFFECT ............................................. 21
STABILITY ............................................................... 21
TECHNICAL PERFORMANCE ............................................... 22
ANALYTICAL SENSITIVITY ............................................ 22
CARRYOVER............................................................. 22
IMPRECISION ........................................................... 23
LINEARITY .............................................................. 27
PRECISION PROFILE ................................................... 29
HOOK/PROZONE EFFECT ............................................. 31
STABILITY ............................................................... 32
OPERATIONAL PERFORMANCE ............................................ 34
COLLECTION DEVICES................................................. 34
UNIT CONVERSION VALUES .......................................... 36
PARTICIPANT SURVEY ................................................. 38
ANALYTICAL SYSTEM USER EVALUATION .......................... 41
EASE-OF-USE ........................................................... 41
CONSUMABLES ......................................................... 47
MAINTENANCE AND SERVICING ..................................... 50
OPERATIONAL CONSIDERATIONS ........................................ 53
ANALYSER REQUIREMENTS ........................................... 53
STAFF .................................................................... 53
ECONOMIC CONSIDERATIONS ....................................... 53
ENVIRONMENTAL CONSIDERATIONS ............................... 54
ACKNOWLEDGEMENTS ..................................................... 55
BIBLIOGRAPHY .............................................................. 56
APPENDICES ................................................................. 57
~4~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
TABLES Page
SUMMARY TABLE: EVALUATION OUTCOMES: TECHNICAL PERFORMANCE .......................................................................... i
TABLE 1: DATA SUPPLIED BY THE MANUFACTURERS THAT ENABLE CONVERSION FROM ng Hb/mL BUFFER
TO µg Hb/g FAECES ............................................................................................................ 10
TABLE 2: SUMMARY OF ANALYSER DESCRIPTIONS ....................................................................................... 13
TABLE 3: MEASURED LOWER LIMIT OF DETECTION FOR EACH ANALYSER. ............................................................ 22
TABLE 4: CARRYOVER ....................................................................................................................... 22
TABLE 5: IMPRECISION OF ANALYSERS .................................................................................................... 23
TABLE 6: SAME-DAY REPEATABILITY ...................................................................................................... 26
TABLE 7: INTRA-DAY REPEATABILITY ...................................................................................................... 26
TABLE 8: ANALYSIS OF THE HOOK/PROZONE EFFECT DETECTED FOR EACH ANALYSER ............................................. 31
TABLE 9: MEASURED STABILITY OF DILUTED Hb AND FAECAL SAMPLES SPIKED WITH Hb......................................... 33
TABLE 10: CONVERSION FACTORS PROVIDED BY MANUFACTURERS AND DETERMINED BY GMEC. ................................. 36
TABLE 11: EASE-OF-USE OF ANALYSERS. ................................................................................................... 43
TABLE 12: CONSUMABLES AND THE STORAGE CONDITIONS REQUIRED FOR EACH ANALYSER ....................................... 48
TABLE 13: MAINTENANCE AND SERVICING OF THE ANALYSERS. ......................................................................... 51
TABLE 14: NUMBER OF ANALYSERS REQUIRED TO COMPLETE AN AVERAGE DAILY WORKLOAD OF 5,000 SAMPLES .............. 53
FIGURES
FIGURE 1: LINEARITY: HM-JACKarc ...................................................................................................... 27
FIGURE 2: LINEARITY: NS-PLUS C15 .................................................................................................... 27
FIGURE 3: LINEARITY: OC-SENSOR DIANA ........................................................................................... 28
FIGURE 4: LINEARITY: FOB GOLD/BIOMAJESTY ......................................................................................... 28
FIGURE 5: PRECISION PROFILES OF ALL ANALYSERS FROM 0 – 400 µg Hb/g FAECES.. ........................................... 30
FIGURE 6: PRECISION PROFILES OF ALL ANALYSERS FROM 0 – 100 µg Hb/g FAECES. ............................................ 30
FIGURE 7: HM-JACKARC COLLECTION PROBE ............................................................................................ 34
FIGURE 8: NS-PLUS COLLECTION PROBE ................................................................................................. 34
FIGURE 9: OC-AUTO SAMPLING BOTTLE 3 PROBE ...................................................................................... 35
FIGURE 10: SENTINEL FOB GOLD COLLECTION PROBE.................................................................................... 35
FIGURE 11: PRECISION PROFILES USING GMEC-MEASURED /MANUFACTURERS’ CONVERSION FACTORS ........................... 37
FIGURE 12: PARTICIPANT SURVEY: HOW EASY WAS IT TO OPEN THE DEVICE? ........................................................ 39
FIGURE 13: PARTICIPANT SURVEY: HOW EASY WAS IT TO COLLECT THE SAMPLE? .................................................... 39
FIGURE 14: PARTICIPANT SURVEY: HOW EASY WAS IT TO REPLACE THE SAMPLE AND SAMPLE PROBE
WITHIN THE COLLECTION TUBE? ............................................................................................... 39
FIGURE 15: PARTICIPANT SURVEY: HOW EASY WAS IT TO CLOSE THE DEVICE? ....................................................... 40
FIGURE 16: PARTICIPANT SURVEY: WHICH DEVICES ARE YOU HAPPY TO USE? ........................................................ 40
FIGURE 17: PARTICIPANT SURVEY: WHICH IS YOUR PREFERRED DEVICE? ............................................................. 40
APPENDICES
APPENDIX 1: INTEGRATED SPHERE TURBIDIMETRY .......................................................................................... 57
APPENDIX 2: IMPRECISION DATA FOR BUFFER SPIKED WITH HAEMOGLOBIN (µg Hb/g FAECES). .................................... 58
APPENDIX 3: IMPRECISION DATA FOR FAECAL SAMPLES SPIKED WITH HAEMOGLOBIN (µg Hb/g FAECES) .......................... 59
APPENDIX 4: PRECISION PROFILE DATA FOR BUFFER SAMPLES SPIKED WITH HAEMOGLOBIN (µg Hb/g FAECES) .................. 59
APPENDIX 5: HM-JACKarc STABILITY DATA ................................................................................................ 60
APPENDIX 6: NS-PLUS C15 STABILITY DATA ............................................................................................... 61
APPENDIX 7: OC-SENSOR DIANA STABILITY DATA ...................................................................................... 62
APPENDIX 8: FOB GOLD/BIOMAJESTY STABILITY DATA.. ................................................................................. 63
APPENDIX 9: NUMBER OF ANALYSERS REQUIRED FOR AN AVERAGE DAILY WORKLOAD OF 5,000 SAMPLES .......................... 64
APPENDIX 10: NUMBER OF ANALYSERS REQUIRED FOR AN AVERAGE DAILY WORKLOAD OF 9,000 SAMPLES .......................... 65
APPENDIX 11: RESPONSES FROM THE FIT COMPANIES ....................................................................................... 66
KYOWA MEDEX ................................................................................................................... 66
ALERE/ALFRESAPHARMA ........................................................................................................ 68
EIKEN .............................................................................................................................. 70
SENTINEL .......................................................................................................................... 71
APPENDIX 12: REVISIONS MADE TO THE ORIGINAL REPORT (DECEMBER 2014) ......................................................... 73
PLEASE NOTE:
This is a revised version of the report that includes revised summary text and a new Summary Table (pages i-iii)
to assist direct comparison of the analysers’ technical performance. All revisions made to the original report are
described in Appendix 12. Page and original table numbering are unchanged.
~5~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
The evaluation
This evaluation was commissioned by the NHS Bowel Cancer Screening Programme (BCSP)
in England.
Scope of use
The 2010 European guidelines for quality assurance in colorectal cancer screening and
diagnosis recommended the adoption of quantitative FIT as the primary screening modality
for colorectal cancer [2,3]. The evidence presented in these guidelines demonstrated that
FIT was analytically more sensitive and specific than the guaiac faecal occult blood test
(gFOBT) and in population screening it showed greater clinical effectiveness in the detection
of cancer and advanced adenoma. In recommending the adoption of FIT, the guidelines
state that FIT have other significant practical benefits for population screening. The merits
of FIT have been promoted by the World Endoscopy Organization (WEO) Colorectal Cancer
Screening Committee’s Expert Working Group (EWG) on ‘FIT for Screening’
(http://www.worldendo.org/weo-crcsc-expert-working-group-fit-for-screening.html); FIT
enables objective and automated measurement, a single FIT device requiring a single faecal
sample is acceptable for screening, it uses a simpler and more attractive sampling technique
than that used by the gFOBT, it allows the positivity rate to be adjusted to meet local
circumstances and the Hb concentration can be incorporated into a multivariate risk score to
enable a higher positive predictive value for cancer or advanced adenoma.
Evaluation systems
Evaluation of quantitative FIT products commenced in November 2012, at which time four
products met essential criteria identified by BCSP. The Guildford Medical Device Evaluation
Centre (GMEC) team commenced evaluation of the following four products; the HM-JACKarc,
the NS-PLUS C15, the OC-SENSOR DIANA and the Sentinel FOB Gold NG for Hb analysed on
the BioMajesty.
The collection devices and analysers were recommended and provided by the
manufacturers. The analysers were installed into the GMEC research laboratory at the
University of Surrey by the suppliers and training was provided to two members of the
GMEC team. Cascade training was then used to train a third member of the team. The
practical evaluation work took place between December 2012 and August 2013.
~6~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
The evaluation used both faecal and aqueous Hb samples. Faecal samples had human blood
added (spiked) at specified Hb concentrations, as did the aqueous solutions that were
prepared in sample collection buffer. These two preparations were used to assess analytical
sensitivity, carryover, imprecision, precision profiles, linearity and Hb stability in the sample
collection devices (stability defined for this evaluation as a fall in Hb concentrations to <50%
of initial concentrations). Questionnaire surveys were performed to assess user preferences
for the four collection devices. Existing laboratory users of the evaluated analytical systems
were contacted to gather information and pool their experience.
~7~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
The HM-JACKarc collection tube has flattened sides, which make it easy to write on,
although it proved the most difficult device to use. A paper tab connects the lid to the
collection tube and this is a valuable feature that provides a useful check to the receiving
laboratory that the device has been opened and used. The device collects only 2 mg of
faeces, which is the smallest amount for the devices evaluated, and some participants in the
questionnaire survey thought that they should sample the faeces multiple times to ensure
that it was adequate to give a reliable result.
NS-PLUS C15
The NS-PLUS collection device was very easy to use. The lid has an easy-to-grip flat surface,
and there is an audible ‘click’ on closure. The blue coloured collection buffer disguises the
addition of faecal matter, and changes to green once a sample has been added. This is a
unique feature of this system and provides the laboratory with a means of checking that the
sample collection device has been used.
The analyser is a small bench-top automated clinical chemistry analyser and therefore has a
large capacity for a range of reagents. Unlike the other systems evaluated, the NS-PLUS C15
uses antibodies bound to colloidal gold particles.
The analyser proved very easy to use, and the software was very easy to follow. Due to the
use of reusable cuvettes, more water and wash solution are required than for the HM-
JACKarc, and more liquid waste is produced. The NS-PLUS C15 can be attached to mains
drainage to avoid frequent manual emptying of the waste container. This reduces the
amount of plastic clinical waste produced.
OC-SENSOR DIANA
The ‘OC-Auto Sampling Bottle 3’ collection device is well-designed with an easy-to-open top
and the flat surfaces of the tube make it easy to write on. The device incorporates a small
filter that removes faecal particulate matter from a sample before analysis, a feature that
reduces the possibility of miss-sampling. The tube has a narrow neck that, whilst it might
mitigate against collection of excessively large samples, does make replacement of the
sample probe after faeces collection more difficult (an observation made by participants in
the questionnaire survey). It is important to note that in December 2012 the buffer used in
this collection device was changed to a buffer that increased sample stability; this significant
change was not marked by a change in product designation.
~8~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
The OC-SENSOR DIANA is a small bench-top analyser dedicated to FIT analysis that uses
latex-coated beads in a turbidimetric assay. Whilst the analyser is easy to use, some of the
menu systems are complex and some procedures proved more difficult to navigate than on
the other analysers evaluated. The OC-SENSOR DIANA, as with the NS-PLUS C15, uses
reusable cuvettes between each test, which reduces the consumption of plastic consumables
but increases the consumption of wash solution and produces large volumes of liquid waste.
The OC-SENSOR DIANA can be attached to mains drainage to avoid frequent manual
emptying of the waste container.
SENTINEL/BIOMAJESTY
The FOB Gold NG collection device is produced by Sentinel and uses a conventional blood
sample-sized tube, which makes FOB Gold suitable for analysis using a wide range of clinical
chemistry analysers. The collection device is different from the others and the curved
surface makes writing a name or date on the device label more difficult than on flat
surfaces. The device has a larger probe than the other devices making it easier to see when
a sample has been collected and easy to reinsert after faecal sampling. The device has a
second screw cap at the base of the tube (at the opposite end to the sample probe). The
cap is removed before analysis but in the tube’s current design the cap could be removed
mistakenly by a screening participant, resulting in total or partial loss of the sample and
collection buffer and rendering the sample unsuitable for analysis.
The BioMajesty analyser was provided by Sysmex UK Ltd, Milton Keynes, for FOB Gold
analysis. It is a conventional floor-standing fast-throughput automated clinical chemistry
analyser. It requires a large volume of purified water and therefore is suitable for use only in
a well-equipped laboratory. The Sentinel FOB Gold reagents are latex-bound polyclonal
antibodies used for conventional turbidimetric analysis. The BioMajesty is a multi-analyte
analyser, much larger and necessarily more complex than the other analysers evaluated.
The analyser software is complex beyond what is necessary to meet the needs of just FIT
analysis and more appropriate for situations when FIT is one test amongst a repertoire of 20
or 30 others; the analyser was not evaluated in that context. The throughput of the analyser
with reusable cuvettes requires two cycles of washing each day (25-30 minutes each), which
uses a substantial volume of deionised water.
~9~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
Measured lower limit of detection Measured lower limit of detection Measured lower limit of detection Measured lower limit of detection
met manufacturer’s claims. Detects met manufacturer’s claims. Detects met manufacturer’s claims but does met manufacturer’s claims. Detects
very low Hb concentrations. very low Hb concentrations. not detect Hb at concentrations as very low Hb concentrations.
low as that seen with the other
analysers.
Very little carryover of sample Very little carryover of sample Very little carryover of sample Very little carryover of sample
between samples with a high and between samples with a high and between samples with a high and between samples with a high and
low concentration. low concentration. low concentration. low concentration.
Repeatability of aqueous results was Repeatability of results and within- Positive bias of results prevented a Positive bias of results prevented a
shown to be consistent with laboratory imprecision of the simple comparison of imprecision simple comparison of imprecision
manufacturer’s claims, although analyser not as good as those with that claimed by the with that claimed by the
there was no within-laboratory claimed by the manufacturer for manufacturer and the Diana had the manufacturer although it had good
imprecision data provided by the both buffer and faecal material. At most imprecise method of the imprecision at lower concentrations,
manufacturer. Data from one faecal lower concentrations (around analysers evaluated. but imprecision of measurement
sample consistent with claims, whilst possible cut-off values) imprecision increased with increasing Hb
data from other sample not was good, however at higher concentration.
consistent. concentrations measurement more
imprecise.
~i~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
Slight positive bias of Hb Results closer to the expected Hb Results showed a positive bias and Results showed a positive bias and
concentration in aqueous solutions. concentrations than was seen with gave results higher than expected. gave results higher than expected.
the other products evaluated.
Good linearity (quoted measurement Good linearity across the quoted Good linearity (quoted measurement Not linear, particularly above 120 µg
range 7-400 µg Hb/g faeces); it has measurement range (3.8-228 µg range 10-200 µg Hb/g faeces). Hb/g faeces (quoted measurement
no system for diluting samples that Hb/g faeces); there is a facility for Dilution semi-automated (manual range 2.55-153 µg Hb/g faeces).
have a high Hb concentration. automatic dilution of samples with identification of ‘over-range’ samples The instrument can dilute high
high Hb concentrations extending and manual replacement of the samples automatically but, even
the dilution range to >500 µg Hb/g sample tube on the analyser). when this facility was enabled, the
faeces. Dilution extends the range beyond assay remained markedly non-linear
the undiluted upper limit to at high concentrations.
50,000 µg Hb/g faeces.
Precision profile (variability of results across the claimed measurement range for aqueous samples)
Good precision through the Good precision through the Good precision through the Good precision but results above 100
measurement range. measurement range. measurement range. µg Hb/g faeces were inaccurate.
~ ii ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
Hook/prozone effect (potential for erroneously low results at exceptionally high concentrations for aqueous samples)
No evidence of hook/prozone effect. Evidence of the hook/prozone effect No evidence of hook/prozone effect; Evidence of a hook/prozone effect
(two cases of false negative results all results were reported as ‘Over (some very high concentrations of
in very highly concentrated Range’. Hb reported as ‘weakly positive’);
samples); whilst only at high whilst only at high concentrations,
concentrations, there is potential for there is potential for misleading
misleading results. results.
Stability (of faecal and aqueous samples in the collection devices at -21˚C, 6˚C, 20˚C, 35˚C)
Consistent with manufacturer’s Consistent with manufacturer’s Consistent with manufacturer’s Consistent with manufacturer’s
claims (stability 120 days at 4˚C and claims (stability 7 days at 2-8˚C and claims (stability 14 days at 2-10˚C claims (stability 14 days at 2-8˚C
14 days at 25˚C); at a sustained 3 days at 18-25˚C); at a sustained and 7 days at room temp); at a and 7 days at 15-30˚C); at a
temperature of 35˚C the stability of temperature of 35˚C the stability of sustained temperature of 35˚C the sustained temperature of 35˚C the
faecal samples was less than 3 days. faecal samples was less than 3 days. stability of faecal samples was less stability of faecal samples was less
than 2 days. than 9 days.
~ iii ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
SUMMARY
FIT Units
Table 1: Data supplied by the manufacturers that enable conversion from ng Hb/mL buffer
to µg Hb/g faeces.
~ 10 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
INTRODUCTION
The NHS Bowel Cancer Screening Programme (BCSP) in England is preparing to replace
guaiac faecal occult blood tests (gFOBT) with quantitative faecal immunochemical tests for
haemoglobin (FIT), commencing with a 6-month pilot from March 2014. As well as being
easier to use than gFOBT [4], quantitative FIT provide a numerical haemoglobin (Hb)
concentration and an opportunity for fast objective automated analysis. FIT provide the
opportunity for the faecal Hb concentration to contribute to a multivariate measure of
colorectal cancer (CRC) risk with further enhancement to its positive predictive value. The
NHS BCSP FIT working group has agreed a software specification with the Health and Social
Care Information Centre which makes provision for the implementation of new CRC risk
algorithms as they are developed by the screening research community. The working group
will finalise the organisational model, develop new programme literature, monitor and
evaluate the pilot and progress analytical arrangements necessary to support FIT system
procurement. The Guildford Medical Device Evaluation Centre (GMEC) has undertaken three
previous evaluations for the NHS BCSP and was commissioned again by the Programme to
provide a detailed evaluation to support the FIT pilot, procurement and rollout. GMEC has a
close association with the BCSP Southern Hub, one of the five Hubs that support the BCSP in
England. The Southern Hub typically receives between 2,000 and 6,000 samples a day, and
must be able to analyse up to 9,000 samples a day in exceptional circumstances.
GMEC had the remit to evaluate all FIT systems that purport to be suitable for automated,
quantitative, population-based colorectal cancer screening using FIT analysis. The faecal
sample collection systems need to be suitable for unaided home faecal sample collection,
meet EU requirements for the mailing of human tissues by the Royal Mail (UN3373 and
P650) and have adequate sample stability. The analytical instrumentation should be suitable
for reliable analysis of between 2,000 and 9,000 faecal samples a day, although workloads
may be split between several analysers if necessary.
~ 11 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
PRODUCT CHOICE
Following a review of commercially available FIT, conducted under the auspices of the WEO
EWG, four FIT products were identified by GMEC as potentially suitable for use in the NHS
BCSP in England:
HM-JACKarc, Kyowa Medex Co Ltd, Japan
NS-PLUS C15 Hb, Alfresa Pharma Corp, Japan
OC-SENSOR DIANA, Eiken Chemical Co. Ltd, Japan
FOB Gold NG, Sentinel CH. SpA, Italy, analysed on BioMajesty, Jeol, Japan.
These were the only products that were identified in October 2012 as able to provide
laboratory-based, automated, quantitative analysis and a faecal collection device suitable for
home use and postal transport. All are CE-marked with respect to the In Vitro
Diagnostic Medical Devices Directive (98/79/EC) [5]. Table 2 describes the analyser
specifications.
~ 12 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
PRODUCT DESCRIPTION COMPARISON TABLE
Manufacturer Kyowa Medex (Japan) Alfresa (Japan) Eiken (Japan) Jeol (Japan)
UK supplier Alpha Laboratories Ltd Alere Ltd Mast Diagnostics Division Sysmex UK Ltd
World launch February 2010 December 2004 2007 2010
UK launch October 2013 January 2007 (CE Mark) 2007 European distributor 2010
World launch of assay February 2010 December 2004 2007 (updated buffer 2012) November 2003
Analyser information
Type of analyser Bench top Bench top Bench top Floor standing
Analyser size (mm) WxDxH 500 x 600 x 610 435 × 430 × 580 630 x 560 x 560 1220 x 850 x 1108
Space for accessories (mm) 300 x 250 450 x 350 x 350 for 300 both sides of analyser 740 x 730 x 1108
deionised water, drain and deionised water, drain and
wash solution bottles wash solution bottles
Weight of analyser 56 kg 43 kg 60 kg 450 kg
Power supply-sockets 1 3 (analyser, PC and display) 1 4
Water requirements N/A 2.4 L/hr 1.1 L/hr 20 L/hr
Wash requirements 5 L /1,000 tests 0.4 L/hr 0.5 L/hr Detergents
Waste Solid waste: non-recyclable Liquid waste: gravity drains Liquid waste: pumped into Liquid waste: 20.2 L/hr,
cuvettes and cuvette holders to 10 L collection bottle. Can 10 L collection bottle. Can be pumped directly into mains
Liquid waste: gravity drains be fed directly into mains fed directly into mains drains drains
to 5 L collection bottle drains
~ 13 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
PRODUCT DESCRIPTION COMPARISON TABLE
Analysis information
Method Polyclonal antibodies. Latex Polyclonal antibodies. Polyclonal antibodies. Latex Polyclonal antibodies. Latex
immunoturbidimetry with Colloidal gold immunoturbidimetry immunoturbidimetry
detection by integrated immunoturbidimetry
sphere turbidimetry1
Calibration traceable to In-house reference Hb Not known WHO BCR-522
Reference material Human Hb (commercial Human Hb (commercial Accuglobin BCR-522
source) source)
Reference method Cyanmethaemoglobin Cyanmethaemoglobin Cyanmethaemoglobin Cyanmethaemoglobin
Calibration range 25-400 ng/mL 0-1200 ng/mL 50-1,000 ng/mL 0-800 ng/mL
Number of QCs Two (high and low) Three (high, low and -ve) Two (high and low) Two (high and low)
Concentration range of QCs 20-30 ng/mL (low) 80-150 ng/mL (low) 111-160 ng/mL (low) 64-87 ng/mL (low)
80-120 ng/mL (high) 200-350 ng/mL (high) 379-513 ng/mL (high) 240-360 ng/mL (high)
Reported measurement 7-400 3.8-228 (without dilution) 10-200 (without dilution) 2.55-153 (without dilution)
range (µg Hb/g faeces)
1
Integrated sphere turbidimetry – see Appendix 1.
~ 14 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
PRODUCT DESCRIPTION COMPARISON TABLE
~ 15 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
PRODUCT DESCRIPTION COMPARISON TABLE
Sampling device design Round stick with two dimples Round grooved stick Round grooved stick Round grooved stick
Device closing system Twist and click Twist and click Push and click Twist
Product support
Finance options Subject to local distributor Available Available Subject to local distributor
Warranty 12 mths for analyser, service Not available for 12 mths 12 mths
parts from the day of NS-Plus C15
purchase
Maintenance contract Subject to local distributor Yes Subject to agreed Terms and Subject to local distributor
Conditions
~ 16 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
STUDIES UNDERTAKEN
The following studies were undertaken to assess both faecal sample collection devices and
analysis.
Technical performance:
Analytical sensitivity
Carryover
Imprecision
Precision profile
Linearity
Hook/prozone effect
Stability
Operational performance:
Analyser requirements
Staff requirements
Economic considerations
Environmental considerations.
~ 17 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
EVALUATION METHODS
Faeces
Faecal samples from 10 healthy volunteers were collected and analysed on all evaluation
systems to ensure that the samples were negative for occult blood. The samples were then
mixed by hand in stomacher bags for five minutes to create a master pooled sample. The
pool was kept at 4˚C before use in the stability study; remaining pool sample was frozen at
-20˚C before use in other aspects of the evaluation.
Faecal/Hb preparation
The required faecal/Hb preparations were produced by adding lysate to aliquots of the
faecal pool followed by thorough manual mixing for 5 minutes. After mixing, the collection
devices for each analytical system were used to sample the faecal/Hb preparation, adhering
strictly to manufacturers’ instructions for use, and stored as appropriate for each study. Any
remaining faecal matter was frozen at -20˚C.
Buffer/Hb preparation
The buffer/Hb preparations were prepared using the red blood cell lysate by pipetting
diluted lysate directly into each system-specific collection buffer to give solutions at the
desired concentrations. These samples were used for evaluation of each analyser’s technical
performance; precision, precision profile, linearity and hook/prozone effect.
~ 18 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
EVALUATION METHODS
Analytical sensitivity
The lower limit of detection for each product was determined by measuring Hb
concentration in 20 unused (no sample added) collection devices. The lower limit of
detection was defined as the mean of those 20 measurements plus two standard deviations.
Carryover
All of the evaluated analysers use reusable probes to transfer sample from the collection
device to the reaction cuvette, and between samples these probes are cleaned
automatically. To determine the thoroughness of probe cleaning, sample carryover was
assessed using the protocol described by Broughton et al. [6].
Three aliquots of a high Hb concentration and three aliquots of a low Hb concentration were
prepared in sample buffer. The concentrations were chosen to enable comparison of the
four analytical systems. Three samples with high Hb concentrations (a1, a2, a3) followed by
three samples with low Hb concentrations (b1, b2, b3) were measured. This set of six
measurements was repeated 10 times. The carryover factor, k, was calculated as follows
using the mean of the 10 samples:
Imprecision
Imprecision is a “measure of the closeness of agreement between independent
measurement results obtained under stated conditions” [7]. The 2006 Clinical and
Laboratory Standards Institute (CLSI) EP15-A2 protocol for user verification of performance
for precision and trueness [8] was used to determine repeatability and within-laboratory
precision (see below for definitions). The protocol enables users to demonstrate that the
reliability and accuracy of their analyser is consistent with claims by the manufacturer. The
comparison assumes that the manufacturer has assessed the instrument using either CLSI
EP05-A2 [9] or a similar protocol to establish and validate the analytical performance of the
method.
Three replicate samples of Hb diluted in each product collection buffer contained in their
collection tubes, at selected concentrations, were analysed on a single analytical ‘run’ (same
date and time), on each of five days.
All aliquots were prepared and stored at 4˚C on the first day of the evaluation. Each day 16
samples were removed from storage, allowed 15 minutes to come to room temperature and
then analysed.
Faecal samples were prepared to compare the four analysers at the same concentrations.
Samples of Hb diluted in each manufacturer-specific buffer were prepared to give expected
concentrations of 15, 30, 60, and 120 µg Hb/g faeces within samples collection devices. Ten
samples were prepared for analysis on day one, when each sample was measured ten
times; a further ten samples were prepared for analysis in duplicate on five consecutive
days.
~ 19 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
EVALUATION METHODS
The CLSI protocol EP15-A2 states that the estimated repeatability standard deviation can be
larger than the manufacturer’s claim, but not statistically significantly larger. To determine
whether this was the case, a verification value was calculated using the formula specified in
the protocol [8]. These values were then used to calculate a verified standard deviation
value specific for each concentration against which the GMEC-determined value could be
compared.
Precision profile
The precision profile is a series of measurements of imprecision across a range of
concentrations. The precision profile for this evaluation spanned the range of concentrations
reported for each of the four products. The profile for each analyser was determined using
Hb diluted in manufacturer-specific buffer, and measured in analyser cups. The profile was
determined between 0 and 400 µg Hb/g faeces; each concentration was measured ten
times, and the mean and standard deviation plotted for all analysers across the
measurement range.
Linearity
To determine the linearity of the measurement method, duplicate serial dilutions of Hb in
manufacturer-specific buffer were analysed on all instruments. The range of the
concentration of the samples used in this study was determined to cover a range that is
inclusive of that for each analyser. This enabled comparison between devices.
~ 20 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
EVALUATION METHODS
Hook/prozone effect
The prozone or high-dose hook effect produces false negative results in samples with a very
high concentration of analyte. This analytical problem can occur with some immunoassay
methods when the amount of antigen present (in this case Hb) is markedly greater than the
amount of antibody present. This excess of antigen produces a fall in the turbidity of the
solution.
Five very high concentrations of human Hb diluted in manufacturer-specific buffer, and all
above the quoted measurement ranges for each analyser, were assayed to look for the
presence of a hook/prozone effect, and to determine how each of the analysers responds to
out-of-range results.
Stability
For the purposes of this evaluation, stability was defined as a fall in Hb concentrations to
below 50% of initial concentrations. The evaluation examined both the stability of Hb diluted
in manufacturer-specific buffer and the stability of Hb diluted in faeces. Both evaluations
used samples collected into the manufacturer’s sample collection device.
~ 21 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Analytical sensitivity
The lower limit of detection for each product was determined using the method described in
the Evaluation Methods section of this report. For all analysers the measured lower limit of
detection was lower than that quoted by the manufacturer (Table 3), with the NS-PLUS C15
and HM-JACKarc being the most analytically sensitive.
Table 3: Measured lower limit of detection for each analyser. Quoted lower limits of
detection were provided by each manufacturer in their data sheets.
Mean
Lower limit of Quoted lower
concentration of
Standard detection limit of detection
20 un-spiked
deviation (µg Hb/g (µg Hb/g
collection tubes
faeces) faeces)
(µg Hb/g faeces)
HM-JACKarc 0.3 0.1 0.6 7
NS-PLUS C15 0.0 0.0 0.0 4
OC-SENSOR DIANA 2.1 0.9 3.8 10
FOB Gold/BioMajesty 0.5 0.4 1.3 2.55
Carryover
Carryover was determined using the Broughton method [6] described in the Evaluation
Methods section of this report. All analysers showed very little carryover of sample between
the high and low concentration samples (Table 4). A k value of less than 5 indicates
acceptable performance [5]; all were well within this limit.
Table 4: Carryover determined using the equation described by Broughton et al. [6]
~ 22 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Imprecision
Tables 5 a, b and c show the GMEC-measured imprecision and that quoted by the
manufacturer. Where mean values are similar the table indicates whether the GMEC and
quoted imprecision were consistent. For this study we present the data using the
concentration units quoted by the manufacturer.
FOB Gold/
114.1 4.3 81.2 2.5 3.7 -
BioMajesty
(ng Hb/mL buffer) 192.0 9.3 133.8 3.6 5.3
510.1 51.6 327.7 5.2 7.7 -
~ 23 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
FOB Gold/
114.1 3.6 81.2 2.2 3.1 -
BioMajesty
(ng Hb/mL buffer) 192.0 6.4 133.8 3.1 4.6 -
510.1 43.0 327.7 4.5 6.6 -
FOB Gold/
82.6 16.6 N/A N/A N/A -
BioMajesty
(ng Hb/mL buffer) 365.5 89.6 N/A N/A N/A -
1099.6 86.6 N/A N/A N/A -
~ 24 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Tables 5a and 5b provide the data derived from a comparison of the imprecision and within-
laboratory repeatability of the analysers using Hb diluted in buffer (also see Appendix 2).
The repeatability of the HM-JACKarc was consistent with the manufacturer’s claimed
repeatability. No data were provided for within-laboratory precision. Data from one faecal
sample were consistent with claims, whilst the other was not. In general the HM-Jack
displayed good imprecision at all concentrations studied.
The NS-PLUS C15 was inconsistent with claimed values for both repeatability and within-
laboratory precision in both buffer and faecal material. At the lower concentrations studied
(around possible cut-off values) the imprecision was good, however at the higher
concentrations measurement was more imprecise.
The OC-SENSOR DIANA has the most imprecise method of analysis, with high sr and sl at
both concentrations measured.
The BioMajesty had good imprecision at the lower concentrations studied, but imprecision of
measurement decreased with increasing Hb concentration.
In all analysers studied, when faecal samples were spiked with Hb, the imprecision was
much poorer (Table 5c and Appendix 3). This can be explained partly by the introduction of
sampling variation when using the collection probe and faecal matter.
The OC-SENSOR DIANA and BioMajesty could not be compared with manufacturers’ claimed
values for Hb in buffer, because the measured concentrations were not within two standard
deviations of the expected, calculated concentrations. Neither of the manufacturers provided
data for the imprecision of measurements of Hb in faeces and therefore comparison with
GMEC data was not possible.
~ 25 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
When Hb was directly diluted in manufacturers’ buffer (to reduce sampling imprecision) the
imprecision of analysis with each method could be measured and compared across
analysers. The analytical imprecision of all analysers was limited, both when samples were
measured within the same run, and when measured on consecutive days. The NS-PLUS C15
showed a negative bias in both investigations, whilst the other three analysers all showed a
positive bias with respect to the expected concentration.
Expected 15 µg Hb/g Faeces 30 µg Hb/g Faeces 60 µg Hb/g Faeces 120 µg Hb/g Faeces
concentration
Mean SD %CV Mean SD %CV Mean SD %CV Mean SD %CV
HM-JACKarc 20.4 0.8 4.1 39.6 1.6 4.0 74.3 2.2 3.0 147.6 6.4 4.3
NS-PLUS C15 10.3 0.9 8.5 25.4 2.0 7.7 51.9 2.4 4.5 104.6 7.2 6.9
OC-SENSOR
20.4 0.9 4.5 49.1 2.3 4.6 89.4 2.9 3.3 179.7 11.6 6.4
DIANA
FOB Gold/
14.5 1.0 7.1 36.8 1.5 4.1 90.4 3.3 3.7 189.7 3.7 1.9
BioMajesty
Expected 15 µg Hb/g Faeces 30 µg Hb/g Faeces 60 µg Hb/g Faeces 120 µg Hb/g Faeces
concentration
Mean SD %CV Mean SD %CV Mean SD %CV Mean SD %CV
HM-JACKarc 18.9 1.0 5.1 38.0 1.8 4.6 71.5 3.0 4.6 139.2 10.8 7.8
NS-PLUS C15 9.7 0.8 8.8 26.5 2.2 8.2 50.8 2.2 4.3 103.1 6.8 6.6
OC-SENSOR
21.2 1.8 8.4 47.2 2.3 4.8 86.4 3.8 4.8 170.4 10.2 6.0
DIANA
FOB Gold/
17.7 1.0 5.7 39.0 1.8 4.5 93.3 3.8 4.1 195.8 4.3 2.2
BioMajesty
~ 26 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Linearity
Figures 1-4 show the linearity of each analytical system across the expected measurement
range. The figure on the left shows the whole concentration range (0-500 µg Hb/g faeces)
and that on the right the range 0-120 µg Hb/g faeces.
The FOB Gold/BioMajesty was linear in the range 0-120 µg Hb/g faeces. The analyser
remained non-linear for all concentrations >120 µg Hb/g faeces even after auto-dilution.
The other three analysers all showed good linearity in the range 0-400 µg Hb/g faeces. All
analysers displayed a positive bias; that shown by NS-PLUS C15 was very small and that by
FOB Gold/BioMajesty was the largest.
140
500 (µg Hb/g faeces)
(µg Hb/g faeces)
120
400
100
y=x y=x
300 80
60
200
40
100
20
0 0
0 100 200 300 400 500 0 20 40 60 80 100 120 140
Expected Concentration (µg Hb/g faeces) Expected Concentration (µg Hb/g faeces)
120
(µg Hb/g faeces)
400
(µg Hb/g faeces)
100
y=x y=x
300 80
60
200
40
100
20
0 0
0 100 200 300 400 500 0 20 40 60 80 100 120 140
Expected Concentration (µg Hb/g faeces) Expected Concentration (µg Hb/g faeces)
~ 27 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
600
Measured Concentration
140
(µg Hb/g faeces)
400 100
y=x y=x
80
300
60
200
40
100 20
0 0
0 100 200 300 400 500 0 20 40 60 80 100 120 140
Expected Concentration (µg Hb/g faeces) Expected Concentration (µg Hb/g faeces)
Measured Concentration
400
(µg Hb/g faeces)
350 250
(µg Hb/g faeces)
y=x
300 200
250
200 150
150 100
100 y=x
50
50
0 0
0 100 200 300 400 500 0 20 40 60 80 100 120 140
Expected Concentration (µg Hb/g faeces) Expected Concentration (µg Hb/g faeces)
~ 28 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Precision profile
Precision profiles show the likely variability (precision) of the measurement system across
the reported measurement range (see Appendix 4 for data in a table).
Precision profiles were determined for each analyser. The HM-JACKarc, NS-PLUS C15 and
OC-SENSOR DIANA were all linear throughout the range studied, as previously described;
the FOB Gold/BioMajesty was linear up to 100 µg Hb/g faeces (Figures 5 & 6).
The HM-JACKarc had the best precision with CV below 6% at all concentrations.
The NS-PLUS C15 showed good precision from 20-100 µg Hb/g faeces with a CV below
10%, although below and above this range (0-20 µg Hb/g faeces and 120-400 µg Hb/g
faeces) the CV increased to between 10 and 20%.
The OC-SENSOR DIANA had very good precision above 20 µg Hb/g faeces with a CV of less
than 5%. At 10 µg Hb/g faeces and 5 µg Hb/g faeces the precision was poor, (15 and 27%
respectively), although the quoted lower limit of detection for the OC-SENSOR DIANA is 10
µg Hb/g faeces indicating that the system is not designed for use at these low
concentrations.
~ 29 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Figure 5: Precision profiles of all analysers from 0 – 400 µg Hb/g faeces. (Hollow markers
indicate samples that were diluted; error bars show ± 1 standard deviation.)
700
HM-JACKarc NS-PLUS
Measured Concentration (µg Hb/g faeces)
400
300
200
100
0
0 50 100 150 200 250 300 350 400 450
Expected concentration (µg Hb/g faeces)
Figure 6: Precision profiles of all analysers from 0 – 100 µg Hb/g faeces. (Error bars show
± 1 standard deviation.)
250
HM-JACKarc NS-PLUS
Measured Concentration (µg Hb/g faeces)
y = 1.49x - 2.08
150
R² = 0.99
y = 1.24x - 0.62
R² = 0.99
100 y = 1.02x - 2.03
R² = 0.99
50
0
0 20 40 60 80 100 120
Expected concentration (µg Hb/g faeces)
~ 30 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Hook/prozone effect
Table 8: Analysis of the hook/prozone effect for each analyser. Analytical results are in
black and error codes in red.
Key: P = prozone detected; OR = over range; HR = the result was high and
automatically diluted.
The HM-JACKarc is not designed to give accurate results in samples above 400 µg Hb/g
faeces, and in general use the analyser will give a result of ‘>400 µg Hb/g faeces’ rather
than giving numerical values. For this study numerical values were made available by
changing the analyser set-up. ‘P’ indicates that the analyser detected a hook/prozone effect
and warns the user that the result could be falsely low.
The NS-PLUS C15 showed evidence of hook/prozone effect on two out of four samples
measured at the highest concentration (200,000 µg Hb/g faeces), presenting false low
results with no indication that they were inaccurate, although this is an unexpected
concentration in the normal sampling procedure and the results were positive. However, the
other two samples measured at that concentration and also all samples measured at
100,000 and 50,000 µg Hb/g faeces gave the error code 90, which shows these results were
picked up as over range hook/prozone effect samples.
The OC-SENSOR DIANA gave the ‘over range’ error code ‘OR’ on all samples. The OR code is
displayed when the result is higher than the upper limit of the analytical range. On dilution,
the re-tested samples at concentrations of 12,500 and 25,000 µg Hb/g faeces gave
concentrations of 18,725 and 36,150 µg Hb/g faeces respectively. All samples measured at
50,000, 100,000 and 200,000 µg Hb/g faeces gave the OR code on the re-tested diluted
samples.
The BioMajesty consistently gave a result of 785 ng Hb/mL of buffer (± 14.0) (equivalent to
149 µg Hb/g faeces, ± 2.6) for all very highly concentrated samples. This result was
flagged with ‘h’ which indicated that the concentration was above the cut-off limit, but did
not require diluting; this error is typical of the hook/prozone effect. All other results were
flagged with ‘H’ and subsequently diluted (giving the result code ‘HR’), but still gave results
well below the expected values, with no indication that there was the possibility of
hook/prozone effect. If these results were to be incorporated into a risk score (see
Introduction) this could contribute to a misleading measure of CRC risk.
~ 31 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
Stability
Stability of measured Hb was assessed in samples with blood added directly to the collection
buffer, and in samples where faecal samples had been added to the collection devices. The
study examined four concentrations of Hb at four different temperatures. The results are
provided in Table 9.
Haemoglobin in the HM-JACKarc collection tubes (in the absence of faeces) is quoted by
Kyowa Medex to be stable for 120 days at 4˚C, and 14 days at 25˚C. These claims are
supported in this study, although this evaluation did not examine samples exposed to a
sustained temperature of 25˚C and it did show that at 35˚C stability was poor, particularly
in faecal samples spiked with Hb. At 35˚C the concentration of Hb decreased by at least
50% within 3 days at all concentrations tested. The faeces-free solution was much more
stable, particularly at high concentrations – it took more than 10 days for the concentration
of Hb to decrease below 50% of the initial concentration. Samples that were initially positive
became negative after 2 days (at the two middle concentrations) and after 6 days with the
highest concentration, which could result in falsely negative results if sampled after this
time.
Hb in the NS-PLUS C15 collection tube (in the absence of faeces) is quoted by Alfresa to be
stable for 7 days at 2-8˚C, and 3 days at 18-25˚C. These claims were supported in this
study although the evaluation did not examine samples exposed to a sustained temperature
of 25˚C. GMEC found that at 35˚C in faecal samples spiked with Hb, samples at highest
concentration dropped by 50% in less than three days. During the period tested (28 days)
the samples remained positive when using the manufacturer’s suggested cut-off of 20 µg
Hb/g faeces.
Eiken quote stability for Hb in the OC-SENSOR OC-Auto Sampling Bottle 3 as 7 days at room
temperature, and 14 days between 2 and 10˚C. These claims were confirmed. At 35˚C, as
with the NS-PLUS C15, the faecal sample with the highest initial concentration decreased to
below 50% of the initial concentration within 2 days. However, all samples positive at the
start remained above a cut-off of 20 µg Hb/g faeces after 30 days. The OC-SENSOR was
similar to the NS-PLUS C15 in that faecal Hb degraded faster than Hb diluted in buffer, with
the level dropping below 50% within 4-18 days (at 35˚C).
Sentinel state that samples are stable in the sample collection devices for 14 days at 2-8˚C,
or 7 days at 15-30˚C. These claims were confirmed and Hb in sample buffer with no faecal
material showed very little sample deterioration over 30 days. However, at 35˚C in a faecal
sample the Hb decreased by 50% within 9 days at all concentrations.
~ 32 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
TECHNICAL PERFORMANCE
KEY: STS – Stable throughout study (i.e. the concentration of Hb did not fall below 50% of the initial
concentration during the study). Conc – concentration.
Numerical values show the day on which the measured concentration fell below 50% of the
initial measured concentration.
~ 33 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Collection devices
All collection devices consist of a vessel that contains a buffer (a solution designed to limit
the degradation of Hb), and a probe to collect the faecal sample. All the probes are
connected to the lid of the device.
The neck of the collection tube has a rubber septum that scrapes off excess faecal sample
to control the amount that enters the buffer. The tube is placed on the analyser rack with
the lid pointing downwards; the other end is covered by a small paper and then a plastic
seal. The analyser specimen probe pierces the plastic seal to reach the sample.
The blue-coloured buffer changes to green after a sample has been added, which is a useful
indicator that the device has been used by the screening participant. The sample tube is
placed with the lid downwards in the analyser rack and the foil seal facing upwards. The
analyser specimen probe pierces the foil to collect the sample below.
~ 34 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Figure 9: OC-AUTO Sampling Bottle The collection device contains a plastic narrow neck
3 probe with and without sample; and to control the amount of sample entering the
the sampling bottle neck, narrowed to buffer. The collection tube is placed on the analyser
remove excess sample. rack with the lid downwards and a foil seal
upwards.
The analyser pierces the foil and uses a hammer to squeeze the bottle and force the buffer
through a filter into the analytical compartment. The filter removes particulate matter
ensuring that the analyser sample probe does not become blocked. Once the pressure is
removed any solution remaining in the analytical compartment returns to the main
compartment for repeat analysis, if required.
The other end of the device has a white plastic screw-on lid with a small seal to hold it in
place. The white plastic lid must be removed before placing on the analyser, and should be
replaced following analysis. If the white lid is removed by a screening participant, the
collection buffer can be lost easily making the device unusable.
~ 35 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
The adoption of these units requires an accurate measure of the volume of buffer in the
collection tube, and the mean mass of faecal sample that is likely to be added to the
collection device buffer. In product descriptions, each manufacturer provides quoted values
of the volume of buffer used in their device and their estimate of expected faecal sample
mass. These manufacturers’ values have been used throughout this report and have enabled
us to convert Hb concentrations in buffer (ng Hb/mL buffer) to Hb in faeces (µg Hb/g
faeces).
GMEC has performed an independent assessment of sampled faecal load and buffer volume
using the same method for each device. The buffer volume was determined using 20
collection devices that were weighed, their liquid contents removed and the devices dried in
an incubator for 1 week. The devices were then re-weighed and the difference in initial and
final weights used to give an approximate buffer volume (assuming a specific gravity for the
buffer solution of 1.0 (the same as water)).
The weight of sample collected was determined using an artificial faecal matrix (Kyokuto
Pharmaceutical Industrial Co., Ltd.) prepared to a consistency similar to soft faeces. A
collection device was cut in half to allow access to the tip of the probe. The matrix was
sampled, the probe pushed into the device and then the tip of the probe was cut off (at the
point at which it emerged from the septum) and weighed. The tips were then washed,
dried and re-weighed. The difference in mass between the two measurements was thereby
determined as an estimate of the weight of the sample collected. The GMEC-derived
conversion factors are provided in Table 10.
The mass of faeces collected by a sample probe will be dependent upon a range of factors
including the nature of the faecal sample. For the GMEC estimation, the artificial faecal
matrix was a rice-based product with a specified volume of added liquid. Further work needs
to be undertaken to determine the effect of different matrices on the sample probes to
provide confidence in the assigned sample mass.
~ 36 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Application of the GMEC conversion factors (Table 10) that are different from those quoted
by manufacturers (see also Table 1) will change the interpretation of some of the evaluation
data. For example, compare the graphs in Figures 11a and b. Figure 11a illustrates precision
calculated using the conversion factor provided by the manufacturer (as per
Figure 6). Figure 11b illustrates precision calculated using the GMEC-determined conversion
factor. The differences are described below.
The BioMajesty continues to have a positive bias, but the measured concentration is
much closer to the expected concentration.
The HM-JACKarc, NS-PLUS C15 and OC-SENSOR DIANA all become negatively
biased, the OC-SENSOR DIANA results becoming much closer to expected
concentrations and the HM-JACKarc and NS-PLUS C15 moving further away from the
expected concentrations.
Figure 11a: Precision profiles for all analysers between 0 and 100 µg Hb/g faeces using
manufacturers’ conversion factors. Error bars show ± 1 standard deviation.
250
HM-JACKarc NS-PLUS
Measured Concentration
y = 1.94x - 8.41
200 OC-Sensor Diana FOB Gold/BioMajesty R² = 0.99
(µg Hb/g faeces)
0
0 20 40 60 80 100 120
Expected concentration (µg Hb/g faeces)
Figure 11b: Precision profiles for all analysers between 0 and 100 µg Hb/g faeces using
GMEC-measured conversion factors. Error bars show ± 1 standard deviation.
250
HM-JACKarc NS-PLUS
Measured Concentration
Expected (y=x)
y = 1.25x - 5.44
150
R² = 0.99
y = 0.97x - 1.35
100 R² = 0.99
y = 0.75x - 1.49
R² = 0.99
50 y = 0.62x - 0.31
R² = 0.99
0
0 20 40 60 80 100 120
Expected concentration (µg Hb/g faeces)
~ 37 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Participant survey
The four FIT collection devices and a short questionnaire were sent to 25 volunteer
participants aged between 25 and 85 years. Each participant was asked to collect a sample
of faeces with each device and then answer the questions.
Participants were asked to record their satisfaction with the mechanism/method for:
Participants were invited to comment on the strengths and weaknesses of each device and
where they would like to see improvements. Participants were also asked which devices they
would be happy to use, and which of the four devices they preferred. The results are
presented in Figures 12-17.
Participants found the HM-JACK device small and awkward to use. Replacement of the probe
into the device was made more difficult because it has three potential entry holes at the
head of the device and poor eye sight was thought likely to accentuate this design
weakness. Participants reported incidents where faeces did not appear to adhere to the
dimples that are a unique feature of this device. Participants were left concerned that kits
would give negative results because no faeces would be present in the test buffer.
Participants found the size and shape of the lid of the NS-PLUS C15 collection device easy to
hold and open. Participants were concerned that unscrewing the lid might prove difficult for
individuals with restricted dexterity, such as those with arthritis, due to stiffness of the
screw thread. Overall the NS-PLUS C15 collection device performed well and was thought
the easiest to use.
Most participants were happy to use the OC-SENSOR device and it was thought to be the
easiest to open. The size of the entry hole at the top of the device was thought by some
participants to be too small and individuals with restricted dexterity could find it difficult to
return the collection probe to the tube.
Participants found the FOB GOLD device easiest to use for sample collection, returning the
probe to the tube and closing the device. Participants repeated the concern reported in the
earlier GMEC evaluation [11] that it was easy to open the wrong end of the device and
release the sample buffer. When asked which device participants preferred, this device
ranked just behind NS-PLUS C15.
~ 38 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
20
HM-JACK
15
Number of people
HM-JACK
10 NS-PLUS
OC-Sensor
5 FOB Gold
0
Very easy Satisfactory Difficult
NS-PLUS
Figure 13: How easy was it to collect the sample?
25
20
Number of people
HM-JACK
15 NS-PLUS
OC-Sensor
10
FOB Gold
5 OC-Sensor
0
Very Easy Satisfactory Difficult
Figure 14: How easy was it to replace the sample and sample probe
into the collection tube?
25
20
Number of people
HM-JACK
FOB Gold
15 NS-PLUS
10 OC-Sensor
FOB Gold
5
0
Very Easy Satisfactory Difficult
~ 39 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
25
20
Number of people
HM-JACK
15 NS-PLUS
OC-Sensor
10
FOB Gold
5
0
Very Easy Satisfactory Difficult
Figure 16: Which devices are you happy to use, which ones are you not happy to use?
25
20
Numebr of people
HM-JACK
15 NS-PLUS
OC-SENSOR
10
FOB Gold
5
0
Happy Not Happy
12
10
Number of people
HM-JACK
8
NS-PLUS
6
OC-Sensor
4 FOB Gold
0
HM-JACK NS-PLUS OC-Sensor FOB Gold
~ 40 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
The suppliers and manufacturers were asked to provide a list of current users of their
systems who would be willing to complete an evaluation questionnaire. We acknowledge
that this may not be a representative sample of users.
Only one reply was received from a laboratory analysing more than 1,000 samples per
week; the other replies came from laboratories analysing 50–1,000 samples per week. No
replies were received from laboratories using the BioMajesty.
Overall, there were very few technical issues reported, and those reported to the supplier
had been dealt with in a timely and satisfactory manner. Most users were happy with their
analyser and the support they receive. One user of an NS-PLUS C15 supplied by Alere in
Canada, mentioned that consumables were frequently not available when ordered and were
placed on ‘back order’.
Users stated significant differences in the length of time that they required to perform
routine maintenance tasks. Such differences are surprising and it suggests that users were
trained differently or that the description of tasks in the user manual may be ambiguous and
interpreted differently by different users (instruments affected: HM-JACKarc and NS-PLUS
C15).
Ease-of-use
Table 11 compares procedures necessary to use the four analysers and highlights where
improvement could be made.
HM-JACKarc
The HM-JACKarc is an easy-to-use bench-top analyser, with a simple touch screen and an
easy-to-follow system of menus. Analyser start-up is simple, and it has an automatic feature
to prepare the analyser so that it is ‘on’ and ready for use at a time programmed by the
operator. Maintenance is easy to perform and analyser start-up and shut-down procedures
are very simple.
GMEC found this analyser simple to use and suitable for use by staff with some laboratory
experience. The small capacity of the HM-JACKarc makes it best suited to a small to
medium rather than a large screening laboratory.
NS-PLUS C15
The NS-PLUS C15 is a very simple to use bench-top analyser, which is well designed with
easy-to-use software. The analyser has good monitoring systems for reagent and waste
levels, and barcodes on reagents to make it easy to replenish them during analysis. The
software easily records and tracks the use of different reagent LOTs. The water and wash
solutions can be topped up easily during analysis, and the waste has a monitor and alarm to
alert the user when it needs to be emptied.
The simplicity of the NS-PLUS C15 software and analyser make it suitable for use by staff
with limited laboratory experience, as part of a large screening programme.
~ 41 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
OC-SENSOR DIANA
The OC-SENSOR DIANA is also an easy-to-use bench-top analyser. Analyser start-up is
simple, and it has an automatic feature to prepare the analyser so that it is ‘on’ and ready
for use at a time programmed by the operator. The calibrator and QC material are supplied
already prepared, which removes the possibility of introducing errors during the process of
reconstitution.
The OC-SENSOR DIANA analyser supplied for the evaluation was replaced during the period
of the evaluation due to a failure of internal plumbing. The instrument was replaced within
two days of the fault being identified. GMEC noted that the wording on some of the menus
was slightly different, which could lead to confusion or the requirement to rewrite standard
operating procedures when analysers are changed.
FOB Gold/BioMajesty
The BioMajesty is a large floor-standing analyser, which is complex to use, but enables
monitoring and checking of systems during operation. The instrument monitors reagent
levels and informs the operator when the analysis will be completed. Similar to the OC-
SENSOR, the BioMajesty has liquid calibrators and QC materials, which obviates the need
for, and potential errors associated with, reconstitution.
Whilst it is possible to connect the BioMajesty to a track system, this facility has not been
evaluated by the GMEC team.
Overall, the GMEC team found the BioMajesty a challenging analyser to use; it was
unnecessarily complex for the analysis of a single marker (FIT). Staff using the analyser
would require extensive training.
~ 42 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Calibration/QC Requires reconstitution with Requires reconstitution with Liquid calibrant and QC – no Liquid calibrant and QC.
analysis accurate pipette and then wait accurate pipette and can use preparation; easy to dispense Compared with other systems
total of 30 mins; two levels of immediately. Single level of with built in dropper and levels the metal foil lids that cover
calibrant need to be prepared. calibrant auto-diluted to seven are easily identified. Single the rubber stoppers are
Calibrant aliquots can be concentrations by analyser. level of calibrant auto-diluted difficult to remove. Foil lids are
frozen. Calibration software Aliquots can be frozen. to seven concentrations by different colours but once
easy to set-up with handheld Calibration software easy to analyser. Calibration software removed rubber stoppers same
barcode reader. set-up. difficult to set up. colour. Six vials are required
for calibration and must be put
into cups in correct order.
Calibration software difficult to
set up.
QC result monitoring Daily QC results are printed by Daily QC results are printed Daily QC results are printed Several steps to find daily QC
the analyser on thermo paper and cumulative data are easy and cumulative data are easy results and more to find
and cumulative data are easy to see on screen. to see on screen. cumulative data - not intuitive,
to see on screen. data do not print
automatically.
~ 43 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Sample unloading Racks move to different area Blue indicator light shown Racks move to different area Samples need to be unloaded
of analyser when finished. Can when racks can be unloaded, of analyser when analysed. individually and lids replaced
then remove 100 samples on a (removed individually). Can then remove 100 samples (care to avoid cross-sample
tray. on a tray. contamination).
Dilution of out-of- Automated dilutions are not Analyser dilution can be Analyser dilution performed Analyser dilution can be
range samples possible on this analyser. performed without sample after reloading sample on a performed without sample
reload - reduces analytical rack with diluent and dilution reload - reduces analytical
productivity. cups. productivity.
~ 44 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Maintenance Quick and easy to perform Quick and easy to perform Quick and easy to perform Quick and easy to perform
daily maintenance; minimal daily maintenance; other daily maintenance; other daily maintenance; other
additional maintenance maintenance not assessed. maintenance minimal. maintenance not assessed.
~ 45 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Instructions for use Some on-screen help; manual On-screen training videos On-screen help available and Can be difficult to find required
and Quick Use guide easy to useful and clear, good manual useful; can be difficult to find section in paper manual and
use. Improved index of manual and good package inserts. required section in manual but when found is difficult to
required. when found is useful; package understand; good package
inserts good but reagent insert inserts for FOB Gold.
has unnecessary information
about other components.
Troubleshooting Easy to understand error Easy to understand error Most error messages easy to Error messages difficult to
messages and to find messages and to find understand and find understand and difficult to find
information in manual. information in manual. information in manual though further information in manual.
some more difficult.
Control of analyser Integrated computer, touch Separate PC required, no Integrated computer, touch Separate PC required, no
screen, easy to follow touch screen, but easy to use screen, software menu can be touch screen, software
software/menu. software with aid of colour confusing but analysis menu is complex to use.
coding and animations. simple.
Staff training and Can be used by staff with basic Can be used by staff with basic Can be used by staff with Needs to be used by
experience required analyser training/experience, analyser training/experience, intermediate analyser training/ experienced laboratory staff
when supervised by staff when supervised by staff experience, with supervision provided with extensive
experienced with auto- experienced with auto- by staff experienced with auto- analyser training.
analysers. analysers. analysers.
~ 46 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Consumables
Table 12 below describes the storage requirements of the reagents and consumables
required for each analyser, and the volume each required to perform 5,000 tests, a typical
daily total for the BCSP Southern Hub.
The HM-JACKarc would require 25 bottles of latex for a day’s work, each placed on the
analyser one at a time. The NS-PLUS C15 requires 17 bottles if no dilutions are carried out;
four bottles of each reagent can be held on the analyser giving a capacity of 1,200 tests (if
dilutions are enabled only three bottles of each reagent can be held on the analyser,
reducing onboard capacity to 900 tests). The OC-SENSOR DIANA and the BioMajesty would
each require 20 bottles of reagent. The OC-SENSOR DIANA can have two bottles on the
analyser at a time (500 tests).
Two of the current Sentinel 20 mL reagent bottles can be emptied into the analyser reagent
container and three of these containers can be held on the analyser, providing a capacity of
1,500 tests. Sentinel has now developed replacement bottles for the BioMajesty with three
times the volume of reagent, which increases onboard capacity to 2,250 tests. The new
containers have a barcode to enable easier loading of the reagents, but these have not been
evaluated by GMEC.
The HM-JACKarc produces the largest mass of solid waste. The analyser uses non-recyclable
plastic for its single-use cuvettes and single-use cuvette racks.
The three other analysers evaluated use reusable cuvettes, which are cleaned between each
analysis. All reusable cuvettes have a recommended lifetime and require replacement: NS-
PLUS C15 after 200 analyses in a cuvette which is approximately 10,000 on the analyser
(every 2–3 days for the Southern Hub workload); OC-SENSOR DIANA after 100,000 analyses
on the analyser (every 20–30 working days for the Southern Hub); BioMajesty every two
years, depending on throughput and evidence from quality control monitoring.
~ 47 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Table 12: Consumables and storage conditions required for each analyser, provided by manufacturers (N/A: not applicable)
Diluent Unopened N/A 2-8˚C until expiry date 2-10˚C until expiry date 2-8˚C until expiry date
(up to 12 mths)
Opened N/A 2-8˚C until expiry date 2-10˚C until expiry date 30 days at 2-8˚C
(up to 12 mths)
On analyser N/A 2-8˚C until expiry date N/A 30 days at 2-12˚C
~ 48 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Table 12: Consumables and storage conditions required for each analyser, provided by manufacturers (continued) (N/A: not applicable)
Cuvettes Number required for 125 racks of cuvettes Change whole ring One set (approximately One set (change every 2
5,000 sample tests (40 per rack) after 10,000 tests 100,000 reactions) years, or as throughput
demands)
Reaction tips Number required for Reusable probes Change after 5,000 Reusable probes Reusable probes
5,000 sample tests tests
~ 49 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
All analysers evaluated for this report require daily maintenance and cleaning. The
BioMajesty is designed for high volume multi-analyte analysis and requires the most
cleaning, including two long analyser washes every day (27-35 minutes each). The
frequency of refilling wash solution and water bottles, and emptying waste containers
depends on the workload, and whether the analysers are connected to mains water and
drainage.
The HM-JACKarc required the least cleaning because it had single-use cuvettes; only
reagent lines and reagent and sample probes need to be cleaned daily. This analyser
therefore created the least liquid waste.
The NS-PLUS C15 and OC-SENSOR DIANA both created more liquid waste because of the
need to wash the reusable cuvettes. The OC-SENSOR DIANA had additional wash
procedures that used more deionised water and created more liquid waste.
The BioMajesty required washes in addition to those for cuvettes and for conditioning and
these washes used more hazardous solutions than those used by the other analysers.
Details of the start-up, daily, weekly and monthly maintenance/procedures, are given in
Table 13. Details are also provided on the preparation and frequency of analysis of
calibrators and quality control material.
~ 50 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Table 13: Maintenance and servicing of the analysers (N/A: not applicable)
Weekly Wash through the system N/A Clean external surfaces Perform a
after replacing the buffer with and the W, R- and S- measurement of the
water. Remove cuvette waste nozzles, the racks and optical density of each
and replace waste bag. the trays. cuvette.
Monthly Clean probes with warm Clean fan filter. Soak cell, nozzle soak Soak and wash mixing
water. Wash waste bottle. and perform a cuvette rods (15 mins).
Clean cuvettes and blank measurement.
reagent lines. Clean the tanks.
Start up One hour wait to allow lamp 12 mins to allow At least 30 mins for the Initialise, prime reagent
output to stabilise. 1-3 system analyser to reach buffer to warm up (can lines and wash. Total
washes (if buffer replaced correct temperature. take up to 50 mins). time 30-40 mins.
between uses, each wash
takes 4 mins).
Shut down One wash, approximately Switch off the analyser. Wash takes 10 mins. Wash, takes 27-35
3 mins. Switch off the mins.
analyser.
~ 51 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL PERFORMANCE
Frequency of calibration Instructions for use state At change of reagent At change of reagent At change of reagent
every 2 days. LOT. LOT. LOT or maximum of 30
days.
Quality Preparation of material Add distilled water to Add diluent to Provided in liquid form Provided in liquid form
Control lyophilised material, leave to lyophilised material, ready to use. ready to use.
stand for 20 mins, mix, leave mix.
to stand for further 10 mins.
~ 52 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL CONSIDERATIONS
Analyser requirements
The five BCSP Hubs in England have different analytical workloads. For an average daily
workload of 5,000 samples, such as might be experienced by the Southern Hub, the
estimated number of analysers required to comfortably complete the workload on the day of
receipt is given in Table 14 (based on an 8-hour working day, during which sample analysis
would be undertaken for 7 hours). The number provided includes an extra analyser, which
increases capacity and allows for a back-up analyser in case of instrument failure or the
need for extensive maintenance or servicing.
Table 14: Number of analysers required to complete an average daily workload of 5,000
sample requests. (Appendix 9 provides a detailed calculation of these numbers;
Appendix 10 provides the calculation for extreme workloads of 9,000 samples
per day.)
OC-SENSOR FOB
HM-JACKarc NS-PLUS C15
DIANA Gold/BioMajesty
Number of
5 4 4 2
analysers required
Staff
The process of sample handling is likely to be more complex for FIT than for gFOBT, but this
will depend upon the nature of the package used for transport. More staff will be required to
receive, open and log the FIT kits.
Once logged, the automated analysis will require fewer staff than is necessary for gFOBT.
The staff performing FIT analysis will need more technical knowledge and skills than are
currently required. Appropriately trained screening staff who have the necessary technical
aptitude would be able to provide routine analysis using all but the BioMajesty analyser. The
analytical service will require close supervision by experienced healthcare scientists
supported by detailed operating and quality control procedures.
Calibration, quality control and assessment of the analysers and technical validation of
results will need to be undertaken or closely supervised by trained healthcare scientists.
Validated results can then be uploaded to the BCSP database (Bowel Cancer Screening
System [BCSS]).
Economic considerations
A realistic estimate of the cost of using FIT requires detailed cost analysis-based experience
obtained from the planned England FIT pilot programme. The cost of the devices will be
subject to competitive tendering between at least four potential suppliers. The cost of
packaging and postage has yet to be determined and will depend upon the final package
design, economies that can be achieved in the mailing system and postal costs that are
currently being subject to substantial increases.
In the laboratory, fewer staff will be required to perform the analysis but greater supervision
will be necessary from qualified scientific staff. The current time- consuming QC procedure
~ 53 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
OPERATIONAL CONSIDERATIONS
should take less time but will require more experienced staff. Instrument maintenance and
quality checking will be a new activity for the screening Hubs and these procedures can be
exacting and demand time from skilled staff. Receipt and computer logging of FIT kits will
require more staff time than is currently required for gFOBT; the exact requirement will
depend greatly upon the design of the device and packaging.
Whilst the literature has estimates of the cost of the adoption of FIT, figures do not reflect
the likely systems and economies of scale that can be realised in the English programme
[12,13,14]. The most recent estimates of the cost effectiveness of a population-based FIT
bowel cancer screening programme have been made by groups in the Netherlands who will
commence their population screening programme in January 2014.
Environmental considerations
Clinical waste
The collection devices are much smaller than current gFOBT kits, however they contain
liquid buffer that will be contaminated with faecal material. These must be disposed of as
clinical waste.
The HM-JACKarc produces an additional quantity of clinical waste, due to once-only use of
the reaction cuvettes. The Southern Hub would therefore typically dispose of 5,000 plastic
devices and reaction cuvettes each day. The other analysers use reusable cuvettes, thereby
reducing plastic clinical waste.
~ 54 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
ACKNOWLEDGEMENTS
GMEC would like to thank the following individuals for their contributions to this evaluation
and report:
Andrea Cugini, Sentinel, Italy
Anna Ceriani, Alere, Italy
Antonio Pezzutto, Alere, Italy
Arthur Sanchez, Interior Health, Canada
Callum Fraser, Centre for Research into Cancer Prevention and Screening, Scotland, UK
Chiara De Cunto, Sentinel, Italy
Christian Ramaker, Erasmus Medical Centre, Netherlands
Darren Stenlake, Sysmex UK Ltd
David Giles, Alpha, UK
David Smith, Alere, Canada
David Wilkinson, Alere, UK
Franco Melziade, Sentinel, Italy,
Henk Engel, Isala Klinieken, Netherlands
Iain McElarney, Mast Group Ltd, UK
Janice Webber, Siemens Plc, UK
Keith Howes, Sysmex, UK
Koen van Dierman, Sysmex, Netherlands
Lloyd Write, Sysmex UK Ltd
Louise Farrar, Queen Elizabeth Hospital, Canada
Maria Chiara Anelli, Sentinel, Italy
Mario Fangareggi, Sentinel, Italy
Matthew Davis, Alpha, UK
Mikihisa Okuda, Yurin Hospital, Japan
Mr. Takahashi, Tohoku Central Hospital, Japan
Neil Stubbs and his staff, NHS Bowel Cancer Screening Programme, Southern Hub, UK
Nicola Jackson, University of Surrey, UK
Nozomi Kitazawa, Nagano Chuo Hospital, Japan
Roberto Dioli, Sentinel, Italy
Ross Witney, Merck/Millipore, UK
Ruggero Lucini, Sentinel, Italy
Shinobu Kato, Sanyudo Hospital, Japan
Simona Kapus, Institute of Public Health of Republic of Slovenia
Spomenka Lajtner, Diagnostics Laboratory of Community Health, Slovenia
Steve Ohlsen, Alere, UK
Steve Smith, NHS Bowel Cancer Screening Programme, Midlands & North West Hub, UK
Susan Thorpe, National Institute for Biological Standards and Control, UK
Takanori Tsukada, Tannann Regional Medical Centre, Japan
Takuo Ichiyanagi, Eiken Chemical Ltd, Japan
Teri-Lynn Bajkov, BCBio, Canada
Tetsuya Kosaka, Alfresa Pharma Corporation, Japan
Tomoyuki Shimba, Public Moramachi Hospital, Shizuoka, Japan
Tracy Wade, Eastern Health, Canada
Yasunobu Masuda, Kyowa Medex Co, Ltd, Japan
Yoshi Itoh, Eiju General Hospital, Japan
Yukata Nara, Saitama Medical Centre, Japan
Yumi Shimizu, Yamanash Kosei Hospital, Japan
~ 55 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
BIBLIOGRAPHY
~ 56 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
~ 57 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
~ 58 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
400 483.2 8.9 1.8 353.0 42.0 11.9 576.7 24.6 4.3 424.3 6.8 1.6
200 260.7 5.7 2.2 210.0 37.9 18.0 271.8 23.3 8.6 353.9 4.6 1.3
100 124.0 1.4 1.1 100.8 5.5 5.4 146.0 5.1 3.5 196.7 2.6 1.3
75 91.6 1.1 1.2 69.8 2.0 2.8 109.0 3.9 3.6 124. 9 2.1 1.7
50 64.3 0.4 0.7 53.7 1.8 3.4 73.6 1.9 2.6 84.3 1.6 1.9
40 47.5 0.8 1.6 37.8 0.8 2.1 56.8 1.1 2.0 69.4 1.3 1.9
30 34.6 0.6 1.8 28.7 1.2 4.0 44.5 1.0 2.2 47.3 1.0 2.1
20 25.2 0.3 1.3 16.6 0.7 4.2 27.8 0.9 3.2 27.3 0.8 2.8
10 12.3 0.3 2.6 8.1 1.2 14.8 11.3 1.7 14.7 14.5 0.7 5.1
5 5.7 0.3 5.3 3.3 0.7 20.5 5.0 1.3 27.2 7.4 0.6 8.7
~ 59 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Appendices 5-8 show the stability data for the four analysers. In each case the first graph
shows the data from sample collection tubes spiked with known concentrations of Hb (red
cell lysate), and the second graph shows the stability of Hb in faecal samples that have been
spiked with known concentrations of Hb and then collected into the collection devices. The
first result is the mean of 10 samples of the same concentration all measured on the day 1.
Key: LLOD – Lower limit of detection. A-D – concs, and figures the temperature in ˚C.
HM-JACKarc
300
Stability study - samples spiked with red cell lysate
A -20
A4
A 20
250
A 35
Concentration (µg Hb/g faeces)
B -20
200 B4
B 20
B 35
150 C -20
C4
C 20
100
C 35
D -20
50 D4
D 20
D 35
0 LLOD
1 2 3 4 6 7 8 9 10 12 14 16 18 20 22 24 29
Day
HM-JACKarc
Stability study - faecal samples spiked with red cell lysate
180
A -20
A4
160
A 20
A 35
Concentration (µg Hb/g faeces)
140
B -20
B4
120
B 20
100 B 35
C -20
80 C4
C 20
60 C 35
D -20
40 D4
D 20
20 D 35
LLOD
0
1 2 3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
~ 60 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
NS-PLUS
Stability study - samples spiked with red cell lysate
200 A -20
180 A4
A 20
160 A 35
Concentration (µg Hb/g faeces)
B -20
140
B4
120 B 20
B 35
100 C -20
C4
80
C 20
60 C 35
D -20
40 D4
D 20
20
D 35
0 LLOD
1 2 3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
NS-PLUS
Stability study - faecal samples spiked with red cell lysate
120 A -20
A4
A 20
100 A 35
Concentration (µg Hb/g faeces)
B -20
B4
80 B 20
B 35
C -20
60
C4
C 20
C 35
40
D -20
D4
20 D 20
D 35
LLOD
0
1 2 3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
~ 61 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
OC-SENSOR - Diana
250 Stability study - samples spiked with red cell lysate
A -20
A4
A 20
200
Concentration (µg Hb/g faeces)
A 35
B -20
B4
150
B 20
B 35
C -20
100 C4
C 20
C 35
50 D -20
D4
D 20
0 D 35
1 2 3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28 LLOD
Day
OC-SENSOR - Diana
140 Stability study - faecal samples spiked with red cell lysate
A -20
A4
120 A 20
A 35
Concentration (µg Hb/g faeces)
B -20
100
B4
B 20
80 B 35
C -20
C4
60
C 20
C 35
40 D -20
D4
D 20
20
D 35
LLOD
0
1 2 3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
~ 62 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Appendix 8: FOB Gold/BioMajesty stability data. The samples more concentrated than
200 µg Hb/g faeces were not diluted, and their reported results gave no
indication that they were out-of-range. It was therefore not possible to
report the stability of Hb in concentrations greater than 200 µg Hb/g faeces
from this study.
FOB Gold/BioMajesty
160 Stability study - samples spiked with red cell lysate
B -20
140 B4
B 20
Concentration (µg Hb/g faeces)
120
B 35
100 C -20
C4
80 C 20
C 35
60
D -20
40 D4
D 20
20
D 35
0 LLOD
1-3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
FOB Gold/BioMajesty
160 Stability study - faecal samples spiked with red cell lysate
A -20
A4
140
A 20
Concentration (µg Hb/g faeces)
A 35
120 B -20
B4
100 B 20
B 35
80 C -20
C4
60 C 20
C 35
40 D -20
D4
20 D 20
D 35
LLOD
0
1-3 4 5 6 7 8 9 11 13 15 17 19 21 23 25 28
Day
~ 63 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Appendix 9: Number of analysers required for an average daily workload of 5,000 samples.
(*Figures for FOB Gold/BioMajesty calculated assuming use of a track.)
If one analyser breaks down there must be sufficient backup within the laboratory to cover
the extra workload. Number of analysers required to measure 5,000 samples with enough
capacity to ensure that the Hub could cope if one analyser was unavailable:
Total number of
analysers 5 4 4 2
required (d)
Number of hours
to process 5,000
5 hrs 4 hrs 10 mins 4 hrs 10 mins 3 hrs 8 mins
samples with ‘d’
analysers
If one analyser breaks down, how long will it take to analyse the samples using the
remaining analysers?
OC-SENSOR FOB Gold/
HM-JACKarc NS-PLUS C15
DIANA BioMajesty*
Number of
analysers left 4 3 3 1
(e = d-1)
total number of
samples to analyse 5,000 5,000 5,000 5,000
(5,000)
Number of
samples per
1250 1667 1667 5,000
analysers left
(f = 5,000/e)
how many hours
6 hrs 18 mins 5hrs 36 mins 5hrs 36 mins 6hrs 18 mins
(g= f/a)
~ 64 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Appendix 10: Number of analysers required for an average daily workload of 9,000
samples. (*Figures for FOB Gold/BioMajesty calculated assuming use of a
track.)
Total number of
analysers 8 6 6 3
required (d)
Number of hours to
process 9,000
5 hrs 38 mins 5 hrs 5 hrs 3 hrs 45 mins
samples with ‘d’
analysers
If one analyser breaks down, how long will it take to analyse the samples using the
remaining analysers?
HM-JACKarc NS-PLUS C15 OC-SENSOR FOB Gold/
DIANA BioMajesty*
Number of
analysers left 7 5 5 2
(e = d-1)
total number of
samples to analyse 9,000 9,000 9,000 9,000
(9,000)
per analysers left
1286 1800 1800 4500
(f = 9,000/e)
how many hours
6 hrs 24 mins 6 hrs 6 hrs 5 hrs 36 mins
(g= f/a)
~ 65 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
1. Kyowa Medex
Main Concept:
System integration for “large” programmatic population-based colorectal cancer screening
using a faecal immunochemical test for haemoglobin.
Kyowa Medex Co., Ltd., a Japanese based in vitro diagnostic manufacturer, has provided
ideal solutions to colorectal cancer (CRC) screening for more than 20 years with highly
integrated automated immunoturbidimetric analytical systems.
Now, the new concept applied to our successful series of analyzers with excellent
performance characteristics has been developed taking expert opinions and customer
feedback into account. The HM-JACK SP will be available soon to offer more benefits to large
population-based CRC screening laboratories. The planned global launch date will be end of
2014 to early 2015.
Improvements/Benefits are:
*optional
The HM-JACK series of analysers are known as very simple to use, highly sensitive systems
from both analytical and clinical aspects. These are termed middle-sized system in this field.
Now, with the HM-JACK SP, we have combined a high throughput module with our highly
sensitive measurement system with little clinical waste into the ideal solution.
~ 66 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Sample
Sample A Sample B Sample C Sample E
D
1 12.8 27.3 102.0 201.2 401.0
2 13.0 27.2 101.9 200.6 398.6
3 12.9 27.4 102.4 199.9 399.3
4 12.7 27.7 101.7 199.7 399.3
5 12.9 27.0 102.3 199.7 398.2
6 12.6 27.5 102.0 199.2 400.7
7 12.6 27.4 101.7 199.9 398.9
8 13.0 27.5 101.1 202.7 402.4
9 13.1 27.2 101.7 201.6 400.8
10 12.8 27.4 101.1 199.7 400.7
Mean 12.8 27.4 101.8 200.4 400.0
SD 0.18 0.19 0.45 1.10 1.32
CV,% 1.4 0.7 0.4 0.6 0.3
~ 67 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
2. Alere/AlfresaPharma
AlfresaPharma Corporation has launched the next generation model of the NS-Plus called the
NS-Prime January 2013 in Japan. The new system is identical in terms of intended use,
device design, principles of operation and work-flow as those found within the proven
NS-Plus.
There are a few key enhancements made to the NS-Prime, which will further streamline the
workflow process. Below is a summary of these updates including Table 1 which outlines the
differences between the NS-Plus and NS-Prime.
NS-Prime
Increased maximum sample onboard capacity to 220 samples from 160 samples on
the NS-Plus instrument.
100-sample rack available for ease of loading.
24-hr reagent cooling system allows reagents to be left in the reagent rack for up to
5 days.
Reagent dispensing nozzle has been modified to metallic instead of carbon containing
plastic, increasing durability and replacement life.
Added capability of managing barcodes for calibrators and controls instead of manual
entry.
Increased onboard reagent capacity.
The user interface has been updated to allow easier navigation through the software.
Table 1
Product Name NS-Plus NS-Prime
Width (mm) 680 805
Depth (mm) 580 620
Height (mm) 430 400
Weight (kg) 58 70
Rated Input Power
400 Unchanged
(Volt-Ampere)
110-120V (50/60Hz) US,
Rated Canada
100-240V (50/60Hz) – Universal
Voltage/Frequency 220-240V (50/60Hz) EU,
China
Throughput Capacity 300 tests per hour Unchanged
Onboard Sample 160 Max (including 10
220 Max (including 20 STAT)
Capacity STAT)
~ 68 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
In addition to the above, the reagents and wash solution have been optimized for use with
the NS-Prime analyzer. The preservative for Reagent R1 has been modified to be identical to
Reagent R2 (NaN3 0.05%). The antibody of the reagent has not changed and is identical to
that which is used for the NS-Plus. The Wash Solution has also been optimized so that less
bubbling occurs when used with the analyzer. Finally, there have been no modifications to
the Calibrator, Control or Specimen Diluent.
~ 69 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Takuo Ichiyanagi
Eiken Chemical Co. Ltd.
~ 70 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
SENTINEL REAGENTS
Considering the screening environment, the collecting device includes a new buffer
(Screening System) developed in order to reduce hemoglobin decay at room temperature for
many days. During 2013 the following CE marked applications of new FOB Screening System
have been released:
On Board
Analyzer Measuring range Total Imprecision (**) Calibration
Calibration Stability
BioMajesty JCA-6010 10 to 900(*) ng/mL 3.1% CV (<100 ng/mL) 6 points 30 days
2.7% CV (>100 ng/mL)
Beckman AU series 10 to 800(*) ng/mL 3.1% CV (<100 ng/mL) 6 points 30 days
2.5% CV (>100 ng/mL)
Roche Modular P 14 to 750(*) ng/mL 3.9% CV (<100 ng/mL) 6 points 30 days
2.3% CV (>100 ng/mL)
Abbott Architect 10 to 800(*) ng/mL 3.1% CV (<100 ng/mL) 6 points 30 days
series 3.0% CV (>100 ng/mL)
Sentinel SENTiFOB 25 to 750(*) ng/mL 6.4% CV (<100 ng/mL) 6 points 30 days
4.5% CV (>100 ng/mL)
Sentinel SENTiFIT270 10 to 900(*) ng/mL 4.5% CV (<100 ng/mL) 6 points 30 days
2.2% CV (>100 ng/mL)
(*): indicative value, depending on the concentration of highest Calibration level
(**): representative data obtained in defined conditions)
~ 71 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Evaluation of absence of dietary interferences using FOB Gold ® Screen System in the
determination of occult blood in fecal samples.
Gramegna M., La Motta M., Longo G., Anelli M.C., Lucini R. Sentinel CH. SpA, Milan, Italy
(AACC 2013 Huston, poster accepted, to be published on Clin Chem.)
~ 72 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~
APPENDICES
Appendix 12. Revisions made to the original report (revisions published December 2014)
Page 8, Summary
A new subheading ‘Evaluation outcomes: operational performance/considerations’ has been
included in the Summary.
Text describing the analysers is the same as the original text for all four analysers.
A new summary table has been added to assist direct comparison of the analysers’ performance
and characteristics (entitled ‘Evaluation outcomes: technical performance’).
The original Summary text included some errors, now corrected in the new summary table:
HM-JACKarc, Imprecision: no within-laboratory imprecision data provided by the manufacturer.
Measurement range: Reference to low upper limit to the measurement range removed. There is
no system for diluting samples that have a high Hb concentration.
BioMajesty, Imprecision: imprecision of measurement increased (not ‘decreased’ ) with increasing
Hb concentration.
OC-SENSOR DIANA, Linearity/measurement range: The upper limit of the undiluted measurement
range had been quoted incorrectly as 500 µg Hb/g faeces, rather than 200 µg Hb/g faeces. The
new summary table includes the corrected undiluted measurement range (10-200 µg Hb/g faeces)
and a statement that ‘Dilution extends the range beyond the undiluted upper limit to
50,000 µg Hb/g faeces’.
~ 73 ~
~ Evaluation of Quantitative Faecal Immunochemical Tests for Haemoglobin ~