Introduction

Accreditation of laboratories has been successful in fostering quality of analytical laboratories. Consequently, it is widely seen as a very important element in quality assurance and is thus helping to increase comparability of measurement results. One part of laboratory accreditation is to assess if incoming chemicals are of sufficient quality. While this was translated by many laboratories into buying from ISO certified companies only, a blind eye was long cast on the suppliers of reference materials (RMs). Third-party assessment of reference material producers has only recently entered the scene, with Australia's National Analytic Reference Laboratory (NARL) being the first laboratory world-wide to achieve accreditation for the production of reference materials in 2000 [1]. Accreditation was based in ILAC Guide 12, as ISO Guide 34 had not been released at that time, but accreditation to the very similar ISO Guide 34 was granted after a re-assessment in 2001. Since then, a number of reference material producers achieved such accreditation, usually for a narrow range of reference materials. For example, NARL is accredited for pure organic standards, while BTF Ltd. is accredited for the production of microbiological reference materials for a clearly defined number of bacteria [2].

In this paper, we review the advantages and disadvantages of the various quality standards to base a third-party assessment for reference material producers. Moreover, the management system of the Reference Materials Unit of the European Commission's Institute for Reference Materials and Measurements (IRMM) is described. It is the first reference material producer who achieved accreditation for RM production for a wide range of analytes and matrices.

Goals and types of third-party assessment

Third-party assessment is an independent peer-review of activities or documents with respect to a certain standard. The intended outcome is an attestation that the activities performed or the system reviewed conform to that standard.

There are two general types of attestation, namely certification and accreditation. Certification is an attestation of compliance, whereas accreditation is an attestation of competence. Any assessment of competence requires an independent anchor point which is the true value (however assessed) in the field of measurements and certified RMs (CRMs). Accreditation is seen as superior to certification in measurement science, as in the end users of laboratory data are interested in correct measurements. For services, such an anchor point does not exist: there is not such a thing as, e.g. a “true” delivery time. It can only be assessed whether a company's promised service complies with the delivered service.

Significance to RM users

Third-party assessment is performed for numerous purposes. It checks the correct application of a management system and thus ensures the sustainability of its inherent benefits. The main advantages for CRM users are:

  • The guarantee that the materials have been produced according to technically valid and internationally recognized principles.

  • As accreditation will not be granted to producers which make only one time one reference material, users have the guarantee that the material is not a one-off product but comes from a producer with considerable experience in this field that has produced similar materials before.

  • Regular audits which are part of the accreditation procedure and surveillance of an accredited RM producer guarantee that the production of the RMs follows the documented and validated technical procedures. This eliminates the need for supplier audits.

Third-party assessment also signals transparency of and confidence in one's own procedures (“We are not afraid of letting other people look into every detail of our operations”).

All these factors should increase the confidence of users in reference materials provided by an accredited producer, even if they come from a producer with whom the customer does not have any experience yet.

For the RM producer, it is a source of acquiring new ideas through plugging into the knowledge of the auditors and their experience from several other organisations.

In a nutshell, third-party assessment should serve as means to improve customer confidence and convenience.

Third-party assessment of reference material production

Reference material production in the sense of ISO Guide 34 [3] is an integrated process comprising all steps from production planning, processing (milling, sieveing etc.), assessment of homogeneity and stability, characterization to sales, after sales service and post certification monitoring as outlined in Fig. 1.

Fig. 1
figure 1

Steps in the production of a reference material and the ISO documents applicable to each process

Full size image

There has been an intensive discussion over the last years amongst accreditation bodies, RM producers and laboratory organizations, also echoed in the scientific literature [4, 5], on the most appropriate document for third-party assessment. The suggestions most often brought forward were certification to ISO 9001, accreditation as product certification body, accreditation to ISO 17025, accreditation to ISO Guide 34/ILAC Guide 12 and combinations of the options mentioned before.

Certification to ISO 9001 (“Quality management systems - Requirements” [6]) was seen by some as ideal option. Service providers and manufacturers of many different industries are certified to ISO 9001, thus showing the versatility of the standard. This option was however not seen as sufficient by the majority of RM producers and laboratories, as certification is not a proof of competence. If third-party assessment should ensure confidence, the intrinsic quality of the materials needs to be taken into consideration. IRMM therefore did not consider certification to ISO 9001 as meaningful mode of third-party assessment for the production of reference materials.

Accreditation as product certification body seems at first glance the option of choice. After all, RM producers do certify products. However, a product certification body certifies that a product passed performance and/or quality assurance tests or qualification requirements stipulated in regulations. Such widely accepted standards do not exist for reference materials, thus rendering this option meaningless.

Accreditation to ISO 17025 (“General requirements for the competence of testing and calibration laboratories” [7]) has been brought forward as another option. This accreditation is indeed an attestation of competence rather than conformance and is well established and well known to laboratories, thus well suited to enhance confidence. Furthermore, a mutual recognition arrangement is in place that ensures that accreditation in one country is accepted by all other signatories. Last but not least, ISO 17025 refers to measurements, undoubtedly a key point in the production of reference materials. This last point is, however, also the big drawback: ISO 17025 refers only to measurements. As shown in Fig. 1, RM production includes many other activities not covered by ISO 17025, including crucial processes like homogeneity assessment or post-certification monitoring. In this context, it does not make any difference whether the accreditation is granted as a testing or a calibration laboratory, as the technical requirements are the same. IRMM therefore regarded ISO 17025 accreditation usually as not sufficient for RM production.

Accreditation to ISO Guide 34 or ILAC Guide 12 (“General requirements for the competence of reference material producers” [3] and “Guidelines for the Requirements for the Competence of Reference Materials Producers” [8] are two very similar documents explicitly directed towards reference material producers. They include all aspects of RM production and also acknowledge that a RM producer can outsource all measurements, thus does not need to have a “laboratory” of its own. It is also an attestation of competence and uses the term “accreditation” already familiar to laboratories. It is sometimes stated that ISO Guide 34 is weak on measurements. While indeed not many paragraphs in ISO Guide 34 are directly devoted to measurements, it explicitly states that for all measurements the requirements of ISO 17025 must be fulfilled. Therefore, there is no need to repeat this latter standard. According to Pauwels et al., ISO Guide 34 would be sufficient to ensure the competence of RM producers [5].

After extensive discussion amongst accreditation bodies whether RM production could be accredited at all, and if yes, to which document, the International Laboratory Accreditation Cooperation (ILAC) decided in 2004 (ILAC Resolution GA 8.12) that indeed RM production could be subject to accreditation and that accreditation should be performed by using a combination of ISO Guide 34 and ISO 17025.

It was therefore decided to achieve accreditation by using the combination of ISO 17025 and ISO Guide 34.

Scope of accreditation

Organizational scope

The organizational scope refers to the organization to be accredited. Both ISO Guide 34 and ISO 17025 put strong emphasis on organizational matters and the technical responsible needs to have the power of decision for many aspects. The broader the organizational scope is defined (i.e. the higher hierarchical level the technical responsibility is situated in the organization), the more activities can benefit from accreditation. At the same time, a broader organizational scope results also in higher costs for harmonization, e.g. illustrated by the number of signatures required for releasing a document. All of IRMM's matrix CRM activities are concentrated in the Reference Materials Unit (see Fig. 2).

Fig. 2
figure 2

Organisational setting of the RM Unit. Abbreviations used: DG: Directorate General; Env.: Environmental; GMO: Genetically Modified Organisms; Eng: Engineering; Path:pathogen

Full size image

Consequently, defining the Reference Materials Unit as the organizational scope minimized the cost of harmonization while maximized the benefits (extending the scope to other Units would have brought little benefits given their few activities in this field).

Technical scope

A second issue was the technical scope, i.e. the technical field for which competence can be proven and attested. This basically boils down to the following question: What does the successful certification of, e.g., the Pb content in hay powder demonstrate: the ability to produce all CRMs? The ability to produce CRMs from plant origin certified for trace element content? The ability to produce a hay powder reference material certified for its lead content? The answer to this question is of crucial importance as most CRMs by IRMM are produced on an infrequent basis (each batch is made to last about 10 years) and replication of the same material is relatively rare. A very narrow interpretation would make accreditation meaningless, as a new assessment for an extension of the scope would be required for every material. After intensive discussions with Beltest, the Belgian accreditation body, Beltest decided to form groups of materials that pose similar technical difficulties. Some groups represent matrix/analyte combinations as shown in Fig. 3 for materials certified for chemical composition. More groups can be created when the need arises to accommodate e.g. elements in fuel oils.

Fig. 3
figure 3

Example for an analyte-material matrix for accreditation to ISO Guide 34 for CRMs certified for chemical composition. Additional categories, e.g. elements in solutions (e.g. S in petrol) etc., may be added at a later stage

Full size image

For elements, many different measurement principles can be employed and the methods are usually well understood. For “small organic molecules", loosely defined as molecules not being macromolecules, often only one separation principle exists (either GC or HPLC) and the extraction procedures are frequently similar. However, the molecules are well known and well defined. In contrast, macromolecules often undergo conformation changes in their tertiary structure during processing or the various steps of the analytical procedure, which leads to additional traceability issues. Finally, method defined properties rely on the approach that all laboratories use exactly the same method.

Structure of the RM Unit management system

The structure of the management system of the RM Unit follows the usual three-level structure. On top is the quality manual which outlines general policies and principles. It describes the activities in a very general way and makes extensive use of links to procedures. Procedures describe the main processes still in a rather general way whereas working instructions give step by step descriptions of particular activities. These three types of documents are supported by forms and other documents to record data etc.

RM production always includes research and development activities, because never two materials are exactly the same. Given the wide variety of RMs produced by IRMM, it is obvious that detailed step-by-step descriptions of the activities in RM production (e.g. homogeneity testing) will only be applicable to a very small range of materials. To avoid unnecessary proliferation of working instructions, it was decided to base RM production on procedures, as they provide the goal, but do not prescribe a definitive way to achieve this goal. The 14 procedures which are describing in total all steps of RM production are listed in Table 1. All of these procedures prescribe a goal that must be achieved and describe a “standard” approach that usually works to achieve such goals. However, deviations of this standard approach are acceptable as long as the goals are achieved and as long as these deviations are scientifically sound, approved and documented. The goals and the main lines of the approach are described in Table 1 and follow the recent version of ISO Guide 35 [9].

Table 1 Procedures governing CRM production
Full size table

Particular emphasis is put on RM project planning: analytes to be certified, range of the certified values and their traceability are defined in the very beginning of a project. All studies are then planned with respect to measurement methods and sampling/statistical setup to achieve the envisaged uncertainty and traceability of the certified value and this information is laid down in a written and approved project plan.

Selection of collaborators is described in a separate procedure as many measurements are performed by collaborators. However, there is no difference between the requirements for in-house measurements and measurements performed by collaborators: all need to comply with ISO 17025. The technical competence of the laboratory in question and the appropriateness of the method must be checked before starting the measurement campaign. Compliance with ISO 17025 does not mean that accreditation had to be achieved, but fulfillment of the requirements of staff qualification, method validation, instrument maintenance, method documentation etc. is mandatory. For laboratories without a formal management system, method information and raw data can be stored at IRMM to ensure proper archiving.

No procedure was written for material processing (milling, sieving etc.), as this varies strongly from material to material. However, processing requirements (contamination control, avoiding cross-contamination, avoiding degradation, target characteristics) are described in the project plan. The technical part of ISO 17025 is covered by 11 procedures and most working instructions refer to the testing activities.

Preparation for accreditation and accreditation

Official start of the campaign for accreditation was April 2003 although contacts with Beltest, the Belgian accreditation organisation, had been established already in 2001 and many of the processes required for accreditation were already common practice, as they relied on the former BCR-guidelines for CRM production [10] and on the ISO Guide 35 [9], to which IRMM contributed significantly. What was lacking was explicit documentation and stricter enforcement of the procedures. As many staff members as possible were invited to comment on draft procedures to increase staff involvement. In addition, a dedicated task force had been formed to discuss all documents before final drafting. Only procedures and the management manual were centrally drafted. Working instructions were in all cases prepared by the staff members performing the actual work. The management manual and procedures were completed and approved by autumn 2003. Each document was implemented as soon as the general principles were agreed upon. At this moment, documents were presented to all staff and internal auditing started. These internal audits were also used as training and instruction sessions, making the process very effective albeit tedious. Training sessions (5 times 1/2 day) were given in October 2003 to introduce all staff members involved in CRM production to the management system. Another one-day session on laboratory procedures was given to all staff performing laboratory work in December 2003. The request for accreditation could then be submitted in January 2004, only 10 months after the official start of the project.

The initial accreditation assessment itself took place in April 2004 and comprised 5 technical assessors, namely one lead assessor, two assessors for CRM production and two assessors for the testing activities. Actual CRM projects were scrutinized and the processing, storage, distribution, sales and after-sales activities were audited within the frame of the technical assessment of CRM production. Correction of a few (testing related) non-compliances was found satisfactory in a re-audit in August 2004 and accreditation was granted in October 2004. The scope of accreditation of both testing and CRM production was extended during the surveillance audit 2005.

Granted scope of accreditation

The scope of accreditation (status: spring 2006) is shown in Table 2 and Table 3. These two tables serve as good illustration for the relationship between ISO 17025 and ISO Guide 34.

  • Impact toughness: Certification of secondary batches of Charpy samples is performed by comparing the new batch with a master batch certified by intercomparison. All measurements in the certification of a secondary batch are performed at IRMM since the installation of a reference pendulum at IRMM. As IRMM's measurements must comply with ISO 17025 requirements to be used for an ISO Guide 34 compliant RM certification, it made sense to achieve formal accreditation also for the impact toughness testing of steel samples.

  • Materials certified for the GMO (genetically modified organism) content: At present, RMs are certified for their content of genetically modified organisms on a mass basis by gravimetric preparation of GM/non-GM mixtures. Internal quality control measurements of the GM content with the help of PCR techniques are performed by IRMM. The methods used for this internal quality control are very similar – the main differences are usually the PCR primers and probes used. It is therefore advantageous to have formal accreditation also for testing of GM mixtures.

  • Solutions of small organic molecules: While the RM Unit of IRMM has demonstrated its competence in preparing such solutions, it does not hold formal accreditation. Such solutions are often certified by intercomparisons, where the RM Unit would be one laboratory amongst several others. Accreditation for the measurement is therefore not yet absolutely required.

  • Matrix materials certified for chemical composition: These materials are usually certified by intercomparison amongst expert laboratories. IRMM does not participate in all of these studies. While the RM Unit achieved ISO 17025 accreditation for some analytes and matrices, the frequency of other measurements at the RM Unit is not high enough to allow accreditation, as measurements are only performed during a certification campaign.

Table 2 Granted scope for testing according to ISO 17025
Full size table
Table 3 Granted scope for RM Production according to ISO Guide 34
Full size table

The independence of the scopes for CRM production and testing acknowledges that one can be a CRM producer without doing measurements of its own.

Conclusions

The RM Unit of IRMM obtained accreditation for its RM production activities in a combination of ISO 17025 and ISO Guide 34 (accreditation 328-T). The scope for RM production was defined in an approach comparable to the “flexible scope” for testing, thus being broad enough to make accreditation useful for future projects. The scope for RM production is independent from the one for testing, acknowledging the possibility of also using outside experts for measurements.