Draft standard CEI-MS-15: Protocol for independent… — submissions
The 12 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
12 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The limitations section states what the method cannot do but not what a reader will wrongly conclude
The respondent has read CEI-MS-15, Protocol for independent third-party verification of a supplied preparation in draft and makes one submission.
The limitations section is accurate and abstract. The respondent states that the misreadings this standard should prevent are specific and predictable, and that naming them would be more use than a general statement of scope.
The respondent proposes that the limitations section list the specific inferences a conformant result does not support.
The secretariat accepts this submission. A limitation stated abstractly is a limitation a reader will not apply to the case in front of them.
The limitations section now lists the specific conclusions a conformant determination does not support, each stated as a sentence a reader could otherwise have written, and the list is placed before the references rather than in an annex.
A determination does not record the version of the standard it was made under
This submission concerns the draft standard CEI-MS-15, Protocol for independent third-party verification of a supplied preparation. The respondent works in medicines information and answers questions from people holding certificates they cannot interpret.
The respondent states that the standard will be revised and that results reported under different versions will circulate together, indistinguishable from one another.
The respondent proposes that the standard identifier and version be carried on the face of every determination.
The secretariat accepts this submission. A conformance claim against an unspecified version is not a conformance claim.
Every determination reported under this standard now carries the standard identifier and version on its face, and a claim of conformance that does not identify the version is recorded by the Institute as not conformant.
The matrix interferes with the determination and no control is required
The respondent has read CEI-MS-15, Protocol for independent third-party verification of a supplied preparation and submits on a matter of presentation.
The respondent states that the matrices in scope can suppress or enhance the response, and that without an inhibition and enhancement control a result cannot be attributed to the analyte.
Data showing recovery outside acceptable limits in an uncontrolled matrix accompanied the submission.
The secretariat accepts this submission. A determination on an interfering matrix without a validated control establishes nothing, and the draft permitted one.
An inhibition and enhancement control is now required for every determination under this standard, with stated recovery limits, and a determination reported without it is recorded as not conformant rather than as a low result.
A report may be issued without naming who performed the determination
This is a submission on the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation, made by a respondent who assesses supplier documentation professionally.
The draft specifies the content of a result and not its provenance. A report bearing a supplier’s name may record a determination performed by that supplier, by a contract laboratory, or by neither, and the reader cannot tell which.
The respondent proposes that the name and address of the performing laboratory be a required field, and that a report which does not carry one be treated as not conforming.
The respondent notes that submission 001 has already been made and confines this submission to a matter not covered by it.
The secretariat accepts this submission. The independence of a determination is the substance of what a third-party report offers.
The performing laboratory is now a required field, and a report without one is recorded as non-conforming with the standard rather than as a report of unknown provenance.
A sortable table implies a comparison the underlying data do not support
The respondent submits on the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation. A documentation standard is the one standard whose failure a purchaser can detect without a laboratory, which makes it the most consequential of the series.
The draft presents a sortable table whose columns are drawn from sources of differing quality. The respondent states that sorting on such a column produces an ordering that looks like a ranking and is not one.
The respondent proposes that sorting be disabled on any column whose values are not commensurable.
The secretariat notes this submission and records that the point is correct in principle.
No amendment arises here because every sortable table in the document set already carries a standing statement above it that the ordering is not a ranking and that the values in each column are commensurable only where the column header says so. The proposal to disable sorting was considered and not adopted, because a reader who cannot sort a table generally sorts it elsewhere and without the statement.
Nothing prevents a chromatogram being reprocessed until it passes
The respondent’s comment on the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation arises from handling certificates of this kind in practice.
Integration parameters can be adjusted after acquisition, and nothing in the draft requires the sequence of processing to be recorded. A result presented as a single determination may be the last of several.
The respondent proposes that the standard require an audit trail of processing, retained with the primary data.
The secretariat accepts the principle and adopts a narrower requirement.
The standard now requires the processing method and its version to be recorded with every result and the primary data to be retained. A full audit trail is recommended and not required, since it depends on software many laboratories in scope do not run.
The standard should state a pass limit
The draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation was read from the standpoint of a hospital pharmacy that receives certificates with material.
The respondent states that a standard which reports a figure without a limit leaves the reader without a criterion, and that suppliers will set their own.
The respondent proposes a universal acceptance limit for the determination.
The respondent has read submission 003 with interest and adds one observation the secretariat may find useful.
The secretariat does not accept this submission. An appropriate limit depends on the compound, the presentation and the intended route, none of which the standard fixes.
The standard states performance and reports the figure as determined. Setting a universal limit would attach a pass or fail to materials whose acceptable ranges genuinely differ, which would be a stronger claim than the Institute can support. The submission remains published in full and the point is recorded in the limitations.
Acceptance criteria are expressed in terms specific to one instrument family
This submission addresses the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation from the standpoint of a reader who will rely on a conformance claim against it.
The respondent states that two criteria are expressed in units reported by one manufacturer's software and not by others, which makes conformance a function of the instrument rather than of the determination.
The respondent proposes that criteria be expressed in instrument-independent quantities.
The secretariat accepts this submission in part. Both criteria are restated in instrument-independent terms. The conversion from the manufacturer-specific quantity is given rather than removed, because laboratories will continue to read it from their software.
The acceptance criteria are now expressed in instrument-independent quantities, with the conversion from the common manufacturer-specific reporting units given in a note so that a laboratory can check conformance without recalculating by hand.
The draft prescribes a procedure where it should state performance
This is a submission on CEI-MS-15, Protocol for independent third-party verification of a supplied preparation.
Several clauses specify a gradient, a column chemistry and a temperature. The respondent states that fixing these prevents a laboratory from using a validated equivalent, and that the Institute settled on stating performance when it scoped the series.
The respondent proposes that the prescriptive clauses be converted into performance requirements with the prescribed conditions retained as a worked example.
The secretariat accepts this submission. The draft departed from the position settled when the series was scoped, and the departure was not deliberate.
The prescriptive clauses are converted to performance requirements, the specific conditions are retained as a worked example clearly labelled as one, and a laboratory using a different validated procedure that meets the stated performance reports a conformant determination.
The standard permits a purity figure to be described as an assay
Having read the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation, the respondent puts one point to the secretariat.
The terms are used interchangeably in the draft. They are different quantities: one is the proportion of the chromatographic response attributable to the principal peak, the other is the mass of the named substance per unit mass of the preparation. A certificate that reports the first under the name of the second overstates content by the mass of counter-ion and water.
The respondent proposes that the two be defined separately in the terminology section and that the standard prohibit the second term being used for the first.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 007.
The secretariat accepts this submission. The conflation is the commonest arithmetic error in this supply channel and the standard should not license it.
The two quantities are now separately defined and the standard states that a chromatographic purity may not be reported as an assay result.
The reporting threshold is not tied to a demonstrated limit of quantitation
The respondent read the draft of CEI-MS-15, Protocol for independent third-party verification of a supplied preparation and has confined this submission to a single requirement.
The draft sets a reporting threshold as a fixed percentage. The respondent states that whether a laboratory can quantify at that threshold depends on the signal-to-noise achieved, and that the threshold should be conditional on demonstrating it.
The respondent proposes that a determination be reportable only where the signal-to-noise at the threshold is demonstrated in the run.
This point is adjacent to the one made in submission 008 and the respondent puts it in a form the secretariat can act on.
The secretariat accepts this submission. A threshold no one has demonstrated is a convention rather than a limit.
The reporting threshold is now conditional on demonstrating the required signal-to-noise at that threshold within the run, and a determination that cannot demonstrate it reports the achievable threshold instead of the nominal one.
Point estimates are given without an interval
The respondent notes that CEI-MS-15, Protocol for independent third-party verification of a supplied preparation will be used as a checklist by people who have not read the rest of the series, and submits with that in view.
Several estimates in the draft appear as single figures. The respondent states that a point estimate without an interval invites a precision the underlying data do not support, and that the effect is worst where the estimate is drawn from a small contributing set.
The respondent proposes that no point estimate appear anywhere in the document set without its interval, including in summary tables and in the abstract.
The respondent notes submission 001 above and does not repeat the ground it covers.
The secretariat accepts this submission in part. Intervals are added wherever the source reports one. The proposal is declined for figures the source published without an interval, because the Institute will not compute an interval a source did not report.
Every estimate now carries its interval where the source reported one, and where it did not, the estimate is annotated as reported without an interval rather than left to appear as a precise figure.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.