Draft standard CEI-MS-13: Development and validation of a… — submissions
The 13 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
13 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
The document should state what a reader ought to do
The respondent submits on the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide…. A method that has not been shown to detect the degradation products of the compound it measures is not measuring what a certificate says it is.
The draft assesses evidence and stops. The respondent, a practising clinician, states that a reader arriving at the document with a decision to make is left to convert an assessment into an action without help, and proposes that each document close with a recommendation.
The respondent argues that other evidence bodies issue recommendations and that declining to do so transfers the difficult part of the work to the reader.
The secretariat does not accept this submission, and records that the point is a reasonable one rather than a misunderstanding.
The Institute assesses evidence and does not issue recommendations, because a recommendation embeds values and a resource context that the Institute does not hold and cannot state. That constitutional limit is published on the methodology page and is not varied by consultation. The submission remains published in full.
A determination does not record the version of the standard it was made under
The respondent submits on CEI-MS-13, Development and validation of a stability-indicating method for peptide…. The point would apply equally to any document in the series.
The respondent states that the standard will be revised and that results reported under different versions will circulate together, indistinguishable from one another.
The respondent proposes that the standard identifier and version be carried on the face of every determination.
The secretariat accepts this submission. A conformance claim against an unspecified version is not a conformance claim.
Every determination reported under this standard now carries the standard identifier and version on its face, and a claim of conformance that does not identify the version is recorded by the Institute as not conformant.
Nothing prevents a chromatogram being reprocessed until it passes
This submission concerns CEI-MS-13, Development and validation of a stability-indicating method for peptide… and makes one point.
Integration parameters can be adjusted after acquisition, and nothing in the draft requires the sequence of processing to be recorded. A result presented as a single determination may be the last of several.
The respondent proposes that the standard require an audit trail of processing, retained with the primary data.
The respondent notes that submission 002 has already been made and confines this submission to a matter not covered by it.
The secretariat accepts the principle and adopts a narrower requirement.
The standard now requires the processing method and its version to be recorded with every result and the primary data to be retained. A full audit trail is recommended and not required, since it depends on software many laboratories in scope do not run.
The matrix interferes with the determination and no control is required
The respondent notes that CEI-MS-13, Development and validation of a stability-indicating method for peptide… will be relied on by laboratories operating under different conditions from the one that developed the method, and submits with that in view.
The respondent states that the matrices in scope can suppress or enhance the response, and that without an inhibition and enhancement control a result cannot be attributed to the analyte.
Data showing recovery outside acceptable limits in an uncontrolled matrix accompanied the submission.
The respondent endorses the general approach taken in submission 001 and asks that it be extended to the matter identified here.
The secretariat accepts this submission. A determination on an interfering matrix without a validated control establishes nothing, and the draft permitted one.
An inhibition and enhancement control is now required for every determination under this standard, with stated recovery limits, and a determination reported without it is recorded as not conformant rather than as a low result.
Nothing addresses the container the material was held in
This submission addresses the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide… from an analytical standpoint.
Adsorption to the container and oxidation in the headspace both change what is measured, and both depend on the container rather than on the material. The standard treats the vial as neutral.
The respondent proposes that container type and headspace atmosphere be reported where they are known.
The secretariat accepts the reporting requirement where the information is held and declines to require it where it is not.
Container type and headspace atmosphere are now reported where known, and recorded as not held where they are not, rather than omitted.
The draft prescribes a procedure where it should state performance
The draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide… was read from the standpoint of a consultancy that performs method validation.
Several clauses specify a gradient, a column chemistry and a temperature. The respondent states that fixing these prevents a laboratory from using a validated equivalent, and that the Institute settled on stating performance when it scoped the series.
The respondent proposes that the prescriptive clauses be converted into performance requirements with the prescribed conditions retained as a worked example.
The secretariat accepts this submission. The draft departed from the position settled when the series was scoped, and the departure was not deliberate.
The prescriptive clauses are converted to performance requirements, the specific conditions are retained as a worked example clearly labelled as one, and a laboratory using a different validated procedure that meets the stated performance reports a conformant determination.
Results are reported without an estimate of measurement uncertainty
The respondent’s comment on the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide… arises from validating methods for materials in its scope.
The respondent, an analytical chemist, states that a figure reported to one decimal place implies an uncertainty the standard nowhere estimates, and that two determinations differing within the uncertainty will be read as a difference.
The respondent proposes that an uncertainty budget be required for every determination.
The secretariat accepts this submission in part. A repeatability figure and a stated reporting precision are required. A full uncertainty budget is not, because the standard is written to be usable by laboratories that do not maintain one and the requirement would exclude them without improving the reading.
Every determination now reports the repeatability observed in the run, the reporting precision, and a statement that two results differing by less than the stated repeatability are not distinguishable by this method.
Nothing states how long a prepared standard solution remains usable
This is a submission on CEI-MS-13, Development and validation of a stability-indicating method for peptide….
The procedure requires a reference solution and does not say for how long it may be used. Peptide solutions at working concentration lose material to surfaces and to hydrolysis, and a solution used for a week will bias every result obtained against it in the same direction.
The respondent proposes that the standard require solution stability to be demonstrated over the period of use, or that a maximum period be stated.
The secretariat accepts this submission. A systematic bias applied uniformly to a sequence is the hardest kind to notice.
The procedure now requires solution stability to be demonstrated over the period of intended use, and the demonstration to be recorded with the result.
Point estimates are given without an interval
The respondent read the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide… and has confined this submission to a single requirement.
Several estimates in the draft appear as single figures. The respondent states that a point estimate without an interval invites a precision the underlying data do not support, and that the effect is worst where the estimate is drawn from a small contributing set.
The respondent proposes that no point estimate appear anywhere in the document set without its interval, including in summary tables and in the abstract.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 006.
The secretariat accepts this submission in part. Intervals are added wherever the source reports one. The proposal is declined for figures the source published without an interval, because the Institute will not compute an interval a source did not report.
Every estimate now carries its interval where the source reported one, and where it did not, the estimate is annotated as reported without an interval rather than left to appear as a precise figure.
The standard should state a pass limit
This submission concerns the draft standard CEI-MS-13, Development and validation of a stability-indicating method for peptide…. The respondent’s interest is in what a validation package has to demonstrate before a figure derived from it may be reported.
The respondent states that a standard which reports a figure without a limit leaves the reader without a criterion, and that suppliers will set their own.
The respondent proposes a universal acceptance limit for the determination.
Submission 004 concerns the same document. The respondent’s point is a different one.
The secretariat does not accept this submission. An appropriate limit depends on the compound, the presentation and the intended route, none of which the standard fixes.
The standard states performance and reports the figure as determined. Setting a universal limit would attach a pass or fail to materials whose acceptable ranges genuinely differ, which would be a stronger claim than the Institute can support. The submission remains published in full and the point is recorded in the limitations.
Nothing in the standard addresses transfer of the method between laboratories
Having read the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide…, the respondent puts one point to the secretariat.
The respondent states that determinations under this standard will be performed by different laboratories on the same material and that without transfer criteria their results are not comparable.
The respondent proposes a transfer protocol with acceptance criteria for inter-laboratory agreement.
The secretariat accepts this submission in part. Acceptance criteria for agreement between laboratories are added. A full transfer protocol is not, because transfer is a quality-system activity rather than a determination and the series does not address quality systems.
The standard now states the maximum difference between laboratories analysing the same sample at which results are treated as agreeing, and requires that a comparison across laboratories state whether that criterion was met.
Nothing requires a laboratory to demonstrate ongoing performance
This is a submission on the draft of CEI-MS-13, Development and validation of a stability-indicating method for peptide…, from a laboratory that has transferred methods of this kind between instruments.
The respondent states that suitability criteria assessed within a run demonstrate that the run worked and not that the laboratory performs consistently over time.
The respondent proposes participation in a proficiency scheme as a condition of conformance.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 011.
The secretariat accepts this submission in part. A trend requirement on system suitability data is added. Proficiency scheme participation is not made a condition of conformance, for the same reason competence requirements are not.
The standard now requires that system suitability results be trended across runs and that a determination report whether the trend was within the laboratory's stated limits, which places the demonstration inside the document the Institute can read.
The document set should be published in translation
The respondent has read CEI-MS-13, Development and validation of a stability-indicating method for peptide… in draft and makes one submission.
The respondent notes that the assessments concern compounds supplied internationally and that publishing only in English restricts access to the assessment to readers who work in it.
The respondent proposes machine translation of the document set as an interim measure, with human review of the certainty language.
The secretariat does not accept this submission, and records that the underlying point is sound and that the proposed remedy is the difficulty.
A translation whose certainty language has drifted is a different assessment carrying the Institute's name, and the Institute cannot review translations it does not have the capacity to review. The documents remain in English. The submission is published in full because the access problem it identifies is real and unresolved.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.