Draft scope of the analytical and methodological standards… — submissions
The 9 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
9 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A printed copy loses the document identifier
The respondent submits on scope of the analytical and methodological standards series. A document series is a set of promises about where things will be found, and this submission concerns one of them.
A page printed and circulated arrives without the identifier, the version or the date, so a recipient cannot tell whether it is current or find it again.
The respondent proposes that the print stylesheet carry all three in a footer on every page.
The secretariat accepts this submission.
The print stylesheet now carries the document identifier, the version, the publication date and the address in a footer on every printed page.
A printed copy loses the information that makes it checkable
The respondent has read scope of the analytical and methodological standards series in draft and makes one submission.
The respondent states that printing a document produces pages without the identifier, the version or the retrieval date, so that a printed copy cannot be tied back to what it was printed from.
The respondent proposes that these be printed in a fixed position on every page.
The respondent read submission 001 after drafting this one and has not altered it, the two points being distinct.
The secretariat accepts this submission. A printed assessment circulates for longer than the browser tab it came from.
The print stylesheet now carries the identifier, version, publication date and retrieval date on every page, expands link destinations into the text, and prints the full reference list rather than a link to it.
References should carry a persistent identifier for every cited source
The respondent’s comment on scope of the analytical and methodological standards series arises from consulting the series for reference rather than writing for it.
Several references in the draft carry a journal, a year and a volume but no persistent identifier. The respondent states that retrieval of such a reference is materially slower and that identifiers should be supplied throughout.
The respondent asks in the alternative that where an identifier exists but is not carried, the omission be explained rather than left as a gap the reader must interpret.
The secretariat accepts the second limb of this submission and declines the first. Identifiers are supplied wherever the Institute holds one. Where the Institute does not hold an identifier it will not supply one, because a reconstructed identifier that resolves to the wrong record is a worse defect than an absent one.
Every reference without a persistent identifier now carries an explicit statement that the identifier is not held by the Institute, so that its absence is a recorded fact rather than an apparent oversight.
Defined terms are defined in a glossary the reader has to know exists
Having read the draft of scope of the analytical and methodological standards series, the respondent puts one point to the secretariat.
The glossary is thorough. A reader meeting indirectness in §3 has no indication that it has a specific meaning here.
The respondent proposes that every defined term be linked at first use in every document.
The secretariat accepts the principle and applies it once per document.
Defined terms are linked at first use in each document. Linking every occurrence was rejected because a page in which a third of the words are links is harder to read than one in which none are.
The registry is published only as pages and not in a form that can be processed
This submission addresses scope of the analytical and methodological standards series from the standpoint of a reader arriving from a search engine rather than from the front page.
The respondent, who works with the register programmatically, states that examining the document set as a whole requires extracting fields from rendered pages, and that a structured record would make the registry usable for research about the Institute itself.
The respondent proposes that the registry fields be published in a structured form alongside the pages.
The secretariat accepts this submission. A body that asks others to publish checkable records should publish one.
The registry fields are now emitted in a structured form alongside the rendered pages, carrying the identifier, series, type, version, dates, certainty rating and status for every document, and the structured record is regenerated with the site so that it cannot drift from the pages.
A figure carries information that appears nowhere in text
This is a submission on scope of the analytical and methodological standards series, from a respondent who reads these documents on paper.
Several figures present the only statement of a quantity in the document. A reader using a screen reader, or printing in monochrome, does not receive it.
The respondent proposes that no figure carry information not stated in the surrounding text or in a data table.
The secretariat accepts this submission.
Every figure in the series now restates its data in the caption or in an accompanying table, and no quantity appears only in a figure.
The same quantity is rounded differently in two parts of one document
The respondent notes that scope of the analytical and methodological standards series will govern a great many documents, and submits accordingly.
A figure appears to one decimal place in the results table and to none in the summary, and in one case the rounding changes which of two values is larger.
The respondent proposes a single rounding convention applied at the point the quantity is computed rather than at each point it is displayed.
Submission 005 concerns the same document. The respondent’s point is a different one.
The secretariat accepts this submission.
Quantities are now rounded once, at computation, and displayed identically wherever they appear. The convention is stated in the series description.
A field the Institute could not verify should be shown rather than omitted
The draft of scope of the analytical and methodological standards series was read from the standpoint of someone who will navigate the series rather than write for it.
The respondent states that omitting an unverifiable field leaves a reader unable to distinguish between a fact that does not exist, a fact the Institute did not seek and a fact it sought and could not confirm.
The respondent proposes that unverified fields be displayed with their status.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 004.
The secretariat accepts this submission. Three different situations were being rendered identically as blank space.
Every field now carries one of four states, being verified, claimed and unverified, sought and not established, or not applicable, and the state is displayed rather than inferred from an empty cell.
An index organised by certainty invites comparison between ratings that are not comparable
The respondent read scope of the analytical and methodological standards series in draft and has confined this submission to a single provision.
The respondent states that a rating assigned to an analytical question and a rating assigned to a clinical outcome sit in the same band in the index, and that placing them together implies a comparability the framework does not claim.
The respondent proposes that the index be organised by domain first and certainty second.
The secretariat accepts this submission in part. The index is grouped by domain. It continues to offer a certainty view, because the whole purpose of the index is to show how much of the output rests on weak evidence.
The index is now grouped by domain with certainty as the ordering within each group, and a standing statement records that ratings in different domains are not directly comparable and states why.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.