Draft policy: labelling of Institute-constructed programme… — submissions
The 13 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
13 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A party that is assessed and also sponsors should be disclosed on its own assessment
The respondent submits on policy: labelling of Institute-constructed programme records.
Disclosure on the funding page is necessary and is not sufficient. The reader who most needs to know is the one reading that party’s assessment, and that reader has no reason to visit the funding page.
The respondent proposes that the disclosure appear on the assessment itself, in the same visual weight as the assessment summary.
The secretariat accepts this submission without qualification. It was a real defect.
The disclosure now appears as a numbered section on the dossier of any assessed party that sponsors the site, as well as on the funding page.
The document is unreadable without specialist training
The respondent notes that policy: labelling of Institute-constructed programme records governs situations that will arise rarely and matter greatly when they do, and submits accordingly.
The respondent states that the draft is written for a reader who already understands certainty grading, and that the people most affected by the subject matter will not reach the assessment at all.
The respondent proposes a plain-language summary at the head of every document, written to the same standard of accuracy as the document itself and not as a promotional abstract.
This submission is made by a respondent with a different professional interest in the outcome from the one behind submission 001.
The secretariat accepts this submission in part. A plain-language summary is added. The proposal that it replace the technical abstract is declined, because the abstract is the part of the document other assessors read and cite.
Every document now opens with a plain-language summary of not more than 150 words, placed above the technical abstract and carrying the same certainty language, so that the two cannot diverge.
There is no route to challenge a disposition
This submission concerns the draft of policy: labelling of Institute-constructed programme records. The respondent has served on committees operating comparable regimes and the observation draws on that.
The respondent states that a submission not accepted receives a reason and nothing further, and that a respondent who considers the reason mistaken has nowhere to go.
The respondent proposes a review of disposition by a person who did not make it.
The secretariat accepts this submission. A disposition that cannot be challenged is a decision without a check.
A disposition may now be referred for review to a member of the assessment committee who did not author the response, once per submission, with the referral and its outcome published alongside the original disposition whether or not it changes.
The Institute publishes no telephone number and no messaging channel
The respondent has read policy: labelling of Institute-constructed programme records in draft and makes one submission.
The respondent states that a body inviting submissions ordinarily provides a channel for questions about the process, and that the written address alone is a high barrier for a first-time respondent.
The respondent asks that a route for procedural questions be provided.
The secretariat notes this submission. The written route is the only route and that is a deliberate constitutional position rather than an omission.
No amendment arises. The Institute holds no telephone number and uses no messaging channel, because a submission made in writing is a document that can be published with its response and an oral exchange is not. Guidance on making a submission has been expanded so that the written route requires no prior enquiry to use.
The search date is not on the face of the document
This submission concerns policy: labelling of Institute-constructed programme records and makes one point.
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
A four-week window is too short for a respondent without institutional support
The respondent submits on policy: labelling of Institute-constructed programme records. Governance documents are read by very few people and relied on by everyone who reads anything else the Institute publishes.
An organisation with staff can answer a consultation in four weeks. An individual reading in their own time, particularly one who needs the document explained before they can respond to it, generally cannot.
The respondent proposes a minimum of eight weeks for any consultation on a document of general application.
The secretariat accepts the principle and adopts a shorter minimum than proposed.
The minimum window is now six weeks for a document of general application and four for a document-specific draft. A respondent who asks for more time before a window closes is given it, and that is now stated in the policy.
The policy does not say what happens to a document the Institute is asked to remove
Having read the draft of policy: labelling of Institute-constructed programme records, the respondent puts one point to the secretariat.
The respondent states that assessments of named parties will attract removal requests, and that a policy silent on them will be applied inconsistently under pressure.
The respondent proposes that the grounds on which a document may be withdrawn be stated in advance and be exhaustive.
The secretariat accepts this submission. A policy written after the first request will be written under the conditions least suited to writing it.
The grounds for withdrawal are now stated exhaustively, being a factual error that cannot be corrected in place and a legal obligation, a withdrawn document is replaced by a notice recording the fact and the ground rather than removed, and every withdrawal is recorded in the corrections log.
The terms on which the documents may be reused are not stated
This is a submission on policy: labelling of Institute-constructed programme records, from a respondent representing readers rather than practitioners.
The respondent states that the site carries no statement of reuse terms and that an assessment which cannot be quoted with confidence will be paraphrased instead, which is worse for accuracy.
The respondent proposes an explicit reuse statement.
The respondent read submission 007 after drafting this one and has not altered it, the two points being distinct.
The secretariat accepts this submission. An unstated position on reuse is a position that will be guessed at.
The reuse terms are now stated explicitly on the legal page and in the document footer, permitting quotation and reproduction with attribution and the document version, and requiring that any reproduction carry the certainty rating that accompanied the quoted statement.
The secretariat answers submissions and is not itself named
The respondent has read policy: labelling of Institute-constructed programme records in draft and makes a single submission.
The respondent states that responses are attributed to the secretariat as a body, and that a reader cannot tell who wrote a response or whether that person had a relevant interest.
The respondent proposes that responses be signed individually.
The secretariat accepts this submission in part. Responses are attributed to the named members of the secretariat responsible for the document rather than to individuals per response, and the interests of those members are published.
Every comment period now names the secretariat members responsible for it, with their declared interests, in the consultation front matter, so that the responses can be read against the interests of the people who wrote them.
A reader who acted on a document is not told when it is corrected
This submission addresses policy: labelling of Institute-constructed programme records from the standpoint of an outside body that would have to rely on it.
Corrections are published in a log. Nobody who read the original page is notified, and the corrected page carries the correction in a form that a reader arriving fresh will read past.
The respondent proposes that a corrected document carry a visible notice at the point of the correction for a stated period, in addition to the log entry.
The secretariat accepts this submission. A correction filed where only the diligent will find it is a correction the reader who needs it will miss.
A corrected passage now carries an inline notice for twelve months from the date of correction, naming the log entry, in addition to the entry itself.
A party criticised in an assessment should have a right of reply
The respondent read policy: labelling of Institute-constructed programme records in draft and has confined this submission to one clause.
The respondent states that a supplier or sponsor whose documentation is assessed adversely has no route to respond other than a comment period that may not be open.
The respondent proposes a standing right of reply published alongside the assessment.
The secretariat accepts this submission in part. A route to submit a correction of fact is opened at any time and is not dependent on a consultation window. A general right of reply published alongside the assessment is not adopted, because it would give an assessed party the last word on the Institute's own document.
A standing correction route is now published, corrections of fact are assessed and applied at any time with the change recorded in the amendment log, and a submission of substance received outside a window is held and published in the next comment period on that document.
Funding should not be accepted from any party the Institute assesses
The draft of policy: labelling of Institute-constructed programme records was read for what it obliges the Institute to do rather than for what it says the Institute intends.
The respondent states that disclosure does not neutralise an incentive, and that accepting funding from an assessed party creates one whether or not it is acted on.
The respondent proposes an outright prohibition on funding from any assessed supplier or manufacturer.
The secretariat accepts this submission in part. Funding from an assessed party is not accepted for assessment work. A general prohibition covering all activity is not adopted, because it would extend to unrestricted contributions the Institute can accept without any assessment consequence.
The policy now prohibits funding from an assessed party for any activity bearing on that party's assessment, requires that any other relationship be published on the funding page and on every page it could bear on, and records the sums involved rather than the fact of a relationship alone.
Whether external reviewers are paid is not stated
The respondent’s comment on policy: labelling of Institute-constructed programme records is made from the standpoint of a person who might be affected by a decision taken under it.
The respondent states that payment or its absence bears on who is able to review and on what the Institute may expect of them, and that the policy does not address it.
The respondent proposes that the position be stated and that any payment be disclosed at the level of the individual document.
The respondent has read submission 012 and asks that this submission be considered with it.
The secretariat accepts this submission. Payment is an interest and the policy treats interests as facts to be published.
The policy now states whether external review is remunerated, records any payment made in respect of a document in that document's front matter, and treats a payment received from the Institute as a declarable interest in the same way as any other.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.