Draft conflicts-of-interest policy — submissions
The 12 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
12 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
Independence is asserted and the mechanism securing it is not described
The draft of conflicts-of-interest policy was read for what it obliges the Institute to do rather than for what it says the Institute intends.
The document states that sponsorship does not influence assessment. It does not describe what prevents it from doing so, and an assertion without a mechanism is not verifiable.
The respondent proposes that the separation between the party accepting funding and the party making assessments be described in operational terms.
The secretariat accepts this submission.
The governance document now describes the separation in operational terms: who accepts funding, who makes assessments, that the two sets do not overlap, and that assessment staff are not informed of placement decisions before publication.
What the site collects about a reader is not stated in terms a reader can check
This is a submission on conflicts-of-interest policy, from a respondent representing readers rather than practitioners.
The respondent states that a body assessing evidence should be checkable in its own conduct, and that a general privacy statement does not permit a reader to verify what is collected.
The respondent proposes that the statement enumerate what is collected and that it be verifiable from the page itself.
The respondent supports submission 001 so far as it goes and adds the matter set out here.
The secretariat accepts this submission. An unverifiable statement about data is exactly the kind of claim the Institute declines to accept from others.
The statement now enumerates what is collected and what is not, and is written so that a reader can verify it from the page source, which is the only form of the statement that can be checked without trusting the Institute.
Whether external reviewers are paid is not stated
The respondent read conflicts-of-interest policy in draft and has confined this submission to one clause.
The respondent states that payment or its absence bears on who is able to review and on what the Institute may expect of them, and that the policy does not address it.
The respondent proposes that the position be stated and that any payment be disclosed at the level of the individual document.
The secretariat accepts this submission. Payment is an interest and the policy treats interests as facts to be published.
The policy now states whether external review is remunerated, records any payment made in respect of a document in that document's front matter, and treats a payment received from the Institute as a declarable interest in the same way as any other.
The search date is not on the face of the document
Having read the draft of conflicts-of-interest policy, the respondent puts one point to the secretariat.
The draft carries a publication date and a review date but not the date on which the evidence was last searched. Those are three different dates and only the third tells a reader how current the assessment is. A document published in one quarter may rest on a search run two quarters earlier, and nothing on the page allows that gap to be measured.
The respondent proposes that the search date be printed adjacent to every certainty rating rather than in the methods section, on the ground that a reader who acts on a rating is unlikely to have read the methods section first.
The secretariat accepts this submission. The distinction between publication, review and search dates is real and the draft did not make it visible where it mattered.
The search date is now printed adjacent to every certainty rating and is carried in the document metadata, so that the interval between the search and the reading is visible without reference to the methods section.
There is no route to challenge a disposition
The respondent’s comment on conflicts-of-interest policy is made from the standpoint of a person who might be affected by a decision taken under it.
The respondent states that a submission not accepted receives a reason and nothing further, and that a respondent who considers the reason mistaken has nowhere to go.
The respondent proposes a review of disposition by a person who did not make it.
This point is adjacent to the one made in submission 004 and the respondent puts it in a form the secretariat can act on.
The secretariat accepts this submission. A disposition that cannot be challenged is a decision without a check.
A disposition may now be referred for review to a member of the assessment committee who did not author the response, once per submission, with the referral and its outcome published alongside the original disposition whether or not it changes.
Absence of evidence is presented in a form a reader will take as negative evidence
The respondent notes that conflicts-of-interest policy governs situations that will arise rarely and matter greatly when they do, and submits accordingly.
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
The materiality threshold for a declarable holding is set too high
This submission concerns conflicts-of-interest policy and a convention used across the Institute’s output.
The respondent states that the threshold in the draft excludes holdings a reader would regard as relevant, and that the cost of declaring a small holding is negligible against the cost of a reader discovering an undeclared one.
The respondent proposes that all holdings be declared regardless of size.
The secretariat accepts this submission in part. The threshold is lowered. Declaration of every holding without threshold is not adopted, because an undifferentiated list obscures the holdings that matter.
The materiality threshold is lowered and is now expressed as a proportion of the declarer's holdings as well as an absolute figure, so that a holding material to the person is declarable whether or not it is large in absolute terms.
A four-week window is too short for a respondent without institutional support
The respondent submits on conflicts-of-interest policy. Governance documents are read by very few people and relied on by everyone who reads anything else the Institute publishes.
An organisation with staff can answer a consultation in four weeks. An individual reading in their own time, particularly one who needs the document explained before they can respond to it, generally cannot.
The respondent proposes a minimum of eight weeks for any consultation on a document of general application.
Submission 004 concerns the same document. The respondent’s point is a different one.
The secretariat accepts the principle and adopts a shorter minimum than proposed.
The minimum window is now six weeks for a document of general application and four for a document-specific draft. A respondent who asks for more time before a window closes is given it, and that is now stated in the policy.
Funding should not be accepted from any party the Institute assesses
This submission addresses conflicts-of-interest policy from the standpoint of an outside body that would have to rely on it.
The respondent states that disclosure does not neutralise an incentive, and that accepting funding from an assessed party creates one whether or not it is acted on.
The respondent proposes an outright prohibition on funding from any assessed supplier or manufacturer.
The secretariat accepts this submission in part. Funding from an assessed party is not accepted for assessment work. A general prohibition covering all activity is not adopted, because it would extend to unrestricted contributions the Institute can accept without any assessment consequence.
The policy now prohibits funding from an assessed party for any activity bearing on that party's assessment, requires that any other relationship be published on the funding page and on every page it could bear on, and records the sums involved rather than the fact of a relationship alone.
Where a document is generated rather than authored, the reader should be told
The respondent submits on conflicts-of-interest policy, on a matter that is not specific to this draft but is visible in it.
The respondent states that some records in the document set are constructed by crossing existing records, that the prose in them is templated, and that nothing on the page distinguishes them from an authored assessment.
The respondent proposes a standing notice on every generated record.
The secretariat accepts this submission. The distinction is material to how much independent judgement a reader should assume.
Every generated record now carries a notice stating that its prose frame is templated and that its included studies, summary of findings and certainty rating are computed from the underlying records, so that the reader can see which parts were authored and which were derived.
No term limit applies to membership of the assessment committee
This submission concerns the draft of conflicts-of-interest policy. The respondent has served on committees operating comparable regimes and the observation draws on that.
Members are appointed and the document does not say for how long. A body whose membership does not turn over converges on a settled view and stops noticing that it has one.
The respondent proposes a fixed term, renewable once, with the appointment date published against each member.
The respondent notes submission 003 above and does not repeat the ground it covers.
The secretariat accepts this submission.
Members are appointed for a four-year term renewable once. Appointment and expiry dates are published in the register of members.
A reader who acted on a document is not told when it is corrected
The respondent has read conflicts-of-interest policy in draft and makes one submission.
Corrections are published in a log. Nobody who read the original page is notified, and the corrected page carries the correction in a form that a reader arriving fresh will read past.
The respondent proposes that a corrected document carry a visible notice at the point of the correction for a stated period, in addition to the log entry.
The secretariat accepts this submission. A correction filed where only the diligent will find it is a correction the reader who needs it will miss.
A corrected passage now carries an inline notice for twelve months from the date of correction, naming the log entry, in addition to the entry itself.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.