Draft citation policy — submissions
The 9 submissions received, published in full with declared interests and secretariat responses.
§2Submissions and responses
9 submissions were received. Each is published in full below with its declared interest, the secretariat response and the disposition. The Institute publishes submissions it did not accept in the same form as those it did.
A party that is assessed and also sponsors should be disclosed on its own assessment
The respondent submits on citation policy. Governance documents are read by very few people and relied on by everyone who reads anything else the Institute publishes.
Disclosure on the funding page is necessary and is not sufficient. The reader who most needs to know is the one reading that party’s assessment, and that reader has no reason to visit the funding page.
The respondent proposes that the disclosure appear on the assessment itself, in the same visual weight as the assessment summary.
The secretariat accepts this submission without qualification. It was a real defect.
The disclosure now appears as a numbered section on the dossier of any assessed party that sponsors the site, as well as on the funding page.
Where a document is generated rather than authored, the reader should be told
The draft of citation policy was read for what it obliges the Institute to do rather than for what it says the Institute intends.
The respondent states that some records in the document set are constructed by crossing existing records, that the prose in them is templated, and that nothing on the page distinguishes them from an authored assessment.
The respondent proposes a standing notice on every generated record.
The secretariat accepts this submission. The distinction is material to how much independent judgement a reader should assume.
Every generated record now carries a notice stating that its prose frame is templated and that its included studies, summary of findings and certainty rating are computed from the underlying records, so that the reader can see which parts were authored and which were derived.
The position of a contributor who withdraws consent is not stated
The respondent has read citation policy in draft and makes one submission.
The respondent states that consent to be named is obtained at submission and that the policy is silent on what happens if it is later withdrawn, which leaves both the Institute and the contributor without a rule.
The respondent proposes that the position be stated in advance and be the same for everyone.
The secretariat accepts this submission. A rule made at the point of the first withdrawal is a rule made about a particular person.
The policy now states that a contributor may withdraw from the register at any time and that the register entry is removed, while a signed submission already published remains published, because the consultation record is a record of what was said and by whom. The position is stated at the point consent is given.
The decision rule for a contested certainty rating is not stated
This is a submission on citation policy, from a respondent representing readers rather than practitioners.
The document describes the committee and not how it decides. Where members disagree about a rating, a reader cannot tell whether the published rating is a majority, a consensus, or the view of the chair.
The respondent proposes that quorum, the decision rule and the treatment of a tie be stated, and that a rating carried on a divided committee be marked as such.
The respondent’s submission overlaps with submission 001 and was prepared without sight of it.
The secretariat accepts this submission. A rating is a judgement and how it was arrived at is part of what a reader is entitled to weigh.
Quorum, the decision rule and the treatment of a tie are now stated. A rating carried on a divided committee is marked and the dissenting position is published with it.
Nothing states what happens to the document set if the Institute ceases to operate
The respondent read citation policy in draft and has confined this submission to one clause.
Documents are cited. A citation to a document that has ceased to exist is worse than no citation, and the policy is silent on what would become of the archive.
The respondent proposes a stated succession arrangement and an open licence that permits the set to be mirrored.
The secretariat accepts this submission.
The governance document now records that the document set is licensed for reuse and mirroring, and names the deposit arrangement under which it would be preserved.
No period is stated within which a submission will be answered
This submission addresses citation policy from the standpoint of an outside body that would have to rely on it.
A respondent who has made a submission has no basis for knowing whether silence means the submission is being considered or has been lost.
The respondent proposes a stated period, with an acknowledgement on receipt and a substantive response within it.
The secretariat accepts this submission.
Receipt is acknowledged within five working days and a substantive response is published within eight weeks of the window closing. Where that period cannot be met the reason is published in its place.
The disclosure requirements exceed those of comparable bodies
The respondent’s comment on citation policy is made from the standpoint of a person who might be affected by a decision taken under it.
The submission is made on behalf of a manufacturer. It states that the per-page disclosure requirement is more extensive than that applied by comparable assessment bodies and that the repetition may be read as implying a relationship of greater significance than exists.
The submission does not object to disclosure and objects to its placement and frequency.
The respondent has read submission 002 and asks that this submission be considered with it.
The secretariat notes this submission and records that the point about repetition is a fair one.
No amendment arises. Per-page disclosure was consulted on separately and retained, on the ground that a reader arriving at a single page from a search engine does not see a disclosure held elsewhere. The wording has been standardised so that the repeated statement is identical at every occurrence and is not read as a series of different claims. The submission is identified as an industry submission.
The materiality threshold for a declarable holding is set too high
The respondent notes that citation policy governs situations that will arise rarely and matter greatly when they do, and submits accordingly.
The respondent states that the threshold in the draft excludes holdings a reader would regard as relevant, and that the cost of declaring a small holding is negligible against the cost of a reader discovering an undeclared one.
The respondent proposes that all holdings be declared regardless of size.
The secretariat accepts this submission in part. The threshold is lowered. Declaration of every holding without threshold is not adopted, because an undifferentiated list obscures the holdings that matter.
The materiality threshold is lowered and is now expressed as a proportion of the declarer's holdings as well as an absolute figure, so that a holding material to the person is declarable whether or not it is large in absolute terms.
Absence of evidence is presented in a form a reader will take as negative evidence
Having read the draft of citation policy, the respondent puts one point to the secretariat.
Where the Institute has identified no study, the draft states that no evidence was found. In several places that sentence sits immediately after a paragraph describing an adverse outcome, and the juxtaposition invites the reading that the compound was studied and found wanting.
The respondent proposes a standing formulation, used identically wherever the situation arises, distinguishing an outcome that was studied and not demonstrated from an outcome that has not been studied at all.
The secretariat accepts this submission. The two states are different, they support different decisions, and the draft rendered them in language a reader could not reliably separate.
A standing formulation has been adopted and is applied wherever an evidence gap is recorded, distinguishing an outcome assessed and not demonstrated from an outcome not assessed. The formulation is identical at every occurrence so that it can be recognised at a glance.
References cited on this page
References are numbered in order of first citation in this document. Each superscript in the text links to its entry below.
- International Organization for Standardization. ISO/IEC 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories. ISO/IEC Standard 2017;3rd edition. identifier not held by the Institute
Identifiers are reproduced only where the Institute holds them. Where a digital object identifier or PubMed identifier is not shown, the Institute has recorded the journal and year and has not constructed an identifier.