--- title: "Computing a Base Rate Inside an Institution That Only Recounts Its Successful Pilots" description: "Publication bias surfaces inside organisations as the systematic reporting of favourable outcomes alongside the silent closure of negative or ambiguous ones, which leads the institution to overestimate its own historical success rate. The neutralising mechanism is not individual candour but record architecture: opening the decision log at the moment of proposal rather than approval, and requiring a closure note before any terminated initiative leaves the budget." url: https://www.beirek.com/en/blog/publication-bias-corporate-decisions canonical: https://www.beirek.com/en/blog/publication-bias-corporate-decisions published: 2025-05-02 modified: 2025-05-02 category: "Organisational Psychology" category_url: https://www.beirek.com/en/blog/category/organisational-psychology language: en-US reading_time_minutes: 8 publisher: BEIREK LLC publisher_url: https://www.beirek.com license: "© BEIREK LLC — citation with attribution and link permitted" keywords: ["publication bias","base rate","decision log architecture","due diligence denominator","pilot to scale conversion"] topics: ["Organisational decision-making","Investment committee governance","Due diligence and deal structure","Programme reporting discipline"] alternate_language_url: https://www.beirek.com/tr/blog/publication-bias-corporate-decisions --- # Computing a Base Rate Inside an Institution That Only Recounts Its Successful Pilots > **In short:** Publication bias surfaces inside organisations as the systematic reporting of favourable outcomes alongside the silent closure of negative or ambiguous ones, which leads the institution to overestimate its own historical success rate. The neutralising mechanism is not individual candour but record architecture: opening the decision log at the moment of proposal rather than approval, and requiring a closure note before any terminated initiative leaves the budget. *An institution's evidence base is assembled not from everything it attempted but from the portion deemed worth recounting. As a discipline of presentation this selection costs nothing; once the same set hardens into the base rate governing capital allocation, the decision has been made without a denominator.* --- In year-end portfolio review sessions, the overwhelming majority of pilots presented report a favourable outcome; as the running order advances, one deck follows another showing that the target was met, that conversion exceeded the planning assumption, and that the team is requesting incremental budget to scale. What the same session almost never contains is a count of how many initiatives were launched during the year and how many never reached the presentation stage at all, largely because no one owns that figure — the team behind a terminated effort has been redistributed across other lines, the budget code has been quietly closed, and the file has landed not in the corporate archive but in an unnamed subdirectory of a shared drive. Nothing in the resulting picture is untrue; every outcome shown was genuinely observed. The picture as a whole, however, describes not what the institution attempted that year but which portion of what it attempted was found worth recounting, and those are different populations with different arithmetic behind them. The same pattern operates on the evidence an institution imports from outside its own walls. A supplier's reference list is drawn not from the full book of engagements but from the subset of counterparties satisfied enough to take the call; a technology vendor's case studies are selected from the sites where the deployment held; a consulting benchmark set is compiled from organisations documented well enough to be measurable in the first place. None of these three sources need contain a single misstatement, yet all three are drawn from ground that slopes in the same direction, and when those slopes stack, what the institution holds is not a map of the observed world but a map of the reported surface of that world — a distinction invisible in any individual document and decisive across the set of them. The pattern has a name — publication bias, the tendency of significant, favourable or expectation-confirming results to be recorded and circulated while negative, ambiguous or expectation-defeating results are never written down at all. Its mechanics inside a corporation are structural rather than cognitive, arising from an asymmetry in the private economics of reporting: the return on disclosing a favourable outcome is measurable in budget, headcount, visibility and promotion track, whereas the return on disclosing an unfavourable one is, in most organisations, either zero or equivalent to volunteering one's own funding for reduction. Under those conditions the typical observed behaviour is not the suppression of information but simply its non-production; no one conceals anything, because no one carries an obligation to write it down. This filter is entirely functional under a defined set of conditions, and treating it as an error would misdescribe what it does. Management attention is a scarce input, and a reporting regime in which every experiment reaches the board in full detail loses the signal inside the noise while slowing the cadence of decision. Selection is, on precisely those grounds, defensible as a discipline of presentation. The difficulty does not reside in the selection itself but in where the selected set subsequently travels: once that set ceases to be presentation material and becomes a base rate — that is, once it is invoked as the empirical foundation for the sentence "we generally succeed with initiatives of this kind" — the filter no longer manages attention but distorts the allocation of capital. The technical form of that distortion is a denominator problem. What is visible is the numerator: initiatives concluded, reported and presented. What is not visible is the denominator: everything that was ever started. Where the denominator is held nowhere, the success rate becomes uncomputable, though uncomputable is not the same as unestimated; decision-makers proceed from the only population available to them, estimate the ratio implicitly, and that estimate departs upward in a predictable direction. The magnitude of the departure moves in proportion to how quietly terminated initiatives are closed, so that the quieter the closure convention, the more optimistic the institution's belief about its own record — a relationship that holds regardless of how conservative the same institution believes its planning culture to be. On the capital side, all of this concentrates in a single line of the business plan: the assumed transition rate from pilot to scale. That assumption claims descent from the institution's own accumulated experience, yet the population from which it descends already consists of favourable outcomes, so the assumption manufactures its own supporting evidence from within. The same distortion is carried forward into gate decisions, stage-based budget releases and the approval memoranda that travel to the investment committee, since at every stage the material offered as evidence comprises the examples that survived the preceding stage. A hurdle rate cleared under these conditions is cleared in appearance; what has actually been cleared is the performance of a pre-selected subset, measured against a threshold calibrated to a population that was never assembled. On the transaction side, the same mechanism produces a far more direct valuation effect. A target company's schedule of completed projects, its reference customer set and its backlog breakdown are, as a rule, presented without omission; what is absent is the schedule of engagements cancelled, suspended, rescued through scope reduction, or carried into dispute, and that second schedule does not appear in the data room unless it is explicitly requested. Requesting the two schedules separately during diligence creates a material information difference in a single line item, since the first schedule demonstrates capacity while the second demonstrates the boundary of that capacity. Where the second is missing, the structure built around the transaction typically attempts compensation through earn-out thresholds, the breadth of representations and warranties, and the escrow percentage — which is to say the information deficit is collected through structure rather than price. At the operating surface, the same tendency erodes incident reporting itself. Near-miss logs, quality deviations, supplier delivery slippage and rework hours are recorded less and less frequently to the extent that recording them imposes a cost on the reporting unit while returning nothing measurable to it; the thinning record is then read, after an interval, as evidence that performance has improved. The cost of that reading surfaces on a lag — in insurance renewal, in warranty scope negotiation, and in site safety audit — because underwriters and auditors tend to interpret a falling frequency of recorded events not as improvement but as a weakening of reporting discipline, and they price the ambiguity in the direction that protects them rather than the insured. The mechanism that neutralises this tendency is record architecture rather than individual awareness, and it separates into three components. The first is opening the decision log at the moment of proposal rather than approval: an initiative receives a reference number when it is proposed, not when its budget clears, so that the denominator accumulates independently of any approval gate. The second is writing the success criterion and its threshold before measurement begins, because when the definition of success and failure is fixed in advance rather than after the outcome is observed, the room available for retrospective reframing closes. The third is making the termination note mandatory: no closed initiative may be released from its budget line without producing a short but standardised closure record. A fourth component can be added — separating ownership of reporting from ownership of the initiative — which removes the structural interest of anyone assessing their own work. In programmes BEIREK manages, this architecture is established by opening the project register at proposal rather than at approval; every initiative enters the record while the decision remains open and stays in it until closure, so that the ratio of everything started to everything completed can be read from a single table at period end rather than reconstructed from memory. The success threshold and the measurement definition are committed to writing before resources are released, and where a termination decision is taken, the closure note operates as a precondition of closure approval — the file of a discontinued engagement does not enter the archive without one. That record set is then reviewed on the monthly programme rhythm across the closed population as well as the active lines, since a register examined only along its live entries reproduces the very filter it was built to remove. The same discipline is applied to externally sourced evidence. A reference list submitted by a supplier, an EPC contractor or a technology provider is not converted into a diligence finding until the scope of unfinished work has been requested alongside the scope of completed work, and benchmarking claims enter the memorandum only to the extent that their denominator can be shown. The consequence is not a gloomier picture of the counterparty; the consequence is a picture the counterparty is also able to compute, which is where the negotiating leverage actually resides, since a claim that both sides can reconstruct from the same population stops functioning as an asymmetry and starts functioning as a shared premise. What an institution believes about its own history derives less from what it did than from which portion of what it did was made an obligation to write down. An organisation that keeps no record of its discontinued work becomes, over time, an organisation that remembers only the bets it won, and a memory constituted that way is already optimistic, requiring no separate act of optimism to produce the effect. ## Key Points - An institution's evidence base consists not of the full set of initiatives attempted but of the subset judged worth reporting, and the denominator is typically held nowhere at all. - The economics of reporting are asymmetric, since disclosing a favourable outcome converts into budget, headcount and visibility while disclosing an unfavourable one returns either nothing or the withdrawal of the reporting unit's own funding. - The filter remains functional so long as it governs presentation only; once the same filtered set becomes the empirical base rate behind an investment decision, the distortion migrates into capital allocation. - In diligence, the schedule of cancelled, suspended and disputed engagements is rarely volunteered alongside the schedule of completed ones, and the gap between the two schedules translates directly into valuation and deal structure. - The corrective is architectural rather than attitudinal: register initiatives when proposed rather than when approved, fix success thresholds before measurement begins, and make the closure note a precondition of budget release. ## Questions ### How does publication bias appear in a company's internal reporting? Initiatives reporting favourable outcomes are carried onto the management agenda, while those terminated or producing ambiguous results close without entering any record. No one conceals information; the reporting obligation simply arises only for work that survives. Over time the historical population available to the institution represents not everything attempted but the subset judged worth recounting, and the success rate is estimated systematically high as a result. ### Why does keeping a record of failed projects produce a valuation effect? The record of terminated, cancelled and disputed engagements demonstrates the boundary of a company's capacity rather than its extent. Where that record is absent, the buy side tends to collect the resulting risk through structure rather than price: earn-out thresholds tighten, representation and warranty scope widens, and the escrow percentage rises. A complete closure archive frequently opens more negotiating room than an additional exhibit of performance evidence. ### How can the pilot conversion assumption in a business plan be verified? Verification requires examining every pilot launched during the period rather than the outcomes of those completed. Where the denominator consists only of approved or presented initiatives, the calculated rate already describes the performance of a pre-selected population. Opening the register at proposal rather than approval accumulates that denominator without any later attempt at reconstruction, and renders the assumption auditable by a party who did not participate in the original decision. ### What additional information should be requested when assessing supplier reference lists? The schedule of completed engagements is, as a rule, presented in full; the material information difference is carried by the schedule of work cancelled, suspended, rescued through scope reduction, or carried into dispute. That second schedule does not appear in the data room unless explicitly requested. Reading the two together reveals the boundary of the contractor's capacity rather than its extent, and feeds directly into liquidated damages design, security packages and milestone structure. --- Source: https://www.beirek.com/en/blog/publication-bias-corporate-decisions Publisher: BEIREK LLC — https://www.beirek.com