--- title: "The Model That Explains the Past Too Well: Overfitting in Corporate Decision-Making" description: "Overfitting occurs when a decision rule learns the coincidences of a past sample as though they were structural mechanism, and then fails once conditions change. Its institutional form is the highly detailed success recipe drawn from a narrow window of data that does not hold in a new market, a new client base, or a new capital structure. The neutralising mechanism is architectural, not attitudinal: recording how many observations a rule actually rests on." url: https://www.beirek.com/en/blog/overfitting-in-corporate-decision-making canonical: https://www.beirek.com/en/blog/overfitting-in-corporate-decision-making published: 2025-04-25 modified: 2025-04-25 category: "Organisational Psychology" category_url: https://www.beirek.com/en/blog/category/organisational-psychology language: en-US reading_time_minutes: 8 publisher: BEIREK LLC publisher_url: https://www.beirek.com license: "© BEIREK LLC — citation with attribution and link permitted" keywords: ["overfitting","decision architecture","founder dependency","due diligence","valuation discount","assumption register","pre-mortem","selection bias"] topics: ["Organisational decision-making","Investment readiness and valuation","Corporate governance and decision records","Behavioural bias in capital allocation"] alternate_language_url: https://www.beirek.com/tr/blog/overfitting-in-corporate-decision-making --- # The Model That Explains the Past Too Well: Overfitting in Corporate Decision-Making > **In short:** Overfitting occurs when a decision rule learns the coincidences of a past sample as though they were structural mechanism, and then fails once conditions change. Its institutional form is the highly detailed success recipe drawn from a narrow window of data that does not hold in a new market, a new client base, or a new capital structure. The neutralising mechanism is architectural, not attitudinal: recording how many observations a rule actually rests on. *A method that accounts perfectly for a company's historical performance has usually encoded the coincidences of that period rather than the mechanism that produced it. In institutional decision-making, overfitting appears not as a statistical error but as an over-calibrated recipe for success, and it fails quietly the moment conditions move.* --- Almost every board pack assembled at the end of a strong three-year run contains a single slide on which the common attributes of the most profitable engagements are set out side by side: clients within a particular revenue band, sales cycles of a particular length, delivery concentrated in a particular geography, contracts written on a particular payment structure. The conclusion drawn from that slide is typically expressed as a targeting rule, and the following year's commercial plan is narrowed to fit it. What makes the presentation persuasive is that the rule captures every successful engagement in the historical record while excluding every unsuccessful one; that clean separation, however, tends to reflect not the strength of the rule but the fact that it was fitted to the data once the outcomes were already known. The question rarely put in the same room is how many independent observations the rule rests on, and how many conditions it required in order to draw the line at all. A structurally similar pattern appears on the operational side, in the control mechanisms erected immediately after the root cause of a failure has been identified. A delivery slips, the investigation isolates a particular supplier, a particular season and a particular order size, and an approval threshold is defined that requires all three conditions to be present at once. The threshold would have prevented the case under examination; most of the delays that surface in the following periods, however, share none of the three attributes, and because the control never produces the event it was built to catch, it is neither removed nor revisited, merely accumulating in the procedure manual alongside a dozen others of the same lineage. The mechanism underneath both patterns is the institutional counterpart of what is termed overfitting in statistical modelling — a model that learns not the systematic relationship present in a sample but the noise particular to that sample, and which therefore performs almost flawlessly in-sample while collapsing to something near chance out-of-sample. Inside a company the model is rarely an equation; it is carried as an intuition, a targeting rule, a hiring profile or an investment thesis, though the mechanics are unchanged. Explanatory power over the past rises not with the number of observations but with the number of conditions the rule is permitted to contain, and beyond a certain point that rise is entirely spurious. What the decision-maker sees, however, carries no marker of the distinction: there is a rule in hand that accounts for the historical record with unusual precision, and precision of explanation is, predictably, mistaken for capacity to predict. A second source of reinforcement lies in how the sample itself came to exist. The deals a company sees, the clients it wins and the employees it retains are already the output of a selection process, which means the historical record is not a random slice of the universe but the residue left by earlier rules. Because the reasons a competitive bid was lost, the way a rejected candidate would have performed, and the behaviour of a market never entered are never observed, the rule is fed exclusively by data that its own filter has generated. Each year, in this loop, the rule grows sharper and more specific while the genuine diversity of observation supporting it remains flat or narrows further. It is worth seeing that this tendency is not an error but a shortcut that is functional under identifiable conditions. Where the cost of deliberation is high and the decision window narrow, repeating a configuration that has previously worked lowers search cost and simplifies organisational coordination; a recipe everyone knows is considerably cheaper than renegotiating every decision from first principles. The difficulty lies not in the shortcut but in its persistence after the conditions that produced it have moved. When the rate environment, the geography of the supply chain, the regulatory regime or the purchasing behaviour of the buyer shifts, an over-calibrated rule does not merely become less accurate; it does something more expensive, in that it conceals its own loss of accuracy for several cycles before the evidence becomes unambiguous. The institutional cost accumulates less in the forecasting error itself than in the commitments attached to it. A commercial rule calibrated to a narrow client profile shapes the composition of the sales team, the commission plan and the field structure of the CRM around that profile, so that when the profile ceases to hold, what has to change is not an assumption but an organisation chart. A multi-year supply agreement signed against a three-year demand pattern enters the balance sheet together with its minimum offtake obligation, and once demand migrates, that obligation compresses the working capital cycle not in the inventory line as a whole but in the portion of inventory that can no longer be converted. Rework cost, staff turnover and delayed entry into an adjacent market advance in the shadow of those two line items, generally without being attributed to the rule that produced them. At the valuation table the same phenomenon takes a sharper form. An acquirer or a credit committee attends less to the magnitude of three years of growth than to whether the mechanism generating it can be described, since growth that cannot be described is, by definition, growth whose repeatability cannot be demonstrated. Where revenue proves to be concentrated in a narrow set of accounts, a single channel configuration or a single window of regulatory incentive, the response is typically not an open argument about price but a structural rearrangement of the deal: an earn-out stretched over a longer horizon, an escrow ratio moved upward, a representation and warranty package extended to survive until the underlying customer contracts have been renewed at least once. Sellers habitually read these adjustments as an expression of distrust, whereas what the counterparty is doing is managing out-of-sample uncertainty through structure rather than through headline value. The most expensive version of this cost arises where the rule sits inside the founder or a small senior core rather than inside a process. A founder's judgement may have been distilled over many years from a large and varied body of observation, or it may be an unusually detailed recipe extracted from three or four decisive transactions; from the outside the two are indistinguishable, since both are expressed with the same confidence and both account for the history equally well. The impossibility of telling them apart follows directly from institutional memory residing in individuals rather than in documents, and it is precisely this that is priced, under a different name, as founder dependency during diligence. The intervention that neutralises the tendency is not an appeal for greater scepticism on the part of decision-makers but an architecture that records the basis of a rule alongside the decision it supports. Four components carry that architecture. The first is an explicit statement, for every decision rule, of how many independent observations it rests on and how many conditions it contains, on the understanding that as the count of conditions approaches the count of observations, the probability that the rule's explanatory power is spurious rises steeply. The second is the definition of the boundary conditions under which the rule is to be treated as void, written when the rule is written rather than after it has failed. The third is the capture of rationale at the moment of proposal rather than the moment of approval, since a rationale composed at approval is generally a narrative reconstructed in a world where the outcome is partly known. The fourth is a separate record of the unobserved sample — the bids lost, the candidates declined, the markets not entered. The method BEIREK operates on capital-intensive projects embeds these four components in the process rather than appending them to it. Before an investment thesis or an operating assumption is carried into a model, we fix in a distinct assumption register both the body of observation it derives from and the range of conditions within which it is held to be valid, so that every material input in the model remains traceable to its source and to its stated range of applicability. On top of that register sits a pre-mortem layer run before the decision: the thesis is assumed to have failed, and the assumption from which the failure begins is written down while the thesis is still unapproved, which forecloses the retrospective rationale. The third layer defines the post-installation rhythm, in that the review of assumptions is tied not to a calendar meeting but to deviation thresholds fixed in advance, so that a review does not require anyone to carry bad news into a room; the breach of a threshold is sufficient. The same discipline works symmetrically on the preparation side, facing an acquirer or a lender. A company able to present its growth mechanism in a form separated from the founder's judgement and written down as process converts a substantial share of the questions asked in diligence from price negotiation into document exchange; where it cannot be presented in that form, the identical uncertainty reappears in the duration of the earn-out, in the escrow ratio and in the conditions precedent to closing. The record built for internal decision quality therefore does double duty, since it also carries the negotiating position outward. What remains open is not how much confidence an institution places in its own recipe for success, but whether it still remembers the conditions under which that recipe was written. An organisation that has forgotten the limits of validity of a rule begins to carry the rule as an identity rather than as a piece of knowledge, and a rule that has become identity is defended rather than revised. Where the difference between how many observations a decision rule was drawn from and how many years it has been repeated is not written down anywhere, that difference eventually surfaces in a more expensive place — as a valuation discount, or as the cost of a restructuring. ## Key Points - A decision rule that explains the prior period without error usually derives that explanatory power from the idiosyncrasies of the sample rather than from any mechanism that will recur. - The cost of overfitting rarely shows up in the forecast error itself; it shows up in the capital, the hiring profile and the supply contract duration committed against that forecast. - Unless the number of independent observations behind a rule is recorded, a recipe distilled from three cases and one distilled from three hundred circulate through the organisation with identical authority. - The question actually being asked at the diligence table is not how much growth occurred but whether the mechanism producing it can be described and repeated without the founder in the room. - The architecture that neutralises overfitting records the rationale for a decision at the moment of proposal rather than the moment of approval, together with the boundary conditions under which the rule is to be treated as void. ## Questions ### How is overfitting recognised inside a company's decision processes? The most reliable indicator is the number of conditions a decision rule contains approaching the number of independent observations behind it. A rule that captures every successful engagement in the historical record while excluding every unsuccessful one generally derives its precision from having been fitted after the outcomes were known. The typical consequence is that the same rule fails to hold its accuracy across a new period, a new client segment or a new geography. ### Why does a growth thesis built on historical data generate a valuation discount? An acquirer or lender examines whether the mechanism producing growth can be described, not how large the growth was. Where revenue is concentrated in a narrow set of accounts, a single channel or a single incentive window, repeatability cannot be demonstrated. The response is usually structural rather than a direct reduction in price: an extended earn-out, a higher escrow ratio, and a representation and warranty package widened to survive the renewal of the underlying customer contracts. ### Why should decision records be captured at proposal rather than at approval? A rationale written at approval tends to be a narrative reconstructed in a world where the outcome is already partly known, and it therefore records the defence of the decision rather than its actual basis. A record made at the point of proposal fixes which observations the assumption was drawn from, the range of conditions under which it is held valid, and the boundary at which it should be considered void, allowing any later review to be conducted against a stable reference. ### What purpose does a record of the unobserved sample serve? The clients a company sees, the deals it wins and the employees it retains are the residue left by earlier filters, while lost bids, declined candidates and markets never entered are never observed at all. Where this missing portion is not kept in a separate record, the rule is fed only by data that confirms its own filter, growing sharper with each cycle while the genuine diversity of observation supporting it remains unchanged or narrows. --- Source: https://www.beirek.com/en/blog/overfitting-in-corporate-decision-making Publisher: BEIREK LLC — https://www.beirek.com