In a follow-up to the previous David Manheim discussion, a blog-post from 2016/09/29 entitled Overpowered Metrics Eat Underspecified Goals, Manheim analyzes examples of twitter use and startups to get a handle on how goals ought to be formalized, especially when transitioning to a corporate structure — here, Peter Drucker's SMART goals (Specific, Measurable, Achievable, Realistic, and Time-bound) versus the BHA goals (Big, Hairy, Audacious) that startups use.
Manheim reminds us that
metrics work because they help ensure that the tasks aligned the intuition of the workers with the needs of the company, create trust between workers and their management, and reduce the complexity of larger goals into manageable steps.
Manheim points out that, in their existing formulation, Goodhart's Law, which probably derived from Donald T Campell's formulation, are at least sometimes false. This follows from the good regulator work of Roger Conant and Ross Ashby, which show an isomorphism between the model of the regulator and the system regulated and which lead to the internal model principle in control theory.
Since Conant's and Ashby's good regulator cannot existing in a process that exceeds modeling complexity, simplified models are targeted by the regulators instead, which can then be exploited.
This means any simplified model used by a regulator can be exploited, especially when the agents understand the model and metrics used. This happens almost everywhere; employees understand the compensation system and seek to maximize their bonuses and promotion, drug manufacturers know the FDA requirements and seek to minimize cost to get their drug approved, and companies know the EPA regulations and seek to minimize the probability and cost of fines. The tension created by the agents is what leads to Goodhart’s theorem; whatever simplifications exist in the model can be exploited by agents.
Manheim now shows how this interacts with the principal-agent problem. [Fn1] Manheim argues that where the story of the individual agent and the bigger story of the cooperative collide, that's too bad for the bigger story.
In companies, the discrepancy between the metrics used and the goal isn’t maximized by the agents: the agents aren’t necessarily against the larger goal, they just pursue their own goals, albeit subject to the regulator’s rules. Goodhart said the correlation doesn’t reverse, it simply collapses.
The outcome is a mismatch between the company's space of possibilities and
the subspace induced by agents’ maximization behaviors.
In other words, even metrics that are aligned well with agents whose goals are understood, they are distorted by the agents whose motives or goals are different than the ones used to build the metric. And because all metrics are simplifications, and all people have their own goals, this is inevitable.
This puts the onus on the model to be as explicit as possible (I think that is what Manheim means with legibility, but I am not 100% sure).
If the model is explicit, game-theoretic optima can be calculated, and principal-agent negotiations can guarantee cooperation. This is equivalent to saying that simple products and simple systems can be regulated with simple metrics and Conant and Ashby style regulators, since they represent the system fully.
Manheim then suggests that Wilson in his discussion of bureaucracy and organizational theory made a useful contribution by replacing the goals with missions (Manheim is persuasive that complexity is often irreducible, thereby curtailing Wilson's other suggestion of how to remedy organizational misalignments.)
[Wilson writes:] "The great advantage of mission is that… operators will act… in ways that the head would have acted had he or she been in their shoes.” But that requires alignment not of metrics and goals, but of goals and missions.
When saddled with unclear goals, metrics begin to take on the role of (self-)justification.
And as Abram Demski pointed out to me, this is an even deeper point; Holmström’s theorem shows that when people are carving a fixed pie, it’s impossible to achieve a stable game-theoretic equilibrium and be efficient too, unless you ignore the budget constraints.
A corporation's solution to this conundrum is
... to make sure people can contribute to growing the size of the pie, making it a non-zero-sum game. Creating this non-zero-sum game to serve as a context for goals is the function of the mission; it’s something that everyone wins by furthering.
To put matters into my own words, missions are supposed to be goal generators.
For Manheim, this is how to turn the old adage from management theory
To motivate a team, you need goals that are clear, and metrics that support them.
into something actionable.
Failure to use metrics well means that motivations and behaviors can drift. On the other hand, using metrics won’t work exactly, because complexity isn’t going away. A strong-enough sense of mission means it may even be possible to align people without metrics.
(This may explain why start-ups and open source projects work.)
The solution may well be to hybridize them, or turn them into a flywheel process.
It makes sense, however, to use both sets of tools; adding goals that are understood by the workers and aligned with the mission, which clearly allow everyone to benefit, will assist in moderating the perverse effects of metrics, and the combination can align the organization to achieve them. Which means ambitious things can be done despite the soft bias of underspecified goals and the hard bias of overpowered metrics.
Footnote 1: I find his point that author and characters in fiction-writing exhibit this interaction as well rather helpful, since I have encountered that resistance in my own writing and often benefited from that, letting my characters and their background constraints take the tale into directions I had not imagined.
No comments:
Post a Comment