David Manheim on Underspecified Goals

 In a follow-up to the previous David Manheim discussion, a blog-post from 2016/09/29 entitled Overpowered Metrics Eat Underspecified Goals, Manheim analyzes examples of twitter use and startups to get a handle on how goals ought to be formalized, especially when transitioning to a corporate structure — here, Peter Drucker's SMART goals (Specific, Measurable, Achievable, Realistic, and Time-bound) versus the BHA goals (Big, Hairy, Audacious) that startups use.

Manheim reminds us that

metrics work because they help ensure that the tasks aligned the intuition of the workers with the needs of the company, create trust between workers and their management, and reduce the complexity of larger goals into manageable steps.

Manheim points out that, in their existing formulation, Goodhart's Law, which probably derived from Donald T Campell's formulation, are at least sometimes false. This follows from the good regulator work of Roger Conant and Ross Ashby, which show an isomorphism between the model of the regulator and the system regulated and which lead to the internal model principle in control theory.

Since Conant's and Ashby's good regulator cannot existing in a process that exceeds modeling complexity, simplified models are targeted by the regulators instead, which can then be exploited.

This means any simplified model used by a regulator can be exploited, especially when the agents understand the model and metrics used. This happens almost everywhere; employees understand the compensation system and seek to maximize their bonuses and promotion, drug manufacturers know the FDA requirements and seek to minimize cost to get their drug approved, and companies know the EPA regulations and seek to minimize the probability and cost of fines. The tension created by the agents is what leads to Goodhart’s theorem; whatever simplifications exist in the model can be exploited by agents.

Manheim now shows how this interacts with the principal-agent problem. [Fn1] Manheim argues that where the story of the individual agent and the bigger story of the cooperative collide, that's too bad for the bigger story.

In companies, the discrepancy between the metrics used and the goal isn’t maximized by the agents: the agents aren’t necessarily against the larger goal, they just pursue their own goals, albeit subject to the regulator’s rules. Goodhart said the correlation doesn’t reverse, it simply collapses.

The outcome is a mismatch between the company's space of possibilities and

the subspace induced by agents’ maximization behaviors.

In other words, even metrics that are aligned well with agents whose goals are understood, they are distorted by the agents whose motives or goals are different than the ones used to build the metric. And because all metrics are simplifications, and all people have their own goals, this is inevitable. 

 This puts the onus on the model to be as explicit as possible (I think that is what Manheim means with legibility, but I am not 100% sure).

If the model is explicit, game-theoretic optima can be calculated, and principal-agent negotiations can guarantee cooperation. This is equivalent to saying that simple products and simple systems can be regulated with simple metrics and Conant and Ashby style regulators, since they represent the system fully.

Manheim then suggests that Wilson in his discussion of bureaucracy and organizational theory made a useful contribution by replacing the goals with missions (Manheim is persuasive that complexity is often irreducible, thereby curtailing Wilson's other suggestion of how to remedy organizational misalignments.) 

[Wilson writes:] "The great advantage of mission is that… operators will act… in ways that the head would have acted had he or she been in their shoes.” But that requires alignment not of metrics and goals, but of goals and missions.

When saddled with unclear goals, metrics begin to take on the role of (self-)justification. 

And as Abram Demski pointed out to me, this is an even deeper point; Holmström’s theorem shows that when people are carving a fixed pie, it’s impossible to achieve a stable game-theoretic equilibrium and be efficient too, unless you ignore the budget constraints. 

A corporation's solution to this conundrum is

... to make sure people can contribute to growing the size of the pie, making it a non-zero-sum game. Creating this non-zero-sum game to serve as a context for goals is the function of the mission; it’s something that everyone wins by furthering.

To put matters into my own words, missions are supposed to be goal generators.

For Manheim, this is how to turn the old adage from management theory

To motivate a team, you need goals that are clear, and metrics that support them.

into something actionable.

Failure to use metrics well means that motivations and behaviors can drift. On the other hand, using metrics won’t work exactly, because complexity isn’t going away. A strong-enough sense of mission means it may even be possible to align people without metrics.

(This may explain why start-ups and open source projects work.) 

The solution may well be to hybridize them, or turn them into a flywheel process.

It makes sense, however, to use both sets of tools; adding goals that are understood by the workers and aligned with the mission, which clearly allow everyone to benefit, will assist in moderating the perverse effects of metrics, and the combination can align the organization to achieve them. Which means ambitious things can be done despite the soft bias of underspecified goals and the hard bias of overpowered metrics.

 

Argument Structures around the Kirtland Safety Society

While historians want to be both educated and corrected by their sources, it is often difficult to pin down that that exactly means. In the following, I want to look at a few examples of how this plays out in the case of the economic history of the Kirtland Safety Society (this is Chapter 10 in my dissertation).

The backbone of course is the chronology. Many narratives already fall apart due to not aligning with the basic chronology of the matter. Of course the selection of what elements to include in the chronology brings us back to the problem of enplotment.

It is important to realize that a chronology is not a sequence of dates, but a sequence of dated events. The distinction is crucial, because dates are just chronometric units, while events have actors and properties and are enmeshed in a event sequence, as well as a geometric positioning. The chronometric units have a canonical order, but it is independent of all semantic contents---which is precisely what makes them units.

Perhaps it is best to first to tease apart the pieces that can go into a historical argument. For an intentional action, we have to presuppose some kind of script, that is, a codification of the "usual" way to do this. This would be the usual way from the point of view of the contemporaries, of course, not from the point of view of the historian. 

The "usual way" is typically distilled in some form from another event that has a similar structure. As Eduard Meyer pointed out (1912) [cited in the dissertation], it is this compare-and-contrast operation that is at the root of the historical research effort for distinguishing the common from the unusual. In the Kirtland case, I used the Owl Creek Bank, which did occur in Ohio about two decades before the Mormon arrival, as well as the Old Bank of Michigan, in which Joseph Smith~Jr's uncle Stephen Mack had participated. In the end it was the cross-comparison between these three instances that provided the fodder for the historical reconstruction.

These three elements were underpinned on the one hand by a history of banking in Ohio, mainly a sequence of state legislature actions and bank foundations (which had to be chartered by the legislature), and a history of the more global economic settings both in Ohio proper and in the US. This was the place for wealthy East Coast magnates and European importers, mainly from England, that had a strong influence on the money supply. This was also the place for Jacksonian interventionism at the national level and the ins and outs of the federal land sale that had such an impact on the monetary supply. All of these lines had their own event sequence with its own "arc" so to speak, mostly expressed as supporting or blocking specific aspirations on the legislative side and injecting or removing money on the financiers' side. 

The final element was the chronology of the creation and dissolution of the Kirtland safety (anti-)banking society itself. The comparison with the other banking charters in Ohio as well as with the Owl Creek Bank highlighted the inversion of the order of steps (charter last) as well as the enormous leverage that had been prototypical of other bank failures in the past and would eventually be outlawed in 1839 by the legislature, requiring 1:3 at the most.

For all of these elements, the observation of the overdetermined asymmetry of causation was in effect. For example, only a small fraction of the notes issued by the Kirtland society survived, supporting reconstructive efforts such as simulations how many bills of which type might have been issued. Also, the stock subscription book survived, allowing to determine which families of the Mormon elite had taken out the most stock and subscribed how much of their money. Some newspaper editorials and similar statements survived, but many other actions and their direct and indirect effects remain inaccessible, lost to time.

Islamic State and Modernity

In preparation for considering the problem of secularization, I am reading a set of essays edited by Friedrich Wilhelm Graf and Heinrich Meier, Politik und Religion: Zur Diagnose der Gegenwart, published in 2013 by C.H.Beck.

The essay by Hillel Fradkin, Die lange Suche nach dem islamischen Staat,  (the long search for the Islamic state), and argues that the Islamic world, after the conquering successes of Mohammed, reigned such a large stretch of land for such a long time (1400 years from Hegira to the Fall of the Ottoman Empire) that Modernity for them is like a nightmare, nibbling away their success first in colonialism and then in modern republican nationalism (pp.136-139). That the infidels should rule the faithful appears like a devilish perversion, because it negates the trope of the superior political organization.

After discussing the Brotherhood in Egypt and the Iranian Revolution, Fradkin turns to the problem of what modernity means as a configuration for Islam.

Denn objektiv gesehen, und nach Erfahrung der westlichen Politik zu urteilen, welche die muslimische Macht zurückdrängte, ist die Macht, deren es heute bedarf, um die Weltpolitik zu dominieren, eine Funktion der Aneignung der Moderne. Damit ist nicht nur die moderne Technologie, sondern zuallererst der moderne Republikanismus und das moderne Denken gemeint, das beides hervorbrachte. Nicht zufällig war dieses Denken von Anfang an in hohem Maße von der Frage der Macht präokkupiert. Zumindest was die Macht betrifft, hat das moderne Denken sein Versprechen gehalten, insbesondere gilt dies für die ökonomische Macht, auf der andere Formen der Macht beruhen. (p.159)

[Viewed objectively, and judged by the experiences of western politics, which have pushed back Muslim power,  is the power that is necessary these days, to dominate world politics, a function of the acquisition of modernity. This does not mean just modern technology, but before all else, modern republicanism and the modern thinking that brought these two about. It is no accident that this thinking was preoccupied from the get-go to a high degree by the question of power. At least as far as power is concerned, the modern thinking kept its promise, and this is especially true for economic power, on which the other forms of power are founded. (p.159)]

But that seems to invert several things, in my mind. While economic power is clearly not independent of resources and raw materials, is it not most heavily tied to technological advances and a workforce that can participate in their use and construction?  Now, one can argue that modern republicanism is a prerequisite for that, but the Chinese system suggests that this is not necessarily so.