Quantitative Management - Myth and Reality

Today measures and numbers have become essential inputs to any decision, whether it is personal decisions, organizational decisions, or decisions by experts like doctors. This has significantly improved the ease of taking decisions as well as the objectivity of decision making. People today can look at the performance numbers  for their MPs before they vote, doctors have many vital measures and counts that they look at before they diagnose or prescribe remedy, managers have many performance numbers that they can look at before they take their decision. But like all good things, this has some bad side effects as well. The sheer availability of so many numbers and the over-dependence on ‘objectivity’ sometimes lulls us into letting numbers take its own decision. Here I have attempted to list down some cautions that we need to exercise to avoid the common pitfalls of over-dependence and misuse of data. It’s mostly written in the context of management decision making, with occasional examples from other areas.

1.     Measures are Approximates

There is a famous quote from Einstein that “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. “ Since most of the goals cannot be directly measured or counted, organizations and managers employ measures that approximate the goal. Customer feedback for customer satisfaction and reported defects for quality are some of such approximations.

Many of the errors that occur while using data for decision making stem from forgetting the fact that measures approximate your goal. In many instances, they are very unsatisfactory approximations at that. So keep the goal in its native form always in sight, and when there is a conflict between the goal and its measure, always align with the goal. Remember, religion is an approximation to god’s truth. A huge part of the current misery could be avoided if people aligned with god’s truth rather than religious dogma in the events of conflicts.

2.     Uncertainty principle

The very act of measuring and reporting will result in a continuous improvement of the said measure, without any material change in the performance with respect to the goal. This is very similar to the illusion of YoY salary increases in an inflationary environment.

This principle explains how organizations have reported continuous improvement of customer satisfaction over many years to the dismay of the customers themselves or how all the internal efficiency measures of an organization show improvement without any change to the bottom-line of the organization.

To take care of this, any reported YoY improvement needs to be inflation-adjusted. The rate of inflation itself will depend on the data discipline of the organization just as the price inflation depends on the fiscal discipline.

3.     Goal vs Path

Myth: Achieving the goal matters, the path taken is incidental.

All paths that lead to the same goal are not equally satisfactory; especially so when what is measured is only an approximation of the goal. Some of the paths achieve the measure but not the goal.

E.g. Smoking might reduce your weight (the measure) but will not achieve your goal (health)

In cases where team members are primarily assessed against targets on multiple parameters, the problem becomes more severe. Each team member starts improving their performance numbers through their individual ways. The impact of some of these improvements on the team goal could be nil or even negative. Significant efforts are expended in re-classifying and re-casting cost, effort, defects just so that some of the individual targets are achieved. In some cases this even crosses over to violation of data, necessitating stricter policing and data-harassment policies from the organization.

4.     Objectivity of measures

Myth: Setting numeric targets and measuring performance blindly by comparing the achieved number with the target is the most objective form of assessment

While managing people one of the irresistible temptations is to set numeric targets against every objective and to measure them against those numbers. This is typically done for the sake of objectivity. Please note that this approach does not make the assessment more objective – instead it takes away the opportunity of the manager to make any assessment. The assessment is replaced by a numeric comparison.

This approach is justified only if

·         the managers cannot be trusted to be fair in their assessment or if

·         the managers are incapable of making any assessment

But then, we have replaced managers with excel sheets. The reality is that without subjective application of context, these numbers do not mean much. To take an example from medicine, how advisable would it be to derive the prescriptions directly from the clinical report, without relying on the ‘subjective’ opinions of a doctor?

5.     Temptation of sub-goals/sub-measures

The problem of individual performance measurement mentioned in the last column deepens when derived goals/sub-goals are used to distribute targets within the team.

The manager gets assigned goals for the team. The lazy management option is to divide these team goals into sub-goals and setting targets on these sub-goals for the team members

Team members should be primarily measured by their contribution towards team goals and not on the achievement of the individual against the sub-goal

There are many reasons why:

·         In an ideal world the sub-goals perfectly integrate into the team goal. But hardly so in reality. The number of passes made by the team members does not add up to the number of goals scored by a soccer team.

·         Promotes unhealthy competition in the team. There is little motivation to help teammates achieve their goals. Some have tried to circumvent this problem by introducing separate targets for teamwork. Well, that is another story for another day. Let me not get started on that!

·         Team goals are more ‘real’ than the derived sub-goals. It is easier and more practical to motivate people with ‘real’ goals than derived ones. Teams lose pride in what they are doing if they see themselves as pushing some ‘faceless’ numbers.

·         Derived goals impact the decision making capability of managers. When the goals are ‘real’, managers can use their good judgment in prioritizing and in making compromises.

6.     Corrosion of management decision making

Over-emphasis of numbers in the organizational culture will impact the management decision making in many indirect ways. Some of these are listed below. Organizations need to watch out for these.

We saw that the urge to make decisions ‘objective’, dissuades managers from using their judgment and makes them depend lazily on the infallibility of numbers.

When goals divide into sub-goals and sub-sub-goals, the manager is turned into an operator looking at a dash-board of some 100 dials. The manager becomes obsessed with keeping each of these dials out the red zone. Over a period of time, these 100 dials replace the big picture, the values, organizational goals and even basic good sense.

Some of the data-driven organizations have the unwritten motto “in numbers, we trust’. There is nothing wrong in trusting the numbers. But when it is reflected as lack of trust in the employees, it pollutes the employee-manager relation.

7.     What cannot be measured

There is a management saying that what cannot be measured, cannot be managed. This sounds good at a training session on quantitative management. But taking this as a religious dogma does not serve you well. That will leave you with two possible approaches on the aspects of your business that cannot be readily measured:

·         Come up with an approximate measure of the aspect that you want to manage – In real life, some of these approximations fall too far away from the goal so that controlling the measure eventually causes more harm than any good. Measuring employee satisfaction by attrition level is a good example of such a case

·         Ignore the aspect as it cannot be managed – in many cases, this is not an explicit decision. But in organizations that are managed through dashboards, all the aspects that have no representation on the dashboard will promptly be ignored. Many of the strategic and long term goals fall off the camel-back this way, unnoticed.

8.     Objectivity vs Accuracy

Numbers are very self-assured. A 4.237 is a 4.237, whichever way you look at it and it knows it! This confidence translates into an assurance of authenticity and accuracy in the mind of the reader. But in some cases, this is misplaced confidence.

This lack of accuracy can creep in through multiple holes.

Myth: The more objective the measure is, the more accurate it is likely to be

In some cases, subjective judgment can add to the accuracy of the measure. When you have made some 100 deliverables to a customer, the average schedule deviation and percentage on-time deliveries are very objective measures of schedule performance. But the customer might not be equally sensitive to schedule slips of each of these milestones. ‘Subjectively’ classifying these into schedule performance of sensitive and not-so-sensitive deliverables might give you a more accurate picture of how frustrated or happy is your customer on your delivery timeliness.

Myth: All numbers are equally objective

The fact is that there pigs among numbers who are more objective than others. Feedback/perception numbers are obviously subjective to an extent. Many other manually collected inputs carry some amount of subjectivity in them. When a user logs efforts she uses subjective judgment to divide effort between multiple activities and projects. In cases when the user is measured based the data that is provided by the user, the levels of inaccuracy tend to creep up.

These aspects make some of the numbers more accurate than the others. It is not wise to trust all numbers equally.