30 Jun Banzo’s sword and metric fixations
Matajuro Yagyu was the son of a famous swordsman. His father, believing that his son’s work was too mediocre to anticipate mastership, disowned him.
So Matajuro went to Mount Futara and there found the famous swordsman Banzo. But Banzo confirmed the father’s judgment. “You wish to learn swordsmanship under my guidance?” asked Banzo. “You cannot fulfill the requirements.”
“But if I work hard, how many years will it take me to become a master?” persisted the youth.
“The rest of your life,” replied Banzo.
My story with the GRE
When I finished the general part of the GRE for math, I knew I had a perfect score. I felt certain the moment I realized one question had an error. I knew all the answers presented were wrong and I knew which one was considered the correct one. For a nerd like me, this was as triumphant of a moment as I could ever live. I felt like I had defeated the system. I know, I know, you are probably thinking, Juan, amigo mío, the idea is to keep the reader interested, math tests, especially standardized tests, not so sexy! But bear with me for a second.
At this point in my life, I had this inner need to leave my country, and academia. Good grades, drunk salsa-dancing, and pot was all I knew how to do. Getting a fellowship in the U.S. was my ticket out, and acing the GRE was an important step towards that. American standardized testing in my upbringing was uncommon and greatly frowned upon. My teachers thought multiple choice questions were a poor form of evaluation. Hence this context was really foreign to me. As you can see, the emotional drive to annihilate the test was huge.
Now, I am not bragging. It was not a special feat. I did answer every question correctly, and so did 14% of all the other people that did the test. What was interesting to me was that I scored higher than all of my math friends that year. And, here’s the caveat, they all were way better mathematicians than me. The difference was simple, for them, the general GRE test was pretty irrelevant, since they were more focused on a specific GRE. For me it was different: I was changing my field from math to literature I did not want the reviewers of my grad school applications to think I was leaving math because I couldn’t cut it . I wanted them to know I was awesome at it, and that my change of interest was just that. In my head, a very high score on the general math section of the GRE was the best way to accomplish that.
Later in life, I realized none of it mattered very much–my application was selected because of my writing, my GRE scores weren’t even considered. However, the experience proved to me in a tangible way something that was very important in my development. Most artificial tests –especially standardized tests– seldom capture the skill or knowledge tested, they just capture the ability to perform in that test. All my math friends had the skills and knowledge to answer every single question right. I had practiced enough to be faster and to use the structure of the test to my advantage. It was not my math knowledge, it was the practice I had put into taking that test. And it really helped that I had family members and American friends who had taken the thing and so could recommend the right preparation tools. I knew how to game the test.
“I cannot wait that long,” explained Matajuro. “I am willing to pass through any hardship if only you will teach me. If I become your devoted servant, how long might it be?”
“Oh, maybe ten years,” Banzo relented.
“My father is getting old, and soon I must take care of him,” continued Matajuro. “If I work far more intensively, how long would it take me?”
“Oh, maybe thirty years,” said Banzo.
“Why is that?” asked Matajuro. “First you say ten and now thirty years. I will undergo any hardship to master this art in the shortest time!”
“Well,” said Banzo, “in that case, you will have to remain with me for seventy years. A man in such a hurry as you are to get results seldom learns quickly.”
Goodhart’s Law and Metric Fixation
One thing that it is important to highlight is that there was nothing unethical about my behavior studying for the GRE. I gamed the test, for sure, but that’s what the test wants you to do. The higher you score the higher the chances are that you’ll receive a scholarship. The test greatly rewards scoring higher. The thing is that the skills that helped me there, had to do more with access and know-how than with math, which is what the test is supposed to measure. And then again If I had not taken advantage of those, I would not be where I am today.
My experience with the GRE proved to me something that now as a coach I live day in and day out. It is expressed succinctly in economics by Goodhart’s law, which states, “Any measure used for control is unreliable.” Putting it in another way: “anything that can be measured and rewarded will be gamed.” (Muller) And putting in yet another way: “When a measure becomes a target, it ceases to be a good measure.” (Strathern)
There is a lot to parse there. However the point is simple, metrics of performance are complicated and often-but-not-always can hide and hinder more than they help. Culturally, we have a very naive understanding that anything measurable is objective. From there we jump right into reality. If it’s measurable it must be real. Metrics of performance can then tend to become targets. This is inescapable in the fitness and nutrition world. Nobody comes to me saying I want to live a long and healthy life, I want to gain strength and balance so that I can be more autonomous for longer. People come to me with metrics they want to improve: I want to lose body-fat, build a heavier deadlift, or hold a handstand for longer. The main problem is, and here’s the peanut of the thing, most of us don’t have a clear understanding of what those metrics actually measure.
When a runner comes to me and tells me, “I just want to decrease my marathon time by 20 minutes”, in their mind that is obviously better. If I decrease my time, I am fitter Juan, period. However, they have never thought about what is being measured. Does that 20-minute “improvement” really mean better cardiovascular health? Are you really a better runner? Are you healthier and fitter? My answer to all that is: it depends. It depends on the context and the person. And my experience as a coach is that I can game that too. I can provide that 20-minute improvement without providing any real improvement other than affecting the measurement.
The same happens with another metric that is the most susceptible to becoming a target in itself: weight. I can totally help you lose the 15 pounds you think are the thing that will give you confidence, or that make you feel better-looking, or that will make you run faster. And yet, none of those things that you wanted to improve will improve. You might lose the 15 pounds and your confidence will still be determined by how many pounds you are up or down each morning. You might lose those 15 pounds and you will not think you are any better-looking because you still think your body is broken. Weight does not measure the health of your relationship with your body. I could go on, and on but you get the point. When the measurement becomes the target it stops being a good measure.
“Very well,” declared the youth, understanding at last that he was being rebuked for impatience, “I agree.”
Matajuro was told never to speak of fencing and never to touch a sword. He cooked for his master, washed the dishes, made his bed, cleaned the yard, cared for the garden, all without a word of swordsmanship.
Three years passed. Still Matajuro labored on. Thinking of his future, he was sad. He had not even begun to learn the art to which he had devoted his life.
Metric Fixation in Fitness and Nutrition
At this point, there is probably a lot of resistance in you. You are probably screaming your ass off at me. “Are you telling me that metrics are not good?” You might even want to say to me: “yes Juan, it just means that there are different kinds of metrics and you need to choose the right ones.” Here’s the thing, the problem is more complex than that (check here and here). It is not just the quality of the metric itself. As with any tool, the problem is how we use it, how we do the thing, and how the thing does us. A metric is a mathematical device. However its interpretation is not statistical or mathematical in nature; its interpretation depends solely on what’s being measured and our relationship to it. And this part is the hard part because “number people” tend to forget the “people” part of the equation. In the case that pertains to us in this blog, if the metric we are choosing measures your capacity, the interpretation of that metric involves knowing about human behavior, psychology, and biological metabolic health. An increase in your deadlift might be good or bad, depending on a plethora of variables that are not measured by the weight on the bar.
Muller has a pretty cool taxonomy of the problems that can arise from what he calls metric fixation. In other words, he has articulated the problems that arise from using metrics without a proper and deep understanding of the thing being measured, when only the numbers determine the judgment of the thing being measured. Some of these problems are fundamental for anybody measuring human performance in fitness, nutrition, or mindset, and so I want to present them to you:
- Goal displacement through diversion of effort to what gets measured. “When performance is judged by a few measures, and the stakes are high (keeping one’s job, getting a raise, raising the stock price at the time that stock options are vested), people will focus on satisfying those measures—often at the expense of other, more important organizational goals that are not measured” (Muller) This is fundamental in fitness and nutrition. The stakes here can be really high. For some, it can be a fundamental health issue –type II diabetes. For others it can be that they can have a strong need to be the fittest mdfkr in the room. In both cases, it is easy to confuse the measurement with the goal.
- Promoting short-termism. When you have a very clear and measurable performance metric in front of you, your brain and soul will try to move the needle right meow. You will focus on the immediate letting go of long-term considerations. Think for example about trying to push your limits on every workout, disregarding the role of your workout in the greater scheme. Another example: restricting your diet intake really hard right meow, disregarding how this restriction will play out next week.
- Discouraging innovation. “When people are judged by performance metrics, they are incentivized to do what the metrics measure, and what the metrics measure will be some established goal. But that impedes innovation, which means doing something that is not yet established, indeed hasn’t been tried out. Innovation involves experimentation. Trying out something new entails risk, including the possibility, perhaps probability, of failure-When performance metrics discourage risk they inadvertently promote stagnation.” (Muller) Have you been endlessly trying to count calories and are stuck not losing weight? This is where you are. Your approach to your health is determined by the measure of mass (weight), which determines that the only way you can address the problem is with the use of calories. And here, my friend you could stay for the rest of your life.
The list goes on. I invite you to check it out and make sure you are not a victim of those problems in other areas of your life. And if you want to see the whole problem working at a macro level don’t be shy and read O’Neil’s book. Now let’s move forward.
But one day Banzo crept up behind him and gave him a terrific blow with a wooden sword.
The following day, when Matajuro was cooking rice, Banzo again sprang upon him unexpectedly.
After that, day and night, Matajuro had to defend himself from unexpected thrusts. Not a moment passed in any day that he did not have to think of the taste of Banzo’s sword.
Metrics Done Right in Fitness and Nutrition
We have made it to the good stuff. But I am aware I have not talked about the swordsman story yet. We will, hang on. Let me first define the principles on how to use metrics in fitness and nutrition:
- Metrics in human performance are unreliable. You need to accept this. If what you are measuring is yourself, then the metric has an effect on what is being measured. That is a super big problem. No scientist wants their measurements to have an effect on what they are measuring. Yet that is exactly where we are in metrics and human performance. A reading of the scale or in a workout that you perceive as negative will have an impact on your behavior whether you want it to or not. There is no way around it.
- Make sure the metric measures the right thing. What is it that you really want? Tell me how you imagine you will feel and what it means for you to be 15 pounds lighter or 20 minutes faster in your marathon. Do you want to feel more confident? The find a metric that measures confidence directly not through a performance or body composition metric. Pro tip: look for the emotional drive behind what you want, that’s the real goal, redefine your metric to measure that.
- Add non-standardized metrics. Are there other sources of information based on experience and judgment that will help you know you are moving in the right direction? These ones are fundamental. State them, define them, and track them. Do you spend a lot more time on Instagram looking at what others are doing when you are anxious about your training? Then time on the IG’s is an important non-standardized metric for you.
- Appeal to intrinsic motivation and self-affirmation. In the self-change business autonomy is both the best tool and the ultimate goal. Make sure your metrics appeal to internal motivations that are independent of the metric itself. This will foster your autonomy and will prevent goal dispersion. Instead of setting a goal of running your next marathon 20 minutes faster, define what kind of runner you want to be. Then and only then define your metrics and make sure they give you information on your goal.
- Always stay vigilant in watching for creaming and gaming. You cannot fully escape this. Use it to your advantage. Be always vigilant of this and when you find yourself being more worried about the metric than about the thing itself, it is time to revisit your goals. Keep in mind, gaming is not cheating, and it is often not very conscious. It is just a misplaced emphasis. Track it and use it as a diagnosis of your mindset.
Matajuro learned so rapidly he brought smiles to the face of his master. Then one day, before completing ten years after his arrival, the master told him he had nothing more to teach him. Matajuro became the greatest swordsman in the land.
Matajuro’s story is a Zen Koan. Koans are narratives used between Zen masters and their students to create an opportunity for reflection. There are several versions of this story. In one of them, Banzo tells Marajuro: “With one eye fixed on the destination, there is only one left to guide you along the journey.” That’s an excellent metaphor for metric fixation and goal displacement.
I really love Matajuro’s story because the first entry point he is required to let go of is the time to completion (an efficiency metric). Only then is he allowed into training. And the second entry point is to let go of training itself. This second point is so fundamental for us in modern-day fitnessing. We are so worried about our workout metrics, that we forget that we train, not to be better at training, but to be better at life –at least, most of us. Matajuro’s training is a reflection of that. Banzo’s sophisticated methodology is awesome: just attack him with a sword and let him learn how to defend himself. That it is the skill he needs to learn to be a better swordsman. There is no other metric than how well he defends himself. In our modern world we tend to become meta-athletes, we want to improve our metrics for the sake of the metrics and we forget that what matters is what brought us to the thing in the first place.
Matajuro’s Koan is an old formulation of Goodhart’s law in a narrative form. I could have started there. Or I could have started with Campbell’s law which is another version of Goodhart’s law coming from the psychology world. However, I chose the economics formulation of this principle because I know my present-day reader trusts things that seem numeric and more scientific over narrative forms. I know that for most, a law in economics bears more weight than a law in psychology. And I know this is because most of us have a very naive and esoteric faith in numbers. I know that Zen Koans have been saying what the Mind and Body Lab at Stanford University is only starting to really grasp and measure.
Learn how to use metrics to your advantage.