Bias

Recently, I’ve had to make a lot of decisions. I’m the type who can’t make quick judgments because I overthink things.
So I’m somewhat weak at making split-second decisions, and to train myself, I’m organizing my decision-making process.
Perhaps the reason judgments take so long is that I haven’t established clear criteria or thought processes leading to decisions.

Most situations requiring instant decisions are those where immediate action must follow. For example, like a wrong order, there have been situations where if time passes, you can no longer take action and must bear the risks caused by delayed judgment. Interestingly, delayed judgment has also sometimes prevented overspending.

Another reason is that time is delayed in the process of self-reviewing whether my judgment is biased. As a light example, when deciding whether to buy or sell stocks, since I’m not a professional analyst, I find it difficult to determine whether I’m making biased judgments or which decision is right. Or when discussing other people’s concerns or social issues. These days, GPT helps somewhat when considering various cases, but it’s still just a tool for easily finding grounds for judgment - the difficulty of making the final decision remains.

Why is it so hard to escape from various biases, large and small? Though I didn’t major in neuroscience, I think it’s because the brain wasn’t designed that way. Since most choices and judgments occur within options created by humans, I wonder if a perfectly ideal machine for judgment could even exist. Rather than bias being a bad phenomenon, isn’t it more like a shepherd keeping us from straying beyond the fence of thought necessary for survival?

If we could create a perfect simulation considering all variables in the world, judgment would be easy, but that’s realistically impossible. To interpret the complex world, we look for appropriate variables and patterns of dynamics. And we create interpretable models mediated by them. Thus, we make decisions called appropriate judgments based on probability on a reasonable physics engine mediated by appropriate variables and dynamics.

Since the elements of these models differ for each person, situation, and time, we approach relationships with different models.

In the physics engine thus equipped, it’s convenient to see and interpret things as I want to see them. However, it’s only a sampled physics engine and doesn’t perfectly explain the world. When we face things that violate our world, we seem to either ignore them or give them wrong interpretations. Readjusting my well-established model and giving it new logic costs a lot. Of course, that doesn’t mean seeing only what you want to see is bad. What is the standard of good and bad in the first place? That too is ultimately just a standard I’ve interpreted. On the other hand, such outliers that violate my world aren’t necessarily absolutely right either. Sometimes these things give the feeling that only I have awakened to the existence of the Matrix. In the end, it’s all individual choices for oneself, and we’re only responsible for our own choices.

To make a long story short, what I wanted to say is that what’s important is not the judgment criteria of what’s right or wrong itself, but developing metacognition that allows me to recognize my own biases.

While how fast we’re heading somewhere is very important, how well we recognize when the direction is wrong and how well we can correct it is also very important.

In that sense, it’s important to be able to judge based on what variables I have at this point, what I haven’t recognized, and what mediations are occurring.

When such analyses work well, couldn’t we move forward faster with stronger convictions? (Of course, such things may work by intuition, but since intuition is an implicit? tacit? black box, we need explicit judgment criteria.)

So I’ve been trying to look at myself more objectively lately. As an extension of that, I discovered keywords while asking various questions about judgment. I don’t remember exactly in which conversation I encountered it, but it was a document about training that CIA analysts undergo.

Cognitive Prison

This is a book called “Psychology of Intelligence Analysis” by Richard J. Heuer Jr., used as educational material by the CIA. The author points out that when analysts interpret reality, the ‘mental models’ already equipped in their minds act like prisons, distorting or ignoring new evidence. He calls this chronic limitation the ‘structural defect of our mental machinery’ and says it’s the primary cause of analytical failure.

The two documents above contain content about how our thinking develops and what training can overcome which biases. I read them but thought it would be too long to organize, so I asked notebookLM to make a podcast. The two files below are general summaries of the above content.

That’s the content of the books, and I’ve additionally organized what biases they discuss.

CIA 'Cognitive Prison' and Major Cognitive Biases Summary

ChatGPT helps you get answers, find inspiration and be more productive. It is free to use and easy to try. Just ask and ChatGPT can help with writing, learning, brainstorming and more.

https://chatgpt.com/s/dr_6831906cbcac8191bacd46afdf743b4c

The main biases mentioned in the book are:

1. Expectations (Expectation Bias)
2. Resistance (Existing Belief Maintenance Bias)
3. Ambiguities (Ambiguity First Impression Bias)
4. Consistency (Consistency Overconfidence Bias)
5. Missing Information (Missing Information Bias)
6. Discredited Evidence (Persistence of Discredited Information)
7. Availability (Availability Bias)
8. Anchoring
9. Overconfidence
10. Rationality Assumption (Excessive Rationality Assumption)
11. Attribution Bias (Fundamental Attribution Error & Mirror Bias)

While these biases can be overcome through training, I thought there might be more convenient methods in this era, so I tried using AI. It’s like having a dedicated analyst.

Based on the conversation above, I asked it to create project instructions, one of GPT’s features.

**ChatGPT Project Instruction - "CIA Cognitive-Bias Interactive Reviewer"**

---

## Purpose

Create an interactive assistant that automatically checks and corrects the 11 'cognitive prison' cognitive biases defined by the CIA Sherman Kent School whenever users share ideas, hypotheses, or judgments.

---

## Cognitive Bias Catalog

(Each item described only with *heading + bullet*)

### 1. Expectations

- Easily perceive only information that matches initial expectations/scenarios
- Undervalue contradictory information even with strong evidence

### 2. Resistance

- Existing mindset resists new evidence → modifies thinking slowly/minimally

### 3. Ambiguities

- If first exposure is unclear, cling to initial interpretation even when clarified later

### 4. Consistency

- Illusion of "certainty" with only a few pieces of consistent information

### 5. Missing Information

- Unable to evaluate meaning of "absence of evidence" and just ignore it

### 6. Discredited Evidence

- Impressions created by evidence proven false persist to the end

### 7. Availability

- If easily recalled, assume actual frequency/probability is also high

### 8. Anchoring

- Cling to initial numbers/judgments and under-adjust subsequent modifications

### 9. Overconfidence

- Confidence exceeds actual accuracy

### 10. Rationality Assumption

- Think others are as consistent and rational as us

### 11. Attribution Bias (Fundamental Attribution + Mirror Bias)

- Interpret others' behavior as disposition, our behavior as situational
- Assume others think and prioritize like us

---

## Response Rules (One Conversation Analysis Output)

### Heading Arrangement

1. **Relevant Biases** - Listed in order of *highest priority*
2. **Not Relevant Biases** - Biases not revealed in this input with brief reasons
3. **(Optional) No Obvious CIA Bias Detected** - When no relevant biases at all

### Bias Section Format

"""

### Anchoring

**Why Suspected**

- …
  **Check Questions**
- …
  **Verification Actions**
- …
  """

_No tables, numbered lists, or long narratives._

### Required Elements

- **Why Suspected**: 1-2 lines of bias clues from user's statement
- **Check Questions**: 2-3 questions for user to self-review/refute
- **Verification Actions**: 1-3 actionable behaviors (data collection/comparison/Peer Review, etc.)
- **Not Relevant** section: One line per bias "Not revealed in this input because of ○○"

### Priority Assessment Guide

- Direct conflict with core logic → **High**
- Auxiliary information or speculation level → **Medium**
- No clues → **Not Applicable**

---

## Conversation Flow Logic

1. **Statement Summary** - Internally identify key claims/assumptions/numbers
2. **Bias Scan** - Compare 11 biases with statement, classify as Relevant/Not Relevant
3. **Priority Sorting** - Sort Relevant Biases by evidence strength/impact
4. **Heading-Bullet Output** - Write according to rules
5. **Reassess Follow-up Input**
   - If user shares **additional evidence/action results**
     ‣ **Rescan** bias list
     ‣ Update whether each bias is **mitigated/worsened/newly occurred**
     ‣ Explain change reasons in "Why Suspected"

---

## Prohibitions

- No mention of heuristics outside CIA 11 biases (framing, sunk-cost, etc.)
- No definitive assertions or accusatory tone
- No tables, numbered lists, or long paragraphs

---

## One Line to Remember

> "The output of this instruction is *a list of questions and actions* — the user verifies the answers themselves."

I’ve created this and am using it to create training data when I need to make judgments. For example, it responds like this:

Since I’ve set it to provide check questions and verification actions, they’re surprisingly not absurd and help awaken things I hadn’t considered.

So

In the AI era, access to information has broken down to the point where there are hardly any barriers, but contaminated information is pouring out just as much.

The content researched by ChatGPT that I referenced while writing this could all be lies. That’s why there’s a greater probability of becoming more confident in our biases. Our speed is accelerating more and more. That’s why I think we need to recognize ourselves again for wise judgment.
Then wouldn’t the answer to where we’re going naturally emerge?

Do not go gentle into that good night
Old age should burn and rave at close of day
Rage, rage against the dying of the light.

Reference

• https://www.cia.gov/resources/csi/static/Pyschology-of-Intelligence-Analysis.pdf
• https://www.cia.gov/resources/csi/static/Tradecraft-Primer-apr09.pdf