Introduction
This book is a compilation of the course Richard Hamming taught at the Naval Postgraduate School in 1995. Its core starting point is that “A life without a struggle on your part to make yourself excellent is hardly a life worth living,” its core claim is “Luck favors the prepared mind,” and its core content is the author’s introspection on his own distinguished research career, plus his observations of famed Bell Labs colleagues (Claude Shannon, John Tukey, etc.), to answer how to become a prepared mind (as a style of thinking).
Another central aim is to inspire you to find your own style of thinking and set a mission and direction. You must think, and you must accomplish something meaningful that transcends yourself. For example, a drunken sailor’s random walk covers an average distance of \sqrt{n}, but if a beautiful girl stands in some direction, the sailor’s average distance becomes proportional to n. Hamming also observes that whether your initial vision is “right” doesn’t matter much. There are many paths to excellence, and any of them beats drifting with the tide.
In other words, this book tries to prepare you for your future. By 1995 Hamming had already observed that technological progress was accelerating and knowledge was iterating ever faster, i.e. half of what you learned 15 years earlier might already be obsolete. Yet the future still holds infinite opportunities, so he shows how he reasoned from first principles to anticipate nascent fields (fiber optics), how a deep grasp of binary mathematics enabled him to invent world‑changing technology (error‑correcting codes), how to research effectively without “fully understanding” (quantum mechanics), how to build with a systems‑engineering mindset, and how to extract design insights from simulation (NIKE missile). His answer to “how to cope with the rapid technological change” is learning to learn—the book’s subtitle.
The target audience was the students at the Naval Postgraduate School, but I think it speaks to anyone who aims to make a significant contribution in a technological society, especially scientists and engineers (undergrads and graduate students). Two reasons: first, you need to feel a professor’s encouragement to pursue excellence; second, you need guided, systematic exposure to the thinking and methods showcased above.
The book leads readers across multiple highly technical modern research areas to extract the “style of thinking” the author emphasizes. Hence it contains hard‑core topics like information theory, coding theory, and digital filters etc.
When choosing a vision/direction, you will inevitably have to answer these three questions:
- What is possible? (Science)
- What is likely to happen? (Engineering)
- What is desirable to happen? (Ethics)
These future‑facing questions are hard. Our only guide is history, and because computers are transforming every field, the history of computing becomes the backbone context of the book.
Across all topics, the distinction between education and training runs through the chapters and is a key separator of leaders from followers. Once you enter a field, you are either a leader or a follower, and Hamming wants you to be a leader.
Reading suggestions
I’d recommend reading “You and Your Research” first with a mindset of reading a blog (30‑minute read) to feel Hamming’s hopes for the next generation. If you’re interested in this book, I suggest splitting it into blocks, then reading each block as a blog or series. For instance, History of Computers is a series; AI is a series; Coding Theory and Information Theory is a series; Simulation is a series. Others, like System Engineering, read more like single posts. Each “series” or “post” has meta points—focus on understanding those meta points, not on exhaustive technical details and anecdotes.
Below I piece together my selected notes I found useful or interesting; for the full version please check this whiteboard.
Education vs Training
Education is what, when, and why to do things. Training is how to do it.
From a systems‑engineering perspective, training teaches students to solve a well-defined problem, i.e. what technique fits which situation and how to apply it. Education prepares students to face ambiguous problems: to iteratively frame the problem using knowledge from multiple courses and then choose suitable techniques to solve it.
For leaders in a technological society, what, when, and why to do things matter more than how (though it still matters). Hamming reminds us: “What you learn from others you can use to follow; what you learn for yourself you can use to lead.” The real purpose of education is to cultivate vision, judgment, and self‑correction, such that when faces with uncertainty, you will will have the ability to choose a direction and adjust your path along the way. In a world with a rapidly shrinking half‑life of knowledge, education should focus on “preparing for the future,” not clinging to the details of “how”.
From the fitness point of view, lifting 40 kg twice usually grows more muscle than lifting 20 kg four times. Does this “physical muscle” training apply to “mental muscles”? They aren’t perfectly equivalent, but the analogy could be useful: deep cognitive challenges prepares you better than shallow repetitions.
For learning and day‑to‑day thinking, Hamming suggests:
- Build deep foundations: technical details iterate rapidly; fundamentals transfer across eras.
- Learn new things continually: knowledge grows exponentially; you must proactively track emerging fields.
- Model quickly: do back‑of‑the‑envelope calculations, use quick mathematical modeling to estimate and test claims.
- Regularly think about big questions: spend an afternoon each week (about 10% of work time) on forward‑looking, open problems. For example, “What is the nature of computing? How will it affect science? What is the natural role of computers at Bell Labs? What will computers do to AT&T? To science at large?” This clarifies where humanity is headed so you can move in the right direction, thus avoiding the drunken sailor’s \sqrt{n} meander and progressing more linearly. It also keeps attention on the crucial problems so your main effort goes where it matters.
The invention of error‑correcting codes
Error‑correcting codes transformed communication and storage, enabling automatic error detection and correction under noise at very low redundancy. It’s a bedrock of modern technology.
One of the main reasons Hamming discovered error-correcting codes was that he was repeatedly frustrated by machine failures caused by coding errors. Machines would stall because a 0 flipped to 1 or vice versa. After repeated schedule slips from such failures, he got angry and asked: “If machines can detect errors, why can’t they locate them and automatically fix them?”
He knew you could store three copies of the code and vote to correct errors, but the code redundancy was too expensive (memory was precious). He knew there had to be a better way so he started working on this. His understanding of two‑out‑of‑five codes, parity checks, and binary arithmetic was decisive. At the time, most scientists and engineers weren’t comfortable with binary; people still favored decimal, unlike today where binary is the natural language for computing.
Hamming’s key insight in inventing error‑correcting codes was a simple yet profound one: treat the detected error “syndrome” itself as a binary number, and that number points directly to the position where the error occurred. Building on that, he went on to propose that, to intuitively explore the power and limits of binary codes, one should shift the metric space for measuring code distance from the traditional Euclidean space to the binary metric of Hamming space, thereby establishing an entirely new geometric framework for designing and analyzing error‑correcting codes.
True creativity comes from a subconscious saturated by the problem
“Originality” isn’t just “never done before.” You can multiply two random ten‑digit numbers, but that has no meaningful consequence. Creativity seems to be combining elements not previously seen as related in a “useful” way, which is often hinging on the initial psychology distance between them (as with choosing Hamming distance rather than Euclidean distance for comparing bit strings).
Can people become more creative? Most creativity courses and tools on the market don't seem particularly helpful, because they never produce a large number of dominant figures in any field. Throughout the book, creativity is called "style," and Hamming believed people can become more creative. A creative act usually begins with a vaguely conscious identification of a problem, followed by a process of converging on the problem. During convergence, don't frame it too quickly, because you'll put it into a conventional form and get conventional conclusions. At this stage, a deep emotional commitment is needed to truly engage. Long incubation of the problem can lead to temporary solutions or a temporary abandonment of the problem. This is the common process behind many great creative acts, and it is essential for letting the subconscious think about the problem.
Next comes the moment of insight or “creativity.” Wrong starts and wrong solutions often make your next attempt sharper. Now you know which approaches are wrong! There are fewer directions left to explore. You gain a better understanding of what doesn’t work and why it might not work.
If the solution comes from the subconscious, then we can saturate the subconscious with the problem. If we stop thinking about anything else for hours, days, or even weeks, we may even chew on the problem in our sleep! Luck favors the prepared mind, and you can be prepared.
During the creative process, reasoning by analogy is one of the most important tools, therefore broad knowledge across different fields helps a great deal, provided that knowledge can be accessed flexibly. That depends on studying new material from multiple angles and building many “knowledge hooks” for it. That way, when needed, you can hook that crucial idea out of your mind by analogy.
If you don’t work on important problems, how can you expect to do important work?
(From You and Your Research)
“It seems to me it is better to do significant things than to just get along through life to its end. Certainly near the end it is nice to look back at a life of accomplishments rather than a life where you have merely survived and amused yourself. (1) it is worth trying to accomplish the goals you set yourself, and (2) it is worth setting yourself high goals.”
So admit boldly: “I want to do first‑class work!” If you do not work on important problems how can you expect to do important work?
Many excuse themselves with “I’m not smart enough,” but many who didn’t look like geniuses did great things. At Bell Labs, Bill Pfann didn’t seem especially brilliant at first, yet his work on zone melting won all the prizes in the field; as his confidence grew, he became more articulate. Ability comes in many forms, not just IQ.
Confidence, perhaps more than high IQ, is the more critical trait, which you could also call "courage." This courage can be cultivated through focus on your successes and less concern for your failures. Claude Shannon, for example, would often boldly put his queen on the line in chess, saying, "I ain't scared of nothing." When you find yourself in a difficult situation, you might tell yourself the same thing.
With ambition and courage, the next step is to ensure you are spending your energy on the "right" problems. To do this, you should regularly set aside time to think about the bigger "great ideas": What is the essence of your field? How will it affect the development of science? If you want to be a leader rather than a follower, you must break free from the sea of details that most people are trenched in and constantly examine the big picture. One concrete method is to keep a running list in your mind of about 10 to 20 important problems that you care about but don't yet know how to solve. Great scientists do this; they keep these problems in mind, and when a clue appears, they would dive in immediately.
The desire for excellence is a fundamental characteristic of great work. Without such a goal, you will wander aimlessly like a drunken sailor (averaging to ( \sqrt{n} )).
How to watch an emerging field and prepare for it
How should we approach a new emerging field with great potential? Using the rise of fiber optics technology as an example from his personal experience, Hamming explained his thinking process for dealing with a completely new field.
First, Hamming started from "first principles" to quickly identify the importance of fiber optics. The principles of physics revealed that optical frequencies are much higher than the electrical frequencies used at the time, thus offering greater bandwidth. Moreover, its raw material is sand, which is everywhere, making its cost advantage obvious compared to the increasingly scarce copper ore. Additionally, he noticed that the conduits for telephone lines in Manhattan, New York, were nearing saturation. If the city were to continue developing, fiber optics, with its smaller diameter, was the only solution that wouldn't require massive street excavations. These reasons convinced him that his employer, Bell Labs, would go all out to invest in this technology.
Next, he continuously tracked the technical bottlenecks he foresaw and updated his knowledge as the field evolved. For instance, he realized early on that "how to splice hair-thin optical fibers" would be a major practical challenge. He also observed that existing systems needed to convert optical signals to electronic signals for amplification and then back to optical signals, which he considered a poor design. He foresaw the inevitable emergence of "optical amplifiers." Furthermore, he realized that fiber optics are immune to electromagnetic interference (such as from nuclear explosions or lightning strikes), meaning they would certainly receive strong support from the military. By focusing on these key issues, the development roadmap of the entire field became clearly visible to him.
You should always proactively ask questions to better predict and lead the future, rather than passively following trends.
Education and Training in the Computer Age
During the development of computers, numerous educational studies indicated that computers were beneficial for education and could assist in training students. However, most educational experiments failed to address the "Hawthorne effect"—a temporary improvement in performance due to the perception of receiving special attention or observation. Most of the efficiency gains observed with new learning methods typically reflect this bias rather than substantial improvements.
The standards of higher education have been constantly changing. Future education will be vastly different from present education, just as present education is almost irrelevant to that of nineteenth century. Educational systems will leap forward dramatically with technological advancements. How can we evaluate proposed CAI (computer-aided instruction) projects without an appropriate vision for the future form of education? We must establish a vision for what educated individuals in future society will look like; only then can we confidently address the issues that arise in CAI.
Hamming believed that computers are helpful for low-level, conditioned responses (like arithmetic drills, flight training) but less useful for education, partly because, as mentioned, we don't yet know what it means to be "educated" in the new era. (In 2025, Andrej Karpathy of Eureka Labs has something to say.)
To what extent is what we teach/learn training, and to what extent is it education?
Systems engineering
The core philosophy of systems engineering is to constantly focus on the macro-objective and transform local actions into overall results.
The fundamental difference between a systems engineering perspective (system view) and an executor's perspective (task view) lies in "I am building a cathedral" versus "I am carving a stone." Many people mistakenly believe they understand this, but most of the time, they are still immersed in details without considering the bigger picture. Therefore, a systems engineer should be evaluated based on what they produce instead of what they say.
The first rule of systems engineering is: "If you optimize small components, you are likely to ruin the performance of the system." If you focus on optimizing a simple component without considering its impact on the overall objective, you will ruin the overall objective. There are examples of this in engineering, but a more easily relatable example is education: if you constantly look for shortcuts in school (e.g., cramming before tests), your school education will be a failure.
A systems engineering approach to education is to have students pass a comprehensive exam covering nine courses based on their major. In this case, when taking any course, the focus is not on passing the course or pleasing the professor, but on learning the course content so that you will still remember the important knowledge from the course at some later time (2 years later). After all, what matters in your education is what emerged at the end, and what happens at each stage is not as important.
The second rule of systems engineering is that the design must be flexible, prepared for change, and ensure smooth transitions when adjustments are made without compromising the functionality of other parts. The half-life of technical knowledge is about 15 years, which means that many of the details learned today may no longer be applicable in the future. Therefore, the goal of education should be to prepare you for your future, focusing more on fundamental principles rather than changing details.
The third rule of systems engineering is that the design should "gracefully degrade" when overloaded. If a design is too close to its specifications, its failure will be rapid and catastrophic when the load exceeds expectations, much like a bridge designed for a specific load or a telephone system designed for maximum traffic. A good system, however, will gradually degrade in performance when the stress exceeds its limits, rather than collapsing outright.
Systems engineering is a never-ending task. First, the emergence of any solution changes its environment, thereby creating new problems. Second, the process of solving a problem often gives the engineer a deeper insight into the problem itself, leading to dissatisfaction with the existing solution. In systems engineering, there is no such thing as a "final answer" because every solution should provide new insights for the next round of improvements. A solution that does not bring new insights can hardly be called a solution. Therefore, the core of systems engineering is to accept the fact that both problems and solutions are not static; evolution is the norm.
School education typically provides well-defined problems and a single correct answer, but real-world systems engineering work is the opposite. It begins in a chaotic, ambiguous context, requiring engineers to shape a workable solution from it. Hamming believed that the best way to learn systems engineering is to contrast the idealized solutions learned in the classroom with complex real-world situations. The goal of systems engineering is to correctly solve the "right problem," even if the solution itself may not be perfect. Every solution is temporary, but the deep understanding it brings paves the way for the next design iteration.
You Get What You Measure
"Measurement" here refers not only to numbers but to all systems of setting indicators: educational objectives, hiring criteria, performance evaluations, and promotion mechanisms. The tools you use for measurement shape your perception, much like a fisherman inferring a minimum fish size based on the mesh of his net. Design every metric with careful consideration for its second and third order effects.
For example, we can look at how scoring systems influence risk-taking:
- Suppose everyone begins with a score of 95%. There is little room for improvement, but any mistake lowers the score, making risk avoidance the rational strategy. Leadership may desire bold leaders, but promotions ultimately favor shrewd risk-avoiders.
- If the starting average is 50%, behavior tends to be more balanced. To encourage innovation, start at 20% (or lower). The cost of failure is small, while the upside from success is large, motivating people to try. If you want true risk-takers in leadership positions, reward risk-taking early in the evaluation and promotion process.
Short-term indicators for admissions, hiring, or promotions may not reflect your long-term goals. In mathematics and computer science, early selection favors reliable execution of certain mental ability (e.g., cracking leetcode hard). However, later on, the ability to grasp the big picture and predict important new results becomes more important. Similarly, in recruitment, traits that are attractive to a typical recruiter may not be suitable for the right candidate, which is why researchers at Bell Labs do recruitment themselves.
The accuracy of measurement is often confused with the relevance of measurement. For example: measuring "training" is easy, while measuring "education" is difficult. Another classic example is IQ tests. The design and calibration of these tests are inherently intended to make their score distributions conform to a normal curve. Therefore, the conclusion that "intelligence is normally distributed" is less a reflection of the true nature of intelligence than an artificial result created by the measurement methodology itself.
In any organization, you need to be aware of one fact: people will optimize around the metrics you set, rather than toward your original intentions when setting those metrics. Employees at all levels will find ways to make their themselves look good. Worse still, the meaning of metrics becomes distorted as it is transmitted across different hierarchical levels. For example, a lower level operator's assessment report on "organizational capacity" might be misinterpreted at the executive level as the "probability" of success.
When metrics change, behavior adjusts accordingly. Military formal inspections have taught units how to "perform" combat readiness; regular inspections are deliberately prepared for, and even supposedly "random" spot checks are often anticipated in advance. Business or war game training often teaches leaders how to win simulated competitions rather than how to meet real-world challenges. Therefore, when designing any measurement system, you cannot only consider its first-order effects; you must also anticipate and respond to the second-order and even higher-order behavioral responses it will trigger.
Insights from High-Dimensional Space on Complex System Design
When we design a complex system, if we consider all available parameters, it is equivalent to searching for an optimal solution in an n-dimensional space. But when the dimension n becomes very large, where is the main body of this design space located? Is it on the "surface" of this n-dimensional sphere, or in its "interior"? The answer to this question provides important insights for how we search for the best design.
To answer this question, we can compare the volume of the thin shell on the outermost layer of an n-dimensional sphere with the proportion of the total sphere volume. Assuming the sphere has a radius of r, and we consider a sphere with inner radius r(1-\epsilon), then the proportion of the volume of the outermost thin shell to the total volume is:
This indicates that the sphere's volume is almost entirely concentrated in the thin shell near the surface. For example, if we consider the outer half of the radius range (i.e., \epsilon = 1/2), in three-dimensional space (n=3), this thin shell accounts for 1 - (1/2)^3 = 7/8 of the total volume. As the dimension n increases, the ratio 1 - (1/2)^n rapidly approaches 1.
In other words, the optimal design almost certainly lies on the "surface" of the feasible region, rather than in its "interior". This may contradict the optimization methods we learned in basic calculus courses, because those methods typically search for extrema in the interior. However, in high-dimensional spaces, the optimal solution often involves pushing one or more design parameters to their allowable limit values, which naturally means we are on the boundary of the design space.
The timelessness of FORTRAN
In computers, the meaning of an instruction derives from its execution process; the symbol itself have no intrinsic meaning. This is what we call "Semantics comes from implementation." In other words, it is the subroutines written to perform specific operations that define the true meaning of a program.
In the very early era of computing, program control was extremely dependent on human intervention. From desktop calculators to punch card machines controlled by plug boards, then to paper tape and ENIAC's initial control through complex wiring, all of these exemplified this dependency.
To simplify program design, a group of scientists invented symbolic abstraction, such as letting the three letters ADD automatically become machine code like 1011 in binary during compilation. Interestingly, the emergence of this Symbolic Assembly Program (SAP) was viewed by hardline programmers of the time as heretical—"sissy stuff"—which they disdained to use.
Building on the foundation of SAP, John Backus and his team proposed FORTRAN (short for Formula Translation) in the 1950s. Hamming's experience was that after his programmers adopted FORTRAN, their output efficiency improved 10x. However, the professional programmer community of the time widely resisted it, for three main reasons: first, they believed it was impossible to implement; second, even if it could be done, it would extremely waste precious machine time and storage capacity; and third, even if it truly worked, no self-respecting programmer would deign to use it.
FORTRAN eventually became widespread and has withstood the test of time, while many later languages, such as Algol, failed. The fundamental reason of FORTRAN's success lies in its design being more aligned with human psychological intuition. It can directly translated mathematical formulas that people had already become accustomed to in school into programs, without requiring the learning of an entirely new way of thinking. In contrast, Algol, designed by logicians, was more rigorously elegant in logic but ultimately became obsolete due to its lack of human-centric design. This demonstrates that designs aligned with human psychology have far greater vitality than those based on pure logic.
This means that whether a tool can have a profound impact on humanity and withstand the test of time depends on whether it is sufficiently aligned with human nature.
Human-Computer Communication
The core problem of most modern software still remains: "What is the most effective way for human-computer communication?" Its sub-problem is the translation problem, “how to effectively convert human problems into executable instructions?”
Between natural language and binary machine language, there needs to be a substantial number of continuous abstract intermediate layers, which provide humans with a buffer from hardware details, enabling us to express our meaning more directly. The choice of tools (such as interpreters versus compilers) is essentially a choice about who bears the "cost" during translation. If you choose to execute programs with an interpreter, the machine will need to repeatedly execute the same operations, bearing the computational resource cost, but humans can immediately see the execution results; if you choose a compiler, humans need to write complete code in advance, bearing a higher cognitive burden, but the machine can rapidly execute the compiled binary results.
Progress comes from better languages and abstractions, enabling people to express their intentions to machines more clearly and safely.
(Or perhaps by 2025, this problem has already been largely solved..?)
Do machines think?
When exploring the philosophical questions of artificial intelligence, we must first clarify that the definition of "AI" has evolved over time. In 1995, the AI that Hamming discussed primarily referred to computers assisting human intelligence in intellectual domains, such as helping with music composition or exploring chemical synthesis routes. This is quite different from the concepts we are familiar with in 2025, such as generative AI, large language models (LLMs), or AI agents.
Despite these definitional differences, a core philosophical question persists to this day: "Can machines think?" The complexity of this question stems fundamentally from the fact that the words we rely on are themselves poorly defined. For example, computers process "symbols" rather than "information," because we still cannot precisely define what "information" actually is, let alone write programs for it. Similarly, the definition of "thinking" is highly controversial. If we define thinking as "something humans can do that machines cannot," then this question will never have an answer.
One's stance on this question directly influences whether a leader in their field will actively adopt new technologies. Hamming reminds us in his book that harboring the bias that "machines cannot think" will cause you to miss opportunities, but if you too readily believe that "of course machines can think," you are likely to encounter enormous failures. Therefore, the key is not whether to believe or disbelieve, but rather to have your own deep insights into this issue. Today we also clearly understand that if we fail to effectively use AI or related tools, development in any field may stagnate or even become obsolete.
However, despite the widespread application of AI, many philosophical debates remain unresolved:
- Can machines learn from experience? Art Samuel's checkers program was able to adjust its parameters through playing against itself and eventually defeated a state champion. Doesn't this count as "learning from experience"? If you argue that this is merely program instructions, isn't a human being taught by a teacher in a classroom also a form of being "programmed"?
- Do machines possess originality and creativity? A geometric theorem-proving program proposed an elegant proof for "the base angles of an isosceles triangle are equal" that even the program's designer had never thought of. Is this surprise an expression of "originality"? Here lies a curious paradox: once a task can be completed by a program, people tend to view it as a mechanical rote process, thereby denying the possibility that machines possess the ability to think.
Is this kind of surprise brought by machines a logically novel discovery (Logical Novelty), or merely a psychological novelty for humans (Psychological Novelty)? Hamming points out that although program execution results often surprise programmers, strictly speaking, this is not a logical innovation because everything remains within the scope of what the program was designed to do. Yet, don't the great discoveries of humanity also often arise from the accumulation of past experience, producing psychological novelty?
Going further, is "thinking" a binary yes-or-no question, or is it a matter of degree? Perhaps we shouldn't only ask "whether thinking occurs," but rather "to what degree does thinking occur."
Finally, how should we measure thinking? Based on the "result" (what), or the "process" (how)? Hamming observes that when a child learns three-digit multiplication, we feel he is "thinking"; when we adults do the same calculation, it feels more like "conditioned reflex"; and when a computer executes it, we don't feel it's thinking at all. Traditional AI research has overemphasized "results" while overlooking that "process" might be the key to thinking. If a program can accomplish complex tasks that humans cannot, such as designing a chip with a million transistors, does that count as intelligence? Or, precisely because the program was written by humans, are all its behaviors ultimately just mechanical steps following instructions? These are challenging questions worthy of our continued contemplation.
The power and limits of definitions
Before information theory was proposed, discussions about "information" were often vague and abstract. However, a clear definition can transform fuzzy intuitions into concrete constraints that can be calculated, tested, and designed with, thus allowing us to answer questions that were previously unanswerable. Shannon's theory exemplifies this: it transformed vague problems in communications into engineering problems where optimal solutions could be sought, and rigorously proved the boundaries of possibility and impossibility, all without touching on semantics. This is the power of definition.
Yet we must carefully examine the implications of this definition. First, Shannon did not define the "meaning" of information; he merely provided a formula to measure it. Second, this measurement is based entirely on "surprise": the lower the probability of an event, the greater the surprise or information it carries. While this concept works extremely well for communication systems between machines like telephones and computers, it conflicts with how humans intuitively understand information in daily life. This is why Bell Labs' management initially wanted to call it "Communication Theory" rather than "Information Theory." Third, it is a relative measure entirely dependent on your state of knowledge. For instance, when you look at a string of random numbers, you might feel each digit brings surprise; but if you know the formula generating these numbers, the next digit holds no mystery and contains no information.
Shannon's examples reveal an universal phenomenon: any concept formalized through rigorous definition inevitably distorts people's original intuitions about that word. In the long run, the meaning of this precisely defined term will be determined by the definition itself. A famous example is what we commonly call IQ (as discussed before): if we design a test, continuously refine it for internal consistency, calibrate it so scores follow a normal distribution, and then claim this measures "intelligence," we have essentially constructed an almost perfect circular argument.
Therefore, we should cultivate a thinking style that critically examines foundational definitions and their limitations, rather than merely learning the techniques they enable. Physicist Eddington once told a famous story: a group of people fish in the sea with nets, and after checking their catch, they conclude that fish in the sea have a minimum size limit. This conclusion obviously stems from their tools (the nets) rather than the actual ocean. Definitions are also tools, they determine what we can "discover." Shannon's definition of "information" is extremely powerful for calculating communication system capacity, limits, and coding schemes, but it is not a theory about human meaning. We must understand its scope of applicability and avoid over-generalization.
On “Garbage In, Garbage Out” in Simulations
Is the old saying "garbage in, garbage out" always true in computational simulations? In fact, both this principle and its opposite can be wrong. The key lies in determining whether the system is "convergent" or "divergent." (or stable vs unstable)
Divergent systems are extremely sensitive to initial conditions. Small errors are rapidly amplified over time, leading to dramatically different results. Weather prediction is a classic example. In theory, the minute fluttering of a butterfly's wings in Japan could ultimately determine whether a storm occurs in the United States. In such cases, "garbage in, garbage out" holds true, and obtaining accurate results is extraordinarily difficult.
Convergent systems possess inherent stability. Even when initial input data contains large errors (garbage), the system's dynamics will guide it toward a relatively accurate result. In other words, large errors become smaller as the simulation progresses. Consider the early NIKE missile tests, for example: two test missiles disintegrated mid-flight for unknown reasons. To simulate the failure process, the team needed to input the missile's flight data before disintegration, but the telemetry data from that time was unclear. They could only use guessed values (garbage) as initial conditions. However, Hamming realized that the missile's guidance system itself was a powerful feedback mechanism that automatically corrected small deviations in flight trajectory, forming a "strongly convergent direction field." Because of this, despite the inaccurate initial data, the simulation ultimately succeeded in reproducing the failure process and revealed the root cause: engineers had overlooked the interaction between pitch and yaw, causing energy to continuously transfer between the two, ultimately causing the missile to lose control. This is an excellent example of "garbage in, precision out."
This principle is similar to the negative feedback amplifier invented by H.S. Black at Bell Labs: as long as the gain in the feedback loop is extremely high, only one critical resistor in the entire amplifier needs to be high-precision; most other components can be inaccurate. The system's feedback characteristics automatically compensate for these imprecisions. Therefore, before conducting or evaluating any simulation, you should carefully examine this fundamental question: is the system stable or unstable? This will determine both the feasibility of the simulation and the reliability of its results.
What makes a good simulation?
At the heart of a trustworthy simulation is always the question: "Why should we believe this simulation is relevant to reality?" When answering this question, you should not be distracted by arguments about the authorities invested, the computer's performance, or the importance of the problem. Those are merely distractions; the true answer must rest on the soundness of the model and the correctness of the computation.
To build a sound model, you first need domain experts with deep understanding of the problem. Only they know which factors in the model are critical and which can be safely ignored. A common mistake many simulation specialists make is believing they can do everything on their own while overlooking that domain expertise is the foundation of simulation accuracy.
When collaborating with domain experts, you need to master the domain's jargon. This is primarily to avoid misunderstandings of basic concepts. A word or a mathematical symbol can be interpreted very differently in different minds. You must ensure both sides have exactly the same understanding of concepts to avoid ambiguity during implementation.
From a practical (economic and time cost) perspective, a simulation project is usually feasible because it contains highly repetitive computational parts. Whether calculating interactions for every spherical shell in an atomic bomb or the changes of every air parcel in a weather forecast, the same code is applied repeatedly. Without this repetitiveness, the programming cost for a single simulation would be prohibitively high.
A good simulation should help you build intuition about a complex system. Rather than starting with an all-encompassing complex model, you should begin with a simple model that contains only the primary effects. In the early NIKE missile simulations, it was precisely because researchers used slow analog computers that they had enough time to observe and think, thereby "feeling" the missile's flight characteristics. That intuition led to breakthrough design insights such as switching to vertical launch and reducing wing size that are hard to find buried in large amounts of high-speed computation. Gaining insight from simplification first, then gradually evolving to more refined, complete simulations, is a more effective path.
When evaluating the reliability of a simulation, you should establish a rigorous checklist to ask yourself or the simulation provider:
- Does the background field largely support the assumed laws?
- How certain are you that you have not omitted some small but important effects?
- Are the input data reliable?
- Is the simulation stable or unstable?
- What cross-checks do you have against known past experience?
- Can you generate any internal checks such as conservation of mass, energy, or angular momentum? Without redundancy, as you know from lectures on error-correcting codes, there can be no checks on reliability.
As with the principles of error-correcting codes, if there is no form of "redundancy" in the system, we cannot perform any effective checks on its reliability.
“Luck favors the prepared mind”
This saying acknowledges the element of luck, but emphasizes that success largely depends on yourself. At every moment you are, through the way you live, choosing either to prepare for success or to do nothing (people are the sum of their habits).
Hamming gives the example of himself and his colleague Shannon. They worked at Bell Labs around the same time and respectively founded Coding Theory and Information Theory. You might say those ideas were "in the air". That’s true, but why them, and not others who were there then? Perhaps some luck, but more importantly their personalities and ways of thinking made them better prepared than others to discover, study, and create those theories. If great achievements were mainly due to luck, they would not repeatedly come from the same small set of people.
Einstein is another example. Between ages 12 and 14 he asked himself: "If I rode alongside a beam of light, what would the light look like?" He realized he would see a frozen wave crest, yet the mathematics at the time did not permit such a static extremum. That contradiction planted a seed in his mind. Years later, when the time for relativity came and many people were investigating related problems, it was that early thinking that put him in a position to understand how to get to the heart of the matter better than anyone else.
Newton said that if others had thought as hard as he did, they could have done the same. Edison also said, "Genius is one percent inspiration and ninety-nine percent perspiration." Great creations arise from long years of hard work, not from things handed to you. Relying entirely on luck to get through the single life you have is extremely unwise.
So how do you prepare? Summarizing the whole book, you should
(1) Acknowledge that you want to do great work (2) Build a deep knowledge foundation and learn continually (3) Cultivate courage and confidence in yourself (4) Set a vision for yourself and system-engineers towards it (5) Let important, right problems saturate your subconscious so you keep chewing on them even in your dreams (6) Care about the big-picture state of the world in your field and continually make predictions