What Should We Demand From Good Science?

by Morten V. Christiansen

Morten Christiansen is a computer scientist whose interests lie primarily in conceptual modeling — how we express what we understand and model what we see. His father is a physicist so he grew up familiar with discussions of physics.

Most laymen see the scientist as a seeker of truth, and many scientists also see themselves that way. A scientist is a person seeking the true nature of the universe, and good science will tell us what the universe is really like.

This would be fine, if the scientist had some sort of divine answer-list he could check, so he could see if his notions of the universe were really true. The Bible and the writings of Aristotle have both been used as such answer-sheets historically.

But most scientists today have to reject or confirm their ideas about the world based on a limited number of crude observations of how the world appears.

This leads to a couple of problems for the noble idea of finding the veritable truth.

One problem is, that according to the laws of statistics (and inductive logic), it is never possible to conclude anything in general with complete certainty from a limited number of observations. No matter how many white swans we observe, we may not conclude that all swans are white. This problem should only bother a mathematician. Most of us are quite happy to conclude generalities from a limited number of observations, if we are sure that the probability of being wrong is sufficiently small.

A much worse problem is that we would like our knowledge of the world to go beyond the specific details we observe. Rather than theories about how specific objects fall down in specific locations, which is what we observe, we would like a notion like gravity and perhaps a notion of air resistance. Or something like that.

Here we move beyond what we can immediately observe and into the realm of speculation and theory. Most scientist are, whether they agree or not, builders of theories, rather than seekers of absolute truth. Some feel that good theories are also true in some sense of the word, while others feel that the merits of a theory have nothing to do with whether it is true or not in an ontological sense.

In my view this last perspective (epistomology) is the proper one. Science is about building good theories, not about discovering what is really there. If God chooses to clue you in on the true ontological nature of reality, that is a fine thing. But it is not science.

Our question now becomes: "What should we demand of a good Scientific theory?"

This is not a question with a single, obvious answer (like "truth"). It is rather a matter of trying to look at theories we feel are good and which have survived for a long time, and try to express what they have in common. And any rules should be rules of thumb, rather than absolute laws.

One philosopher, Karl Popper, has given rules that seem reasonable to a lot of us.

1. A scientific theory should be as simple as possible

2. A scientific theory should fit observed facts as well as possible

3. A scientific theory should be as falsifiable as possible

Many other criteria might be imagined: A theory should be deterministic. A theory should not disagree with the bible. A theory should be aesthetically pleasing. A theory should describe things as they actually occur. And so on. But Popper's rules turn out to work pretty well.

Popper made it very clear that these rules should apply only to sciences that observe the world. Engineering, for instance, is not a science in Popper's perspective, because it is about creating rather than observing. Neither are such "synthetic" disciplines as logic, philosophy, mathematics and metaphysics. They have no element of observation. The social sciences and most of the humanities are about observing the world, so Poppers rules apply here as well as in the natural sciences.

But let us examine the three criteria which Popper felt were important.

Simplicity

Occam's razor tells us, that the simplest explanation of a fact is most often the truth. In model-building we are not concerned with truth, but we are concerned with eliminating useless complexity. So it seems obvious that simpler theories are more useful than more complex theories, if their contents are identical otherwise.

It is not always possible to make the determination about which of two competing theories are simpler. But often it is possible, and then we can disregard the more complex theory. The Copernican view of the solar system, where the sun is regarded as the center, gave a slightly simpler model of the solar system than the competing earth-centric model. So it was, from Popper's perspective, a better theory. From the perspective back then it was a worse theory, because it did not agree with the writings of Aristotle.

Fitting observed facts

This should be obvious. It is not. We observe that a leaf falls slower than a coin. Yet our preferred theory say that all objects fall with the same speed (Galileo), while the theory it replaced (Aristotle) correctly predicted that the leaf would fall slower. The theory of "air resistance" has to be included to explain what we actually observe. But in most cases things do fall with the same speed, and we can ignore air resistance. And so we consider Galileo's theory to be better that Aristotle's. Few competing theories explain exactly the same set of facts, and so we can sometimes use this as a criterium to prefer one theory for another. And of course some theories fly directly in the face of what we observe. Those we can either dissmiss, or accept in some "weakened" form (for instance by creating supplementing theories, like "air resistance").

Falsifiability

A theory should be as falsifiable as possible. What does this mean ?

It means that the theory should make as many and as specific predictions about the world as possible. The theory should tell us something about the world that we don't learn from other theories, and this "something" should have observable effects. If our theory makes no predictions, yet can "explain" every imaginable observation, does it really tell us anything ? Popper says no. It does not matter how many angels can dance on the head of a pin, if we cannot somehow observe them.

The more new predictions a theory can make, the better the theory. At least until some prediction turns out wrong. Most often this is the criteria a new "intuitively reasonable" theory will fail. Much pseudo-science fail at exactly this point.

Using the criteria

This article started as a discussion of a particular version of the many-worlds interpretation of physics on the fire-list. So let us use that as an example.

We like simple, understandable theories. This means that we don't particularly like quantum mechanics. It is a non-deterministic theory, it claims that particles behave in strange, ghostly ways, it says that there are things we cannot ever know. This is not nice. QM also conflicts with the theory of relativity, because quantum mechanics claim that a particularly phenomenon, the "wave-function collapse" is instant everywhere in the particle system. Relativity says that there is no such thing as instant. We cannot go beyond the speed of light.

Relativity is a much nicer theory than quantum mechanics. It is still counter-intuitive, at least for me, but it follows directly from reasoning I can accept. And it is deterministic, which is a nice property in a physical theory. So I don't like to be told that it is wrong and QM right.

The problem is, quantum mechanics actually describe what we observe, no more and no less. The conflict with relativity has been decided by experiment. QM is right, relativity is wrong. Or, as most prefer to see it, a quantum system is somehow just one place. Locality does not apply in quantum mechanics.

So, what we would like is a theory that would explain the same, or preferably more, behavior that QM does, but without the nasty rules about what we "may say" about particle properties, and also preferably a theory that is easier to grasp intuitively.

Here in Copenhagen such a notion is heresy, of course. The "Copenhagen-interpretation" regards the mathematics of QM as the final truth. Any attempt to understand QM outside the mathematics is meaningless.

But most people not from Copenhagen want a prettier theory, and dislike the "darkness" that QM seem to spread. And so new theories are attempted.

Some theories, like the "hidden variable" models, claim that "there is really something there", even while they accept the fact that we can never measure it. This goes directly against the principle of falsifiability as well as against the idea that our models should be as simple as possible, but it does provide a more intuitively acceptable model.

But it is a bit like claiming that God is really there, even if he never interferes.

Another attempt to explain some of the oddities of QM, such as the fact that particles interact with versions of themselves that only exist as probability (double split experiment) is to postulate that particles interacts with other universes. A particle in fact follows all possible paths in different universes, and these universes interact at the quantum level.

This makes a lot of sense at the intuitive level. Instead of weird and ghostly wave-functions that permeate our entire universe and only collapse at measurement time, we just have to accept multiple universes. We can rescue locality and the theory of relativity. We can even, if we wish, make it a deterministic model. As long as we don't expect anything beyond what QM predicts.

The problem, in Popper's perspective, is mainly falsifiability. We gain no simplicity, because we must construct our many-universe interpretation to carefully match the full predictive power of QM. And a theory that says "The world behaves precisely like QM, but the real truth is something else" is not a better theory, but just metaphysics.

So the real test of the many-worlds model become the predictions. If a many-worlds theory makes testable predictions that are not made outside that theory, and these predictions turn out to hold, then it may someday replace the QM view of the world. I would like that a lot.

Popper's rules do not tell us that we should not speculate. Many speculative and interesting ideas are not falsifiable. But they may still be very inspiring and worthwhile. An example is the notion of super-strings, which are so small that their existence will never be decided. But the mathematics is (or at least started out) nice. Many cosmological speculations about the early or very late universe are not science by Popper's definition. But they are certainly inspiring, and often help in the creation of better theories. But Popper's rules can be useful in helping us know what is speculation, and what is good science.