For anyone looking for a quick and hands-on dive into the world of Bayesian modelling and inference, I can't recommend JASP enough, made freely available by the University of Amsterdam[0]. I've recommended it before, and it's just a breeze to work with, seeing frequentist and Bayesian analyses side-by-side.
There’s no “prior” to the priors, thus the priors are actually posteriors, and follow from habit and not anything deductively necessary. Habit always evades critique, as it explains itself. Thus, we will be forced, ceaselessely, to think in the same ways, just with differing parameters.
For a wonderful biographical take on this topic try "The theory that would not die-how Bayes'rule cracked the Enigma code, hunted down Russian submarines & emerged triumphant from centuries of controversy" by Sharon McGrayne.
She tells a terrific story with a fascinating large cast of characters including Laplace,Bayes,Fisher,Pearson,Jeffries,Savage,Turing and many others. Engagingly told, highly recommended. Could the takeaway "Do you want to solve a practical problem or do you want scientific rigor?"
Relatedly, David Deutsch's "Simple refutation of the ‘Bayesian’ philosophy of science"
> By ‘Bayesian’ philosophy of science I mean the position that (1) the objective of science is, or should be, to increase our ‘credence’ for true theories, and that (2) the credences held by a rational thinker obey the probability calculus. However, if T is an explanatory theory (e.g. ‘the sun is powered by nuclear fusion’), then its negation ~T (‘the sun is not powered by nuclear fusion’) is not an explanation at all. Therefore, suppose (implausibly, for the sake of argument) that one could quantify ‘the property that science strives to maximise’. If T had an amount q of that, then ~T would have none at all, not 1-q as the probability calculus would require if q were a probability.
> Also, the conjunction (T₁ & T₂) of two mutually inconsistent explanatory theories T₁ and T₂ (such as quantum theory and relativity) is provably false, and therefore has zero probability. Yet it embodies some understanding of the world and is definitely better than nothing.
> Furthermore if we expect, with Popper, that all our best theories of fundamental physics are going to be superseded eventually, and we therefore believe their negations, it is still those false theories, not their true negations, that constitute all our deepest knowledge of physics.
> What science really seeks to ‘maximise’ (or rather, create) is explanatory power.
Any refutation that depends on the fundamental unknownability of the Universe rules trivially applies to every single philosophy is science.
Science must work despite it, or you don't have science.
And any other singularity you get from assuming the odds of a hypothesis is infinitely smaller than the odds of it being false is unrealistic. You shouldn't assume that.
Not from a non-bayesian perspective. It's what Deutsch would call a "bad explanation" i.e. it's easy to vary and thus doesn't tell us about how the sun is powered.
It's a bad explanation because the sun (probably) IS powered by nuclear fusion.
Deutsch is confused by this situation because he doesn't have the scientific background to understand the usefulness of negations of hypotheses.
Historically, for example, a lot of people believed the sun revolved around the earth. If we treat this as T, then ~T is "the sun does not revolve around the earth".
~T certainly lacks details, but to say it's a "bad explanation" is rather silly. Obviously it's an incomplete explanation, which is why Galileo presented a full explanation ("the earth revolves around the sun") rather than just saying, "the sun does not revolve around the earth". But in fact, "the sun does not revolve around the earth" was the part that was controversial because it was the bad explanation being presented by the church (who happened to be closer to philosophers than scientists).
Basically, Deutsch is just making a straw man argument. In Deutsch's mind, the fact that "the sun does not revolve around the earth" is an incomplete theory of heliocentrism is somehow a refutation of all science, when in fact that's simply not the sort of hypothesis scientists even explore typically.
Bullshit like this is exactly why I think scientists are better philosophers than philosophers are. The text you've quoted, is, frankly, not the writings of an intelligent person.
The reason I'm being very blunt about this, is because bullshit like this is actively harmful. Science is fucking important. Science is what resulted in the technology you're using to read this. Science is, with non-negligible probability, the basis of medicine that prevented you from dying before the age of 5 to be able to read this. When philosophers posit that they can inspect the their own navels and find deep truths about the world, they are undermining one of the fundamental pillars of society that holds up so much of the positive changes humans have been able to make.
We need to call this what it is--nonsense and misinformation--and stop amplifying its signal.
> By ‘Bayesian’ philosophy of science I mean the position that (1) the objective of science is, or should be, to increase our ‘credence’ for true theories, and that (2) the credences held by a rational thinker obey the probability calculus. However, if T is an explanatory theory (e.g. ‘the sun is powered by nuclear fusion’), then its negation ~T (‘the sun is not powered by nuclear fusion’) is not an explanation at all. Therefore, suppose (implausibly, for the sake of argument) that one could quantify ‘the property that science strives to maximise’. If T had an amount q of that, then ~T would have none at all, not 1-q as the probability calculus would require if q were a probability.
Of course "the sun is not powered by nuclear fusion" IS an explanation, it's just not an explanation of a phenomenon we observe, which is why most scientists don't believe "the sun is not powered by nuclear fusion". If we observed something about the sun that was not consistent with the hypothesis that it is powered by nuclear fusion, "the sun is not powered by nuclear fusion" would indeed be an explanation of what we were observing.
This is all sidestepping the absurdity that Deutsch doesn't seem to understand that "none at all" has a mathematical representation, 0, meaning that if p = 1 - q = 0, then q = 1. This is not difficult math here, folks.
> Also, the conjunction (T₁ & T₂) of two mutually inconsistent explanatory theories T₁ and T₂ (such as quantum theory and relativity) is provably false, and therefore has zero probability. Yet it embodies some understanding of the world and is definitely better than nothing.
Uh sure, which is why nobody with a brain takes the conjunction of those two things. This isn't a criticism of Bayesian philosophy of science, it's a straw man argument.
> Furthermore if we expect, with Popper, that all our best theories of fundamental physics are going to be superseded eventually, and we therefore believe their negations, it is still those false theories, not their true negations, that constitute all our deepest knowledge of physics.
I agree entirely. As a professional scientist who routinely uses Bayesian methods to solve complex computational and staticial problems, with actual real world applications, I cannot stress enough how irrelevant such philosophical musings about the foundations of Bayesian Statistics are for getting actual science work done.
I don't think you know what you're responding to, but in any case, regarding Deutsch, he "laid the foundations of the quantum theory of computation, and subsequently made or participated in many of the most important advances in the field, including the discovery of the first quantum algorithms, the theory of quantum logic gates and quantum computational networks, the first quantum error-correction scheme, and several fundamental quantum universality results."
Deutsch is primarily a physicist, not a philosopher. I'm not a fan of his philosophical takes either but he is not as stupid as you infer. He invented the Deutsch-Josza algorithm along with Richard Josza, the first example of absolute quantum speedup.
Doctor Oz was at one time a doctor, too, but he's also been a blight on society, spreading medical misinformation for decades.
A lot can go wrong with a person's brain between the publication of Deutsch-Josza (1985, 1992) and the writing of the linked post (2014).
I'd also note that the algorithm described is more a work of math than an example of experimental science. It's not creating a hypothesis and testing it, it's writing an algorithm for a (then-theorized) computer system. So I wouldn't say that this lends credence to his ideas on the validity of hypotheses.
Perhaps, but I don't think that's what happened. He continued publishing good quality research in quantum theory and quantum computation until I was doing my PhD, which is just before 2014, and likely still does.
As I say, I also disagree with his philosophical (and political) viewpoints. But your dismissal above is based on a pretty shallow reading alongside unfounded personal attacks.
Thank you for beating me to it. Astounding how those LessWrong dorks were able to revive the corpse of this dead end of epistemology after it was so thoroughly destroyed by Popper. And to what benefit.. sex crimes, massive financial fraud, murder cults and a far right government?
I never thought I'd see a misunderstanding of what "implies" means in science versus in logic be the fundamental mistake made in a paper on logic for science.
Here's the truth table for implies (if) in logic.
| A | B | If A then B |
|---+---+-------------|
| T | T | T |
| T | F | F |
| F | T | T |
| F | F | T |
Show this to anyone in the sciences who hasn't done logic and you'll instantly get the objections "But hang on, the two rows at the bottom don't fit!".
This is where you need to add temporal logic so that the scientific understanding of A casually implies B can be represented in logic.
In short the paper does nothing of the sort of what it says it does because it fundamentally uses the wrong tool for the job.
In the following table we have the other three operations which give different truth values to what you call the vacuous case. If "if" is replaced by "x" or "y" in the following table than that factorization does not work, "z" still does work but I'd be hard pressed to find anyone who thinks it represents causal implication better than "if" or "x".
| A | B | x A B | y A B | z A B | If A then B |
|---+---+-------+-------+-------+-------------|
| T | T | T | T | T | T |
| T | F | F | F | F | F |
| F | T | F | F | T | T |
| F | F | F | T | F | T |
I'm not using the common names for the operations to not bias people.
Or to put it another way: No one outside a logic course seriously thinks that A implies B means the same thing as not A or B.
For anyone looking for a quick and hands-on dive into the world of Bayesian modelling and inference, I can't recommend JASP enough, made freely available by the University of Amsterdam[0]. I've recommended it before, and it's just a breeze to work with, seeing frequentist and Bayesian analyses side-by-side.
[0]: https://jasp-stats.org/
There’s no “prior” to the priors, thus the priors are actually posteriors, and follow from habit and not anything deductively necessary. Habit always evades critique, as it explains itself. Thus, we will be forced, ceaselessely, to think in the same ways, just with differing parameters.
For a wonderful biographical take on this topic try "The theory that would not die-how Bayes'rule cracked the Enigma code, hunted down Russian submarines & emerged triumphant from centuries of controversy" by Sharon McGrayne.
She tells a terrific story with a fascinating large cast of characters including Laplace,Bayes,Fisher,Pearson,Jeffries,Savage,Turing and many others. Engagingly told, highly recommended. Could the takeaway "Do you want to solve a practical problem or do you want scientific rigor?"
Great recommend, thanks!
Going to leave this here: https://plato.stanford.edu/entries/popper/#GrowHumaKnow
Relatedly, David Deutsch's "Simple refutation of the ‘Bayesian’ philosophy of science"
> By ‘Bayesian’ philosophy of science I mean the position that (1) the objective of science is, or should be, to increase our ‘credence’ for true theories, and that (2) the credences held by a rational thinker obey the probability calculus. However, if T is an explanatory theory (e.g. ‘the sun is powered by nuclear fusion’), then its negation ~T (‘the sun is not powered by nuclear fusion’) is not an explanation at all. Therefore, suppose (implausibly, for the sake of argument) that one could quantify ‘the property that science strives to maximise’. If T had an amount q of that, then ~T would have none at all, not 1-q as the probability calculus would require if q were a probability.
> Also, the conjunction (T₁ & T₂) of two mutually inconsistent explanatory theories T₁ and T₂ (such as quantum theory and relativity) is provably false, and therefore has zero probability. Yet it embodies some understanding of the world and is definitely better than nothing.
> Furthermore if we expect, with Popper, that all our best theories of fundamental physics are going to be superseded eventually, and we therefore believe their negations, it is still those false theories, not their true negations, that constitute all our deepest knowledge of physics.
> What science really seeks to ‘maximise’ (or rather, create) is explanatory power.
https://www.daviddeutsch.org.uk/2014/08/simple-refutation-of...
Any refutation that depends on the fundamental unknownability of the Universe rules trivially applies to every single philosophy is science.
Science must work despite it, or you don't have science.
And any other singularity you get from assuming the odds of a hypothesis is infinitely smaller than the odds of it being false is unrealistic. You shouldn't assume that.
Love this. So clearly written. The truest thing we know is how much we don’t know
„The sun is not powered by fusion“ actually contains a little information as to the inner workings of the star, so I’m a bit confused by the argument.
Not from a non-bayesian perspective. It's what Deutsch would call a "bad explanation" i.e. it's easy to vary and thus doesn't tell us about how the sun is powered.
It's a bad explanation because the sun (probably) IS powered by nuclear fusion.
Deutsch is confused by this situation because he doesn't have the scientific background to understand the usefulness of negations of hypotheses.
Historically, for example, a lot of people believed the sun revolved around the earth. If we treat this as T, then ~T is "the sun does not revolve around the earth".
~T certainly lacks details, but to say it's a "bad explanation" is rather silly. Obviously it's an incomplete explanation, which is why Galileo presented a full explanation ("the earth revolves around the sun") rather than just saying, "the sun does not revolve around the earth". But in fact, "the sun does not revolve around the earth" was the part that was controversial because it was the bad explanation being presented by the church (who happened to be closer to philosophers than scientists).
Basically, Deutsch is just making a straw man argument. In Deutsch's mind, the fact that "the sun does not revolve around the earth" is an incomplete theory of heliocentrism is somehow a refutation of all science, when in fact that's simply not the sort of hypothesis scientists even explore typically.
He's talking about Bayesian philosophy of science, not science, which ultimately does not rely on Bayesian epistemology.
He's an astrophysicist, by the way.
Bullshit like this is exactly why I think scientists are better philosophers than philosophers are. The text you've quoted, is, frankly, not the writings of an intelligent person.
The reason I'm being very blunt about this, is because bullshit like this is actively harmful. Science is fucking important. Science is what resulted in the technology you're using to read this. Science is, with non-negligible probability, the basis of medicine that prevented you from dying before the age of 5 to be able to read this. When philosophers posit that they can inspect the their own navels and find deep truths about the world, they are undermining one of the fundamental pillars of society that holds up so much of the positive changes humans have been able to make.
We need to call this what it is--nonsense and misinformation--and stop amplifying its signal.
> By ‘Bayesian’ philosophy of science I mean the position that (1) the objective of science is, or should be, to increase our ‘credence’ for true theories, and that (2) the credences held by a rational thinker obey the probability calculus. However, if T is an explanatory theory (e.g. ‘the sun is powered by nuclear fusion’), then its negation ~T (‘the sun is not powered by nuclear fusion’) is not an explanation at all. Therefore, suppose (implausibly, for the sake of argument) that one could quantify ‘the property that science strives to maximise’. If T had an amount q of that, then ~T would have none at all, not 1-q as the probability calculus would require if q were a probability.
Of course "the sun is not powered by nuclear fusion" IS an explanation, it's just not an explanation of a phenomenon we observe, which is why most scientists don't believe "the sun is not powered by nuclear fusion". If we observed something about the sun that was not consistent with the hypothesis that it is powered by nuclear fusion, "the sun is not powered by nuclear fusion" would indeed be an explanation of what we were observing.
This is all sidestepping the absurdity that Deutsch doesn't seem to understand that "none at all" has a mathematical representation, 0, meaning that if p = 1 - q = 0, then q = 1. This is not difficult math here, folks.
> Also, the conjunction (T₁ & T₂) of two mutually inconsistent explanatory theories T₁ and T₂ (such as quantum theory and relativity) is provably false, and therefore has zero probability. Yet it embodies some understanding of the world and is definitely better than nothing.
Uh sure, which is why nobody with a brain takes the conjunction of those two things. This isn't a criticism of Bayesian philosophy of science, it's a straw man argument.
> Furthermore if we expect, with Popper, that all our best theories of fundamental physics are going to be superseded eventually, and we therefore believe their negations, it is still those false theories, not their true negations, that constitute all our deepest knowledge of physics.
Deutsch apparently doesn't know what an approximation is, and instead thinks of correct/incorrect as a binary. Relevant https://hermiene.net/essays-trans/relativity_of_wrong.html
I agree entirely. As a professional scientist who routinely uses Bayesian methods to solve complex computational and staticial problems, with actual real world applications, I cannot stress enough how irrelevant such philosophical musings about the foundations of Bayesian Statistics are for getting actual science work done.
I don't think you know what you're responding to, but in any case, regarding Deutsch, he "laid the foundations of the quantum theory of computation, and subsequently made or participated in many of the most important advances in the field, including the discovery of the first quantum algorithms, the theory of quantum logic gates and quantum computational networks, the first quantum error-correction scheme, and several fundamental quantum universality results."
Deutsch is primarily a physicist, not a philosopher. I'm not a fan of his philosophical takes either but he is not as stupid as you infer. He invented the Deutsch-Josza algorithm along with Richard Josza, the first example of absolute quantum speedup.
Doctor Oz was at one time a doctor, too, but he's also been a blight on society, spreading medical misinformation for decades.
A lot can go wrong with a person's brain between the publication of Deutsch-Josza (1985, 1992) and the writing of the linked post (2014).
I'd also note that the algorithm described is more a work of math than an example of experimental science. It's not creating a hypothesis and testing it, it's writing an algorithm for a (then-theorized) computer system. So I wouldn't say that this lends credence to his ideas on the validity of hypotheses.
Perhaps, but I don't think that's what happened. He continued publishing good quality research in quantum theory and quantum computation until I was doing my PhD, which is just before 2014, and likely still does.
As I say, I also disagree with his philosophical (and political) viewpoints. But your dismissal above is based on a pretty shallow reading alongside unfounded personal attacks.
Thank you for beating me to it. Astounding how those LessWrong dorks were able to revive the corpse of this dead end of epistemology after it was so thoroughly destroyed by Popper. And to what benefit.. sex crimes, massive financial fraud, murder cults and a far right government?
Here's a fun one https://sci-hub.3800808.com/10.1038/302687a0
I never thought I'd see a misunderstanding of what "implies" means in science versus in logic be the fundamental mistake made in a paper on logic for science.
Here's the truth table for implies (if) in logic.
Show this to anyone in the sciences who hasn't done logic and you'll instantly get the objections "But hang on, the two rows at the bottom don't fit!".This is where you need to add temporal logic so that the scientific understanding of A casually implies B can be represented in logic.
In short the paper does nothing of the sort of what it says it does because it fundamentally uses the wrong tool for the job.
But those two rows are vacuous truths, which Popper obviously doesn't care about in context as they're not falsifiable.
The whole paper depends on the factorization:
In the following table we have the other three operations which give different truth values to what you call the vacuous case. If "if" is replaced by "x" or "y" in the following table than that factorization does not work, "z" still does work but I'd be hard pressed to find anyone who thinks it represents causal implication better than "if" or "x". I'm not using the common names for the operations to not bias people.Or to put it another way: No one outside a logic course seriously thinks that A implies B means the same thing as not A or B.