At the Junge Akademie, scientists of various disciplines have joined together to form the "Eureka" working group in order to pursue the question of when something is considered proven in which science.
© Anamcara - Photocase.com"Eureka", exclaims the researcher, and has an idea. "Eureka", again, and like magic the result is written down. And "Eureka!" exclaim her readers, thrilled, euphoric, and the world is enriched by a profound insight. A dream? Fiction? When do we really exclaim? Is it, for example, already enough for us as readers if the daily press tells us: "American scientists have discovered that..."? We could make it easy for ourselves and blindly believe. Thank goodness for authority. But we do nothing of the sort.
"My dear colleague, Madame Professor, I need facts, images, data! I want to understand your thoughts, and then know - know so that I may follow your reasoning and have faith in your results." But which is the moment at which I give credence to the results? Is it the moment when I discover the asterisk of significance above a column, or is it not until I sense: "Wow, she's had a good idea and thought it through well; it's so convincing that I now find myself thinking in the same way"? How do we know that and when we know something? This is one of the oldest questions of philosophy. We - a group of researchers from the Junge Akademie, scholars of mathematics to medicine, physics to philosophy, the history of art to historic art - have asked ourselves this question anew. Our interest was not in epistemology, nor was it in the history of science; there are competent specialists for these. Our interest was in our practice in day-to-day scientific work, for each of us in his or her own discipline.
Is there an overarching criterion, an order that tells us to what extent a statistical analysis can be used as proof of a hypothesis, to what extent an argument in philosophy is valid, to what extent a proof sketch confirms a conjecture in mathematics? Certainly there are a wide range of methods with which evidence can be created. We learn and teach them in methodology courses. We also categorise them: mathematical methods are proof, error columns or statistics. Graphics, illustrations or tables deliver visual proof. Controlled experiments, the independent reproducibility of experiments or a precise categorisation of synthesis rules are considered necessary in the natural sciences.
Authoritative references include quotes, but also the standing of the publishing journal in its respective discipline or gratitude expressed towards luminaries. An order such as this has the advantage of immediate plausibility, but it precisely does not answer the question that occupies us daily: "What creates the compelling logic causing the human mind to consider individual theoretical and possibly also experimental steps to be coherent, a prerequisite for speaking of them as corroboration, proof, or an argument in favour of something, or indeed refuting it?" In order to approach this question we have, after categorising the methods, attempted to identify various levels of gaining and generating evidence in everyday research.
Gaining evidenceTime-wise, in one's own scientific work it all starts with the eureka experience as a subjective experience of evidence that comprises of a subjective moment of realisation for an assumption, a hypothesis or an argument. Here, the evidence that has sufficed for the researcher him- or herself to develop a new thesis must be distinguished from the accumulated evidence that he or she requires in order to make a hypothesis or conjecture truly unassailable. This first step of the subjective evidence experience gives rise either to a line of reasoning in order to show the necessity of conceptual contexts, or to an experimental design in order to generate evidence for the hypothesis. This is followed in philosophy by extensive argumentation, in biochemistry by the experiment to provide - ideally conclusive - proof, or in mathematics by the proof that confirms the conjecture.
Generating evidenceOnly after this two-step process of "active" evidence gaining can we also produce the eureka experience in our readers or listeners. For this purpose we first present our abstract ideas, hypotheses, concepts or conjectures. Interestingly, this notionally abstract step is usually performed by means of specific examples. The evidence is then conveyed to the reader or listener by presenting the corroboration, proof or concrete results. This "passive" generating of evidence therefore requires that the reader actually follow the logical connections in his or her mind: a step which always at its outset also requires active creativity. But what precisely happens at these levels? When and by what is our sense of understanding justified? In the case of the initial, private, subjective eureka moment this will certainly be our experience as researchers. For the second step of active evidence gaining we then require, as already indicated, a further decisive creative achievement: the experimental design or line of reasoning.
Experimental designThe experimental design describes the relevance relationship between the scientific hypothesis, the suggested experiment and the results of the experiment. It is decisive for whether evidence is generated or not, because data alone stand only for themselves. The evidence they generate is created only through the research question, that is, the relevance relationship within the experimental design, and not by referring to data or corroboration. In particular, it is only this relevance relationship that allows us to unambiguously accept or reject the hypothesis on the basis of the experimental results. Therefore the experimental design determines whether the results of an experiment suffice as evidence for the statement derived from the experiment.
The results are thus the corroboration that is used for a hypothesis, which however cannot by themselves generate evidence. The statistical test in an experiment is therefore also a methodical component which alone by no means generates evidence. Nonetheless it may be required for a review of the experimental design by a statistician before the experiment is performed, as is often practised in biology or psychology. Moreover, other methodical rules must also be observed. For example, experiments should be planned in such a way that as many parameters as possible can be specifically controlled: the number of quantitative measurement points must also be appropriately selected. In order to generate evidence it must also be in accordance with the other expertise of a discipline. Every discipline has rules and conventions which are recognised as state of the art and therefore also expected to be followed. Only in this context is the sense of one's own understanding completed. As an example of such conventions we will describe the evidence levels of philosophical argumentative working:
1. A thesis is professed, or transcendental circumstances are described.
2. The evidence for this thesis is created through argumentation, for example by showing that an example is relevant to proving the thesis. Analogous to experimental design in the natural sciences is the highlighting of conceptual relationships, for which purpose concept analyses are performed.
3. The eureka moment is produced and conveyed in a purely linguistic fashion. As opposed to data proof in the natural sciences, in this process arguments, conceptual analyses and substantiations already represent necessary, compelling or highly probable correlations.
Evidence conveyorsImages or graphics for example play an entirely different role in passive evidence generation than arguments and conceptual analyses. They are solely methods of conveyance. Nonetheless, in practice an increase in visualisation can be observed in argumentation insofar as illustrations are increasingly presented in place of proof in articles. This is remarkable in particular because it is also true of imaging methods that the evidence of a hypothesis is not provided by the image but through the experimental design and the relevance relationship between hypothesis, experiment and results contained therein. Images, graphics and statistics are thus often only the representation of the results of the experimental work. This is by no means a trivial issue: in particle physics, photographs of decay processes prove that a certain particle has been created. Biologists present images of cells that have been manipulated in various ways. And in art history that which is made visible by images is also affected by the viewing method by which they are approached.
Before we came to these conclusions, allowing the book "Heureka - Evidenzkriterien in den Wissenschaften" (Eureka - Evidence Criteria in the Sciences) to be written, we extensively discussed interdisciplinary misunderstandings. "Of course experiments can only become relevant through statistical testing!" says the biologist. "No", objects the chemist, "when I have successfully synthesised a substance, it either exists or it doesn't. I don't need statistics for that."
We also laboured long and hard over concepts; this too is a frequent symptom in interdisciplinary discourse: is a thesis the beginning of new research? Or the end? Or perhaps the central point? In fact, many of the concepts that are popular in this context (corroboration, evidence, hypothesis, thesis, conclusion, proof) are fundamentally badly or imprecisely defined, and used and interpreted in different ways in practice and across disciplines. This is the source of countless misunderstandings. We therefore tested our observations through extensive surveys of our colleagues. We have compiled the results in the volume "Heureka", published by Spektrum Akademischer Verlag, It contains one chapter each for a total of 14 different disciplines, in which the individual authors respond to the question about the eureka experience and the evidence criteria in their disciplines.
In their disparity, the wide range of replies make one thing clearly apparent: it is the same thing that matters to us all, namely to find out what holds the world together in its inmost folds, each discipline in its own way. And in this respect we know: even our colleagues in the same disciplines wish they could add more to our chapters. And that in turn resulted in a new little, big "eureka".
From Forschung und Lehre ::
19. April 2017
University of Amsterdam (UvA)
5. May 2017