Methodology

The research question of this study: To what extent are the presentations of key topics in introductory economics textbooks in line with empirical findings? 

Because of practical constraints, we limit the analysis to ten textbooks and ten topics. We plan to expand the project and encourage further research to analyse more textbooks and topics. 

Textbooks

The analysis of textbooks by the experts will done as a blind review. This means that when the experts review the relevant sections of the textbooks, they do not know from which textbooks the texts come. We analyse textbooks that are used in introductory economics university courses around the world. We analyse the most recent editions of the textbooks. When there is an introductory economics textbook series by the same author(s), we include the introductory principles, microeconomics and macroeconomics textbooks. To safeguard the blind analysis of the textbooks by leading experts, we will only provide more information on the selection of textbooks at a later stage in the research.

Topics

The focus in terms of topics is on recent insights and empirical results that have societal relevance. Over the past decades, there have been many important developments and new findings in economics. Here key recent empirical insights are selected to find out to what extent they have made their way into economics education. In this way, we get a better understanding of the degree to which students around the world learn and benefit from gaining state of the art knowledge on these important economic topics. 

Additionally there are two measurement criteria that are taken into account. The topic has to be discussed in most textbooks to enable meaningful analysis and comparison. We also only selected topics for which we are able to identify clear sections of the textbooks in which it is discussed, rather than underlying assumptions that run through entire textbooks.

Process

  1. The editors select the topics and textbooks

  2. The editors select the relevant texts on the topics in the textbooks

  3. The topic experts review the textbooks on their topic with the selected texts

  4. The editors write the textbook and overview pages summarising the reviews by the experts

Scoring

Each of the ten textbooks is scored by leading experts from 1 (very poor) to 5 (very good) on the following question: Is the presentation of the topic in the textbook in line with empirical findings?

  1. Very poor (i.e. The presentation is fully in contradiction with empirical findings)

  2. Poor (i.e. The presentation is mostly in contradiction with empirical findings)

  3. Neither good nor poor (i.e. The presentation is partially in line and partially in contradiction with empirical findings)

  4. Good (i.e. The presentation is mostly in line with empirical findings)

  5. Very good (i.e. The presentation is fully in line with empirical findings)

When a textbook does not discuss the topic, we note “n.d.”.

What we do not measure

This research does not measure everything. Two large areas left out are how much the textbook goes into empirical evidence, and its didactic qualities.

  • How much empirical evidence is discussed

This would often amount to measuring how extensive and detailed a textbook is as longer textbooks are more likely to devote more text to discussing empirical evidence. Rather than favouring long textbooks, we therefore decide to judge the textbooks solely based on whether the presentation of the topic is in agreement or contradicts empirical findings, not whether it discusses those empirical findings. Of course, it is highly relevant to observe whether textbooks do discuss the empirical findings themselves. We encourage experts to include this in the texts about the good and bad practices. But we do not include it in the scoring.

  • Didactic quality of the explanations of the topic

This is highly relevant for which textbook is most effective at teaching students economics. We have, however, decided to limit the scope of this study to the content itself, rather than the didactic quality of its presentation. An assessment of didactic quality requires different expertise and measurement methods. The didactic aspect of economics education receives substantially more attention in research than its content does. To maximise our added benefit we therefore focus on the less often studied aspect of the content.