SMART – Systematic Mapping and Analysis of Research Topographies

(For in-depth study, see references below.)

SMART differs from most other ways of conducting research reviews in three main respects.

Firstly, SMART involves explicit recognition of the fact that educational science research encompass different legitimate research paradigms, where paradigm refers to a research tradition based on certain assumptions about how the world is and how we can gain knowledge about it. In our studies employing SMART, we have used the distinctions between functionalist (sometimes called positivist), interpretive, and critical paradigms, but different sets of distinctions are possible, of course. Having primarily accessed research for our reviews through the Web of Science, which is dominated by Anglo-Saxon research, we have found a predominance of functionalism in the material, alongside distinctly smaller elements of both interpretive and critical perspectives.

Secondly, we believe that research reviews can never be made from a neutral point of view, and we have chosen pragmatism – and its tenet that research can and should be used to develop society – as our point of departure. Together with the first point, this means we believe that different scientific traditions can make important contributions to the development of society and education. What such a development entails is partly a question of values; therefore it is important to be clear about the values ​​on the basis of which the research is assessed.

Thirdly, we focus on what is central to researchers themselves, in the sense that we map and analyze research that is highly cited by other researchers. This does not mean that we believe that such research is obviously the most important or the best, but rather that it is central to the field. A systematic mapping and reflective analysis facilitates critical examination of the research and making informed suggestions regarding the direction or further development of that research. For example, several of our mappings and analyses reveal that actor perspectives are marginalized in research.

It is therefore important to understand that SMART is not a method but an approach, with explicit meta-theoretical and methodological points of departure. We conceptualize SMART as a way of conducting reflective and systematic reviews. Characterizing the reviews as reflective means acknowledging that there are several legitimate scientific paradigms and that one should relate explicitly to this fact and be as clear as possible about one’s own starting points. Compiling educational research is not considered a technical skill, where given rules of a method are followed, but rather something that requires very good knowledge of the research area being analyzed.

In the compilation of the research, there are certain systematic steps that can be followed:

  • Identify research arenas and select the most cited works. (We have mainly used the databases Web of Science and Scopus, as these have information on how often the works are cited.)
     
  • Compile background data (number of authors, authors’ department affiliations, etc.).

  • Map the material. (What theoretical traditions do the researchers start from? Which article genre dominates? Which subjects are central? Which methods are used? etc.)

  • Make an in-depth analysis of one or some aspects. (How are basic concepts defined? Are they defined in different ways in different theoretical traditions?)

There are no hard and fast lines between mapping and analysis, and SMART provides many opportunities to analyze various aspects of high-impact research.

In central parts of our VR project, we have mapped and analyzed research reviews (and not original empirical research) because research on teaching is so very encompassing. Thus, we have used SMART at what has come to be called the overview level (original research-review-overview).

There are also many other ways to develop SMART, such as by finding ways to measure impact other than by the number of citations in the aforementioned databases. It is also quite possible, for example, to analyze the articles in a particular journal that have had the greatest impact. In addition, one might develop the reflective part by relating research not only to scientific traditions but also to educational ideological points of departure.

The book referred to below also contains sketches of how SMART could be used in the social sciences and humanities.

For further reading

Nilholm, Claes. (2017). SMART – ett sätt att genomföra forskningsöversikter [SMART – a way to conduct research reviews]. Lund: Studentlitteratur. (In Swedish only)

Román, H., Sundberg, D., Hirsh, Å ,. Forsberg, E. and Nilholm, C. (2021). "Mapping and analyzing reviews of research on teaching, 1980–2018, in Web of Science: An overview of a second-order research topography". Review and Education (forthcoming online publication). 

Last modified: 2022-03-10