A collaborative project space developing open tools for
understanding the evolutionary reasoning of students, experts, and AIs.
Understanding agency and evolution
How humans, individually and collectively, come to develop an understanding of our own origins and our capacities to adapt, greatly influences our abilities to work together towards a more sustainable planet and more preferrable futures. Â
Developing a scientifically adequate conceptual undertstanding of evolutionary adaptation requires us to think critically, flexibily, and metacognitively, about the agency (e.g. the goals and behaviors) of the agents we are interested in, as well as many other (i.e. decentralized) casual factors that influence evolutionary outcomes (e.g. mutations, drift, natural selection).
The EvoFlex project has grown as a focal point for developing practical assessment tools within the broader OpenEvo Project, Understanding Agency and our AI Innovations space. Within this broader framework, we have clarified some aspects of understanding agency in the context of evolution education, which has led to the practical focus of the EvoFlex project. Â
Our project aims to support open interdisciplinary research at the intersections of educational innovation sciences, cross-cultural cogntive development, and research in Artificial Intelligences.Â
The project has three core pillars of development, spanning theory, research methods, and educational innovation.
Explore our page by scrolling through the sections below, or use the quick links to navigate directly to your section of interest.
Quick links
Theory development
In line with our educational innovation research model, we are synthesizing and working to inform pedagogical guidance (i.e. theories of schooling) on the role of four foundational human cognitive capacities in relation to evolutionary explanations:
- Agential causal reasoning: our capacities to reason about the role of the agency (e.g. goals and behaviors) of agents, ourselves and others, in influencing outcomes of interest. Â
- Decentralized causal reasoning: our capacities to reason about the many other causes not related to agency that influence outcomes of interest. For example, causes related to context, coincidence, functional consequence, and factors distributed in time and space. Â
- Integrated causal reasoning: our capacities to adequately engage and relate agential and decentralized causal reasoning in the context of evolutionary (or other causal) explanation development. Â
- Metacognition: our knowledge about our own cognition and our capacities to regulate it (e.g. toward integrated causal reasoning).
We are interested in the development of these capacities in relation to flexible, contextual, and conceptual reasoning about the scientific adequacy of culturally diverse conceptions of evolution (i.e. expert conceptions in evolutionary biology and anthropology, as well as related models of cognitive, behavioral, cultural and computational evolution, as compared against more diverse, and, possibly, scientifically inadequate conceptions). Our research methods and related educational innovations help drive empirical insights to improve our collective theoretical understanding of the implications for classroom instruction and curriculum design.Â
Methods development
We are working to advance a range research tools and protocols for exploring the development of relevant conceptions in evolutionary reasoning.
Learn more about initial focal projects here:
Educational innovation
In line with our educational design concept, we are advancing a range of educational resources and conceptual innovations informed by and relevant to the EvoFlex project.
Learn more about initial focal projects here:
The origins and evolution of integrated causal reasoning
Since at least the origins of human tool-making behaviors, our ancestors have been working together to influence and evolve our shared understanding of both the agential and decentralized causes that shape our world, even as we shape it.Â
Research in evolutionary anthropology can itself be understood as a culturally elaborated process for the critical accounting and relating of these two broad classes of causal factors. Many departments, research groups, and labs at the Max Planck Institute for Evolutionary Anthropology contribute to a more fine grained understanding of the origins and functions of agential causal factors within our species’ evolutionary history. In this context, the EvoFlex project aims to provide theory, methods, and innovations to empower classrooms around the world to develop their own metacognitive competencies in these domains, through reflecting on the big questions of understanding the human condition.Â
Examples of relevant research from MPI-EVA:
Ammar, M., Fogarty, L., & Kandler, A. (2023). Social learning and memory. Proceedings of the National Academy of Sciences, 120(33): e2310033120. https://doi.org/10.1073/pnas.2310033120Â
Tomasello, M. (2010). Origins of human communication. MIT press.
Tomasello, M. (2022). The evolution of agency: behavioral organization from lizards to humans. MIT Press.
Proffitt, T., Reeves, J. S., Braun, D. R., Malaivijitnond, S., & Luncz, L. V. (2023). Wild macaques challenge the origin of intentional tool production. Science Advances, 9(10): eade8159. https://doi.org/10.1126/sciadv.ade8159
evoflex-1.0 - evolutionary biology assessment
Our formative assessment instrument and interview protocol to explore student or (pre-service) teacher conceptions and reasoning about the role of agency and more classically defined evolutionary concepts across diverse cases of evolutionary biology
Instrument design
The evoflex-1.0 is a 6-question multiple choice assessment that asks participants to select ALL explanations that they agree with, from 5 options. The evolutionary explanation options are similarly structured across cases. Critically, however, the optimally adequate response varies between integrated causal explanations (5/6 questions) and adequate agential negation (1/6 questions), depending on the specific case context.
The focal case studies for the instrument were chosen on the basis of being relatively ‘classic’ cases in research and education in evolutionary biology, as well as spanning a range of agential influences on decentralized causes (e.g. natural selection, behavioral selection, sexual selection).
In addition to this theory-based development of the test items, we also built on current knowledge in evolution education research and earlier exploratory piloting to develop the distractor items in a unique way.Â
Instrument and protocols
Exploratory work using the evoflex-1.0 with pre-service biology teachers in Germany and Argentina has begun. Use these the links below to access our methods:
The evoflex-1.0 is currently available in:
- English
- German
- Spanish
Contact us if you are interested in translating it to other languages, and/or want access to multi-lingual versions of the survey on https://www.soscisurvey.de/
- Interview protocol [forthcoming]
- Interview slide deck [example, requires adaptation to individual results]
We will continue to make all resources for the project open access through this projectbase page. Feel free to explore and use these tools for your own research. We welcome interested or active researchers to get in touch and explore possible collaborations with our team and network.
Current findings
Pre-service biology teacher student samples
In our first round of data collection with (n=75) pre-service biology teachers in Germany and Argentina, we found some evidence that integrated causal explanations are intuitively palatable for these populations, and that many (but not all) such students can engage in contextually flexible reasoning about the role of agency in evolutionary explanations.
Interview excerpt
Our reflective interview protocol can shed deeper insights into student reasoning patterns and processes. In the case below, a student who was a high performer on the evoflex-1.0 instrument further demonstrates her competencies through this explanation of her own reasoning. Further work is needed to explore student conceptions in more linguistically ambiguous cases.Â
Early insights
This exploratory work has already begun to offer some early views into possiblly warranted insights relevant to science education and curriculum design:
Some evolution education researchers argue that a focus on equating evolutionary explanation with agential negation (even if implicitly) is necessary, because reasoning about the role of agency in evolution is “too complicated” for students, who need to focus on “learning the basics” of evolution “first”. Early work with the evoflex-1.0 suggests otherwise. When presented with a scientifically adequate evolutionary explanation that integrates agential causal factors (as contextually appropriate), this option appears to be the dominant explanation preferred by our undergraduate pre-service biology education students. Prior research in evolution education has not typically included scientifically adequate options that integrate agential and decentralized causal factors.
Evolutionary explanations do not universally negate the role of agency in adaptation, but rather, explanations of different focal traits may require the accounting and causal integration of agencies that originate from either internal (to the focal agents) or external (conspecific, predator/prey, or domesticator) sources. Early work with the evoflex-1.0 suggests that many pre-service biology education students can reason in this contextually flexible way across diverse cases, while others may need further support for adequate evolutionary reasoning.Â
While a majority of students in our current sample did tend to prefer explanations that adequately integrate agential causal factors, and many demonstrate contextual flexibility in their reasoning, many still also selected other explanations, often related to needs and desires of the agents themselves. Interpreted alone, these explanations would represent classic misconceptions in evolution, however, in the context of selecting these options alongside the adequately integrated choice, it remains ambiguous to what degree these students are capable of adequately accounting for and causally integrating the role of agent needs and desires within an evolutionary explanation. This is where follow-up interviews and classroom discussions are providing deeper insights and opportunities for metacognitive competency development in the students themselves.
Within our sample of undergraduate pre-service biology teacher students, the inappropriate selection of agential negation explanations was most commonly 2nd or 3rd in frequency to the adequately integrated option. Some of these students also self-identified has having high levels of expertise in evolutionary biology. This adds evidence to emerging discourse in evolution education research on the possible dangers of students (and experts!) overlearning science communication heuristics or metacognitive regulation strategies. That is, students often chose to negate all forms of agency, possibly on the basis of the simplifed linguistic cues of “random genetic mutation and natural selection”.
EvoFlex AI Studies
As part of our OpenEvo AI Innovations space, we are driving exploratory work and interdisciplinary discussions on the capacities, opportunities, challenges, and threats related to the use of Large Language Models (LLMS), such as ChatGPT and diverse alternatives, in reasoning about evoflex related conceptions, as well as pedagogical content knowledge. That is, we are interested in exploring how models like ChatGPT reason about the scientific adequacy of examples in the evoflex-1.0 survey instrument. Additionally, we want to see how diverse LLM models reflect patterns of reasoning about pedagogical strategies for diverse possible student conceptions in this space. We are aiming to share our project outputs here as early as possible, to foster more open science discourse and research in these directions. We welcome interested or active researchers to get in touch and explore possible collaborations with our team and network.
Can ChatGPT-3.5 reflect adequate agential reasoning and contextually flexible reasoning?
By prompting ChatGPT-3.5 with the evoflex-1.0 question items, we can see interesting patterns in the results. The LLM tended to choose a wide range of the items (often correctly selecting ef_1, integrated causal reasoning, but also often pairing that with the more or less ambiguously incorrect items). Importantly, the LLM appears to (sometimes) flexibly shift to the appropriate negation of agential causes (ef_5) in the case of fur thickness in polar bears.
Ultimately, while some performances from these studies have been impressive, ChatGPT-3.5 is not currently competent in evaluating evoflex items to warrant more general use of the LLM for automating or evaluating evolutionary explanations. Additional work in fine tuning models on causal explanations across diverse cases could strengthen performance on benchmarks like evoflex-1.0.
Study materials
What can we learn from comparative analysis of LLMs and multiple instrument items?
By prompting both ChatGPT-3.5 and Google’s Bard with the evoflex-1.0 question items AND the items from kaevo-2.0 (Kuschmierz et al. 2020), we can see further interesting patterns in the results. While both LLMs frequently declined to choose a best explanation, when items were identified as most scientifically adequate, Bard had a strong preference for the evoflex items ef_1 (integrated causal reasoning) and more so, ef_4 (ambiguous need-based teleology). Similarly, but slightly different, ChatGPT-3.5 also tended to favor evoflex items, including ef_1 and ef_4, but also demonstrated much wider variation, including occaisionally selecting kaevo items, including kvo_2; the scientifically adequate item from the kaevo-2.0 instrument, though not as frequently as kvo_4, a misconception. You can explore the prompts on each model using the buttons below.
Study materials
Data (forthcoming)
Prompt links:
Tools for metacognitive competencies
We are working to support the development of educational innovation research that leads to practical products for use in schools and classrooms globally. Currently our focus is on developing tools with the following pedagogical aims:
Scientific models can be communicated in more precise ways (e.g. formal models) or more accessible ways (e.g. verbal explanations in everyday language). While there may or may not be a fundamental tradeoff between precision and accessibility, it is often the case that using everyday language to communicate science may introduce ambiguity in terms of the precise causal model being described. This can ambiguity can be relevant, both in understanding how students might interpret a given evolution explanation task, as well as how educators can or should interpret the explanations preferred or produced by students.
- “Why” questions (e.g. Why does a giraffe have a long neck) may be interpreted as a request for information about the origins of the trait, OR the functions of the trait.
- Phrases such as “in order to”, or references to “desires” or “needs” may or may not include adequate explication of the viable causal chain between agency and evolutionary outcomes.
Educators can use the phenomena of linguistic ambiguity to drive discussions on the nature of science generally, and the nature of evolutionary explanations in particular.Â
Nehm & Kampourakis (2023) characterize the distinction between novice and expert reasoning in evolutionary domains in terms of a conceptual structure of knowledge. They argue that while novices will use a range of different, case or context-specific explanations to explain diverse phenomena, experts will recognize and utilize the higher-level generalized principles (e.g. mutation and natural selection) to explain the full range of diversity of interest. While we respect the intent of their argument, we emphasize a need for more nuanced thinking about the actual conceptual structures of knowledge in evolutionary biology (and evolutionary anthropology). We suggest expert reasoning entails recognizing the full landscape structure of generalizable concepts, including higher and more mid-level explanatory principles. Evolutionary biology is already structured this way, at least implicitly, in terms of agential and decentralized causal factors. Natural selection is a very high level generalization, and is sometimes even used by experts as a general term even to refer to cases involving more agential influence. Concepts such as sexual selection, behavioral selection, and niche construction are all examples of more mid-level generalizations to explain particular classes of agential influence on the otherwise decentralized process of natural selection. Thus, contra to Nehm & Kampourakis (2023), expert reasoning in evolutionary biology requires understanding the structured landscape of generalized principles and the ability to apply them critically across appropriate case contexts.
Evolutionary explanations can take many forms, and because of this linguistic ambiguity, experts can potentially disagree about the best structure or components of an adequate or optimal explanation. Some in evolution education present frameworks that may be at odds with our approach to integrated causal reasoning (see e.g. Kampourakis 2020; Eirdosh & Hanisch 2022). In this way, some experts may prefer evolutionary explanations that abstract organismal agency out of the equation, while others may see value in articulating and clarifying what functions, if any, organismal agency may play in evolutionary adaptation. We suggest that scientific literacy and understanding the nature of science itself requires understanding this aspect of expert disagreement, and the plausible value of pluralistic explanations of a given set of phenomena. Â
EvoFlex-based teacher education resources
We are working to support the development of educational innovation research that leads to practical products for use in schools and classrooms globally.Â
The basic evoflex survey can provide an accessible entry point to classroom dicsussions with pre-service or in-service teachers regarding the causal dynamics and causal reasoning entailed in evolutionary reasoning.
In line with with our broader framework for metacognitive competencies ( see Hanisch & Eirdosh 2023), reflecting on the evoflex project and resources can empower students with new tools to critically inspect and revise their own causal models of evolution. Building on González-Galli et al (2020) and Perez et al. (2022), we suggest that evolutionary literacy or competency requires metacognitive regulation, but critically, it is not simply the down regulating of agential concepts, but the co-regulating of agential and decentralized causal concepts, that is required.
EvoFlex Team and Collaborators
Our project began through discussions between our educational innovation labs in Germany, and our collaborators in biology teacher education in Argentina. Learn more about our collaborators below, and join us in developing this open, collaborative research space!
Project leads
EvoFlex is centrally coordinated by the OpenEvo project within the Department of Comparative Cultural Psychology at the Max Planck Institute for Evolutionary Anthropology, in cooperation with teacher education research groups at University of Leipzig and the Friedrich-Schiller University of Jena, Biology Education research groups.
University research collaborators
Universidad de Buenos Aires
Prof. Leonardo González Galli
Dr. Gaston Perez
Betina Cupo, M.Sc.
References
We maintain an open access Zotero collection of references relevant to the EvoFlex project. These references are not comprehensive in scope, but reflect some core and helpful literature for understanding the context and aims of our project.