Using Pre and Post-Tests to Close Gaps in Knowledge

Tags: , , , , ,

Description

A challenge most academics experience is delivering courses which assumes that the student has the required prerequisite knowledge.  A great example of this is a course which assumes that a student has completed the prerequisite course at a previous level.  This challenge more often than expected, occurs and with key topics, where the prerequisite knowledge is either missing or incorrect thus delaying the student’s progress as they pursue their course.  When the student acquired the prerequisite knowledge the information may have been designed through different learning activities and was then organized into a specific schema (Minsky, 1975) to have meaning to the student.  As this knowledge structure becomes complex, through the addition of new concepts, the encoding that occurs creates a schema for what was once an informational item (Rumelhart 1984; Rumelhart and Norman, 1978).  What is of particular concern for this situation is the influence of an existing schema on new knowledge that is being encoded (Anderson, 1994).  Knowing that as a student learns, any new knowledge will be guided by the categories and levels of abstraction that already exist and thus foundational concepts may be guided by either misrepresented or missing information – thus creating gaps in knowledge.    This strategy is used in a second year quantitative methods course in an undergraduate business program as a blended learning solution.  The course is a compulsory prerequisite for other core courses hence its high enrollment numbers (e.g. 900 students) .  This course is also taken as a compulsory course by students from other disciplines as well as some students transfer into programs from other institutions, thus contributing to the diversity in knowledge and person in the student population.

The proposed strategy uses a common experimental design (pretest, treatment and post-tests) as a way to consolidate and/or close the gaps in knowledge. It requires students to:

  1. [PRE-TEST] Take a short test online
  2. [TREATMENT] Use the results of the test to guide learning activities to correct any misconceptions or missing information.  These learning activities will occur in a face to face environment
  3. [POST-TEST] Retake the short test online with the outcome hopefully being a closed gap in knowledge.

Link to example artifact(s)

The strategy is employed as a blended learning intervention for the economics course titled “Quantitative Methods II (QM2)”.  This course is used as a compulsory pre-requisite for the Bachelors of Commerce and the Bachelor of Arts programs which can have an annual enrolment of up to 1500 students.  Each week, students are required to physically attend a two hour lecture and a one hour tutorial, making each week three hours of contact.  The strategy is implemented whereby it is assumed that a student will not have the required prerequisite knowledge before attending a class (see Figure 1).  This assumption is used to initiate the pre-testing phase of the strategy whereby all of the students are alerted to the possibility of them not having the prerequisite knowledge and then [they are] given the option to ensure they have the knowledge before most of the content is covered.  The entire intervention is conducted using Smart Sparrow and is delivered via the Learning Management System (LMS) via LTI integration.  The entire intervention requires that self-directed actioning is used by the student to

  1. take the pre-test during the first ten minutes of the tutorial session in week 1
  2. receive the test results in a report (includes a summative score, an outline of all of the topics represented in the pre-requisite knowledge, and highlights to the topics which were incorrectly answered by the student)
  3. use the results to study in week 2
  4. take the post-test (same test as the pre-test) to gauge whether gaps in knowledge are closed (see Figure 2) during the first ten minutes of the tutorial session in week 3

For each of the weeks (i.e., weeks one to three) the curriculum for QM2 was delivered as expected during the scheduled two-hour lecture time.  The report with the test results, has hyperlinked pages with notes to all of the topics that are required for the pre-requisite knowledge, thus forming a study guide for the post-test.  This intervention seem to produce the expected results with there being an increase in student scores when the pre-test is taken in week 1 and the post test is taken in week 3 of the same semester (see Figure 3).

An additional concern of when the correct knowledge is shared creates more complexity in the design.  Should the correct knowledge be assessed before the semester begins or during the semester as part of the course requirements? This design-decision is based on a number of factors

  • What is the topic being tested? Is it important to immediately correct misconceptions thus forming new schemas? What support is available to the student as these new schemas are being formed?
  • How many students are completing this activity? It is important that the number of students completing the activity is monitored to ensure that the correct support systems are in place.
  • What are the characteristics of the students? Did they learn the concepts in a different language? Would the translation of these concepts into a another language result in the same or similar meaning?
Figure 1: Student begins the course with misconceptions or missing prerequisite knowledge
Figure 1: Student begins the course with misconceptions or missing prerequisite knowledge
Figure 2: Student takes a pretest, the treatment that provides study material and then a post-test to close the gap
Figure 2: Student takes a pretest, the treatment that provides study material and then a post-test to close the gap
Figure 3: Results shown seem to suggest that some gaps were closed
Figure 3: Results shown seem to suggest that some gaps were closed

Link to scholarly reference(s)

Anderson, R. (1994). Role of the reader’s schema in comprehension, learning, and memory. In Ruddell, R., Ruddell, M., and Singer, H. (Ed.), Theoretical Models and Processes of Reading(4th ed.), (pp. 469-482). Newark, NJ: IRA.

Minsky, M. (1975). A framework for representing knowledge. In P. Wilson. The psychology of computer vision,2, (pp. 211-277). New York: McGraw-Hill.

Rumelhart, D. & Norman, D. (1978). Accretion, tuning and restructuring: Three modes of learning. In. J.W. Cotton & R. Klatzky (eds.), Semantic Factors in Cognition.Hillsdale, NJ: Erlbaum.

Rumelhart, D. E. (1984). Schemata and the cognitive system.

Citation

Dickson-Deane, C., Karunarathne, W. 2018. Using pre and post-tests to close gaps in knowledge. In Chen, B., deNoyelles, A., & Thompson, K. (Eds.), Teaching Online Pedagogical Repository. Orlando, FL: University of Central Florida Center for Distributed Learning. Retrieved November 22, 2019 from https://topr.online.ucf.edu/r_1npzmc7iisynm2g/.

Post Revisions:

[Total: 2    Average: 3/5]