Comparative Measures in Electromagnetism

Standard Assessments (Concept Inventories)

The School of Physics has used the Brief Electricity and Magnetism Assessment (BEMA) to assess the qualitative understanding gained by introductory electromagnetism (E&M) students. The BEMA is a 30-item multiple choice test focusing on the core concepts of E&M. It contains items that are common to both courses including electrostatics, DC circuits, magnetostatics, and Faraday's Law. A copy of the assessment can be obtained by emailing Danny Caballero.

Students in the M&I course outperform on the BEMA relative to their standard course (TRAD) cohorts. Recent work has probed these performance differences. A preprint of a publication accepted by Phys. Rev. is available here. For more information on this work, please contact Danny Caballero.

Average post-instruction BEMA scores at four academic institutions - The average BEMA test scores are shown for students who have completed a one-semester E&M course with either the traditional (TRAD) or Matter & Interactions (M&I) curriculum. The number of students tested for each curriculum at each institution is indicated in the figure. The error bounds represent the 95% confidence intervals on the estimate of the average score.


Gain in student understanding of E&M at four academic institutions - The increase in student understanding resulting from a one-semester traditional (TRAD) or Matter & Interactions (M&I) course is measured using the average normalized gain g. The number of students tested for each curriculum at each institution is: GT M&I: N=297, GT Trad.: N=887, Purdue M&I: N=76, Purdue Trad.: N=79, NCSU M&I: N=79, NCSU Trad.: N=48, CMU M&I: N=73, CMU Trad.: N=116. The error bounds represent the 95% confidence intervals on the estimate of the normalized gain. The estimates of g require the average BEMA scores for incoming students I; for the NCSU and CMU results, I was computed differently than for the GT and Purdue results. (The average incoming score is relatively stable, that is it doesn't change at a given instituition very much. An incoming average was obtained and used for NCSU and CMU for all semesters tested.)

Pre-test BEMA score distributions for Georgia Tech and Purdue - The distributions of BEMA test scores for students before completing an E&M course with either a traditional (dot-dashed line) or M&I curriculum (solid line) are shown for data from (a) GT (N = 1319 for traditional students, N = 321 for M&I students) and (b) Purdue (N = 78 for traditional students, N = 76 for M&I students). The plots are constructed from binned data with bin widths equal to approximately 6.7% of the maximum possible BEMA score (100%).

Post-instruction BEMA score distributions at four academic institutions - The percentage of students with a given BEMA test score is plotted for students who have completed an E&M course with either a traditional (dot-dashed line) or M&I curriculum (solid line) at (a) GT, (b) Purdue, (c) NCSU, and (d) CMU. The arrows indicate the location of the average score for each distribution. The right-most arrow in each subfigure corresponds to the M&I course. The total number of students tested in each curriculum at each institution is the same as in the first figure. The plots are constructed from binned data with bin widths equal to approximately 6.7% of the maximum possible BEMA score (100%).

Average BEMA scores by section at Georgia Tech - The average end-of-semester BEMA scores for 11 traditional (T#) and 5 M&I (M#) sections at GT are shown. The error bounds indicate 95% confidence intervals on the estimates of the average for each section. The number of students tested in a particular section is given by Nm in the table below.

Fractional difference in performance for E&M subtopics - The fractional difference in performance ΔGi/ΔG between M&I and traditional students at GT is shown for each question on the BEMA. Positive (negative) ΔGi/ΔG indicates superior performance by M&I (traditional) students. The numerical labels indicate the corresponding question number in order of appearance on the BEMA. The ΔGi/ΔG are grouped together into one of four topics: Electrostatics (ES), DC circuits (DC), Magnetostatics (MS), or Faraday's Law and Induction (FL).

Georgia Tech BEMA test results are shown for five Matter & Interactions sections (M1-M5) and eleven traditional sections (T1-T11). Different lecturers are distinguished by a unique letter in column L. (Note that lecturer B in M3 was assisted by lecturer A.) The average BEMA score O for NO students completing the course is shown for all sections. Moreover, in those sections where data is available, the average BEMA score I for NI students entering the course are indicated. Nm is the number of students in a given section who took the BEMA both at the beginning and at the end of their E&M course. GPA is the incoming cumulative grade point average for students in a given section.

BEMA Results (Fall 2008)

Please read the entire caption for explanation. Testing conditions were markedly different.

The results for this semester were strikingly different from the outcome of the large-scale BEMA results above. In the above figure standard course sections are plotted as blue bars and the M&I sections are pltted as red bars. The conditions under which this assessment was given were quite different from the large scle work above. The standard course used the BEMA as part of the Final Exam for the course. The M&I course gave the assemssment in the laboratory with NO incentives. It was assumed the performance would markedly different under these conditions and the results are given for completeness.

CSEM Results (Spring 2009)

Please read the entire caption for explanation. Testing conditions were markedly different.


Another standardized measure was used in the Spring of 2009. The Conceptual Survey of Electricity and Magnetism (CSEM) was given to students in both courses. The results were strikingly different from the outcome of the BEMA. However, the conditions under which this assessment was given were quite different. The standard course used the CSEM as part of the Final Exam for the course. The M&I course gave the assemssment in the laboratory with NO incentives. It was assumed the performance would markedly different under these conditions and the results are given for completeness.

References

There are several references to consider when discussing the BEMA. Those references are listed below:

  1. Ding, L. and Chabay, R. and Sherwood, B. and Beichner, R., Evaluating an electricity and magnetism assessment tool: Brief electricity and magnetism assessment, Physical Review Special Topics-Physics Education Research, vol. 2, pgs 010105, 2006. [ link ]
  2. Pollock, SJ, A Longitudinal Study of the Impact of Curriculum on Conceptual Understanding in E&M, 2007 Physics Education Research Conference, vol. 951, pgs. 172-175, 2007. [ link ]
  3. Pollock, SJ, Comparing student learning with multiple research-based conceptual surveys: CSEM and BEMA, 2008 Physics Education Research Conference, vol. 1064, pgs. 171-174, 2008.
  4. Kohlmyer, M.A. and Caballero, M.D. and Catrambone, R. and Chabay, R.W. and Ding, L. and Haugan, M.P. and Marr, M.J. and Sherwood, B.A. and Schatz, M.F., A Tale of Two Curricula: The performance of two thousand students in introductory electromagnetism, Accepted to Phys. Rev., Arxiv preprint arXiv:0906.0022, 2009. [ link ]

Grades in a College of Engineering Course

Physics 2212 is a prerequisite for the Electrical and Computer Engineering introductory electromagnetics course, ECE 3025, which all electrical and computer engineering majors must take. Student grades were obtained after the completion of ECE 3025. For students who had taken the standard 2212 course, the mean course GPA in ECE 3025 was 2.93 +/- 0.15 (N = 144). For students who had taken the M&I 2212 course, the mean course GPA in ECE 3025 was 2.95 +/- 0.28 (N = 24). The grade distributions for both M&I students and standard course students in ECE 3025 were statistically indistiguishable. For details please contact Danny Caballero.

Common Final Exam Problems

The introductory physics courses have given common final exam problems (free-response). Eric Murray and Danny Caballero have collaborated to develop these questions for the 2212 courses. Common questions were given to 2212 students in the Spring semester of 2009 and Fall semester of 2008. These questions were scored by TAs grading for each course. To compare the performance between the two courses, Eric and Danny agreed on a threshold score within each course that was used to determine the percentage of students who correctly answered the problem sans a minor calculation/clerical mistake. Due to the nuances of the grading scheme used in each course, score distributions cannot be compared. However, both parties agree that the tails of the distribution represent the same type of responses. Linked below are copies of these problems along with percentage of students answering these problems correctly sans a minor mistake.

Spring 2009 - Electrostatics Problem

Problem: [pdf]

Percent Correction: Traditional = 10%, M&I = 17%

Fall 2008 - Electrostatics Problem

Problem: [pdf]

Percent Correction: Traditional = 22%, M&I = 39%

Fall 2008 - Faraday's Law Problem

Problem: [pdf]

Percent Correction: Traditional = 20%, M&I = 29%