Haughton & Keil Mobile Devices and Electonric Performance Assessments of Student Teachers

Haughton, N. A., & Keil, V. L. (2009). Engaging with faculty to develop, implement, and pilot electronic performance assessments of student teaches using mobile devices. The Teacher Educator, 44(4), pp. 275-284.

Summary: This study examined the implementation of mobile devices to standardize supervisors’ observations of PSTs’ practices in the field experience. They designed a standardized form and provided the supervisors with some training on how to use the mobile devices for observations. Essentially the researchers revised the field observation instrument from being paper and pencil and individualized to being electronic and standardized across all disciplines. Supervisors were required to select specific sentences that described what they were observing. The supervisors were required to assign a score and then provide evidence for how that score was assigned rather than previous practice of recording evidence and then assigning a score. Results indicate that the intervention provided a more standardized and comprehensive approach to PST observation across the disciplines. Supervisors want more training and professional development to minimize some of the initial challenges they experienced.

Key Words: supervision as evaluation, student teaching, field experience, technology, observation tools, mobile devices, supervision in teacher preparation,

Conceptual Framework:

None

Research Questions:

• “Is the instrument comprehensive to the extent that it can be used across all program disciplines?
• How did the use of mobile technology influence the communication between supervisors and student teachers?
• Do the supervisors support the continued use of electronic assessment supported by mobile technology for the supervision process?” (p. 277).

Method:

Mixed methods: Field test, survey, and focus group interview

Participants:

20 supervisors received the mobile devices and were required to use the form in their observations

18 participated in a post survey

8 participated in a follow up focus group interview

Intervention Description:

Findings:

1. “Supervisors were overwhelmingly positive about the experience, the value of the updated assessments, and the ongoing plans to support field observations with mobile technologies” (p. 275).
2. Issues of using mobile devices included a need for more training and support for the supervisors.
3. Supervisors perceived the revised instrument was more comprehensive and accurate.
4. The mobile devices required supervisors to select sentences from a library of pre-determined sentences in order to accurately match their observations from a standardized list of possible observations. Supervisors struggled with this concept and were frustrated by the long list of sentences.
5. The tool caused a shift in supervisors’ inductive practices to deductive practices because of the standardized nature of the tool.
   1. Example: “Jane (trained PRAXIS II assessor, early childhood supervisor, and experienced PDA user) struggled with the changes in her observation routine. She was used to observing student teachers in the field and writing copious handwritten notes regarding what was being observed. Jane would refer to the notes to determine whether each criterion within the Pathwise framework was met. The PDA forced a reversal in this habit. She was not being forced to assign a score and then provide qualitative evidence to support the given score” (p. 280).
6. Supervisors expressed a desire to leave a physical artifact as evidence of their observation (like the triplicate forms). Consequently, they used compensatory strategies that increased their workloads.
   1. “Many (supervisors) also saw the use of the mobile technology as a barrier to providing personal and more detailed comments even though a comprehensive sentence library of comments was available” (pp. 280-281).
7. Supervisors had difficulty with technology, workload, work habit, and observation logistics.
8. Supervisors felt training was inadequate and wanted more.
9. Supervisors expressed that using mobile devices was a good decision and encouraged the college to continue pursuing it.

Key Passages:

None