A selection of key figures from my dissertation.
Chapter 4: Explorations: Teaching, Technology, and Performance
Figure 4.3: 4 frames of images generated by Motion Music, as participants walk across a learning space
The initial Motion Music prototype was able to provide useful auditory feedback about the number of participants in a space – a single participant produced a single auditory tone, while multiple participants produced multiple tones resulting in a chording effect. Motion was represented by modifying the pitch and volume of each generated tone in response to the changing X and Y coordinates of the person as they moved through the space (figure 4.3). As a participant moved from one side of the room to the other, the pitch of their associated tone rose and fell accordingly. As participants move together (or apart), their corresponding audio tones and chords merged or separated, giving a sense of the overall motion of all participants in the space who are visible to the software.
Figure 4.5: A still image, exported from a 360 ̊ video recording of a learning space
After developing a prototype and testing the technology to see if it was feasible for this use, I then recorded 360 ̊ videos within a classroom at the Taylor Institute for Teaching and Learning. The still image in figure 4.5 demonstrates the severe warping that occurs as a result of the panoramic lens and the resulting transformation of the 360 ̊ image into a 2-dimensional format to be saved on the camera.
Figure 4.6: A Baxter humanoid robot, guiding a learner through the steps of assembling a mechanical gearbox
I was fortunate to be part of a team that explored a novel application of humanoid robots – in this case, a Baxter industrial robot (figure 4.6) – to see if it could be used to teach novices how to perform a simple mechanical assembly task.
Figure 4.8: Three versions of media for review by the instructor: high definition video (left), edge-detection “synthetic” video (centre), and Nao robot performance (right)
Through software processing, recorded video can be converted from a high fidelity, realistic repre- sentation and into other interpretations such as a line-drawing or cartoon version of the recording (figure 4.8)
Figure 4.9: The Choregraphe software interface, used to sequence actions for perfor- mance by a Nao humanoid robot
Every action, movement, gesture, vocalization and inflection must be carefully and explicitly programmed – whether into the control system for a remotely tele-operated robot, or into the software and artificial intelligence developed to enable an autonomous robot performer (figure 4.9).
Chapter 5: Teaching and Games: Connecting Performance-Centred Research Disciplines
Figure 5.1: A mockup of video and audio recordings analyzed using video game research methods.
These research methods are complementary and could be employed together to help avoid “blind spots”. SoTL research is best suited to address pedagogical design questions, and evaluation of the impact of those design decisions on student learning. HCI research is best suited to understanding the outcomes of pedagogical design as implemented in the classroom, and how they shape the actions and interactions of all participants.
Chapter 7: Design Study: Exploring Online Teaching Performance
Figure 7.6: Maps of the online learning environment for five courses, as described during the initial interviews
A simple map (figure 7.6) of the key applications or online locations that are used in a course may provide some structure to the experience of remote teaching and learning.
Figure 7.7: Course timelines for courses described by five instructors
Figure 7.7 presents a combined visualization of the courses described by all five instructors. Each week of the semester is presented on the horizontal axis, with activities presented on the vertical axis – grouped by “instructor” and “student” formal roles. This combined view provides a way to quickly compare patterns of activity across all five courses – the patterns for instructor 1 and 4 are similar, while instructor 2 shows a unique pattern with different portions of the semester described.
Chapter 8: Integrating HCI and SoTL: A Framework
Figure 8.7: Five dimensions drawn from the design study, expanded to include concepts from instructional- and video game design
During the design study, a number of dimensions emerged from the interviews with instructors, and from the development of a number of mockups to explore concepts from each course. These warrant inclusion in this chapter because they provide an opportunity to form the foundation of a new framework to describe teaching and learning. Each of these dimensions can be expanded to include several concepts drawn from both instructional design and from the design of video games (figure 8.7).
Figure 8.8: A framework for integrating HCI and SoTL concepts to describe course designs
However, these dimensions do not exist in isolation, but are interconnected and overlapping (figure 8.8). It is this integrated view of the framework that provides useful means to inform our design and description of teaching and learning. Each of the dimensions – Player, Performance, Narrative, Environment, and System – describe different aspects of the lifecycle of a course.