Interpreting Movements
As mentioned, it has largely been the case in the medical education literature that the number of movements an individual makes is indicative of their overall efficiency. That is, the fewer the movements, the more efficient the performance. While, construct validation studies show this to be the general case when expert and novice performances are compared,27, 28 there is less clarity on the way that efficiency develops through the intermediate stages of learning. In particular, one perspective on the study of human motor control describes learning with respect to the way in which individuals vary and explore components of the action as they search for the optimal approach to performance.42-45 As a consequence, different components of action can shift between phases of stability (i.e., low movement variability) and instability (i.e., high movement variability) as learners attempt to organize their movements. The idea is that these shifts occur in response to constraints placed on the performance by the task or environment, and even as a function of changes in the learner’s ability and motivation. A good example of this is Guerin and Kunkle’s (2004) study of individuals learning to kick a ball over a barrier onto a target. Measurement of the participants’ kick height and accuracy over 12 extensive practice sessions demonstrated that they focused initially on ensuring that the ball crossed over the barrier, with little concern for accuracy.46 However, as they became able to clear the height of the barrier consistently, their focus shifted to landing the ball accurately on the target. That is, as the learning experience progressed, the height constraint deteriorated in importance and the accuracy constraint emerged as increasingly more pertinent.
With respect to metrics of efficiency, such as number of movements, this type of shifting means that skill assessors need to understand the performance constraints to which learners are currently attending. A small study exploring the validity of an instrumented simulator for the assessment of surgical knot tying skills provides a nice example.8 In this study, the simulator incorporated flexometer technology that measured the quality of the knots tied by pre-medical undergraduate students (i.e., novices), medical clerks (i.e., intermediates), and senior medical residents (i.e., experts). Interestingly, the technology also permitted the experimenters to measure the economy of action via the amount the walls of the simulated wound moved while the participants performed. Not surprisingly, the results showed that the completed knots were tighter and more sustainable as the performers increased in expertise. That is, the experts’ sutures were better than those of the intermediates, and the intermediates’ sutures were better than those of the novices. However, the movement economy metric revealed that the intermediates performed far more inefficiently than both the experts and the novices. Taken together, the two metrics reveal how the intermediate group had sacrificed attention to efficiency in order to achieve stronger final products. Given that learners will shift focus from outcome to efficiency at different stages of practice, it is essential that the assessor of clinical technical skills remembers that an objective computerized assessment of performance efficiency– defined in terms of number of movements or otherwise - exists independently of the outcome of the performance. As a consequence, the relationship between efficiency and expertise is not always direct.47 Thus, as a construct for assessment, it may be necessary to avoid motion efficiency as a measure of competence until after the learner’s proficiency at reliably producing quality outcomes is well established. In surgery, this may be particularly important, as learners will often value accuracy at the expense of efficiency.
Furthermore, when interpreting number of movements, one should also keep in mind the way in which contexts of performance and individual performer differences can impact how a metric such as number of movements is used as a competency standard. With respect to the former, differences in task rules or equipment can have significant impacts on movement performance. In this regard, for example, it would not be fair to compare the number of movements associated with open surgical performance to those associated with laparoscopic surgical procedure.48 Indeed, even changes in the time a performer is allotted to complete a task has shown to disrupt the metrics of efficiency that so often characterize the most expert performers.49 Moreover, skills that occur in closed, static environments are fundamentally different than those that occur in open, dynamic environments. The former afford approaches to skill performance that are largely anticipatory in nature, which allows for fuller action plans to be generated prior to initiation, while the latter requires more attention to be given to online control processes that determine the movements needed as a procedure unfolds.31 Thus, it would also be unfair to compare performances that permit and don’t permit prior planning, as the latter would most certainly be associated with higher movement counts.