Search for Articles
Effect of Multiple Object Tracking Training on Visual Search Strategies
Journal Of Digital Life.2021, 1,3;
Received:September 10, 2021 Revised:October 5, 2021 Accepted:October 8, 2021 Published:October 14, 2021
- Ryousuke Furukado
- Faculty of Engineering, Integrated System Engineering, Nishinippon Institute of Technology
- Daisuke Akiyama
- Faculty of Human Science, Department of Sport Science and Health, Kyushu Sangyo University
- Tomohiro Sakuma
- Nihon University, College of Engineering
- Ryota Shinriki
- Tokoha University, Faculty of Health Promotional Sciences
- Goichi Hagiwara
- Faculty of Human Science, Department of Sport Science and Health, Kyushu Sangyo University
- Hirohisa Isogai
- Faculty of Human Science, Department of Sport Science and Health, Kyushu Sangyo University
This study aimed to understand changes in visual search strategies before and after intervention, in addition to examining the training effect of multiple object tracking (MOT) skills. Twenty-nine male university baseball players were divided into two groups: a experiment group (EXP, n = 19) and a control group (CON, n = 10). The three-week intervention comprised MOT tasks. Because of the intervention, a significant interaction was observed between the groups, and before and after intervention. EXP Group showed a significant increase in scores on the MOT tasks after intervention compared to those before the intervention. Subsequently, we examined the changes in visual search strategies among six participants with a large training effect (LTE group) and six participants with a small training effect (STE group). Consequently, the gaze travel distance of the post-intervention in the LTE group was short, and the eye’s angular speed between gaze points was large – moderate effect size without a significant difference. These results suggest that MOT skills can be acquired through training for baseball players. Furthermore, we did not observe any clear change in visual search strategies brought about by MOT skill training.
In ball sports such as baseball and soccer, a player is in possession of the ball for a limited amount of time, and a snap decision made during that time can affect winning or losing. In baseball, runners need to quickly judge their situation and act by predicting the next play. In close-score situations, the pressure placed on the runners becomes very large. Therefore, runners need to carefully observe many objects, such as the position of opponents, teammates, and the ball, within the visual field, interpreting them within the constantly changing context. Tracking several objects moving within view is called multiple online tracking (MOT). Pylyshyn and Storm (1988) examined this mechanism using an MOT task, which clarifies that MOT uses selective attention since it is impossible for a human to track every moving object simultaneously. Several models have been proposed as models of attention during the performance of MOT tasks (Meyerhoff et al., 2017), and studies have been conducted in the field of cognitive psychology.
Research on MOT has also been conducted in the field of sports. Faubert and Sidebottom (2012) state that MOT is a necessary ability in open-skill sports and is important for advanced visual processing in situations where more than one object moves.
MOT also involves elements such as accuracy and speed, each of which is a trainable skill, understood as the ability to achieve the target action (Maruyama, 2001). Neuroscience studies suggest that MOT skills can be trained (Mahncke et al., 2006). For example, a study of ice-hockey athletes by Faubert and Sidebottom (2012) found that training in 15 MOT skills sessions improved performance on the MOT task by a mean of 40%. A study that conducted training on MOT skills at a frequency of twice a week for five weeks (Parsons et al., 2014) reported improvement in working memory, attentional function, and visual information processing speed; a significant decrease in absolute and relative power of the EEG delta (1–4 Hz), theta wave (4–8 Hz), and alpha (8–12 Hz) bands; a significant increase in the relative power of the beta band (13–30 Hz); and a significant increase in the absolute/relative power of the gamma band (30–50 Hz). In terms of the correlation between MOT skills and performance, significant correlations were found between MOT skills and basketball performance (Mangine et al., 2014), and improvement in MOT skills influenced performance improvement in soccer passing accuracy (Romeas et al., 2016).
However, because an MOT task involves tracking the movements of multiple objects with the eyes, movement of gaze is considered an important defining factor in MOT skills. One method of moving the line of sight is called the visual search method, which is defined as “the process of selecting specific information among many pieces of visual information present in a field of view spreading in front of one’s eyes and accurately capturing an object” (Anii, 2008, p.367-370). Both MOT and visual searches involve eye movements used to select information by attention to different sources of visual cues. Additionally, efficient training methods in MOT skills could be added to sports practice if the pattern of visual search strategies during the performance of MOT tasks can be classified. However, previous MOT studies have not examined the relationship between MOT skills and visual search strategies.
Therefore, it would seem useful to improve the performance of visual search strategies used in sports in an ever-changing situation. More specifically, advantages could be gained, such as having more time available in sports plays by reducing the time spent in judgment and decreasing error frequencies by increasing judgment accuracy. Therefore, in this study, in addition to examining the training effects of MOT skills, we aimed to clarify the factors that improved MOT skills from changes in visual search strategies.
2-1. Test participants
Participants were 29 university students (M = 19.69; SD = 1.07) playing baseball. They were randomly assigned to either a experiment group (EXP, n=19) and a control group (CON, n=10). None of the experimental participants had prior training experience in MOT skills. To participate in the experiment, the participants had to have a static visual acuity of ≥ 1.0 which included correction, normal color vision, and visual function.
2.2. Study protocol
First, pre-intervention measurements were administered to 29 participants. EXP group was given MOT skill training based on Parsons et al. (2014), which demonstrated training effects in MOT skills, with training given twice a week for five weeks. In this study, MOT skill training was conducted twice a week for a three-week period. Finally, post-intervention measurements were taken, and the data obtained for both groups were compared.
2.3. Intervention method
1) Training task: The Neuro Tracker (hereafter, NT) MOT task (CogniSens Athletics, Inc., Neuro Tracker Pro 65) was adopted for the training task. This task featured eight yellow spheres (10 cm in diameter) on an immersive cube (1 side = 1.5 m) screen (Figure 1a). First, four spheres were randomly selected from the eight spheres. These spheres and depths changed to red, and lit up for 1 sec (Fig. 1 b). All spheres then returned to yellow and were indistinguishable from the others. Then, for approximately 8.70 seconds, the spheres moved randomly, repeatedly colliding with the walls and one another on the cube screen (Fig. 1c). Numbers 1 to 8 were assigned to all spheres once movement had ended, and the participant responded by identifying the numbers of the four yellow spheres. The correct answer was then shown to the participant (Fig.1 d). The correct answers were shown by a change in the color of the particular spheres and their depth to red. This series of events made one trial, a total of 20 trials were administered following this session. Each trial was performed using the staircase method; sphere speed is increased when the participant correctly selects all four spheres in the prior trial or a decrease in speed if one sphere is incorrectly selected. Velocity thresholds were calculated using the 1-up 1-down staircase method of Levitt (1971). The movement of the spheres increased in 0.05 log increments when answered correctly and decreased the same when answered incorrectly. This staircase procedure was interrupted when it had flipped eight times, and the final velocity threshold was calculated using the average speed during the last four inversions. The movement speed of the sphere in the first trial was 0.3 m/s.
2) Training frequency: EXP group performed two sessions of an NT MOT task on the day of training, with training provided twice a week for a three-week period (12 sessions in total). The time required for one training period was approximately 20 min and to perform one session was approximately 8 min. There was an adequate break in between sessions.
2.4. Measurements before and after training
1) Measurement Items: Measurements of MOT skills and eye movements were performed simultaneously. The NT MOT task was used to measure MOT skills before and after training, a 47-inch 3D television (Panasonic Co., Ltd., TH-47AS650) was used for presentation of the images. An eye movement measurement device (nac Image Technology Inc., EMR-9) was used to measure eye movements during the performance of the NT MOT task. The sampling rate was 60 Hz, with a minimum resolution of 0.1°. A visual field lens of 62° was also used.
Experimental participants were asked to wear 3D glasses (Panasonic Co., Ltd., TY-ER3D4MW) and an eye movement measuring device in the laboratory. The participants were instructed to sit on a chair 1.5 m away from the television screen, and their head was fixed to a chin table. A calibration task of the eye movement measurement device was then performed. When presenting images in the laboratory, a dark curtain was placed on the wall surface to eliminate the effects of external light.
Rate thresholds after completion of one session were used for the assessment of scores on the NT MOT task. Additionally, if an angle deviation occurred during the experiment, offsetting was carried out each time to maintain the validity of the data.
2) Analysis Methods: To investigate the training effects of MOT skills, a two-factor analysis of variance (ANOVA) was conducted between groups and pre- and post-intervention on speed thresholds for the NT MOT task.
Subsequently, to clarify the training effects for MOT skills and the associated visual search strategies, the correlation coefficients (Pearson’s product-moment correlation coefficients) were calculated and compared for the mean pre-and post-training changes in scores of the NT MOT task and eye movement data for the EXP group. The change in each item was calculated by subtracting the mean value before training from the mean value after training.
EXP group was divided into two subgroups: those whose training had a large effect and those whose was minimal. This allowed for the examination of how the visual search strategy changed, focusing on the effect of MOT skill training. The grouping was based on a ±0.5 SD from the differences in velocity thresholds before and after training on the NT MOT task, with participants categorized into two groups: six in the large training effect group (LTE group) and six in the small training effect group (STE group).
Eye movement data for each trial of the NT MOT task was extracted from how far the sphere moved on the screen (510 frames) until it stopped. This task was performed for one session (20 trials). Then, eye movement data from one session (20 trials) extracted per experimental participant were integrated (10,020 frames) and included in the analyses. Blinking or a misalignment of the measuring device may have caused errors to the eye movement data. Therefore, this study analyzed eye movement data in which valid eye movement data accounted for more than 90% of a session. The eye movement data for all the experimental participants were valid.
Analytical items for eye movement data were calculated for channel data from both eyes with disparity correction processing for distance traveled (m), number of gaze points, gaze duration time (s), and eye’s angular speed between gaze points (deg/sec) of gaze for one session. The distance traveled by the gaze was calculated from the x- and y-coordinate values between each frame for each trial to calculate the Euclidean distance. In this study, gaze status was defined as a state in which the gaze was retained for at least 133 ms within 2° of a particular visual object (Kato and Fukuda, 2002). The visual angle for the sphere presented on the anterior side of the cube on the television screen was approximately 3.82°. EMR-dFactory, made by nac Image Technology, Inc., was used for these eye movement data analyses. If there was a significant interaction in the two-factor ANOVA, a simple main effect test was performed. In all tests, IBM SPSS Statistics (version 20.0) was used. Additionally, the effective dose was calculated by referring to the study by Mizumoto and Takeuchi (2010), a standardized index that does not change depending on the sample size. The indices used for the effective dose were partial eta-squared (ηp2) in the ANOVA, and Cohen’s d index (d) was used for comparison between the groups.
3.1. Training effects of MOT skills
The two-factor ANOVA for the training effect of MOT skills revealed a significant interaction between group and pre- and post-intervention (F(1,27)=7.46, p<.05, ηp2=.22). The score of the NT MOT task greatly improved post intervention (M=1.92, SD=0.51) in comparison to pre intervention (M = 1.35, SD = 0.37) when the simple main effect test was executed (p<.01, d=1.27). Additionally, after the intervention, EXP group (M=1.92, SD=0.51) had significantly higher scores on the NT MOT task than CON group (M=1.08, SD=0.41), and the effect size was also sufficiently large (p<.01, d=1.76). These results reveal that three weeks of training with the NT MOT task improved MOT skills in EXP group (Figure 2).
3.2. Correlation between MOT skills and eye movement measures before and after training
Correlation coefficients were obtained for the change in the score for the NT MOT task and in the eye movement data before and after training. The correlation coefficient between the scores of the NT MOT task and the gaze travel distance for one session was r=-.08, correlation coefficient with the number of gaze points in one session was r=-.04, correlation coefficient with gaze duration time in one session was r=-.06, and the correlation coefficient with the eye’s angular speed between gaze points in one session was r=.24.
3.3. Differences in visual search strategies by training effect
The mean values and standard deviations for each item are listed in Table 1. A two-factor ANOVA was performed on the distance traveled by eye gaze in one session, which revealed no significant interaction between group and pre- and post-intervention (F(1,10)=2.06, n.s., ηp2=.17), and no significant main effect for each factor, but the effect size in the STE group pre- and post-intervention (pre-intervention: M=66.00, SD=10.59; post-intervention: M=83.8, SD=19.91) was sufficiently large (d=1.12). The effect size in the LTE group before and after the intervention (pre-intervention: M=78.4, SD=21.46; post-intervention: M=70.76, SD=25.35) was moderate (d=0.33). A two-factor ANOVA for the number of gaze points per session revealed no significant interaction between group and pre- and post-intervention (F(1,10)=0.01, n.s., ηp2=.001), with no significant main effect for each factor. The STE group also had a smaller effect size before and after the intervention (pre-intervention: M=243.50, SD=34.00; post-intervention: M=249.83, SD=53.01) (d=0.14) and a smaller effect size before and after the intervention in the LTE group (pre-intervention: M=241.17, SD=45.22; post-intervention: M=244.50, SD=54.36) (d=0.17).
Two factorial analyses of variance for gaze duration time in one session revealed no significant interaction between group and pre- and post-intervention (F(1,10)=0.86, n.s., ηp2=.08) and no significant main effect for each factor, but the effect size in the STE group pre- and post-intervention (pre-intervention: M=158.36, SD=4.08; post-intervention: M=154.29, SD=6.06) could be interpreted as large (d=0.79). The effect size in the LTE group before and after the intervention (pre-intervention: M=157.68, SD=3.77; post-intervention: M=156.81, SD=7.74) was small (d=0.14).
A two-factor ANOVA was performed on the eye’s angular speed between gaze points in one session, which revealed a significant interaction between group and pre- and post-intervention (F(1,10)=7.62, p<.05, ηp2=.43). The test of simple main effects showed a significant decrease in the eye’s angular speed between gaze points after the intervention (M = 151.79, SD = 21.43) in comparison to before (M=174.86, SD=17.89) (p<.05, d=1.17). The effect size in the LTE group before and after the intervention (pre-intervention M=155.78, SD=21.80; post-intervention: M=164.64, SD=19.00) was moderate (d=0.43).
Eye Movement Data Before and After the Intervention
4.1. Training effects of MOT skills
These skills significantly improved in EXP group (mean 142%) after the intervention compared to before the intervention. The effect of training on MOT skills has reportedly improved performance in basketball (Mangine et al., 2014) and soccer (Romeas et al., 2016). Additionally, MOT skill training is effective in reducing human errors (e.g., missing catches, inaccurate passes, and missing shots) among professional athletes (Faubert and Sidebottom, 2012). It is believed that training MOT skills will reduce missed catches and judgement during defense scenarios in baseball. However, at this time the specific aspects in baseball have not been determined. To clarify this, it will be necessary to add performance variables before and after the measurement of MOT skills and examine their impact.
4.2. Eye movement indices associated with MOT skills
By comparing the correlation coefficient for the scores on the NT MOT task and the change in the extent of eye movement data before and after the intervention, the visual search strategy correlating with MOT skills could be clarified. The results showed that the correlation coefficient between the change in MOT skills and the change in gaze travel distance, number of gaze points, and gaze duration time of the eye was less than 0.10, and the correlation could not be clarified. Furthermore, the correlation coefficient was 0.27 on the change in the extent of MOT skills and change in the extent of the eye’s angular speed between gaze points, indicating a weak positive correlation. Thus, this possibly relates to the improvement in MOT skills and improvement of the eye’s angular speed between gaze points. It is inferred that with the improvement of MOT skills, the movement speed of the sphere during the MOT task increased, and the eye’s angular speed between gaze points also increased to correspond to the speed. However, the correlation coefficients between pre- and post-intervention for both variables were small, so it is unlikely that the MOT skills and the eye movement indices measured in this study are correlated. In contrast, there was a difference in the effect size of the training in MOT skills even among EXP group. EXP group was divided into groups considering the magnitude of the training effect, and an analysis was conducted to identify any difference in visual search strategy.
4.3. Differences in visual search strategies by training effect
A comparison of the gaze travel distance in one session of the NT MOT task revealed no significant interaction and main effects between group and pre- and post-intervention. The mean MOT score in the LTE group was M=1.27 before the intervention and M=2.41 after the intervention, and in the STE group M=1.49 before the intervention and M=1.42 after the intervention. Therefore, the traveling speed of the sphere after the intervention during the NT MOT task was larger in the LTE group and smaller in the STE group. Previous studies have reported the shrinking of gaze movement when the information entering the visual field becomes complex (Ando and Watanabe, 2002). In the LTE group, it was predicted that the movement of the gaze would shrink owing to the increasing movement speed of the sphere and the movement distance would become shorter. However, the results of this study suggest that the movement speed of the sphere when performing the MOT task is not dependent on the movement distance of the gaze.
No significant interactions or main effects were observed between groups and pre- and post-intervention for the number of gaze points and gaze duration time in one session. These results show that the number and time of eye gaze points do not change before and after the intervention, depending on the MOT skill training. The lack of significant differences in the number of gaze points and gaze duration time in this study could be attributed to the NT MOT task that was difficult in terms of predicting the objects’ motion. For example, when the shortstop throws a ball in a defensive situation with runners on the first and third bases in baseball, it is possible to respond systematically to the situation. Such specific scenarios in sports include information necessary to predict the situation, and proficiency, such as the experience of the athlete, that may influence eye movement data. In contrast, the NT MOT task did not include predicable information similar to that in a sports context with dynamic multiple-object movement, because in addition to spheres moving randomly in each trial, each sphere is indistinguishable from the others. A previous study that examined visual search activity with multiple objects in the field of vision noted the importance of fixing the gaze on a specific object (Kikumasa and Kokubu, 2018). Therefore, it is necessary to examine the position of the gaze points in the future.
Considering the eye’s angular speed between gaze points, a significant reduction in the STE group was found post-intervention. In contrast, no significant difference was observed in the LTE group between pre- and post-intervention. From these results, as a visual search strategy to enhance the effect of MOT skill training, the speed of gaze movement from one gaze point to the next gaze point seemed to be irrelevant.
To explore factors that improved MOT skills, we compared changes in MOT skills and visual search strategies before and after the intervention. We considered visual search strategies before and after the intervention in the LTE and STE groups but could not clarify changes in visual search strategies for either group. Future studies should address the following points to advance this research.
Intra-individual comparisons may be performed to clarify the change in visual search strategies before and after intervention. For example, examining gaze at locations where objects are crowded together, where objects collide, or where crossings occur, may reveal visual search strategies that lead to efficient enhancement of MOT skills. Additionally, although this study’s population was composed of baseball players, it is important to collect data from a population of diverse participants and increase the sample size.
However, given the results obtained in this study, factors other than visual searching may have improved MOT skills. For example, the finger of installation (FINST) model (Pylyshyn and Storm, 1988), used during an MOT task, states that during an MOT task, subjects do not move repetitively between tracking objects by eye movements but index the objects in advance in their heads to retain and track each object. Additionally, attention, working memory, and information processing speed are of particular interest as indices related to MOT skills (Parsons et al., 2014).
In this study, we hypothesized that improved MOT skills would alter visual search strategies; however, in view of the results, the variables are not believed to be correlated. We believe that this data will be useful for future experiments that can test different variables contributions to MOT skills using other variables, like attention.
In this study, we conducted an eye movement measurement experiment to investigate the training effect of MOT skills on baseball athletes and how visual search strategies change before and after training. Training twice a week for three weeks resulted in a significant improvement in MOT skills (142%) after the intervention. The correlation coefficient of the change in the extent of MOT skills and eye movement index before and after the intervention was small, and no correlation between the variables seemed to exist. Therefore, we concluded that MOT skills are likely not correlated with visual search strategies. Future research will be needed to advance the study on the model of attention style during MOT task execution, especially variables other than visual search strategy, clarify the training effect of MOT skills on actual baseball performance, and construct a guidance method for actual sporting situations.
Conceptualization, R.F., H.I.; methodology, R.F., D.A., T.S., R.S., G.H.; software, R.F.; validation, R.F.; formal analysis, R.F., T.S., G.H.; investigation, R.F., D.A., T.S., R.S., G.H; resources, R.F., D.A.; data curation, R.F., D.A., T.S.; writing—original draft preparation, R.F.; writing—review and editing, R.F., H.I.; visualization, R.F., D.A., T.S., R.S., G.H.; supervision, R.F.; project administration, R.F., H.I.
Conflicts of Interest
No potential conflicts of interest are disclosed.
Institutional Review Board Statement
This study was conducted at a time when the ethics organization change at the Kyusyu Institute of Technology, Graduate School of life Science and Systems Engineering, to which the author belonged at the time of the primary publication (2018), was undergoing review, so it did not undergo ethical review. However, we conducted our research in accordance with the ethical guidelines for Ergonomics Research on Human Subjects. Specifically, we explained the purpose of the research in writing and orally to the participants of the experiment when we asked for their cooperation. At that time, we explained to them that their cooperation in the study was not mandatory, and they could stop cooperating at any time, and that the data would be processed statistically so that no individual would be identified. Finally, if the participants understood these explanations and were willing to cooperate in the study, they were asked to fill out a consent form with their own signature.
Informed Consent Statement
Informed consent was obtained from all participants involved in the study.
Data Availability Statement
This paper is a secondary publication of an article entitled ” Training Effect of Multiple Object Tracking Skills: An Investigation on Visual Search Strategies “, which appeared in the Japanese Journal of Ergonomics, Volume 55, Number 2, Pages 33-39. It has been approved by The Japan Ergonomics Society (Permit No. 21002).
Ando, T., & Watanabe, K. (2002). Automobile driving security－Evaluation of visual activity－, Material for the 19th Sensing Forum of the Society of Instrument and Control Engineers, 19, 245-250.
Anii, A. (2008). perceptual/cognitive abilities. Japanese Society of Sport Psychology (Eds.), In Encyclopedia of sport psychology (pp.367-370). Tokyo: Taisyukansyoten.
Faubert, J., & Sidebottom, L. (2012). Perceptual-cognitive training of athletes. Journal of Clinical Sport Psychology, 6(1), 85-102. https://doi.org/10.1123/jcsp.6.1.85
Kato, T., & Fukuda, T. (2002). Visual search strategies of baseball batters during the preparatory phase of batting. The Japanese Journal of Ergonomics, 38(6), 333-340. https://doi.org/10.5100/jje.38.333
Kikumasa, S., & Kokubu, M. (2018). Decision-making and Visual Search Strategy of Baseball Catchers in a Situation of Giving Directions on a Play, Japanese Journal of Sport Psychology, 45(1), 27-41. https://doi.org/10.4146/jjspopsy.2017-1707
Levitt, H. C. C. H. (1971). Transformed up‐down methods in psychoacoustics. The Journal of the Acoustical society of America, 49(2B), 467-477. https://doi.org/10.1121/1.1912375