I just finished reading the following paper:
"An e-coaching ecosystem: design and effectiveness analysis of the engagement of remote coaching on athletes."
Boratto, L., Carta, S., Mulas, F. et al. Pers Ubiquit Comput 21, 689–704 (2017).
https://doi.org/10.1007/s00779-017-1026-0Overview
Upon finding this paper I was super excited, because it is directly related to my existing project on remote coaching of athletes by human coaches. This paper examines the usefulness in behavior change of an existing application for remotely coaching runners. The app allows users to be linked with a human coach who programs their training program and analyzes their performance. One interesting aspect of this app is the coach can also setup a virtual trainer that can communicate with the athlete while they workout. The virtual trainer can read notes or give feedback or give encouragement to the athlete. The users can also opt to create their own workouts as well including using the virtual trainer.
One important point that I will discuss further below is that in order to utilize the human coach users had to subscribe to the coaching service.
The authors conduct a study on the apps user data to gain insight into the following research questions:
RQ1: Are the human-coach created workouts preferred by athletes? If so, to what extent?
RQ2: Do athletes train more often when under the supervision of a human coach?
RQ3: Do athletes complete more human-coach programmed workouts or more athlete-programmed workouts?
They answer these questions using quantitative methods.
The results of their analysis is summarized as follows:
RQ1: Users prefer human-coach programmed workouts:
“75% of athletes performed 54% or more of their workouts by completing those assigned by a human coach” [p. 698].
“50% of athletes performed 94% or more of their workouts by completing those assigned by a human coach” [p. 698].
“25% of the athletes performed athlete-programmed workouts” [p. 698].
RQ2: Yes, users train more when they are supported by a human coach. The authors state: “The human-coach is fundamental to keep users motivated.” [p. 700]
“25% of the athletes only worked out during their subscription period” [p. 700].
“50% of the athletes were most active during their subscriptions period which consisted of 78 to 100% of their active training weeks” [p. 700].
RQ3: Yes, athletes complete more workouts when they are programmed by a human coach.
“56% of the athletes completed their workout sessions when using a human coach.” [p. 701]
“12% of the athletes completed athlete-programmed workouts.” [p. 701]
Study Critique
These are very nice results, but I have a hard time putting a lot of faith in them even though I want to, because they support my intuition. However, there are two main issues I have with this study:
Paid Subscriptions: The paper does not discuss the impact a paid subscription model has on the results of the study.
Ethics: The paper does not discuss the means to which the authors gained access to the data for the study. Nor does the paper discuss participant consent.
The subscription model has an affect on participant engagement. The very act of being willing to financially invest in coaching implies a higher level of engagement by the athlete. This could support the high engagement of subscribers versus the low engagement of non-subscribers. It is my opinion that a better study would follow a free model to remove this potential bias.
The main ethical issues I have with this study is there is no discussion of participant consent. The study was conducted on the data of real paid subscribers of the mobile app u4fit. Since the participants are actual paid subscribers did they know that their data was being used as part of a research study? Did they have an option to opt out of the study? Unfortunately, none of this was discussed in the paper.