Wearables and mobile health (Mhealth) apps collect subject/patient data from mobile or purpose built devices to record data in real time. The rationale behind this type of technology is to reduce the burden on subjects by eliminating unnecessary procedures, streamlining routine procedures and reducing time spent at clinical trial sites. It is evident from the review of a range of literature that studies integrating some form of mobile health technology can be broadly categorised into a few phases of development: studies on the development of new devices, studies on the validity of functional wearable devices, studies comparing new device and conventional endpoints, and finally those studies which trial the device as a health intervention. This article aims to briefly discuss these phases with reference to examples of recent studies demonstrating some safety or efficacy endpoint relating to a newly developed device.
New developments in health devices and apps have been passing through the market for many years and the advent of sensors has enhanced their functionality in recent years. One such recent device is a novel sensorized shoe system called ActiveGait. This device, which is essentially a specialist shoe, was developed by Simbex LLC to measure gait severity in children with cerebral palsy (CP). A study by Mancinelli et al1 measured gait severity in eleven children with CP to detect areas where pressure was applied on the foot. The sensors were particularly accurate in detecting children who displayed a toe-walking pattern. Classification of the gait severity was done via a random forest (RF) algorithm that classified the gait severity based on the Edinburgh Visual Scale (EVS). The results from the classification algorithm were comparable against previous studies that used a camera motion analysis system and also to Experts who classified gait severity based on video recording made during data collection. It was concluded that a larger dataset of patients using the ActiveGait in a home setting over a period of several months, could help with the longitudinal monitoring of gait severity on patients which will provide valuable information when making therapeutic decisions.
Studies of this sort demonstrate that the results from new devices such ActiveGait are proving to be a valuable method of recording and analysing data, however this may require specialist knowledge on data mining and machine learning techniques that may not be easily acquired.
The next stage of development we identified from the literature was validating a wearable device against an industry standard. This is an important phase of development as it ensures the new technology can be replicated in a safe and reliable manner which is comparable to the current industry standard. Examples of such studies include Kwasnicki et al’s2 study where sensor based mobility score (the Hamlyn Mobility Score or HMS) was used to assess postoperative functional mobility in patients with open tibial fractures. The comparator assessment was the ‘gold standard’ Quality of Life Questionnaire, SF-36. Correlation between the combined HMS scores and the SF-36 questionnaire were assessed to determine the extent of concurrency, and Internal validity of the HMS was determined by calculating an intraclass correlation coefficient, which turned out to have p-value < 0.001, suggesting high validity. Concurrent validity between HMS and SF-36 scores was demonstrated by Spearman correlation coefficient, which turned out to have a p-value of 0.005, again confirming significant validity. It is evident from the results that the HMS has demonstrated reliability and validity in terms of performance compared to a standard method. Furthermore, the HMS demonstrated a higher receptiveness to changes in mobility compared to the SF-36. It was concluded that, where standard assessment such as SF-36 lack of specificity and repeatability, a sensor based mobility assessment like HMS is able to provide objective and detailed information about the patient’s rehabilitation allowing for adaptable, personalized and cost effective postoperative follow-up services.
This study is a good example of a successful validity study, where the new assessment compares well to the standard, and in some cases, proves to be even better than its predecessor.
Diabetes as a therapeutic area has seen a few interesting devices over the years. Gadgets like Bayer’s Contour NEXT USB allow for the facilities of a traditional glucose meter reader with USB functionality, intended for “plug & play” access to blood sugar trends that can be read fast and easily off one’s computer. These standard glucose monitoring systems are numerically and clinically accurate and have been the gold standard within the medical industry since 1991. However, at its core, they remain a painful and invasive monitoring method.
So is there any non-invasive technology out there? And do their efficacy endpoints differ from traditional ones? The answer to both those questions is ‘yes!’, in the next stage of development where new and conventional endpoints are compared.
Sobel et al3, in their pilot study, have been investigating a novel, non-invasive armband device (SenseWear Pro Armband, SWA) to estimate levels of plasma glucose in patients. The SWA measures endpoints such as galvanic skin response (similar to lie detector tests), heat flux, and body motion, which is then converted to signals using a series of Mathematical algorithms to predict physical activity and caloric expenditure. These endpoints are very different to the simple test of the level of blood glucose on a test-strip. The performance of the SWA was compared to a standard system in 2 separate conditions: an oral glucose tolerance test (OGTT, which is a fasting test to elicit rapid glucose excursions) and a treadmill test (TT). Significant correlation between the SWA and a glucose monitoring system were found; with a correlation coefficient of 0.65 (p < 0.05) during OGTT and 0.91 (p < 0.05) during TT.
New solutions for a painless and non-invasive self-monitoring blood glucose system is still in its infancy, and there’s still very much a gap in the market for this type of technology. The biggest concern about such non-invasive technology is clinical accuracy. Since there is no direct measurement of the endpoint, in this case plasma glucose concentration, there is still a need to find a cut-off point that defines the minimally acceptable accuracy of a reading compared with the existing clinically approved monitoring device.
Studies at the fourth stage of development, which trial the device as an intervention, are only just starting to appear in the literature:
Mention has to be made of smart phones and how they have contributed to the design of studies in recent years. Mobile phones are ubiquitous and economical, but how efficacious are they as a monitoring device? Seto et al carried out a randomized controlled trial where a mobile phone-based telemonitoring system was implemented as an intervention device to manage heart failure. The telemonitoring group took daily readings such as weight, blood pressure and single-lead ECGs. Readings were automatically transmitted wirelessly to the mobile phone and then to data servers. The patients were sent instructions via their phone and alerts were made to the cardiologist’s phone when necessary. The findings from the study reflect that a mobile phone-based telemonitoring system improved the quality of life of the patients through improved self-care and clinical management. It also proved to be highly feasible for patients, including the elderly and those with no experience with mobile phones.
What happens when the viability of the data collected by a Mhealth device is under investigation? Fitbit are currently facing a class action lawsuit regarding the accuracy of their heart rate data following the findings of researchers at the California State Polytechnic University who have showed a margin of inaccuracy of up to 20 beats per minute. Does this lawsuit spell the end of Mhealth technologies or the data they collect? They should not. In fact it should be an indicator of the need to have in place rigorous standards for data collection, such as those mandated by CDISC.
Right now there is a gap between the technology and experience, and a gap between modern consumer-focused apps, devices and sensors and traditional medical counterparts, in terms of user experience. This difference in usability and accuracy is shrinking. 
Learn more about how our statistical consultancy group could support your clinical trial by scheduling a call with one of our sales representatives.
References:
- Mancinelli, Chiara, et al. "A novel sensorized shoe system to classify gait severity in children with cerebral palsy." 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2012.
- Kwasnicki, Richard M., et al. "Assessing functional mobility after lower limb reconstruction: a psychometric evaluation of a sensor-based mobility score." Annals of surgery 261.4 (2015): 800-806.
- Sobel, Sandra I., et al. "Accuracy of a novel noninvasive multisensor technology to estimate glucose in diabetic subjects during dynamic conditions." Journal of diabetes science and technology 8.1 (2014): 54-63.
- Vyas, N., et al. "Development of a Personalized Non-invasive Glucose Monitoring System for Free-living Environments." Annual Meeting of Diabetes Technology. 2009.
