Using consumer sleep technologies to characterize sleep in medical research; options and limitations.

Sleep is fundamental for health. Sleep wearables may have a key role to better characterize and understand sleep. Within the research and clinical sleep communities, there is a growing recognition of the potential benefits of using Consumer Sleep Technologies (CST). Collection of continuous data, day and night, could also lead to better understanding of links between sleep and daytime behaviors such as physical activity and exercise. The number of available devices is continually growing, and a multitude of them claim to track and define sleep-related metrics [1]. The multisensory (accelerometer in combination with hearth rate, PPG, temperature, …) CSTs open a path for greater accuracy in measuring sleep, as compared to the first single sensor CST. Unfortunately, minimal validation data exist regarding the ability of CSTs to accurately perform sleep characterization. The CST-data is not standardized, and raw data and proprietary algorithms are typically unavailable. In addition, wearable companies can change their algorithms without notice, an important aspect to consider when using a device over a certain period, and particularly for longitudinal studies. Although the number of validation studies is growing, validation clearly moves at a slower pace than the wearable industry. Validation studies typically demonstrate sensitivity around 90%, but wider ranges for specificity (20% – 80%).
Further work is needed to investigate the potential use and performance, pros and cons, and limitations of these novel CSTs, particularly in sleep disorder populations [2-4]. Currently, these devices should be used cautiously, and interpretation of their outcomes should be carefully considered to avoid large inaccurate data sets leading to potential misleading scientific conclusions, assessment of sleep disturbances, and therapeutic decisions.

Polysomnography (PSG) as a gold standard

Polysomnography (PSG) is the gold standard method for assessing sleep and is the main reference for device validation. Following the standard recommendation sleep is manually scores in 30-s intervals by visual identification of the specific phasic and tonic features of multiple EEG and physiological channels to assign each epoch as either: wake, N1, N2, N3, or REM sleep. PSG is usually confined to sleep laboratory research and clinical settings because it requires specialized equipment and expertise for recording, scoring and interpreting PSG data [2].

Actigraphy based sleep assessment

The accepted alternative to PSG for nonlaboratory settings is actigraphy. Actigraphy devices rely on an accelerometer to measure patterns of activity and estimate sleep/wake states accepting the simple assumption that motion implies wake, and no-motion implies sleep. The device’s accelerometer detects the occurrence and degree of motion in multiple directions, which is converted into a digital signal to derive an activity count. Then, depending on the sleep-wake threshold of the algorithm, an epoch is determined as wake if its activity count exceeds the threshold, or sleep if it is below the threshold. Although the majority of studies report high sensitivity (ability to detect true sleep) and accuracy (overall ability to detect true wake and sleep), actigraphy is inherently impaired in detecting true wake (specificity) as it is unable to identify motionless wake. For studies that have included healthy participants specificity ranged from 26.9% to 77%, whereas other that have included a variety of patient groups report specificity values ranging between 32.5% and 80%. Although many studies report specificity less than 50%, this finding is often minimized or overlooked and actigraphy is accepted as providing an accurate estimate of PSG. There is a trade-off between sensitivity and specificity. Whether researchers should aim for high overall accuracy and sensitivity and acknowledge that sleep is overestimated, or whether they should instead aim to more accurately detect wake at the cost of sleep is still an open question, and is probably best decided based on the objectives of the investigation. Unfortunately, there is still no consensus on specific recommendations for different patient groups, devices and algorithm thresholds for actigraphy [2, 4, 5].

Sleep assessment with the MOX accelerometry platform

At Maastricht University Medical Center+ an automated sleep-wake detection algorithm for actigraphy is available [6]. Based on the raw accelerometer signals and a set of predetermined rules and thresholds bed times and wake times are estimated. From these two times the sleep time (time in bed) can be determined. The algorithm achieved high levels of agreement in determining wake times (median absolute difference of 12 minutes) as well as bed times (median absolute difference of 25 minutes) when compared to self – reported diary times. These results indicate that the algorithm can be used as an accurate method to automatically identify the waking period and separate this from the sleeping period in 24-hour accelerometry data, in middle-aged and older adults when compared to a self-reported method. Consequently, when using the algorithm the amounts of sedentary time and physical activity accumulated during waking time can be estimated correctly, which is of importance for examining associations between sedentary behaviour or physical activity and health outcomes.

+: Outcome measure is feasible for the given method. If available sensitivity is shown between brackets
-: Outcome measure is not feasible for the given method.
N/A: outcome measure is not available for the given method

Conclusion

Over the past years, wearable sleep trackers are being increasingly adopted by the general public, researchers and clinicians. Multisensory sleep trackers open a path for greater accuracy in measuring sleep, as compared to the motion-based approach to sleep/wake assessment. However, the proven theoretical advantage of the multisensory approach to sleep staging needs further empirical validation. Further work is needed to investigate the potential use and performance, pros and cons, and limitations of these novel sleep trackers, particularly in sleep disorder populations.
Actigraphy is limited in terms of sleep related outcome parameters. However, research grade devices have the benefit of providing raw data. This enables researches to stay in control of the post-processing of the data throughout the study. In addition, actigraphy provides reliable insights in the physical activity pattern compared to consumer grade wearables.
It is clear that many different aspects play a role in the selection of an appropriate device. Depending on the research question, study logistics, data safety and privacy regulations one might be more suited than the other might. Besides data quality, usability and compliance are other essential aspects for every research to consider.

References

1. Khosla, S., et al., Consumer Sleep Technology: An American Academy of Sleep Medicine Position Statement. Journal of Clinical Sleep Medicine, 2018. 14(05): p. 877-880.
2. DE ZAMBOTTI, M., et al., Wearable Sleep Technology in Clinical and Research Settings. Medicine & Science in Sports & Exercise, 2019. 51(7): p. 1538-1557.
3. de Zambotti, M., et al., Sensors Capabilities, Performance, and Use of Consumer Sleep Technology. Sleep Medicine Clinics, 2020. 15(1): p. 1-30.
4. Goldstein, C., Current and Future Roles of Consumer Sleep Technologies in Sleep Medicine. Sleep Medicine Clinics, 2020. 15(3): p. 391-408.
5. Scott, H., L. Lack, and N. Lovato, A systematic review of the accuracy of sleep wearable devices for estimating sleep onset. Sleep Medicine Reviews, 2020. 49: p. 101227.
6. van der Berg, J.D., et al., Identifying waking time in 24-h accelerometry data in adults using an automated algorithm. Journal of Sports Sciences, 2016. 34(19): p. 1867-1873.