Authors: Christopher Tack / Submitted: 06 July 2020.
This article is undergoing peer review.
Background: Healthcare digital transformation should focus on the use of innovative technologies to enhance quality, safety, efficiency of care services; and patient experience. Subsequently, the roles and skills of healthcare staff will change, requiring evaluation and elevation of digital literacy across the physiotherapy profession.
Aim: To evaluate the confidence, motivation and competence of digital technologies in a cohort of UK physiotherapists (compared to a wider group of allied health professionals).
Methods: On-line questionnaire of physiotherapists and other allied health professionals (AHPs) in the UK.
Results: 282 responses from AHPs were received, with 279 complete responses for further analysis (including 126 physiotherapists). Physiotherapists report moderate-high levels of confidence in the use of digital devices (7.6 ±1.77); and high levels of motivation in learning about digital technology (8.7 ±1.6). Physiotherapists self-rate their knowledge regarding the benefits of digital transformation as high (72%). Physiotherapists show a strong preference for daily communication via telephone (82%) and email (97%).
Conclusion: Confidence and motivation in digital technology does not fully transfer into high self-rating of competence in many areas of digital application. Educational development should be directed towards cultivating general knowledge and practical skills into more specialist areas.
- Professional development should be focused primarily on the theoretical benefits, risks and ethics associated with emergent technology (e.g. AI).
- Undergraduate and postgraduate learning on statistics should include concepts of data type, quality and evaluation, as well as data use in practice (e.g. health informatics).
In 2016, Professor Robert Wachter, made recommendations highlighting the need for a thoughtful, long term strategy for NHS digital transformation; focusing on using technology as a means to enhance quality, safety, efficiency and patient experience (Wachter, 2016). In 2019, The Topol Review “Preparing the healthcare workforce illustrated how innovative technologies may alter the roles of clinical staff in the next two decades (Topol, 2019). It mapped the skills required to ensure safe, effective and personalised care for patients; and identified gaps which could be filled by alternate professions, or via training and development of current and future staff. In support of the vision for digitally-enabled care in the NHS Long Term Plan (NHS England, 2019a), the review recommends raising digital literacy amongst the current workforce, and the development of knowledge, skills and behaviours to facilitate staff confidence and competency. Organisations are advised to invest in building digital skills within their workforce, and recommends all staff are trained in health data management, critical appraisal and ethics of artificial intelligence (AI) and automated technologies.
The need to upskill healthcare staff has been documented (NHS England, 2019; Topol, 2019; Wachter, 2016) however the mechanism by which education can influence digital implementation remains unclear (Cornford, Klecun & Lichntner, 2014; Hilberts & Gray, 2014). Workforce development is essential to ensure benefits of technology materialise on implementation (Greenhalgh et al., 2017). Health Education England’s Technology Enhanced Learning Programme, and The Building a Digital Ready Workforce programme of the National Information Board, developed the Health and Care Digital Capabilities Framework (Health Education England, 2018). They define digital literacy as “The capabilities that fit someone for living, learning, working, participating and thriving in a digital society.” The framework divides digital literacy into domains of capability by which individuals can be assessed; including communication, data and technical proficiency.
In the Timmus project, Newman, Church and Beetham (2019) investigated the use of digital diagnostic tools to evaluate the digital capabilities of NHS staff, and found that measurement of digital “willingness” (confidence and motivation) is largely successful. However, evaluation tools fail to reliably measure digital “experience” (environment, specific competence within professional roles). They suggest that measurement of competency should focus on the local context of environment and professional role to have meaning (Newman et al., 2019]. Work developing profession-specific competency guidance has begun in post graduate medical education; indicating that 42% of health informatics knowledge domains are not present in UK-based postgraduate curricula (Jidkov et al. 2019). In 2019, NHS England published the Allied Heal th Professional (AHP) Digital Framework; providing guidance for AHPs to develop digital competence at individual, department and organisational levels (NHS England, 2019b). It supports the planning and delivery of training in digital capabilities in order to evolve services into digitally mature and data-enabled entities. The facilitation of digital readiness, competence and literacy in all AHPs is key to this strategy. With a dearth of guidance as to how digital education should be guided for AHPs (and specifically physiotherapists), a two part project was commenced.
The first stage (described here) was a UK-wide survey of digital confidence, motivation and experience within AHPs; using the same question framework utilised by the Timmus project (Newman et al., 2019). Additionally, this survey asked respondents to self-rate their perceived competence relative to a preliminary competency framework, devised from the HEE digital capabilities (Health Education England, 2019) and the NHS England AHP Digital Frameworks (NHS England, 2019b). The results for physiotherapists will be compared to the wider AHP cohort to guide future workforce education and development. The second stage (not described here) is a Delphi study to elaborate on and ratify the preliminary competency framework used in this survey.
The questionnaire was constructed in 4 parts:
- Respondent characteristics
- Self-rating of confidence, motivation and experience (3 questions reproduced with permission from Timmus (Newman et al., 2019).
- How confident are you at using digital devices and software at work?
- How motivated are you to learn how to use new digital devices and software at work?
- How much experience do you have at using a variety of digital devices and systems at work?
- Self-rating of perceived competence in digital competencies (93 competencies over 10 domains, 5 point Likert scale from very poor to very good)
- Description of current digital capability (11 questions, 1 question rating the degree of support provided by host organisation, numerical rating scale 0-10; 10 questions recording frequency of digital activity categorically: daily- never).
Practicing AHPs (including physiotherapists) from the UK, were invited to complete an on-line survey. The survey was disseminated via social media, and a blog post describing the objectives of the project. The NHS England-facilitated AHP Digital Network, and professional groups (e.g. the CSP Digital and Informatics Physiotherapy Group, the Scotland Digital Nursing, Midwifery and Allied Health Professional network, and the Northern Ireland Digital AHP group) were used as a further means of dissemination. This author’s host organisation was a final source of recruitment.
Survey responses were collected from 10th July 2019 to 30th November 2019 (143 days). All responses were anonymous.
Descriptive analysis was undertaken for all respondent characteristics, self-rating of competence, and recording of frequency of activity data. Response percentages in the Likert scale categories of the physiotherapy cohort are compared with the results from AHPs. For the mean average self-rating scores for digital confidence, motivation, experience and organisational support; standard deviations are calculated and a Student’s T-test has been used to calculate degree of statistical significance between the cohorts.
Ethical consideration Ethical review was undertaken internally as part of the governance and quality assurance process within the author’s host institution. Anonymization of responses was used to protect the identities of respondents.
282 responses from AHPs were received; with 279 complete responses suitable for data analysis.
Figure 1a-e illustrates the profile characteristics of the respondents. 45% of respondents (n=126) were physiotherapists, 55% (n=153) were other AHPs. Figures 1b-e illustrate characteristics of the physiotherapists regarding experience (Agenda for Change Band), work setting and location. 44% of respondents were band 8-9, with 50% of respondents between band 5-7. 30% of respondents worked in an inpatient hospital setting, 23% in outpatient clinics, and 14% within the community. 51% of respondents worked in London, England. 9% were situated in Scotland, and 3% from Northern Ireland.
Self-rating of confidence, motivation and experience
Table 1 summarises the self-rating of confidence, motivation and experience of digital technologies. Physiotherapists show moderate-high levels of confidence in the use of digital devices (7.6 ±1.77); and high levels of motivation in learning how to use technology (8.7 ±1.6). No statistical significant difference is found with the AHP cohort. AHPs show a slightly greater rating of experience in using a digital technology at work.
|Mean (SD)||Mean (SD)||p=|
|Confidence||7.6 (±1.77)||7.79 (±1.76)||0.38|
|Motivation||8.7 (±1.6)||8.75 (±1.45)||0.81|
|Experience||6.96 (±1.94)||7.3 (±1.75)||0.13|
Self-rating of competence
Table 2 illustrates rating of perceived competence in various digital competencies. The results are grouped into positive (good to very good knowledge or ability); neutral (fair); and negative ratings (poor to very poor). The results presented cover the most pertinent competency areas relative to the Topol Review Recommendations (Topol, 2019) and the AHP Digital Framework (NHS England, 2019b).
|Good – Very Good||Good-Very Good||Fair||Fair||Poor-Very Poor||Poor-Very Poor|
|Awareness of benefits of digital transformation for own profession and wider NHS||91||110||25||28||10||15|
|Ability to undertake a self-evaluation of digital literacy||79||90||35||47||12||16|
|Understanding of the data protection act and risks associated with data privacy||88||104||32||40||6||9|
|Ability to evaluate data type/ quality towards effective searching and analytics||37||43||46||56||43||54|
|Ability to capture structured & unstructured patient data in the electronic health records (EHR)||54||71||33||50||39||32|
|Understanding of clinical coding terminologies (e.g. SNOMED CT) within the EHR||25||28||29||47||72||78|
|Knowledge of digital tools which support the transfer of patient information at point of referral, admission, handover or discharge.||35||45||57||54||34||54|
|Ability to use electronic systems for efficient medicines management||13||23||20||25||93||105|
|Understanding of digital tools which support visibility, requesting and resulting of testing (laboratory/ pathology and medical imagining)||33||43||45||30||48||80|
|Knowledge and understanding of machine learning and artificial intelligence innovations within digital diagnostic systems||19||25||22||24||85||104|
|Knowledge and understanding of local organisational performance measurement systems||32||40||33||37||61||76|
|Knowledge and understanding of digital tools to advise practice through evidence based guidelines to direct a patient pathway.||29||42||35||40||62||71|
|Knowledge and understanding of machine learning and AI algorithms underpinning clinical decision support system tools within healthcare digital systems||14||19||21||37||91||97|
|Knowledge and understanding of the development and/ or evaluation of clinical decision support systems which utilise machine learning and AI.||13||18||22||36||91||99|
|Ability to identify non-clinically assured/ inaccurate on-line health and care information.||46||62||34||43||46||48|
|Ability to recommend or prescribe a mobile health application||34||41||41||42||51||70|
|Ability to view and/ or capture patient data at the point of care (via hand held device, wearable technology or connected medical device)||24||31||27||35||75||87|
|Ability to develop virtual clinics for direct patient care (using digital media as an alternative to face to face consultation)||25||29||23||42||78||82|
|Knowledge and understanding of the benefits of virtual clinics using secure platforms to provide consultations||38||52||33||47||55||54|
|Capacity to use digital technologies as required within quality improvement programmes with own local organisation||51||56||32||47||43||50|
|Capacity to identify the needs and requirements of the healthcare institution to guide a strategic programme of digital transformation||35||37||38||41||53||75|
|Capacity to direct the research agenda for own department towards topics of digital therapeutics, mobile health and digital transformation||34||41||35||35||57||77|
Figures 2a-b show the self-rating for competencies rated to knowledge and understanding of digital. Both groups show high positive ratings of knowledge associated with the benefits of healthcare digital transformation (72% physiotherapy and AHP), and data protection/ privacy regulation (70% physiotherapy; 68% AHP). Knowledge of clinical coding in electronic health records (e.g. SNOMED CT) is rated positively by only 18% of physiotherapists. In line with recommendations advising upskilling of workers on AI (Topol, 2019); physiotherapists reported only a 10-15% positive rating in this area. Across three competencies associated to machine learning (ML)/AI, 68-72% of physiotherapists reported poor to very poor understanding. Two thirds of knowledge-based competencies showed ratings of poor to very poor understanding in greater than 30% of physiotherapists; demonstrating broad lack of knowledge in variable areas of digital capability. Similar findings are seen in the AHPs.
Figures 2c-d show competencies associated with the ability to perform digital tasks. The highest positive ratings by physiotherapists (good to very good ability to perform task), are the ability to perform a self-evaluation of their digital literacy (63%); and the ability to capture data in electronic health records (43%). These results are similar to the AHP cohort. The lowest rated abilities by physiotherapists were associated with using an electronic medicines management system (10%); data collection via wearable technology (19%); and the development of virtual patient clinics (20%). Whilst medicines management may fall outside of the scope of practice for many physiotherapists; the collection of data via linked, interoperable hardware, and the use of virtual clinics to enhance accessibility are both strongly featured in the NHS Long Term Plan (NHS England, 2019a) and the Topol Review (Topol, 2019). 10/11 skill based competencies showed greater than 30% of the physiotherapy cohort describe their ability as poor to very poor (including data analytics; mHealth app prescription; and virtual care). Similar practical deficiencies are seen in the AHP cohort.
Description of current digital capability
Respondents rated how supportive their organisation is in the workforce development of digital skills. Physiotherapists had an mean average rating of 6.05/10 (±2.33), with AHPs rating 6.5/10 (±2.3) (p=0.11). Thus, indicating professionals consider their organisations to be supportive of digital training to a moderate-high level. Digital capability is estimated by frequency of activities being undertaken. Figure 3a shows that 72% of physiotherapists analyse patient data longitudinally on a monthly basis or greater (66% AHP).
Between group similarities also exist for the use of voice recognition to dictate clinical records, with both groups stating this is never undertaken (75% physiotherapy, 74% AHP). Differences between groups are seen in the application of medical devices daily (19% physiotherapy vs 7% AHP). 65% of AHPs saying they never undertake this activity (40% physiotherapy).
Figure 3d shows the physiotherapy results for the selected skills showing that 40% report they never undertake these activities.
Figures 4a-g compare communication preferences of physiotherapists versus AHPs. Figure 4h shows only physiotherapist preferences. Physiotherapists have a substantial preference to use institutionally-secure (NHS/ Trust) email for daily communication (97%). Telephone is also used daily (82%). AHPs show similar findings with 91% daily frequency for email, and 82% for telephone. 82.53% of physiotherapists report never using fax machines (67% AHPs). Video calling platforms are used by 45% of physiotherapists on a monthly basis or greater (43% AHPs), however 50% of physiotherapists never use this mode of communication. Other methods are used at variable rates, including 73% using WhatsApp (69% AHPs). Only 18% use Slack (13% AHPs) and 16% use Microsoft Teams (20% AHPs).
To the best of this author’s knowledge, this is the first study evaluating the confidence and motivations of physiotherapists, and juxtapose to their perceptions of competence, regarding digital knowledge and skills. Further, the frequency of undertaking specific digital tasks is described.
Confidence, motivation and experience
Physiotherapists describe moderate-high levels of confidence and motivation regarding the use of digital technology; with results similar to “therapists” measured in the Timmus study (Newman et al., 2019). Whilst, Newman et al. (2019) found that confidence and motivation are important in describing digital capability; measuring experience was inhibited by the diversity of roles both within and between professions. They suggest that confidence and motivation (“digital willingness”) are as important as experience when considering engagement with digital technology.
Two competency areas are deemed vital components of digital competency by the guiding literature both in UK and abroad (Baker, Charlebois, Lopatka, Moineau & Zelmer, 2016; Bilimoria et al. 2019; Jidkov et al. 2019; NHS England, 2019; Topol, 2019): machine learning/ AI, and health informatics.
Machine learning/ Artificial Intelligence
AI is a term used to describe the ability of a computer to perform tasks, which if performed by a human, one would consider intelligent. Machine learning (ML) is a subfield of AI where computers are trained how to learn without being explicitly programmed (Samuel, 1967). Physiotherapists in this study report their knowledge regarding AI/ML as poor to very poor across two areas: AI/ML used in diagnostic systems and AI/ML for decision support. These results were similar to AHPs. Ooi et al. (2019) evaluated the attitudes of radiologists regarding AI/ML. Their survey (n=125) found that 64.8% of respondents viewed themselves as novices regarding AI/ML, with 76% planning to include the topic within future personal development. This shows that the perception of competence in a professional group with greater exposure to AI/ML is similar to physiotherapists. The development of training standards in radiology for AI/ML is in the nascent stages. In both the USA and UK, standards require training radiologists to understand basics of imaging informatics (e.g. data privacy, post-processing imaging) (American Board of Radiology (2019). However, syllabi do not explicitly include AI/ML within the curriculum.
Kolachalama and Garg (2018) suggest that concerning AI/ML, curricula should aim for literacy, rather than proficiency; focusing on developing conceptual knowledge to assist clinical practice. Physiotherapy education could follow, by embedding technological knowledge relevant to patient care into workforce development strategies. Undergraduate training should include basic principles of AI/ML, illustrated via simulated cases; alongside the theoretical principles of benefits, risks and ethics. Thus supporting experiential learning in practice. Proficiency with advanced programming skills should be reserved for postgraduate training (Kolachalama & Garg, 2018).
Informatics/ Data Analytics
Health informatics (i.e. clinical/ medical informatics) is an interdisciplinary field concerning the use of biomedical data to improve individual and wider-population health (Kulikowski et al. 2012; Wyatt & Liu, 2002). It involves analysing data to guide evidence based practice (Otero, Hersh & Ganesh, 2014). Systems are being developed which capture, analyse and apply data from various sources (e.g. genomic, public health, electronic health record (EHR) data) at a rate which is unsustainable for the healthcare workforce (Raghupathi & Raghupathi, 2014). However, there is little discussion how practitioners will be supported in the use of this data; and how education may prevent the NHS workforce from “drowning in data” (Oteroet al., 2019).
Physiotherapists report inconsistent knowledge and abilities in areas concerning data management. Lower ratings are reported for perceived knowledge of clinical coding (57% poor to very poor understanding), and in evaluating data type/ quality to assist analytics (35% poor to very poor). Whilst 43% report good to very good ability to capture structured/ unstructured data in the EHR; only 19% have good to very good ability to capture data digitally at the point of care with wireless/ wearable devices (19%). This means data would be manually added into the medical record and may be recorded incorrectly (e.g. structured data stored as unstructured clinical notes). 72% of physiotherapists analyse patient data longitudinally, however 48% perform this only on a monthly basis or less. These results indicate a willingness to undertake informatics activities, but variability exists in competency across the profession.
Clinicians express discomfort in understanding statistics (Krouss, Croft & Morgan, 2016); often seen as a barrier to translating research into practice by physiotherapists (Janssen, Hale, Mirfin-Veitch & Harland, 2016). Educational programmes should include concepts of data analytics beyond simple statistics (Otero et al. 2014). Informatics is a broader domain concerned with understanding types of data; evaluating its quality for specific tasks; and manipulation of data to leverage patient benefit (Dhar, 2013). It is likely that breadth and depth of training may be variable across the workforce; with greater depth for those demonstrating sufficient aptitude (Dhar, 2013).
Capability is estimated based upon frequency by which digital activities are undertaken. Physiotherapists report low levels of voice recognition software use (75% never using), which is chosen as an affordable, mature digital tool used in practice (Bergeron, 2001). This low engagement is contradictory to the confidence and motivation described by the physiotherapy cohort; and may suggest that barriers, such as lack of EHR system interoperability, are inhibiting the use of such tools. This could explain the infrequent use of digital medical devices in practice (40% never used); and would indicate that structure and functionality of EHR systems should be examined to ensure it does not constrain practical application of technology. Physiotherapists should engage with EHR system development, to ensure it meets the needs of practice.
Current communication preferences
The NHS has been aiming to become “paperless” (Macaulay, 2016), with Health Secretary Matt Hancock aiming to “axe the fax” (Department of Health and Social Care, 2019). This data would suggest progress towards this goal with 82.5% of physiotherapists never using a fax machine (67% AHPs). This survey showed communication preferences were biased towards telephone and institutional email. Other digital communication platforms never used by the physiotherapy cohort included: Slack (82%), on-line video conferencing (55%); and Microsoft Teams (84%). However, in response to the worldwide COVID-19 pandemic in 2020, it may be that these preferences change significantly. For example, Microsoft Teams, a secure instant messaging and audio/video calls platform, was made available for free for NHS workers for a limited time during the outbreak (NHS Digital, 2020). Furthermore, NHSX provided guidance elaborating on information governance and use of digital platforms to share data during the unique circumstances with the pandemic (NHSX, 2020). Freedom to use digital tools (e.g. WhatsApp) was expanded for clinical communication where “benefits outweigh the risks” (Digital Health London, 2020). Both internal organisational communication, and patient-therapist interaction were largely transitioned to digital to optimise social distancing. Use of video conferencing platforms (e.g. Zoom or Attend Anywhere) were rapidly and widely adopted. Consequentially, the Chartered Society of Physiotherapy published a guide for the rapid implementation of remote consultations (Chartered Society of Physiotheapy, 2020). How enforced transition to digital care, alters the behaviours of clinicians beyond COVID-19 remains to be seen. It is likely that evolution towards enhanced digital literacy and, in particular, remote consultation and digital communication, remains a legacy of the pandemic. However, there is likely to be variability in which tools are used.
These results are limited by the failure to subgroup respondents by age or gender. Increasing age has been associated with reduced digital literacy and lesser likelihood of using digital tools (Antonio and Tuffley, 2015). Analysing the impact of age may have assisted the development of guidance relative to the varied needs of differing age groups. Similarly, capability may present differently between genders. Whilst actual digital skills may not differ between men and women; the self-perception of women’s capabilities may be lower than men’s (Martínez-Cantos, 2017; van Deursen & van Dijk, 2015). This may subsequently impact their inclusion with digital technologies.
The on-line recruitment process used may have led to sampling bias in respondents who already have greater digital literacy; perhaps skewing results towards higher ratings of knowledge and competence. This was counteracted by sharing the survey internally to the author’s home organisation, however this may also have led to similar bias due to professionals sharing the same environment.
Another limitation is that the framework used for self-assessment has not yet been ratified by independent AHP groups. Whilst, published frameworks of digital literacy were used to construct the competencies, it remains appropriate to have the framework validated for appropriateness to each profession. Further, the digital tools which respondents were questioned about is not an exclusive list of tools where individuals can demonstrate competency. Rather, the most commonly used tools have been used to present an example of current digital skill levels. As such, this data will be biased to these author’s choices.
Physiotherapists demonstrate moderate-high levels of confidence and motivation to use digital technologies, with variability within and between experience levels and professional roles. The results give a snapshot of a spectrum of digital literacy (knowledge and skills), which despite not being exclusive, does provide a measure of the current state of digital capability across the profession. Higher ratings are shown for knowledge associated with theoretical underpinnings of digital transformation and associated policies; and lower ratings for digital skills in practice, and more advanced topics such as AI/ data analytics.
- American Board of Radiology (2019). Noninterpretive Skills Study Guide.
- Antonio, A., & Tuffley, D. (2015). Bridging the age-based digital divide. International Journal of Digital Literacy and Digital Competence (IJDLDC), 6(3), 1-15.
- Baker, C., Charlebois, M., Lopatka, H., Moineau, G., & Zelmer, J. (2016). Influencing change: Preparing the next generation of clinicians to practice in the digital age. Healthcare Quarterly, 18(4), 5-7.
- Bergeron, B. P. (2001). Voice recognition in clinical medicine: process versus technology. The Journal of medical practice management: MPM, 16(4), 213-215.
- Bilimoria K, Harish V, McCoy L, Mehta N, Morgado F, Nagaraj S, et al. (2019). Training for the future: Preparing medical students for the impact of artificial intelligence. Ontario Medical Students Association.
- Chartered Society of Physiotherapy. (2020). COVID-19: guide for rapid implementation of remote consultations.
- Cornford, T., Klecun, E., & LICHTNER, V. (2014). Digital Health Literacy: Beyond IT Skills. Conference on Medical Informatics in Europe. Istanbul, Turkey.
- Department of Health and Social Care. (2019). Health and Social Care Secretary bans fax machines in NHS. London, UK.
- Dhar, V. (2013). Data science and prediction. Communications of the ACM, 56(12), 64-73.
- Digital Health London (2020). Clinicians told they can use WhatsApp to share data in face of Covid-19.
- Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., Hinder, S., … & Shaw, S. (2017). Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. Journal of medical Internet research, 19(11), e367.
- Health Education England. (2018). A Health and Care Digital Capabilities Framework.
- Hilberts, S., & Gray, K. (2014). Education as ehealth infrastructure: considerations in advancing a national agenda for ehealth. Advances in Health Sciences Education, 19(1), 115-127.
- Janssen, J., Hale, L., Mirfin-Veitch, B., & Harland, T. (2016). Perceptions of physiotherapists towards research: a mixed methods study. Physiotherapy, 102(2), 210-216.
- Jidkov, L., Alexander, M., Bark, P., Williams, J. G., Kay, J., Taylor, P., … & Banerjee, A. (2019). Health informatics competencies in postgraduate medical education and training in the UK: a mixed methods study. BMJ open, 9(3), e025460.
- Kolachalama, V. B., & Garg, P. S. (2018). Machine learning and medical education. NPJ digital medicine, 1(1), 1-3.
- Krouss, M., Croft, L., & Morgan, D. J. (2016). Physician understanding and ability to communicate harms and benefits of common medical treatments. JAMA internal medicine, 176(10), 1565-1567.
- Kulikowski, C. A., Shortliffe, E. H., Currie, L. M., Elkin, P. L., Hunter, L. E., Johnson, T. R., … & Smith, J. W. (2012). AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. Journal of the American Medical Informatics Association, 19(6), 931-938.
- Macaulay, T. (2016). Progress towards a paperless NHS. BMJ, 355, i4448.
- Martínez-Cantos, J. L. (2017). Digital skills gaps: A pending subject for gender digital inclusion in the European Union. European Journal of Communication, 32(5), 419-438.
- Newman T, Church S, Beetham H. (2019). An investigation of digital diagnostic tools to introduce NHS staff to digital capabilities. Timmus Ltd.
- NHS Digital. (2020) Messaging tool for NHS to support remote working during coronavirus outbreak.
- NHS England (2019a). The NHS Long Term Plan. London, UK.
- NHS England. (2019b). A digital framework for allied health professionals. London, UK.
- NHSX. (2020). Covid-19 Information Governance advice for health and care professionals. London, UK.
- Ooi, S. K. G., Makmur, A., Soon, A. Y. Q., Fook-Chong, S., Sia, S. Y., Ting, Y. H., & Lim, C. Y. (2019). Attitudes toward artificial intelligence in radiology with learner needs assessment within radiology residency programmes: a national multi-programme survey. Singapore Med J, 1, 22.
- Otero, P., Hersh, W., & Ganesh, A. J. (2014). Big Data: Are Biomedical and Health Informatics Training Programs Ready?. Yearbook of medical informatics, 23(01), 177-181.
- Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health information science and systems, 2(1), 3.
- Samuel, A.L. (1967). Some studies in machine learning using the game of checkers. II—Recent progress. IBM Journal of research and development. 11(6), 601-17.
- Topol, E. (2020) Preparing the healthcare workforce to deliver the digital future: the Topol Review: An independent report on behalf of the secretary of state for health and social care.
- van Deursen, A. J., & van Dijk, J. A. (2015). Internet skill levels increase, but gaps widen: A longitudinal cross-sectional analysis (2010–2013) among the Dutch population. Information, Communication & Society, 18(7), 782-797.
- Wachter, R. (2016). Making IT work: harnessing the power of health information technology to improve care in England. London, UK: Department of Health.
- Wyatt, J. C., & Liu, J. L. Y. (2002). Basic concepts in medical informatics. Journal of Epidemiology & Community Health, 56(11), 808-812.