Analytics —
Training activity: 7

Details

Evaluate the performance of a medical device including sensitivity, specificity and diagnostic accuracy.

Type

Developmental training activity (DTA)

Evidence requirements

Evidence the activity has been undertaken by the trainee​.

Reflection on the activity at one or more time points after the event including learning from the activity and/or areas of the trainees practice for development.

An action plan to implement learning and/or to address skills or knowledge gaps identified.

Reflective practice guidance

The guidance below is provided to support reflection at different time points, providing you with questions to aid you to reflect for this training activity. They are provided for guidance and should not be considered as a mandatory checklist. Trainees should not be expected to provide answers to each of the guidance questions listed.

Before action

What are the intended outcomes of the training activity?

• How have you reviewed the specific learning outcomes to ensure you are prepared to explore and explain the interdependencies between the medical device, the operator, and the clinical environment?
• What essential knowledge have you identified as necessary regarding the principles of diagnostic accuracy metrics (e.g., gold standards vs. index tests) and the specific technical specifications of the device before you begin?
• How do you intend to demonstrate that your statistical selection and critique follow best practice guidelines for clinical and operational datasets to ensure a safe and high-quality service?

What do you anticipate you will learn from the experience?

• What specific insights do you hope to gain regarding how the sensitivity and specificity of a device directly dictate its clinical utility within a specific patient pathway?
• In what ways do you expect this activity to improve your ability to critique statistical interpretations and recognize the limitations of diagnostic data when applied to operational service delivery?
• Thinking about what you already know, which aspects of the diagnostic accuracy evaluation do you find most complex or interesting to explore further?

What actions will you take in preparation for the experience?

• What preparatory discussions have you planned with your Training Officer or clinical lead to gain clarity on the expected depth of the statistical analysis and the clinical relevance of the chosen dataset?
• Which standard operating procedures (SOPs) or national technical standards have you selected to review to ensure your evaluation of the medical device is robust and evidence-based?
• How have you identified your feelings about embarking on this activity, considering the professional responsibility of validating technologies that influence patient management decisions?

In action

What are you doing?

• How are you currently approaching the calculation of performance metrics, and what is the technical rationale for your chosen statistical methodology?
• What real-time decisions are you making regarding the classification of findings (e.g., true positives vs. false positives) as the dataset unfolds?
• Which aspects of the evaluation feel intuitive to you, and which—such as explaining system relationships or determining diagnostic thresholds—require more conscious, focused effort?

How are you progressing with the activity?

• How effective are your current analytical actions in moving towards a definitive interpretation of the device’s accuracy and reliability?
• What technical or clinical challenges are you facing in this moment—such as ambiguous data or a lack of a clear gold standard—and what are you learning from these as they occur?
• How are you applying your specialist knowledge during the analysis to ensure the investigation remains safe, accurate, and of high quality?

How are you adapting to the situation?

• Are there alternative analytical models or data filtering strategies you should consider if the initial results provide an ambiguous or unexpected performance profile?
• What immediate support or guidance do you need from a senior colleague right now to resolve an interpretative ambiguity before finalising your conclusions?
• How are you ensuring that your real-time adjustments and reporting style remain strictly within your professional scope of practice while addressing the clinical question?

On action

What did you notice?

• How would you summarise the key technical findings of the evaluation, specifically detailing the calculated sensitivity, specificity, and diagnostic accuracy of the medical device?
• What specific clinical or technical details—such as data outliers or system interdependencies—did you notice were most significant in influencing your final recommendations?
• Which ‘reflect-in-action’ moments did you notice where you had to adapt, such as re-verifying a dataset because of a discrepancy with the clinical referral?

What did you learn from the activity?

• What specific skills or knowledge did you develop or improve regarding the ability to critique statistical analyses performed on clinical or operational datasets?
• In what ways did your real-time decisions—such as adjusting a diagnostic threshold for better specificity—influence the quality and accuracy of the final clinical report?
• How does the experience of building this integrated performance narrative relate to the requirements for your future post-programme practice as a Clinical Scientist?

What will you take from the experience moving forward?

• What specific areas for continued development in medical device management or advanced statistical interpretation have you identified as a result of this activity?
• How can you apply the learning from this performance evaluation (e.g., more rigorous assessment of diagnostic accuracy) to your routine practice within the department?
• What are the specific ‘next steps’ you will now take, such as reviewing national equipment guidelines or leading a process improvement plan, to support the assimilation of what you have learned?

Beyond action

Have you revisited the experiences?

• How have you re-evaluated your initial design decisions in light of subsequent learning regarding the nuances of device performance and system interdependencies?
• Comparing this evaluation with your Observed Training Activities (OTAs), what specific observable behaviours in technical precision and professional communication have you now fully assimilated?
• In what ways have discussions with peers regarding your statistical strategies changed your view on how to navigate complex or ambiguous technical datasets?

How have these experiences impacted upon your current practice?

• How has this medical device activity supported your development in wider areas, such as writing interpretative technical reports or presenting findings to the multidisciplinary team (MDT)?
• In what ways has the cumulative learning from this evaluation supported your preparation for ‘in-person’ assessments, such as Case-Based Discussions where you must justify your technical choices?
• Can you identify specific instances where your understanding of diagnostic accuracy and statistical caveats has influenced your current prioritisation of technical results or your approach to quality control?

How might these experiences contribute towards your future practice?

• What transferable skills in system exploration and clinical reasoning have you developed that will help you evaluate and adopt new medical technologies in the future?
• How will your mastery of synthesising technical findings for clinical use help you contribute to future service improvements or innovations as you progress toward becoming a Clinical Scientist?
• What clear actions for continued professional development have you identified to ensure you stay current with the evolving landscape of medical device standards and diagnostic algorithms?

Relevant learning outcomes

#	Outcome
# 1	Outcome Explore and explain relationships between elements in a system.
# 2	Outcome Select, perform and critique statistical analyses and interpretations on clinical and operational datasets.