Data and Security —
Training activity: 11

Details

Extract, prepare and process clinical data for further analysis and reporting

Type

Entrustable training activity (ETA)

Evidence requirements

Evidence the activity has been undertaken by the trainee repeatedly, consistently, and effectively over time, in a range of situations. This may include occasions where the trainee has not successfully achieved the outcome of the activity themselves. For example, because it was not appropriate to undertake the task in the circumstances or the trainees recognised their own limitations and sought help or advice to ensure the activity reached an appropriate conclusion. ​

Reflection at multiple timepoints on the trainee learning journey for this activity.

Considerations

• Automated Extract, Transform, Load (ETL) techniques
• Database designed for reporting purposes
• Data lifecycle, including frequency and nature of data updates
• Transforming data
• Data aggregation and reduction
• Statistical techniques
• Manual data analysis
• Governance arrangements, for example: using clinical data for secondary purposes such as management reporting, clinical audit, specialist and quality assurance
• Integration of database reporting, business intelligence or analytics tools

Reflective practice guidance

The guidance below is provided to support reflection at different time points, providing you with questions to aid you to reflect for this training activity. They are provided for guidance and should not be considered as a mandatory checklist. Trainees should not be expected to provide answers to each of the guidance questions listed.

Before action

• Identify what is expected of you in relation to extracting, preparing, and processing clinical data for analysis and reporting.
• Discuss with your training officer to gain clarity of what is expected of you regarding data source, format, required cleaning/processing steps, and the intended format for the prepared data.
• Think about what you already know about data extraction methods (e.g., using SQL1, APIs, file exports), data cleaning techniques, data transformation, and working with clinical datasets. This might relate to academic content on data mining or handling large datasets.
• Consider possible challenges you might face during the activity, such as dealing with missing data, inconsistent formats, large data volumes, complex data structures, or identifying and handling outliers. Think about how you might handle them.
• Recognise the scope of your own practice for this activity. i.e., know when you will need to seek advice or help (e.g., on specific data definitions, appropriate cleaning methods for a particular issue, or using specific data processing tools) and from whom (e.g., data manager, statistician, training officer).
• Consider the specific skills you want to develop, drawing upon previous experiences with data handling. This could include improving efficiency in data cleaning, mastering specific data manipulation libraries or tools, or better understanding the complexities of real-world clinical data.
• Identify the specific insights you hope to gain from engaging with the activity. For example, understanding the typical issues found in clinical datasets, the practical application of data cleaning principles, or how data preparation impacts subsequent analysis and reporting.
• Consult actions identified following previous experience of the activity. Are there specific areas you needed to improve on previously, such as identifying data quality issues or choosing appropriate processing steps?
• Identify important information you need to consider before embarking on the activity. This could include the specific dataset to be used, the required outcome format for the prepared data, any data governance requirements, and the planned downstream analysis or reporting that the data is being prepared for.

In action

• While extracting, preparing, or processing the clinical data, does anything feel surprising or different from what you anticipate happen? For example, is the data format different from expected, do you encounter a higher-than-expected number of missing values or inconsistencies, do errors occur during data cleaning or transformation steps, or does the processing take significantly longer than anticipated?
• Consider how this experience compares with previous experiences you have had handling clinical data or performing similar data extraction, cleaning, or processing tasks.
• Identify how this impacts upon your actions. For example, do you immediately try a different data cleaning method? Do you pause the process to investigate the nature of the unexpected data issues? Do you need to seek advice from a data manager or your supervisor in the moment?
• Do you adapt or change your approach to the extraction, preparation, or processing steps based on the unexpected event?
• Does it affect your ability to undertake the activity independently at that point in time?
• Consider how you feel in that moment. For example, do you find it difficult to adapt to the data quality issues? Does it affect your confidence in your data handling and processing skills? Do you feel positive that you can identify and resolve the data problems?
• As you react in the moment, do you ensure you work within your scope of practice? For example, are you authorised to perform the specific data manipulations or access certain data fields?
• Identify what you learn as a result of the unexpected development. For instance, do you learn a new technique for handling a specific type of data inconsistency, a characteristic of this particular clinical dataset, or a troubleshooting method for data processing errors?

On action

• Begin by summarising the key points of extracting, preparing, and processing the clinical data. Describe the dataset, the steps taken, and the tools or methods used.
• Consider specific events, actions, or interactions which felt important during the activity, including your own feelings.
• Include any ‘reflect-in-action’ moments, where you adapted to the situation as it unfolded. For example, did you have to change your data cleaning script partway through, or rethink your approach to handling missing values based on what you found?
• Identify what learning you can take from the experience. What strengths did you demonstrate in handling the clinical data? What skills and/or knowledge gaps were evident regarding data extraction methods, data quality issues, cleaning techniques, or processing steps?
• Compare this experience against previous engagement with similar activities. Were any previously identified actions for development in data handling or processing achieved? Has your practice in preparing data for analysis improved?
• Identify any challenges you experienced (e.g., unexpected data format, significant data inconsistencies, technical issues with processing tools) and how you reacted to these. Did this affect your ability to deal with the situation? Were you able to overcome the challenges?
• Identify anything significant about the activity. Did you need to seek advice or clarification from a data manager or supervisor regarding data issues or processing methods? Or did you need to escalate anything to ensure that you were working within your scope of practice when handling sensitive clinical data?
• Acknowledge any changes in your own feelings now that you are looking back on the experience. For instance, do you feel more confident in your ability to prepare and process clinical data for reporting after completing this TA?
• Identify the actions / ‘next steps’ you will now take to support the assimilation of what you have learnt. For example, what will you do differently next time you extract or prepare a clinical dataset? Has anything changed in terms of what you would do if you were faced with similar data quality issues? Do you need to practise any aspect of data cleaning, transformation, or processing further?
• Include insights from any feedback you have received on the prepared data or the subsequent analysis/report.

Beyond action

• Have you reviewed your reflections from previous times you undertook this training activity, or similar activities involving clinical data extraction, cleaning, or processing? Have you successfully actioned previous points for improvement related to data quality, cleaning techniques, or processing efficiency?
• Consider this experience in relation to other training activities involving datasets, data integrity, de-identification, or AI data preparation. How does this instance of data handling compare to others you’ve encountered?
• Have you discussed the challenges or methods used in this data handling activity with peers or colleagues? How has their insight influenced your understanding or future approach to data preparation?
• How has the learning from extracting, preparing, and processing clinical data in this instance influenced your approach when dealing with other datasets or analysis tasks?
• Think about how this experience contributes to your preparation for observed assessments that might involve data integrity checks, de-identification, or explaining data flows.
• How has your practice in clinical data handling developed and evolved over time, considering this and previous experiences? Are you more adept at identifying potential data quality issues early or confident in knowing when you need to seek expert advice on data handling within your scope of practice?
• Have you applied the specific techniques or knowledge gained from this training activity in other work since completing it?
• What transferable skills have you developed through this activity, such as data literacy, attention to detail, methodical processing, or understanding data governance requirements?
• How does ability in extracting, preparing, and processing clinical data contribute to your overall role, particularly in supporting analysis, reporting, or AI development?
• Based on revisiting this experience and reflecting on your current practice, what clear actions for continued development in clinical data handling or related areas like data cleaning tools or methods can you identify?

Relevant learning outcomes

#	Outcome
# 9	Outcome Prepare reports against information systems through the development and execution of complex and multi-staged SQL queries.