Optimising feedback for early career professionals: a scoping review and new framework

Meta‐analyses have shown that feedback can be a powerful intervention to increase learning and performance but there is significant variability in impact. New trials are adding little to the question of whether feedback interventions are effective, so the focus now is how to optimise the effect. Early career professionals (ECPs) in busy work environments are a particularly important target group. This literature review aimed to synthesise information to support the optimal design of feedback interventions for ECPs.

CONTEXT Meta-analyses have shown that feedback can be a powerful intervention to increase learning and performance but there is significant variability in impact. New trials are adding little to the question of whether feedback interventions are effective, so the focus now is how to optimise the effect. Early career professionals (ECPs) in busy work environments are a particularly important target group. This literature review aimed to synthesise information to support the optimal design of feedback interventions for ECPs.
METHODS We undertook a scoping literature review, using search terms such as 'feedback' and 'effectiveness' in MEDLINE, MEDLINE-In-Process, PsycINFO, CINAHL, Education Research Complete, Education Resources Information Center, the Cochrane Database of Systematic Reviews, the Social Sciences Citation Index and Applied Social Sciences Index and Abstracts, to identify empirical studies describing feedback interventions in busy workplaces published in English since 1990. We applied inclusion criteria to identify studies for the mapping stage and extracted key data to inform the next stage. We then selected a subset of papers for the framework development stage, which were subjected to a thematic synthesis by three authors, leading to a new feedback framework and a modified version of feedback intervention theory specifically for ECPs.

RESULTS
A total of 80 studies were included in the mapping stage, with roughly equal studies from hospital settings and school classrooms, and 17 papers were included in the framework development stage. The feedback framework comprised three main categories (audit, feedback and goal setting) and 22 subcategories. The review highlighted the limited empirical research focusing solely on feedback for ECPs, which was surprising given the particular nuances in feedback for ECPs identified through this study.
CONCLUSIONS We offer the feedback framework to optimise the design of future feedback interventions for early career professionals and encourage future feedback research to move away from generic models and tailor work to specific target audiences. Meta-analyses in education have shown that feedback can be a powerful intervention to increase learning and performance. 1 In primary and secondary education, feedback has been identified as one of the most impactful education interventions. 2 The importance of feedback has also been recognised in health care and health care education, with audit and feedback generally leading to small but potentially important improvements in professional practice. 3,4 Feedback in both education and health care, however, shows significant variability in impact, with some interventions having no impact or even negative impact. 4,5 In other words, although feedback can be powerful and effective in certain circumstances, the outcomes are inconsistent. This is perhaps not surprising, given the large number of behaviours and settings to which feedback interventions have been applied and the multiple components of feedback that may be altered.
Research exploring the different components of feedback that contribute to effectiveness also faces challenges. The latest Cochrane systematic review on audit and feedback, undertaken by the Effective Practice and Organisation of Care (EPOC) group, 3 synthesised best estimates of effect sizes according to different components of feedback interventions and concluded that feedback may be most effective: when health professionals are not performing well to start out with; when the person responsible for the audit and feedback is a supervisor or colleague; when feedback is provided more than once; when feedback is given verbally and in writing; and when it includes clear targets and an action plan. In 2014, Ivers et al. 4 extended this Cochrane review and a cumulative analysis showed that the effect size became stable in 2003, suggesting that new trials are adding little to the question of whether feedback interventions are effective. As they put it: At this point the appropriate question is not "can audit and feedback improve professional practice" but "how can the effect of audit and feedback interventions be optimised?" 4 They conclude that research is now needed to understand the impact of task characteristics, feedback characteristics, recipient characteristics and context on feedback effectiveness. 4 From a complex systems perspective, 6 these different characteristics are also likely to interact in unforeseen ways, so rich descriptions of the feedback process and its impact (or lack of impact) will be needed. Whereas, the Cochrane reviews have only included randomised controlled trials (RCTs), other types of study designs and theoretical perspectives are now required to explore the professional or organisational processes that may impact substantially on effectiveness and provide greater insights into mechanisms and unintended consequences.
Colquhoun et al. 7 argue that part of the problem is that interventions have typically been designed without underpinning theory from the behavioural and social sciences. This is problematic because theory provides important insights into how change strategies might work and when and why they might not work. Colquhoun et al. 7 analysed randomised controlled trials of audit and feedback and concluded that explicit use of theory in these studies was rare. Colquhoun et al. 8 developed a list of theory-informed hypotheses, based on interviews with theory experts, about how to design more effective audit and feedback interventions. This can inform practical guidance to support those designing feedback interventions. 9 Early career professionals (ECPs), who are transitioning into busy work environments, are a particularly important group of learners. It is increasingly clear that the experiences and needs of those at the beginning of their careers are different from those of experienced professionals; for example, because they may be unfamiliar with the workplace systems and professional norms and may be working within a strong professional hierarchy. 10 To date, there is little research that targets early career professionals specifically and we need to know much more about how the type of feedback received, intentionally or unintentionally, affects the learning or sense of professional identity of qualified professionals entering the workplace. 11 Because feedback is often more effective when baseline performance is low, 3 it is likely that the 'return on investment' from a feedback intervention in terms of impact on professional practice would be high as ECPs learn to undertake the tasks required by their new jobs. In addition, retention of ECPs is problematic in medicine, teaching and beyond, suggesting that additional opportunities to provide support and encouragement are needed. The transition from university to work is challenging in any field but may be particularly daunting in busy environments such as hospital wards and school classrooms, which are what Eraut 12 calls 'hot action' contexts where 'changing conditions feature prominently'. Therefore, the aim of this scoping review was to bring together information in an easily accessible way to support the optimal design, implementation and reporting of workplace-based feedback interventions for early career professionals. This complements previous research, such as that of van der Ridder et al., 13 whose work focuses on undergraduate education and assessments (rather than ECPs in busy workplace environments and naturally occurring workplace-based measures) and whose primary audience is researchers (rather than feedback intervention designers). Given the complexity of feedback interventions and their variable reported impact, our methodological approach sought to identify literature that provided in-depth accounts of feedback strategies that aimed to change the behaviour of ECPs, from research involving a wide range of study designs and from different professions. We then developed a feedback framework, which can be used to optimise the design of future interventions, and developed a theory of feedback that was broad enough to accommodate what we had learned about ECPs.

Aim and research questions
The aim of this literature review was to bring together information that can support the optimal design, implementation and reporting of workplacebased feedback interventions for ECPs (as defined in Table 1).
The research questions were:  15 We considered a number of educational, psychological and sociocultural theories to inform our study but ultimately selected feedback intervention theory from organisational and management research and used their definitions of feedback as our starting point. 5 Feedback intervention theory has five basic arguments: behaviour is regulated by comparisons of feedback to goals or standards; goals or standards are organised hierarchically; attention is limited and therefore only feedback standard gaps that receive attention actively contribute to behaviour change; attention is normally directed to a moderate level of the hierarchy; and feedback interventions change the locus of attention and therefore may affect behaviour.

Definitions
1 Feedback: for the purposes of this study, we see feedback as an intervention comprising those 'actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one's task performance'. 5 Our focus was feedback from workplace measures that related to specific element(s) of authentic practice and targeted learner behaviours relevant to professional practice and clinical workplace outcomes. Assessment feedback was out of the scope, unless it met these specific criteria. Feedback on simulated activities was also out of the scope. 2 Workplace settings: this study was interested only in feedback within authentic, busy workplaces settings comparable to a 'hot action' context, 12 such as hospital wards or school classrooms. 3 Early career professional: a graduate who is <2 years into professional practice.

Developing the search to identify studies
Search terms were identified through background searching in Google and relevant journals, and the specificity and sensitivity of free text (i.e. title and abstract) and indexing (e.g. Medical Subject Headings -MeSH) terms explored using Ovid MEDLINE. Combinations of search terms were benchmarked against prespecified target papers. Our final approach combined terms for 'feedback' and terms which described either the effectiveness of feedback or terms which are used in qualitative study designs such as 'qualitative', 'experience' and 'interview'. 16 The qualitative study design terminology recognised that many of the papers from our pilot searches that were providing the most detailed information in relation to the research questions were qualitative. However, we did not wish to exclude other study designs; hence these terms were combined using OR with terms which described the effectiveness of feedback which are not specifically related to a particular study type. Given the study's resource constraints, we used a date limit of 1990 onwards, selected because it encompassed many key professional education developments, and an English language filter.

Search strategy
The databases searched (Box 1) were selected to provide coverage of medicine, health care more broadly, education and other professions. The Ovid MEDLINE search strategy is presented in Figure 1.
The search results were supplemented by checking the reference lists of the studies included in the synthesis stage of the review.

Mapping stage
The aim of the mapping stage was to describe the inevitably broad and diverse literature, in order to make an informed decision about which studies would contribute substantially to the framework development stage. 17 The inclusion criteria for both stages are described in Table 1 1.
limit 13 to (english language and yr="1990 -Current")  Because the aim of the mapping stage was to characterise existing literature on a particular topic in order to make an informed decision about whether to undertake in-depth review and synthesis, we did not undertake quality assessment at this stage.

Framework development stage
A subset of the included papers was selected, based on specific inclusion criteria (see Table 1), to be subjected to a second phase of data analysis called framework development. All selection decisions were made by three authors (KM, CP and NB) and papers were not excluded based on study design. Our approach to drawing the literature together to create a framework was 'configuring' rather than 'aggregating'. 17 So rather than focusing on 'multiple observations of the same phenomena', the aim of this stage was to place 'study findings alongside one another in order to build up a picture of the whole, and how they relate to one another', which is more achievable for diverse literatures. To 'configure' our studies, we drew on the principles of thematic synthesis.

Thematic synthesis
We read each study line by line, made notes, charted key observations and revisited the text to extract the key ideas, concepts and messages. In doing so, we focused as much on authors' explanations for the observed findings as the findings themselves. We went through each paper  highlighting the sections that contribute to the research questions and making comments and codes about how they do that, which we then discussed and combined, ultimately leading to a feedback framework. The process was not entirely inductive as it was driven by the research questions and feedback intervention theory, and involved a wide range of study designs.

Quality assessment
Scoping reviews are not required to undertake quality appraisal. 14 Thus, although we did not exclude any papers on the basis of quality (because there is little empirical basis on which to do this), we did interpret and make sense of the different aspects that these papers presented throughout the analysis process, taking into account the paradigm they belonged to, how well each of their arguments were made on the basis of data and how the different papers corroborated each other. This was carried out predominantly by one author (KM), with input from three further authors (NB, CP and MP). We have made this process transparent in Appendix S1, which contains a column for 'strength of evidence'.

Papers identified
The total number of hits retrieved from each database search is detailed in Table 2 and a flow diagram showing which papers were included is given in Figure 2. Through searching the reference lists of the included studies, 57 additional references were identified as potentially interesting, of which 24 had already been identified by our database search. Of the remaining 33, 21 did not meet our inclusion criteria, four were duplicates and one (an earlier version of a Cochrane review) had been withdrawn, leaving seven papers (two empirical and five literature reviews) that were read in full and included in the mapping stage. A subset of 17 of the papers from the mapping stage were included in the framework development stage.

Mapping the literature
In total, 80 studies were included in the mapping stage. Of these, 60 were empirical studies (more details about the studies are presented in Appendix S2  17 with only two from schools, and one was a theoretical paper that was not context specific. The number of included studies varied from seven to 140. The literature reviews generally focused on quantifying the effectiveness of feedback rather than optimising the process.

Developing the feedback framework
We reviewed 17 papers in the framework development stage (11 empirical studies [six quantitative, four mixed and one qualitative] and six literature reviews). This subset was narrower in scope, with over half the papers focusing specifically on ECPs (9 out of 17 papers), most coming from a health care context (14 out of 17 papers) and all providing in-depth analysis of contextual features. The included papers did not contain an existing feedback taxonomy or framework that was broad enough to accommodate the literature found. We therefore created one through this research (Appendix S1) and used it to organise the findings. The feedback framework incorporated the findings of the included papers to conceptualise workplacebased feedback interventions as comprising three main categories (audit, feedback and goal setting) with 22 subcategories. The categories and subcategories are summarised in Appendix S1, with an indication of the strength of the evidence underpinning each subcategory, so that researchers can target their future work on aspects that have less robust evidence.
Overall, there appeared to be a lack of clarity and consistency in the use of terminology in the included studies, combined with a lack of specificity in discussing the different components of 'feedback'. The EPOC group, who undertook the most recent Cochrane review of audit and feedback say: In an audit and feedback process, an individual's professional practice or performance is measured and then compared to professional standards or targets. In other words, their professional performance is "audited". The results of this comparison are then fed back to the individual. The aim of this process is to encourage the individual to follow professional standards. 3 However, we feel this definition seems to conflate the process of feeding back the data collected through the audit and the process of reflecting on the data and setting goals and an action plan to bring about a change in performance, even though this latter aspect was deemed important in its own right within the papers we reviewed. We also  27 ) or have only referred to audit implicitly. 5 In the audit category, the six subcategories developed from analysis of our data were as follows: complexity of task chosen, type of task chosen, nature of data to be collected, metric importance to the intended recipient, data credibility, and baseline performance (Appendix S1). According to the literature reviewed, less complex tasks or behaviours are probably easier to change through feedback. Where feedback metrics were aligned to the priorities of the recipient or organisation, they seemed likely to have a greater impact, although feedback messages could be undermined by contextual cues or other feedback sources. Feedback seemed most effective when baseline performance was low, which is likely for ECPs, but if it was very low then rapid improvement might be needed to sustain engagement with the task. Importantly, for ECPs who often work under supervision, or where teamwork or shift work is common, performance data were sometimes deemed an unfair reflection of ECP's own clinical practice, which could undermine the feedback process. For example: Junior doctors explained that because they rarely made prescribing decisions independently, the feedback letters should be sent to all team members, including the senior doctors. 30 In the methods section, we defined feedback as 'actions taken by (an) external agent(s) to provide information regarding some aspect(s) of one's task performance' 5 ; in other words, the way in which audit data are made available to the feedback recipient. Information may be provided via a range of means and can be presented in a range of ways, for example, with or without comparisons to other data. In the category of feedback intervention, the seven subcategories were: feedback format, comparison to other data, judgement made on data, content of feedback, likelihood of feedback being perceived as a threat, correct solution information, and timing and frequency of feedback (Appendix S1). Written feedback seemed more effective than verbal or graphical delivery. Comparison to past performance (e.g. feedback on progress over time) tended to be effective, because it directed attention to the task, whereas public feedback, peer comparisons, praise and discouragement could divert attention to meta-task aspects involving the self (e.g. emotional responses or concerns about implications). Clay et al. propose debriefing cards as 'a tool for deliberate practice' that improves trainee performance by providing: the opportunity for frequent self-assessment, explicit expectations for performance, and feedback on each resident's self-assessment by a supervising physician. 33 Feedback could sometimes be perceived as a threat to self-esteem or to external rewards or punishments, so the benefits of 'authoritative' sources must be weighed up against the risks of this making the feedback seem more threatening. Providing a 'correct solution' as part of feedback was thought to increase its likely impact by focusing attention on target behaviours, but correct solutions may be scarce within professional practice, where complex judgements in a messy practice context are common. Feedback was thought to be most effective when presented more than once and when it occurred soon after the performance event.
Goal setting, the final category in our feedback framework, occurs when the feedback recipient considers behavioural change based on the information received. Providing information alone is often insufficient to change behaviour or outcomes. In this category, the nine subcategories were: presence or absence of goal setting, presence of a reviewer to support goal setting, relationship to reviewer, nature of goal, tailoring of goal-setting conversation, nature of conversation, recipient ownership of goal setting, acceptance of goals suggested, and successful completion of goals (Appendix S1). Knowing what to do with feedback seemed as important as receiving it. The included studies suggested that feedback was more effective when combined with reflection or goals and an action plan. Most included studies involved a reviewer in goal setting, who needed to be perceived as credible. A quality goal-setting conversation could help learners to identify the gap between current and desired performance and agree a strategy for change. Goals seemed unlikely to be accepted or prioritised if the immediate relevance to the practice setting was unclear. As Redwood et al. 27 note: . . . metrics used need to be concrete rather than abstract and must reflect actual work processes which may be different in different clinical contexts (e.g. working on a day or night shift, or in a surgical or medical speciality).
Goals also needed to be within the scope of responsibility or possibility of the ECP. It seemed important for recipients to discuss the performance context, because sometimes apparent 'poor performance' could be explained when placed in context.

Modified feedback intervention theory for ECPs
A key question underpinning our research was the extent to which ECPs were a specific group, with different needs and contextual influences to other professional groups. Therefore, Table 3 outlines some key features of feedback intervention theory, 5 which in its original form was offered as a universal model, and highlights where our research indicates that it might need to be extended or given extra weighting to accommodate the particular situation of feedback for ECPs in busy work environments.

DISCUSSION
The aim of this scoping review was to bring together information that can support the optimal design of workplace-based feedback interventions for ECPs. We mapped the literature that exists already and organised the most relevant literature into a feedback framework (Appendix S1), developed for this study, which we now offer as a tool to optimise the design of future feedback interventions. The detailed analysis allowed us to answer the research questions by identifying the specific features of feedback interventions for ECPs that seemed to underpin their effectiveness (or ineffectiveness) and by trying to explain why these features were important and how they might work (Appendix S1). We addressed the final research question by exploring the extent to which feedback intervention theory in its existing format could accommodate ECPs, which led us to identify some specific nuances associated with ECPs that we felt needed modification, or greater emphasis, in a 'FIT [feedback intervention theory] for ECPs' ( Table 5).
It is clear that feedback interventions have the potential for significant positive impact and that ECPs are an appropriate target group who may stand to gain the most benefit from feedback. However, our review suggests that few studies provided a convincing rationale that their feedback intervention design was optimal, and very few focused on ECPs alone, so we believe it is likely that the impact of current and future interventions can be significantly improved. Our analysis of findings against the existing feedback intervention theory suggests that it would need to be modified in order to take account of the key features influencing feedback for ECPs. Colquhoun et al. 8 suggested that a taxonomy of feedback interventions would improve the design, description and reporting of feedback interventions and we hope our feedback framework might contribute towards this aim.
Like previous researchers, 5,19-21 we note a need for more detailed reporting of feedback interventions, with rich descriptions of the different components and rationale for their combination, in order to share experiences, build theory or synthesise evidence across studies. Although these are challenges in many domains, it is particularly important for complex, multistage interventions such as feedback. In health care the terminology 'audit and feedback' seems to have become established in the literature but seems to underplay the important steps that occur after feedback is received and before behaviour change occurs, which are highlighted through our included papers and other literature. [22][23][24] The literature reviews included in our scoping review typically only synthesised quantitative studies 20 and often only RCTs. 3,4,19,25 A broader range of empirical research methodologies would provide greater insights into mechanisms and unintended consequences, and different types of literature review such as qualitative metasyntheses, realist reviews or other theory-based approaches to evidence syntheses. 26 A stronger theoretical basis is also needed. Colquhoun et al. 8 developed a list of testable theory-informed hypotheses about feedback, which sets out a useful research agenda and makes a first step in linking theory to practice, but evidence syntheses and frameworks are now required to inform the design of feedback interventions for specific recipient groups, because the primary data are challenging and time consuming to interpret. 19 Recommendations for policy and practice Based on the literature included in this study we recommend that: feedback interventions focus on an important aspect of performance; the data selected are credible and reflective of individual performance of the target audience, in our case ECPs; information should be communicated privately and mapped against appropriate external standards (relevant to stage of training) with clear information about the standard sought; supportive, developmental opportunities to discuss feedback and set goals are provided with an experienced professional who is familiar with the specific practice context and has good facilitation skills; and that the goals are within the ECP's scope of practice. However, it is hard to predict effectiveness of feedback interventions 27-30 because subtle changes of context or process can make a big difference 31 and subtle nuances can affect how the feedback message is received. 31 The feedback framework developed through this research is offered to support the design (or redesign) of feedback interventions for ECPs. Through Appendix S1, we have highlighted the strength of the evidence base underpinning each stage of the feedback framework, using a format applied successfully for communicating with policymakers and education professionals. 2 Appendix S1 also signposts the original publications underpinning each summary judgement, so that feedback intervention designers can engage with the evidence firsthand should they wish to. We hope that providing easy access to the theory and evidence underpinning each stage of the feedback framework will enable future interventions to be more likely to have a greater impact.

Recommendations for future research
The feedback framework also serves to highlight those aspects of feedback that have received much attention from researchers and those that have been neglected to date. By mapping which aspects of the framework are well populated with research, we hope to be able to ensure that research efforts are channelled to those areas most neglected currently and do not waste resources duplicating what is already well established in the literature. 4 Topics requiring future attention include the impact of the types and complexities of tasks chosen for a feedback intervention, perceptions of feedback recipients about the nature of the data chosen for feedback, the impact of praise or displeasure on the feedback recipient, and the process and implications of the goal-setting conversation (Appendix S1). For example, it would be interesting to understand how highly experienced mentors, or those who know the individual feedback recipient, implicitly tailor their feedback. But most importantly, we encourage future feedback research to move away from generic models and tailor work to the specific groups targeted by feedback interventions.
Like Hysong 2009, 19 we felt feedback intervention theory provided a useful theoretical framework to guide decisions in designing future feedback interventions. Despite offering a very interesting perspective on feedback, it was cited by very few of the studies included in our scoping review. We recommend that feedback intervention theory or modifications thereof, as well as other behavioural and social science theories, are given greater attention in future research.
Interestingly, in some of the studies included in the synthesis stage, the feedback intervention did not work as anticipated. 27,29 As with all complex interventions, we need to be alert to unintended consequences and see these as important opportunities for learning. Studies involving observation may provide new insights into workplace contexts in which unanticipated events have occurred. Other future studies might interview multiple stakeholders in feedback interventions that have worked particularly well or particularly badly, to try and identify common themes. The future is also likely to involve easier access to electronic performance data, which will provide new opportunities to evaluate feedback interventions, 32 although concerns have been raised about the panoptic gaze on clinical practice. 27

Strengths and limitations of the research
The strengths of this work are: the rigorous twostage scoping review process undertaken by a multidisciplinary team; the deliberately broad methodology to incorporate relevant literature beyond the health care setting; the focus on busy workplaces and naturally occurring measures; and the inclusion of a wide range of study designs. Our team included an information scientist, implementation scientist and social scientist as well as medical educators. Our review allowed a comparison of literature across two key 'hot action' environments, hospital wards and school classrooms, because similar numbers of empirical studies were included in the mapping stage. In general, the number of participants was smaller for school settings in the included papers and there were fewer literature reviews.
As with all research studies, there are also some limitations. The summary of evidence derived from a scoping review can only be as good as the literature it finds. Resource constraints meant we only looked at literature since 1990 and published in the English language. The fact that feedback intervention reports were mostly positive raises concerns about both publication bias (in that unsuccessful or negative impact feedback interventions may not be published 20 ) and selection bias (in that the research participants who engage with feedback may reflect those more motivated trainees 33 ). We believe studies that don't work as intended or participants who aren't keen to participate are well placed to contribute important new insights. Also, given the limitations of reporting and methodologies described, we are unlikely to know whether feedback interventions took place as intended 29 and not all the included literature focused solely on ECPs. More mixed-methods research, for example incorporating a qualitative process evaluation, would help with this. As McLellan et al. put it: the social world is a complex one and we would not therefore be able to explain how or why our intervention had an effect on the basis of numeric data alone. 28 Finally, our search does not claim to be exhaustive and the feedback framework developed is just one interpretation, but we hope it can start an interesting and important conversation about workplace-based feedback for ECPs.

CONCLUSIONS
The aim of this scoping review was to bring together information that can support the optimal design of workplace-based feedback interventions for ECPs. The feedback framework we developed comprised three main categories (audit, feedback and goal setting) and 22 subcategories. The evidence summary highlighted imprecise terminology, patchy research coverage across the feedback framework and limited research focusing on ECPs. Comparing our findings with the existing feedback intervention theory allowed the specific nuances associated with feedback for ECPs to be made explicit. We now offer the feedback framework and our tailoring of FIT for ECPs to help optimise the design of future feedback interventions and to help researchers identify priorities for study.