Much research has gone into the benefits of multi-source feedback and a 360° perspective on a healthcare professional’s practice, the development of MSF360 (formerly MCC 360 and before that, the Physician Achievement Review, or PAR). Below are some research outcomes, case studies and publications that have impacted this program and improved the value of the opportunities it provides for healthcare professionals.
| Title | Published | Year | Link | Citation | Origin |
|---|---|---|---|---|---|
| Co-worker assessment and physician multisource feedback | ARC Journal of Nursing and Healthcare | 2016 | View | Trueman, Gregg & Lockyer, Jocelyn. (2016). Co-Worker Assessment and Physician Multisource Feedback. ARC Journal of Nursing and Healthcare. 2. 1-8. 10.20431/2455-4324.0201001. | Canada |
| Clinical teaching as part of continuing professional development: Does teaching enhance clinical performance? | Med Teach | 2016 | View | Lockyer JM, Hodgson CS, Lee T, Faremo S, Fisher B, Dafoe W, Yiu V, Violato C. Clinical teaching as part of continuing professional development: Does teaching enhance clinical performance? Med Teach. 2016 Aug;38(8):815-22. doi: 10.3109/0142159X.2015.1112895. Epub 2015 Nov 30. PMID: 26618220. | Canada |
| An analysis of long terms outcomes for surgeons from three and four year curriculum | Can J of Surgery | 2012 | View | Lockyer J, Violato C, Wright B, Fidler H, Chan R. Long-term outcomes for surgeons from 3- and 4-year medical school curricula. Can J Surg. 2012 Aug;55(4):S163-70. doi: 10.1503/cjs.37610. PMID: 22854154; PMCID: PMC3432252. | Canada |
| The assessment of pathologists/laboratory-medicine physicians through a multi source feedback tool | Archives of Pathology and Laboratory Medicine | 2009 | View | Lockyer JM, Violato C, Fidler H, Alakija P. The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool. Arch Pathol Lab Med. 2009 Aug;133(8):1301-8. doi: 10.5858/133.8.1301. PMID: 19653730. | Canada |
| The assessment of psychiatrists in practice through multi source feedback | Canadian Journal of Psychiatry | 2008 | View | Violato C, Lockyer JM, Fidler H. Assessment of psychiatrists in practice through multisource feedback. Can J Psychiatry. 2008 Aug;53(8):525-33. doi: 10.1177/070674370805300807. PMID: 18801214. | Canada |
| What multi source feedback factors influence physician self assessments? A five year longitudinal study | Acad Med 2007 | 2007 | View | Lockyer JM, Violato C, Fidler HM. What multisource feedback factors influence physician self-assessments? A five-year longitudinal study. Acad Med. 2007 Oct;82(10 Suppl):S77-80. doi: 10.1097/ACM.0b013e3181403b5e. PMID: 17895697. | Canada |
| The assessment of emergency physicians by a regulatory authority | Academic Emergency Medicine | 2006 | View | Lockyer JM, Violato C, Fidler H. The assessment of emergency physicians by a regulatory authority. Acad Emerg Med. 2006 Dec;13(12):1296-303. doi: 10.1197/j.aem.2006.07.030. Epub 2006 Nov 10. PMID: 17099191. | Canada |
| A study of a multi source feedback system for international medical graduates holding defined licenses | Medical Education, | 2006 | View | Lockyer J, Blackmore D, Fidler H, Crutcher R, Salte B, Shaw K, Ward B, Wolfish N. A study of a multi-source feedback system for international medical graduates holding defined licences. Med Educ. 2006 Apr;40(4):340-7. doi: 10.1111/j.1365-2929.2006.02410.x. PMID: 16573670. | Canada |
| Self and peer assessment of pediatricians, psychiatrists and medicine specialists: Implications for self-directed learning | Advances in Health Science Education | 2006 | View | Violato C, Lockyer J. Self and peer assessment of pediatricians, psychiatrists and medicine specialists: implications for self-directed learning. Adv Health Sci Educ Theory Pract. 2006 Aug;11(3):235-44. doi: 10.1007/s10459-005-5639-0. PMID: 16832707. | Canada |
| The assessment of pediatricians by a regulatory authority | Pediatrics | 2006 | View | Violato C, Lockyer JM, Fidler H. Assessment of pediatricians by a regulatory authority. Pediatrics. 2006 Mar;117(3):796-802. doi: 10.1542/peds.2005-1403. PMID: 16510660. | Canada |
| A multisource feedback program for anesthesiologists | Can J of Anesthesia | 2006 | View | Lockyer JM, Violato C, Fidler H. A multi source feedback program for anesthesiologists. Can J Anaesth. 2006 Jan;53(1):33-9. doi: 10.1007/BF03021525. PMID: 16371607. | Canada |
| An examination of the appropriateness of using a common peer assessment instrument to assess physician skills across specialties | Academic Medicine | 2004 | View | Lockyer JM, Violato C. An examination of the appropriateness of using a common peer assessment instrument to assess physician skills across specialties. Acad Med. 2004 Oct;79(10 Suppl):S5-8. doi: 10.1097/00001888-200410001-00002. PMID: 15383375. | Canada |
| Multi source feedback in the assessment of physician competencies | Journal of Continuing Education in the Health Professions | 2003 | View | Lockyer J. Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof. 2003 Winter;23(1):4-12. doi: 10.1002/chp.1340230103. PMID: 12739254. | Canada |
| Likelihood of change: A study assessing surgeon use of multi source feedback data | Teaching and Learning in Medicine | 2003 | View | Lockyer J, Violato C, Fidler H. Likelihood of change: a study assessing surgeon use of multisource feedback data. Teach Learn Med. 2003 Summer;15(3):168-74. doi: 10.1207/S15328015TLM1503_04. PMID: 12855387. | Canada |
| Multi source feedback: A method of assessing surgical practice | British Medical Journal | 2003 | View | Violato C, Lockyer J, Fidler H. Multisource feedback: a method of assessing surgical practice. BMJ. 2003 Mar 8;326(7388):546-8. doi: 10.1136/bmj.326.7388.546. PMID: 12623920; PMCID: PMC1125427. | Canada |
| Changing physicians’ practices: The effect of individual feedback | Acad Med | 1999 | View | Fidler H, Lockyer JM, Toews J, Violato C. Changing physicians’ practices: the effect of individual feedback. Acad Med. 1999 Jun;74(6):702-14. doi: 10.1097/00001888-199906000-00019. PMID: 10386101. | Canada |
| Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians | Academic Medicine | 1997 | View | Violato C, Marini A, Toews J, Lockyer J, Fidler H. Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians. Acad Med. 1997 Oct;72(10 Suppl 1):S82-4. doi: 10.1097/00001888-199710001-00028. PMID: 9347748. | Canada |
| The Ottawa Resident Observation Form for Nurses (O-RON): Assessment of Resident Performance through the Eyes of the Nurses | J Surg Educ | 2021 | View | Dudek N, Duffy MC, Wood TJ, Gofton W. The Ottawa Resident Observation Form for Nurses (O-RON): Assessment of Resident Performance through the Eyes of the Nurses. J Surg Educ. 2021 Sep-Oct;78(5):1666-1675. doi: 10.1016/j.jsurg.2021.03.014. Epub 2021 Jun 3. PMID: 34092533. | Canada |
| Identifying coaching skills to improve feedback use in postgraduate medical education | Med Educ | 2019 | View | Armson H, Lockyer JM, Zetkulic M, Könings KD, Sargeant J. Identifying coaching skills to improve feedback use in postgraduate medical education. Med Educ. 2019 May;53(5):477-493. doi: 10.1111/medu.13818. Epub 2019 Feb 18. PMID: 30779210. | Canada |
| Facilitated Reflective Performance Feedback: Developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R2C2) | Acad Med | 2015 | View | Sargeant J, Lockyer J, Mann K, Holmboe E, Silver I, Armson H, Driessen E, MacLeod T, Yen W, Ross K, Power M. Facilitated Reflective Performance Feedback: Developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R2C2). Acad Med. 2015 Dec;90(12):1698-706. doi: 10.1097/ACM.0000000000000809. PMID: 26200584. | Canada |
| Sharing Reflections on Multisource Feedback in a Peer Group Setting: Stimulating Physicians’ Professional Performance and Development | Acad Med | 2021 | View | Bindels E, van den Goor M, Scherpbier A, Lombarts K, Heeneman S. Sharing Reflections on Multisource Feedback in a Peer Group Setting: Stimulating Physicians’ Professional Performance and Development. Acad Med. 2021 Oct 1;96(10):1449-1456. doi: 10.1097/ACM.0000000000004142. PMID: 33951680. | International |
| Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study | BMJ Open | 2021 | View | Holm EA, Al-Bayati SJL, Barfod TS, Lembeck MA, Pedersen H, Ramberg E, Klemmensen ÅK, Sorensen JL. Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study. BMJ Open. 2021 Jul 28;11(7):e047019. doi: 10.1136/bmjopen-2020-047019. PMID: 34321296; PMCID: PMC8319975. | International |
| Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles | J Contin Educ Health Prof | 2018 | View | Lockyer JM, Sargeant J, Richards SH, Campbell JL, Rivera LA. Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles. J Contin Educ Health Prof. 2018 Winter;38(1):32-40. doi: 10.1097/CEH.0000000000000183. PMID: 29329147. | Canada |
| A multi-method analysis of free-text comments from the UK General Medical Council Colleague Questionnaires | Med Educ | 2009 | View | Richards SH, Campbell JL, Walshaw E, Dickens A, Greco M. A multi-method analysis of free-text comments from the UK General Medical Council Colleague Questionnaires. Med Educ. 2009 Aug;43(8):757-66. doi: 10.1111/j.1365-2923.2009.03416.x. PMID: 19659489. | International |
| How do physicians assess their family physician colleagues’ performance? Creating a rubric to inform assessment and feedback | J Cont Educ Health Prof | 2011 | View | Sargeant J, Macleod T, Sinclair D, Power M. How do physicians assess their family physician colleagues’ performance?: creating a rubric to inform assessment and feedback. J Contin Educ Health Prof. 2011 Spring;31(2):87-94. doi: 10.1002/chp.20111. PMID: 21671274. | Canada |
| Ratings of performance in multisource feedback: comparing performance theories of residents and nurses | BMC Med Educ | 2020 | View | Tariq M, Govaerts M, Afzal A, Ali SA, Zehra T. Ratings of performance in multisource feedback: comparing performance theories of residents and nurses. BMC Med Educ. 2020 Oct 12;20(1):355. doi: 10.1186/s12909-020-02276-1. PMID: 33046055; PMCID: PMC7549199. | International |
| A qualitative exploration: questioning multisource feedback in residency education | BMC Med Educ | 2018 | View | Yama BA, Hodgins M, Boydell K, Schwartz SB. A qualitative exploration: questioning multisource feedback in residency education. BMC Med Educ. 2018 Jul 24;18(1):170. doi: 10.1186/s12909-018-1270-7. PMID: 30041635; PMCID: PMC6057061. | Canada |
| Perceptions of Emergency Medicine Residents of Multisource Feedback: Different, Relevant, and Useful Information | Ann Emerg Med | 2019 | View | Castonguay V, Lavoie P, Karazivan P, Morris J, Gagnon R. Perceptions of Emergency Medicine Residents of Multisource Feedback: Different, Relevant, and Useful Information. Ann Emerg Med. 2019 Nov;74(5):660-669. doi: 10.1016/j.annemergmed.2019.05.019. Epub 2019 Jul 4. PMID: 31280923. | Canada |
| Collecting validity evidence for an assessment of professionalism: findings from think-aloud interviews | Acad Med | 2008 | View | Mazor KM, Canavan C, Farrell M, Margolis MJ, Clauser BE. Collecting validity evidence for an assessment of professionalism: findings from think-aloud interviews. Acad Med. 2008 Oct;83(10 Suppl):S9-12. doi: 10.1097/ACM.0b013e318183e329. PMID: 18820511. | International |
| Can 360-Degree Reviews Help Surgeons? Evaluation of Multisource Feedback for Surgeons in a Multi-Institutional Quality Improvement Project | J Am Coll Surg | 2015 | View | Nurudeen SM, Kwakye G, Berry WR, Chaikof EL, Lillemoe KD, Millham F, Rubin M, Schwaitzberg S, Shamberger RC, Zinner MJ, Sato L, Lipsitz S, Gawande AA, Haynes AB. Can 360-Degree Reviews Help Surgeons? Evaluation of Multisource Feedback for Surgeons in a Multi-Institutional Quality Improvement Project. J Am Coll Surg. 2015 Oct;221(4):837-44. doi: 10.1016/j.jamcollsurg.2015.06.017. Epub 2015 Jul 23. PMID: 26272014. | International |
| Improving the utility of multisource feedback for medical consultants in a tertiary hospital: a study of the psychometric properties of a survey tool | Aust Health Rev | 2019 | View | Corbett H, Pearson K, Karimi L, Lim WK. Improving the utility of multisource feedback for medical consultants in a tertiary hospital: a study of the psychometric properties of a survey tool. Aust Health Rev. 2019 Jan;43(6):717-723. doi: 10.1071/AH17219. PMID: 30463660. | International |
| Changes in performance: a 5-year longitudinal study of participants in a multi-source feedback programme | Med Educ | 2008 | View | Violato C, Lockyer JM, Fidler H. Changes in performance: a 5-year longitudinal study of participants in a multi-source feedback programme. Med Educ. 2008 Oct;42(10):1007-13. doi: 10.1111/j.1365-2923.2008.03127.x. PMID: 18823520. | Canada |
| A GP fellowship project looking at whether the multi-source feedback and the educational supervisor’s report can be used to identify trainees at risk of difficulty | Educ Prim Care | 2022 | View | Leung C, McGee J, Scallan S. A GP fellowship project looking at whether the multi-source feedback and the educational supervisor’s report can be used to identify trainees at risk of difficulty. Educ Prim Care. 2022 May;33(3):185-187. doi: 10.1080/14739879.2022.2045517. Epub 2022 Apr 20. PMID: 35443891. | International |
| An analysis of long-term outcomes of the impact of curriculum: a comparison of the three- and four-year medical school curricula | Acad Med | 2009 | View | Lockyer JM, Violato C, Wright BJ, Fidler HM. An analysis of long-term outcomes of the impact of curriculum: a comparison of the three- and four-year medical school curricula. Acad Med. 2009 Oct;84(10):1342-7. doi: 10.1097/ACM.0b013e3181b6c08e. PMID: 19881418. | Canada |
| Leadership development in a professional medical society using 360-degree survey feedback to assess emotional intelligence | Surg Endosc | 2017 | View | Gregory PJ, Robbins B, Schwaitzberg SD, Harmon L. Leadership development in a professional medical society using 360-degree survey feedback to assess emotional intelligence. Surg Endosc. 2017 Sep;31(9):3565-3573. doi: 10.1007/s00464-016-5386-8. Epub 2016 Dec 30. PMID: 28039648. | International |
| Can colleagues’, patients’ and supervisors’ assessments predict successful completion of postgraduate medical training? | Med Educ | 2017 | View | Naidoo S, Lopes S, Patterson F, Mead HM, MacLeod S. Can colleagues’, patients’ and supervisors’ assessments predict successful completion of postgraduate medical training? Med Educ. 2017 Apr;51(4):423-431. doi: 10.1111/medu.13128. Epub 2016 Nov 23. PMID: 27882593. | International |
| Self-other agreement in multisource feedback: the influence of doctor and rater group characteristics | J Cont Educ Health Prof | 2013 | View | Roberts MJ, Campbell JL, Richards SH, Wright C. Self-other agreement in multisource feedback: the influence of doctor and rater group characteristics. J Contin Educ Health Prof. 2013 Winter;33(1):14-23. doi: 10.1002/chp.21162. PMID: 23512556. | International |
| Multisource feedback as part of the Medical Board of Australia’s Professional Performance Framework: outcomes from a preliminary study | BMC Med Educ | 2018 | View | Narayanan A, Farmer EA, Greco MJ. Multisource feedback as part of the Medical Board of Australia’s Professional Performance Framework: outcomes from a preliminary study. BMC Med Educ. 2018 Dec 29;18(1):323. doi: 10.1186/s12909-018-1432-7. Erratum in: BMC Med Educ. 2019 Mar 4;19(1):73. doi: 10.1186/s12909-019-1490-5. PMID: 30594157; PMCID: PMC6310994. | International |
| Swedish adaptation of the General Medical Council’s multisource feedback questionnaires: a qualitative study | Int J Med Educ | 2018 | View | Olsson JE, Ekblad S, Bertilson BC, Toth-Pal E. Swedish adaptation of the General Medical Council’s multisource feedback questionnaires: a qualitative study. Int J Med Educ. 2018 Jun 15;9:161-169. doi: 10.5116/ijme.5af6.c209. PMID: 29908128; PMCID: PMC6129165. | International |
| Assessing anaesthesiology and intensive care specialty physicians: An Italian language multisource feedback system | PLoS One | 2021 | View | Carenzo L, Cena T, Carfagna F, Rondi V, Ingrassia PL, Cecconi M, Violato C, Della Corte F, Vaschetto R. Assessing anaesthesiology and intensive care specialty physicians: An Italian language multisource feedback system. PLoS One. 2021 Apr 23;16(4):e0250404. doi: 10.1371/journal.pone.0250404. PMID: 33891626; PMCID: PMC8064525. | International |
| A novel workplace-based assessment for competency-based decisions and learner feedback | Med Teach | 2018 | View | Hicks PJ, Margolis MJ, Carraccio CL, Clauser BE, Donnelly K, Fromme HB, Gifford KA, Poynter SE, Schumacher DJ, Schwartz A; PMAC Module 1 Study Group. A novel workplace-based assessment for competency-based decisions and learner feedback. Med Teach. 2018 Nov;40(11):1143-1150. doi: 10.1080/0142159X.2018.1461204. Epub 2018 Apr 24. PMID: 29688108. | International |
| The intersection of professionalism and interprofessional care: development and initial testing of the interprofessional professionalism assessment (IPA) | J Interprof Care | 2019 | View | Frost JS, Hammer DP, Nunez LM, Adams JL, Chesluk B, Grus C, Harvison N, McGuinn K, Mortensen L, Nishimoto JH, Palatta A, Richmond M, Ross EJ, Tegzes J, Ruffin AL, Bentley JP. The intersection of professionalism and interprofessional care: development and initial testing of the interprofessional professionalism assessment (IPA). J Interprof Care. 2019 Jan-Feb;33(1):102-115. doi: 10.1080/13561820.2018.1515733. Epub 2018 Sep 24. PMID: 30247940. | International |
| A german-language competency-based multisource feedback instrument for residents: development and validity evidence | BMC Med Educ | 2020 | View | Hennel EK, Subotic U, Berendonk C, Stricker D, Harendza S, Huwendiek S. A german-language competency-based multisource feedback instrument for residents: development and validity evidence. BMC Med Educ. 2020 Oct 12;20(1):357. doi: 10.1186/s12909-020-02259-2. PMID: 33046060; PMCID: PMC7552497. | International |
| Validating the Interpretations and Uses of Test Scores | Journal of Educational Measurement | 2013 | View | Kane, M.T. (2013), Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement, 50: 1-73. https://doi.org/10.1111/jedm.12000 | International |
| 2018 Consensus framework for good assessment | Med Teach | 2018 | View | Norcini J, Anderson MB, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Palacios Mackay MF, Roberts T, Swanson D. 2018 Consensus framework for good assessment. Med Teach. 2018 Nov;40(11):1102-1109. doi: 10.1080/0142159X.2018.1500016. Epub 2018 Oct 9. PMID: 30299187. | International |
| Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning | Am Psychol | 1994 | View | Messick, S. (1994), VALIDITY OF PSYCHOLOGICAL ASSESSMENT: VALIDATION OF INFERENCES FROM PERSONS’ RESPONSES AND PERFORMANCES AS SCIENTIFIC INQUIRY INTO SCORE MEANING. ETS Research Report Series, 1994: i-28. https://doi.org/10.1002/j.2333-8504.1994.tb01618.x | International |
| Multisource feedback: can it meet criteria for good assessment? | J Cont Educ Health Prof | 2013 | View | Lockyer J. Multisource feedback: can it meet criteria for good assessment? J Contin Educ Health Prof. 2013 Spring;33(2):89-98. doi: 10.1002/chp.21171. PMID: 23775909. | Canada |
| The construct and criterion validity of the multi-source feedback process to assess physician performance: a meta-analysis | Adv Med Educ Pract | 2014 | View | Al Ansari A, Donnon T, Al Khalifa K, Darwish A, Violato C. The construct and criterion validity of the multi-source feedback process to assess physician performance: a meta-analysis. Adv Med Educ Pract. 2014 Feb 27;5:39-51. doi: 10.2147/AMEP.S57236. PMID: 24600300; PMCID: PMC3942110. | International |
| Multisource feedback to assess surgical practice: a systematic review | J Surg Educ | 2013 | View | Al Khalifa K, Al Ansari A, Violato C, Donnon T. Multisource feedback to assess surgical practice: a systematic review. J Surg Educ. 2013 Jul-Aug;70(4):475-86. doi: 10.1016/j.jsurg.2013.02.002. Epub 2013 Apr 10. PMID: 23725935. | International |
| A contemporary approach to validity arguments: a practical guide to Kane’s framework | Med Educ | 2015 | View | Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015 Jun;49(6):560-75. doi: 10.1111/medu.12678. PMID: 25989405. | International |
| Multisource Evaluation of Surgeon Behavior Is Associated With Malpractice Claims | Ann Surg | 2019 | View | Lagoo J, Berry WR, Miller K, Neal BJ, Sato L, Lillemoe KD, Doherty GM, Kasser JR, Chaikof EL, Gawande AA, Haynes AB. Multisource Evaluation of Surgeon Behavior Is Associated With Malpractice Claims. Ann Surg. 2019 Jul;270(1):84-90. doi: 10.1097/SLA.0000000000002742. PMID: 29578910. | International |
| Colleague appraisal of Australian general practitioners in training: an analysis of multisource feedback data | BMC Med Educ | 2022 | View | Vayro C, Narayanan A, Greco M, Spike N, Hanson J, Mitchell B, Hanson D, Stewart R. Colleague appraisal of Australian general practitioners in training: an analysis of multisource feedback data. BMC Med Educ. 2022 Jun 24;22(1):494. doi: 10.1186/s12909-022-03559-5. PMID: 35751119; PMCID: PMC9233327. | International |
| Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates | J Educ Eval Health Prof | 2011 | View | Nestel D, Regan M, Vijayakumar P, Sunderji I, Haigh C, Smith C, Wright A. Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates. J Educ Eval Health Prof. 2011;8:13. doi: 10.3352/jeehp.2011.8.13. Epub 2011 Dec 17. PMID: 22259672; PMCID: PMC3258549. | International |
| Assessment of international medical graduates and their integration into family practice: the Clinician Assessment for Practice Program | Acad Med | 2008 | View | Maudsley RF. Assessment of international medical graduates and their integration into family practice: the Clinician Assessment for Practice Program. Acad Med. 2008 Mar;83(3):309-15. doi: 10.1097/ACM.0b013e318163710f. PMID: 18316885. | Canada |
| The value of patient and peer ratings in recertification | Acad Med | 2002 | View | Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in recertification. Acad Med. 2002 Oct;77(10 Suppl):S64-6. doi: 10.1097/00001888-200210001-00021. PMID: 12377708. | International |
| Initial evaluation of the first year of the Foundation Assessment Programme | Med Educ | 2009 | View | Davies H, Archer J, Southgate L, Norcini J. Initial evaluation of the first year of the Foundation Assessment Programme. Med Educ. 2009 Jan;43(1):74-81. doi: 10.1111/j.1365-2923.2008.03249.x. PMID: 19141000. | International |
| Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians’ Professional Performance | J Cont Educ Heal Prof | 2017 | View | van der Meulen MW, Boerebach BC, Smirnova A, Heeneman S, Oude Egbrink MG, van der Vleuten CP, Arah OA, Lombarts KM. Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians’ Professional Performance. J Contin Educ Health Prof. 2017 Winter;37(1):9-18. doi: 10.1097/CEH.0000000000000143. PMID: 28212117. | International |
| Longitudinal Outcome of Programmatic Assessment of International Medical Graduates | Adv Med Educ Pract | 2021 | View | Parvathy MS, Parab A, R Nair BK, Matheson C, Ingham K, Gunning L. Longitudinal Outcome of Programmatic Assessment of International Medical Graduates. Adv Med Educ Pract. 2021 Sep 23;12:1095-1100. doi: 10.2147/AMEP.S324412. PMID: 34588836; PMCID: PMC8476105. | International |
| Composite reliability of workplace-based assessment of international medical graduates | Med J Aust | 2017 | View | Nair BKR, Moonen-van Loon JM, Parvathy M, Jolly BC, van der Vleuten CP. Composite reliability of workplace-based assessment of international medical graduates. Med J Aust. 2017 Nov 20;207(10):453. doi: 10.5694/mja17.00130. PMID: 29129176. | International |
| A Multifaceted Organizational Physician Assessment Program: Validity Evidence and Implications for the Use of Performance Data | Mayo Clin Proc Innov Qual Outcomes | 2017 | View | Leep Hunderfund AN, Park YS, Hafferty FW, Nowicki KM, Altchuler SI, Reed DA. A Multifaceted Organizational Physician Assessment Program: Validity Evidence and Implications for the Use of Performance Data. Mayo Clin Proc Innov Qual Outcomes. 2017 Jul 25;1(2):130-140. doi: 10.1016/j.mayocpiqo.2017.05.005. PMID: 30225409; PMCID: PMC6135024. | International |
| Exploring valid and reliable assessment methods for care management education | Leadersh Health Serv (Bradf Engl) | 2016 | View | Gennissen L, Stammen L, Bueno-de-Mesquita J, Wieringa S, Busari J. Exploring valid and reliable assessment methods for care management education. Leadersh Health Serv (Bradf Engl). 2016 Jul 4;29(3):240-50. doi: 10.1108/LHS-09-2015-0029. PMID: 27397747. | International |
| The reliability of a portfolio of workplace-based assessments in anesthesia training | Can J Anaesth | 2019 | View | Castanelli DJ, Moonen-van Loon JMW, Jolly B, Weller JM. The reliability of a portfolio of workplace-based assessments in anesthesia training. Can J Anaesth. 2019 Feb;66(2):193-200. English. doi: 10.1007/s12630-018-1251-7. Epub 2018 Nov 14. PMID: 30430441. | International |
| The reliability of multisource feedback in competency-based assessment programs: the effects of multiple occasions and assessor groups | Acad Med | 2015 | View | Moonen-van Loon JM, Overeem K, Govaerts MJ, Verhoeven BH, van der Vleuten CP, Driessen EW. The reliability of multisource feedback in competency-based assessment programs: the effects of multiple occasions and assessor groups. Acad Med. 2015 Aug;90(8):1093-9. doi: 10.1097/ACM.0000000000000763. PMID: 25993283. | International |
| Factors that might undermine the validity of patient and multi-source feedback | Med Educ | 2011 | View | Archer JC, McAvoy P. Factors that might undermine the validity of patient and multi-source feedback. Med Educ. 2011 Sep;45(9):886-93. doi: 10.1111/j.1365-2923.2011.04023.x. PMID: 21848716. | International |
| Evaluation of and Feedback for Academic Medicine Leaders: Developing and Implementing the Memorial Method | Acad Med | 2017 | View | Rourke J, Bornstein S, Vardy C, Speed D, White T, Corbett P. Evaluation of and Feedback for Academic Medicine Leaders: Developing and Implementing the Memorial Method. Acad Med. 2017 Nov;92(11):1590-1594. doi: 10.1097/ACM.0000000000001722. PMID: 28471781. | Canada |
| Developing a multisource feedback tool for postgraduate medical educational supervisors | Med Teach | 2013 | View | Archer J, Swanwick T, Smith D, O’Keeffe C, Cater N. Developing a multisource feedback tool for postgraduate medical educational supervisors. Med Teach. 2013;35(2):145-54. doi: 10.3109/0142159X.2012.733839. Epub 2012 Oct 26. PMID: 23102057. | International |
| Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors | J Grad Med Educ | 2016 | View | Goldhamer ME, Baker K, Cohen AP, Weinstein DF. Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors. J Grad Med Educ. 2016 Oct;8(4):592-596. doi: 10.4300/JGME-D-15-00543.1. PMID: 27777673; PMCID: PMC5058595. | International |
| A review of multi-source feedback focusing on psychometrics, pitfalls and some possible solutions | SN Soc Sci | 2021 | View | Ashworth, N., De Champlain, A.F. & Kain, N. A review of multi-source feedback focusing on psychometrics, pitfalls and some possible solutions. SN Soc Sci 1, 24 (2021). | Canada |
| Validation of Multisource Feedback in Assessing Medical Performance: A Systematic Review | J Cont Educ Heal Prof | 2018 | View | Stevens S, Read J, Baines R, Chatterjee A, Archer J. Validation of Multisource Feedback in Assessing Medical Performance: A Systematic Review. J Contin Educ Health Prof. 2018 Fall;38(4):262-268. doi: 10.1097/CEH.0000000000000219. PMID: 30157152.. | International |
| Assessment of physician performance in Alberta: the Physician Achievement Review Project | Can Med Ass J | 1999 | View | Hall W, Violato C, Lewkonia R, Lockyer J, Fidler H, Toews J, Jennett P, Donoff M, Moores D. Assessment of physician performance in Alberta: the physician achievement review. CMAJ. 1999 Jul 13;161(1):52-7. PMID: 10420867; PMCID: PMC1232653. | Canada |
| Use of peer ratings to evaluate physician performance | JAMA | 1993 | View | Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA. 1993 Apr 7;269(13):1655-60. PMID: 8240483. | International |
| Ratings of the performances of practicing internists by hospital-based registered nurses | Acad Med | 1993 | View | Wenrich MD, Carline JD, Giles LM, Ramsey PG. Ratings of the performances of practicing internists by hospital-based registered nurses. Acad Med. 1993 Sep;68(9):680-7. doi: 10.1097/00001888-199309000-00014. PMID: 8397633. | International |
| A multisource feedback tool to assess ward round leadership skills of senior paediatric trainees: (2) Testing reliability and practicability | Postgrad Med J | 2015 | View | Goodyear HM, Lakshminarayana I, Wall D, Bindal T. A multisource feedback tool to assess ward round leadership skills of senior paediatric trainees: (2) Testing reliability and practicability. Postgrad Med J. 2015 May;91(1075):268-73. doi: 10.1136/postgradmedj-2015-133308. Epub 2015 May 4. PMID: 25941417. | International |
| A multisource feedback tool to assess ward round leadership skills of senior paediatric trainees: (1) Development of tool | Postgrad Med J | 2015 | View | Lakshminarayana I, Wall D, Bindal T, Goodyear HM. A multisource feedback tool to assess ward round leadership skills of senior paediatric trainees: (1) Development of tool. Postgrad Med J. 2015 May;91(1075):262-7. doi: 10.1136/postgradmedj-2014-132692. Epub 2015 May 4. PMID: 25941418. | International |
| Use of SPRAT for peer review of paediatricians in training | BMJ | 2005 | View | Archer JC, Norcini J, Davies HA. Use of SPRAT for peer review of paediatricians in training. BMJ. 2005 May 28;330(7502):1251-3. doi: 10.1136/bmj.38447.610451.8F. Epub 2005 May 9. PMID: 15883137; PMCID: PMC558096. | International |
| Resident-patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors, and nurses | Acad Med | 1994 | View | Woolliscroft JO, Howell JD, Patel BP, Swanson DB. Resident-patient interactions: the humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors, and nurses. Acad Med. 1994 Mar;69(3):216-24. doi: 10.1097/00001888-199403000-00017. PMID: 8135980. | International |
| Is there an association between nurse, clinical teacher and peer feedback for trainee doctors’ medical specialty choice? An observational study in Taiwan | BMJ Open | 2018 | View | Hsu CM, Hsiao CT, Chang LC, Chang HY. Is there an association between nurse, clinical teacher and peer feedback for trainee doctors’ medical specialty choice? An observational study in Taiwan. BMJ Open. 2018 Apr 12;8(4):e020769. doi: 10.1136/bmjopen-2017-020769. PMID: 29654041; PMCID: PMC5898308. | International |
| Variation in assessment of first-year medical students’ interprofessional competencies by rater profession | Journal of Interprofessional Education & Practice | 2021 | View | Joseph B. House, Lynze R. Franko, Fatema Haque, James A. Cranford, Sally A. Santen, Variation in assessment of first-year medical students’ interprofessional competencies by rater profession, Journal of Interprofessional Education & Practice, Volume 24, 2021, 100424, ISSN 2405-4526 | International |
| Processing multisource feedback during residency under the guidance of a non-medical coach | Int J Med Educ | 2018 | View | Buis CAM, Eckenhausen MAW, Ten Cate O. Processing multisource feedback during residency under the guidance of a non-medical coach. Int J Med Educ. 2018 Feb 23;9:48-54. doi: 10.5116/ijme.5a7f.169d. PMID: 29478041; PMCID: PMC5834823. | International |
| Peer-assisted debriefing of multisource feedback: an exploratory qualitative study | BMC Med Educ | 2018 | View | Francois J, Sisler J, Mowat S. Peer-assisted debriefing of multisource feedback: an exploratory qualitative study. BMC Med Educ. 2018 Mar 14;18(1):36. doi: 10.1186/s12909-018-1137-y. PMID: 29540204; PMCID: PMC5853071. | Canada |
| Assessing interprofessional teamwork: pilot test of a new assessment module for practicing physicians | J Contin Educ Health Prof | 2015 | View | Chesluk BJ, Reddy S, Hess B, Bernabeo E, Lynn L, Holmboe E. Assessing interprofessional teamwork: pilot test of a new assessment module for practicing physicians. J Contin Educ Health Prof. 2015 Winter;35(1):3-10. doi: 10.1002/chp.21267. PMID: 25799967. | International |
| The Influence of Relationship-Centered Coaching on Physician Perceptions of Peer Review in the Context of Mandated Regulatory Practices | Acad Med | 2020 | View | Arabsky S, Castro N, Murray M, Bisca I, Eva KW. The Influence of Relationship-Centered Coaching on Physician Perceptions of Peer Review in the Context of Mandated Regulatory Practices. Acad Med. 2020 Nov;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S14-S19. doi: 10.1097/ACM.0000000000003642. PMID: 32769452. | Canada |
| Composite reliability of a workplace-based assessment toolbox for postgraduate medical education | Adv Health Sci Educ Theory Pract | 2013 | View | Moonen-van Loon JM, Overeem K, Donkers HH, van der Vleuten CP, Driessen EW. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 2013 Dec;18(5):1087-102. doi: 10.1007/s10459-013-9450-z. Epub 2013 Mar 15. PMID: 23494202. | International |
| Validation of a competence-based assessment of medical students’ performance in the physician’s role | BMC Med Educ | 2020 | View | Prediger S, Schick K, Fincke F, Fürstenberg S, Oubaid V, Kadmon M, Berberat PO, Harendza S. Validation of a competence-based assessment of medical students’ performance in the physician’s role. BMC Med Educ. 2020 Jan 7;20(1):6. doi: 10.1186/s12909-019-1919-x. PMID: 31910843; PMCID: PMC6947905. | International |
| Interprofessional assessment of medical students’ competences with an instrument suitable for physicians and nurses | BMC Med Educ | 2019 | View | Prediger S, Fürstenberg S, Berberat PO, Kadmon M, Harendza S. Interprofessional assessment of medical students’ competences with an instrument suitable for physicians and nurses. BMC Med Educ. 2019 Feb 6;19(1):46. doi: 10.1186/s12909-019-1473-6. PMID: 30728006; PMCID: PMC6364398. | International |
| Multisource feedback in evaluating the performance of doctors: The example of the UK General Medical Council Patient and Colleague Questionnaires | Acad Med | 2012 | View | Wright C, Richards SH, Hill JJ, Roberts MJ, Norman GR, Greco M, Taylor MR, Campbell JL. Multisource feedback in evaluating the performance of doctors: the example of the UK General Medical Council patient and colleague questionnaires. Acad Med. 2012 Dec;87(12):1668-78. doi: 10.1097/ACM.0b013e3182724cc0. PMID: 23095930. | International |
| Assessing the professional performance of UK doctors: an evaluation of the utility of the General Medical Council patient and colleague questionnaires | Qual Saf Health Care | 2008 | View | Campbell JL, Richards SH, Dickens A, Greco M, Narayanan A, Brearley S. Assessing the professional performance of UK doctors: an evaluation of the utility of the General Medical Council patient and colleague questionnaires. Qual Saf Health Care. 2008 Jun;17(3):187-93. doi: 10.1136/qshc.2007.024679. PMID: 18519625. | International |
| Exploring Content Relationships Among Components of a Multisource Feedback Program | J Cont Educ Health Prof | 2022 | View | Roy M, Kain N, Touchie C. Exploring Content Relationships Among Components of a Multisource Feedback Program. J Contin Educ Health Prof. 2022 Oct 1;42(4):243-248. doi: 10.1097/CEH.0000000000000398. Epub 2021 Oct 1. PMID: 34609355. | Canada |
| Multisource feedback and self-assessment of the communicator, collaborator, and professional CanMEDS roles for diagnostic radiology residents | Can Assoc Radiol J | 2014 | View | Probyn L, Lang C, Tomlinson G, Bandiera G. Multisource feedback and self-assessment of the communicator, collaborator, and professional CanMEDS roles for diagnostic radiology residents. Can Assoc Radiol J. 2014 Nov;65(4):379-84. doi: 10.1016/j.carj.2014.04.003. Epub 2014 Sep 26. PMID: 25267375. | Canada |
| Using Multisource Feedback to Assess Resident Communication Skills: Adding a New Dimension to Milestone Data | Ochsner Journal | 2020 | View | Byrd A, Iheagwara K, McMahon P, Bolton M, Roy M. Using Multisource Feedback to Assess Resident Communication Skills: Adding a New Dimension to Milestone Data. Ochsner J. 2020 Fall;20(3):255-260. doi: 10.31486/toj.19.0054. PMID: 33071656; PMCID: PMC7529137. | International |
| Developing and Implementing a Multisource Feedback Tool to Assess Competencies of Emergency Medicine Residents in the United States | AEM Educ Train | 2017 | View | LaMantia J, Yarris LM, Sunga K, Weizberg M, Hart D, Farina G, Rodriguez E, Lucas R, Mahmooth Z, Snock A, Lockyear J. Developing and Implementing a Multisource Feedback Tool to Assess Competencies of Emergency Medicine Residents in the United States. AEM Educ Train. 2017 Jun 15;1(3):243-249. doi: 10.1002/aet2.10043. PMID: 30051042; PMCID: PMC6001492. | International |
| How physicians change: Multisource feedback driven intervention improves physician leadership and teamwork | Surgery | 2020 | View | Hu J, Lee R, Mullin S, Schwaitzberg S, Harmon L, Gregory P, Elkin PL. How physicians change: Multisource feedback driven intervention improves physician leadership and teamwork. Surgery. 2020 Oct;168(4):714-723. doi: 10.1016/j.surg.2020.06.008. Epub 2020 Aug 6. PMID: 32773278. | International |
| A Training Intervention through a 360° Multisource Feedback Model | Int J Environ Res Public Health | 2021 | View | Sureda E, Chacón-Moscoso S, Sanduvete-Chaves S, Sesé A. A Training Intervention through a 360° Multisource Feedback Model. Int J Environ Res Public Health. 2021 Aug 30;18(17):9137. doi: 10.3390/ijerph18179137. PMID: 34501727; PMCID: PMC8431571. | International |
| The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review | Acad Med | 2014 | View | Donnon T, Al Ansari A, Al Alawi S, Violato C. The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review. Acad Med. 2014 Mar;89(3):511-6. doi: 10.1097/ACM.0000000000000147. PMID: 24448051. | Canada |
| Evaluation of physicians’ professional performance: an iterative development and validation study of multisource feedback instruments | BMC Health Serv Res | 2012 | View | Overeem K, Wollersheim HC, Arah OA, Cruijsberg JK, Grol RP, Lombarts KM. Evaluation of physicians’ professional performance: an iterative development and validation study of multisource feedback instruments. BMC Health Serv Res. 2012 Mar 26;12:80. doi: 10.1186/1472-6963-12-80. PMID: 22448816; PMCID: PMC3349515. | International |
| Multisource feedback: an overview of its use and application as a formative assessment | Canadian Medical Education Journal | 2022 | View | Lockyer J, Sargeant J. Multisource feedback: an overview of its use and application as a formative assessment. Can Med Educ J. 2022 Aug 26;13(4):30-35. doi: 10.36834/cmej.73775. PMID: 36091727; PMCID: PMC9441111. | Canada |
| A multi-source feedback tool for measuring a subset of Pediatrics Milestones | Academic Medicine | 2016 | View | Schwartz A, Margolis MJ, Multerer S, Haftel HM, Schumacher DJ; APPD LEARN–NBME Pediatrics Milestones Assessment Group. A multi-source feedback tool for measuring a subset of Pediatrics Milestones. Med Teach. 2016 Oct;38(10):995-1002. doi: 10.3109/0142159X.2016.1147646. Epub 2016 Mar 30. PMID: 27027428. | International |
| The Pediatrics Milestones Assessment Pilot: Development of Workplace-Based Assessment Content, Instruments, and Processes | Academic Medicine | 2016 | View | Hicks PJ, Margolis M, Poynter SE, Chaffinch C, Tenney-Soeiro R, Turner TL, Waggoner-Fountain L, Lockridge R, Clyman SG, Schwartz A; APPD LEARN-NBME Pediatrics Milestones Assessment Group. The Pediatrics Milestones Assessment Pilot: Development of Workplace-Based Assessment Content, Instruments, and Processes. Acad Med. 2016 May;91(5):701-9. doi: 10.1097/ACM.0000000000001057. PMID: 26735520. | International |
| Family Physician Quality Improvement Plans: A Realist Inquiry Into What Works, for Whom, Under What Circumstances | J Contin Educ Health Prof | 2022 | View | Roy M, Lockyer J, Touchie C. Family Physician Quality Improvement Plans: A Realist Inquiry Into What Works, for Whom, Under What Circumstances. J Contin Educ Health Prof. 2023 Summer 01;43(3):155-163. doi: 10.1097/CEH.0000000000000454. Epub 2022 Jul 6. PMID: 37638679. | Canada |
| How does multisource feedback influence residency training? A qualitative case study | Medical Education | 2022 | View | Hennel EK, Trachsel A, Subotic U, Lörwald AC, Harendza S, Huwendiek S. How does multisource feedback influence residency training? A qualitative case study. Med Educ. 2022 Jun;56(6):660-669. doi: 10.1111/medu.14798. Epub 2022 Mar 16. PMID: 35263461; PMCID: PMC9314722. | International |
| Optimizing Multisource Feedback Implementation for Australasian Physicians | J Contin Educ Health Prof | 2019 | View | Pooley M, Pizzuti C, Daly M. Optimizing Multisource Feedback Implementation for Australasian Physicians. J Contin Educ Health Prof. 2019 Fall;39(4):228-235. doi: 10.1097/CEH.0000000000000267. PMID: 31567831. | International |
| Doctors’ perceptions of why 360-degree feedback does (not) work: a qualitative study | Medical Education | 2009 | View | Overeem K, Wollersheim H, Driessen E, Lombarts K, van de Ven G, Grol R, Arah O. Doctors’ perceptions of why 360-degree feedback does (not) work: a qualitative study. Med Educ. 2009 Sep;43(9):874-82. doi: 10.1111/j.1365-2923.2009.03439.x. PMID: 19709012. | International |
| The impact of patient feedback on the medical performance of qualified doctors: a systematic review | BMC Medical Education | 2018 | View | International | |
| 360-Degree Physician Performance Assessment | Healthcare Quarterly | 2010 | View | Dubinsky, Isser & Jennings, Kelly & Greengarten, Moshe & Brans, Amy. (2010). 360-Degree Physician Performance Assessment. Healthcare quarterly (Toronto, Ont.). 13. 71-6. 10.12927/hcq.2013.21673. | International |
| Feedback data sources that inform physician self-assessment | Medical Teacher | 2011 | View | Lockyer, J., Armson, H., Chesluk, B., Dornan, T., Holmboe, E., Loney, E., … Sargeant, J. (2011). Feedback data sources that inform physician self-assessment. Medical Teacher, 33(2), e113–e120. https://doi.org/10.3109/0142159X.2011.542519 | International |
| Factors influencing the effectiveness of multisource feedback in improving the professional practice of medical doctors: a systematic review | BMC Medical Education | 2014 | View | International | |
| Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine | J Gen Intern Med | 2020 | View | Rosenthal MA, Sharpe BA, Haber LA. Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine. J Gen Intern Med. 2020 Dec;35(12):3644-3649. doi: 10.1007/s11606-020-06235-w. Epub 2020 Sep 21. PMID: 32959350; PMCID: PMC7728945. | International |
| Evaluation of the physician quality improvement initiative: the expected and unexpected opportunities | BMC Medical Education | 2015 | View | Wentlandt K, Bracaglia A, Drummond J, Handren L, McCann J, Clarke C, Degendorfer N, Chan CK. Evaluation of the physician quality improvement initiative: the expected and unexpected opportunities. BMC Med Educ. 2015 Dec 22;15:230. doi: 10.1186/s12909-015-0511-2. PMID: 26694493; PMCID: PMC4687152. | Canada |
| Family physician practice visits arising from the Alberta Physician Achievement Review | BMC Medical Education | 2013 | View | Lewkonia R, Flook N, Donoff M, Lockyer J. Family physician practice visits arising from the Alberta Physician Achievement Review. BMC Med Educ. 2013 Sep 9;13:121. doi: 10.1186/1472-6920-13-121. PMID: 24010980; PMCID: PMC3846577. | Canada |
| Assessment of physician performance in Alberta: the Physician Achievement Review | Canadian Medical Association | 1999 | View | Canada | |
| Multi-source feedback in undergraduate medical education: a pilot study | Canadian Medical Education Journal | 2025 | View | Bartman I, St-Onge C, Roy M, Gingerich A, Katsoulas E, Chahine S, Gagnon N. Multi-source feedback in undergraduate medical education: a pilot study. Can Med Educ J. 2025 May 1;16(2):25-31. doi: 10.36834/cmej.79283. PMID: 40365596; PMCID: PMC12068204. | Canada |