Published on in Vol 8 , No 5 (2019) :May

Preprints (earlier versions) of this paper are available at, first published .
Testing a Communication Assessment Tool for Ethically Sensitive Scenarios: Protocol of a Validation Study

Testing a Communication Assessment Tool for Ethically Sensitive Scenarios: Protocol of a Validation Study

Testing a Communication Assessment Tool for Ethically Sensitive Scenarios: Protocol of a Validation Study


1Department of Pediatrics, Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada

2Performance and Evaluation, Genome Canada, Ottawa, ON, Canada

3Department of Psychology, Carleton University, Ottawa, ON, Canada

Corresponding Author:

Thierry Daboval, MSc, MD

Department of Pediatrics

Children's Hospital of Eastern Ontario

University of Ottawa

401 Smyth Road

Ottawa, ON,


Phone: 1 613 737 7600 ext 3258

Fax:1 613 737 4847


Background: Although well-designed instruments to assess communication during medical interviews and complex encounters exist, assessment tools that differentiate between communication, empathy, decision-making, and moral judgment are needed to assess different aspects of communication during situations defined by ethical conflict. To address this need, we developed an assessment tool that differentiates competencies associated with practice in ethically challenging situations. The competencies are grouped into three distinct categories: communication skills, civility and respectful behavior, clinical and ethical judgment and decision-making.

Objective: The overall objective of this project is to develop an assessment tool for ethically sensitive scenarios that measures the degree of respect for the attitudes and beliefs of patients and family members, the demands of clinical decision-making, and the success in dealing with ethical conflicts in the clinical context. In this article, we describe the research method we will use during the pilot-test study using the neonatal context to provide validity evidence to support the features of the Assessment Communication Tool for Ethics (ACT4Ethics) instrument.

Methods: This study is part of a multiphase project designed according to modern validity principles including content, response process, internal structure, relation to other variables, and social consequences. The design considers threats to validity such as construct underrepresentation and factors exerting nonrandom influence on scores. This study consists of two primary steps: (1) train the raters in the use of the new tool and (2) pilot-test a simulation using an Objective Structured Clinical Examination. We aim to obtain a total of 90 independent assessments based on the performance of 30 trainees rated by 15 trained raters for analysis. A comparison of raters’ responses will allow us to compute a measure of interrater reliability. We will additionally compare the results of ACT4Ethics with another existing instrument.

Results: This study will take approximately 18 months to complete and the results should be available by September 2019.

Conclusions: ACT4Ethics should allow clinician-teachers to assess and monitor the development of competency of trainees’ judgments and communication skills when facing ethically sensitive clinical situations. The instrument will also guide the provision of meaningful feedback to ensure that trainees develop specific communication, empathy, decision-making, and ethical competencies.

International Registered Report Identifier (IRRID): PRR1-10.2196/12039

JMIR Res Protoc 2019;8(5):e12039



Importance of Ethics and Communication Education in Medicine

Clinicians face ethically challenging situations such as limiting care, engaging in collective or shared decision-making and surrogate decision-making, and participating in end-of-life discussions. In order to engage in successful clinical decision-making during these situations, specific competencies must be identified, education milestones should be established, and learning needs must be monitored. Given the complexity and ubiquity of these cases requiring tactful communication, high-stakes moral and ethical judgment, and mindfulness, training must start early during undergraduate and continue into postgraduate health care professional education [1-4]. The development of these competencies requires the promotion of skills associated with shared decision-making, increased empathetic accuracy of clinicians, and a focus on decreasing psychological distress for severely ill patients and their families [5-7].

Although all health care professions involve difficult situations, neonatology presents a diverse set of considerations including a wide range of ethically sensitive situations and frequent inter- and intradisciplinary communication. Supporting this, there is a growing recognition that Neonatal-Perinatal Medicine (NPM) trainees require a greater understanding of ethical features of clinical situations and that there is a need to increase the availability and efficacy of ethics and communication programs [8]. Our team implemented a neonatal ethics teaching program that integrates knowledge of ethics acquisition and provides opportunities to practice communication skills in a safe, structured environment [9]. This program helps develop ethical competencies by providing constructive feedback and promoting self-reflection [10]. Our neonatal ethics teaching program and other neonatal ethics published curricula focus on helping trainees refine their knowledge of ethics and learn competencies relevant to professionalism and communication [11,12]. While multiple approaches based on adult learning theory support trainees’ education, pedagogical methods best suited to teaching ethics and communication have yet to be identified. Assessment methods that can differentiate specific features of these situations need to be developed. These assessment tools can be used to support three important objectives: monitoring the development of a clinician’s knowledge and skills over the course of their career, supporting the evaluation of teaching methods, and guiding the design of effective education programs [13]. We considered these objectives in the development of our assessment tool.

Need for a Communication Assessment Tool for Ethically Sensitive Clinical Situations

Studies focusing on NPM residency training have identified the need to develop tools supported by empirical evidence that assess knowledge and behavioral learning [8,14]. In addition to NPM, many other subspecialties in medicine need such assessment instruments. While knowledge tests in medical ethics competency and attitudes have been described for pediatrics and internal and neonatal medicine [15-18], they are not sufficient in and of themselves; assessment of general competencies such as communication skills must also be considered [9,11,19-21]. Our literature review identified several well-designed tools to assess communication skills during medical interviews. They were typically designed for medical encounters exploring symptoms or providing difficult news to patients and their family members. We failed to identify any that were directly related to medical ethical situations [22-27]. For instance, the Gap-Kalamazoo Communication Skills Assessment Form (GKCAS), which is used to assess communication competencies during complex encounters across medical subspecialties, includes domains such as building relationships, understanding the family perspective, sharing information, demonstrating empathy, and reaching agreement, but we believe it lacks important topics such as moral judgment and ethical conflicts [28].

Foundation of the Communication Assessment Tool for Ethics Framework

Against this background, we created an assessment instrument designed to differentiate competencies related to ethical judgment and decision-making, respect and empathy, and communication skills during ethically sensitive situations. Although developed in the context of NPM, the Assessment of Communication Tool for Ethics (ACT4Ethics) can be applied to a wide range of ethically challenging situations encountered in many fields in the health care professions. This assessment instrument includes key milestones in the roles of medical professional, communicator, and collaborator [11]. It includes competencies aligned with verbal and nonverbal communication skills adapted to medical interviews [23]; communication skills included in delivering bad news [26], demonstrating awareness of ethical features of a situation including virtue ethics; bioethical principles and communicative ethics [29]; ethical judgment and decision-making [30]; and engaging patients in a decision-making process [31]. We used the academic literature to guide the construction of the ACT4Ethics scale. We grouped our assessment criteria into three broad domains reflecting a proposed set of distinct competencies: communication, civility and respectful behavior, and clinical and ethical judgment and decision-making.

The communication skills subscale evaluates basic verbal and nonverbal skills, while also addressing more complex skills such as conversational pragmatics that might arise during a clinical encounter [22,32]. Given the often ambiguous nature of the clinical setting as well as differences in clinicians’ and patients’ knowledge, this communication subscale also assesses whether a learner effectively closed the loop (ie, sought explicit confirmation that they shared an understanding) during critical periods throughout the encounter as well as at the end of the encounter [33]. Ethically sensitive scenarios regularly include elements such as presenting affectively charged information that would reasonably disappoint patients and requires the consideration of empathy [26] and respect for their values and beliefs.

The civility and empathy subscale assesses empathy-related skills and overt social cues associated with respect. Civility has become a growing concerning in professional environments [34,35] and clinical settings [36-38]. In contrast to respectful and disrespectful behaviors, incivility reflects actions wherein the intentions of the communicator lack clarity. This lack of clarity can lead to negative consequences over lengthy periods [34]. Similarly, empathy can be defined along two separate dimensions. Whereas cognitive empathy requires understanding another’s thoughts and beliefs in order to predict behavior [39], affective empathy involves sharing the emotional response for another’s joy and pain [40]. These appear to have a distinctive neurological basis [41,42]. Crucially, in the context of a clinical encounter, patients and family members might not disclose relevant information or consider clinical alternatives if they do not believe that clinicians respect their emotional responses, beliefs, or choices.

The judgment and decision-making subscale assesses clinical and ethical features of judgment and decision-making. Judgment and decision-making require consideration of a clinician’s awareness of the situation [43] and whether the clinician gathers and assesses evidence and considers diagnostic and treatment alternatives in an unbiased manner [44] given the constraints of the situation [45]. In the clinical context, this involves considering multiple sources of information such as that provided by other health care professionals, patients, and family members. Even if a clinician is an effective communicator and has demonstrated respect and empathy with a patient, failure to integrate the available information will ultimately undermine the clinical encounter.

The Communication Assessment Tool for Ethics Scale

Figures 1 and 2 present the structure of the ACT4Ethics scale.

Figure 1. Assessment Communication Tool for Ethics (ACT4Ethics), page 1.
View this figure
Figure 2. Assessment Communication Tool for Ethics (ACT4Ethics), page 2.
View this figure

The ACT4Ethics scale includes 12 competencies aligned to the three domains included in the framework. The tool includes a 5-level rating scale for each competency with examples of milestones and stages in accordance with the continuum base by design model [11] to guide the rating. A red flag is added to the rating scale to allow the assessing clinicians to note concerns severe enough that a trainee should not currently be permitted to talk to patients or family members without supervision. The tool allows capturing the clinical context of the interaction and the relevant situational factors that might have justifiably interfered with the ethical judgment and decision-making processes. Such qualifications are necessary in order to account for any variability introduced by atypical features of clinical encounters likely to occur outside the simulation context. In the end, ACT4Ethics includes an overall assessment scale. Ideally, ratings on this overall scale should positively correlate with the averages obtained from the individual subscales.

The pilot study will present the tool without displaying the hypothesized targeted competencies depicted in subscale labels in order to ensure that raters will not be biased in assigning their ratings. For instance, while we have grouped certain items within a specific subscale, raters might assign ratings that do not support organizing ACT4Ethics in this manner.

Aim of the Study

The objective of this study is to pilot-test ACT4Ethics. This study will be divided into two parts—part A: training the raters to use the tool and part B: obtaining validity evidence to support the scale construction [46].

Setting and Participants

The study will take place between February and June 2019 at the University of Ottawa Faculty of Medicine adjacent of the Children’s Hospital of Eastern Ontario (CHEO) and the Ottawa Hospital General Campus. CHEO is a tertiary pediatric center that houses a 3B neonatal intensive care unit, and the Ottawa Hospital General Campus is equipped with a birthing unit and a level 3A neonatal intensive care unit. The University of Ottawa Faculty of Medicine’s NPM training program is a 2- to 3-year residency accredited by the Royal College of Physicians and Surgeons of Canada.

The teaching tool used in the context of an Objective Structured Clinical Examination (OSCE) session is designed to test clinical skill performance that trainees in health care professions are expected to acquire [47]. The one-station OSCE will consist of a case focusing on antenatal findings of multiple congenital anomalies without a specific genetic syndrome identified. It will be divided into two parts for a total of 30 minutes. The first 15 minutes of the session will be videotaped and include one trainee and one standardized patient. One of the two coauthors (TD or EF) who directly observed the interaction will provide confidential formative feedback in the second 15 minutes of the session, which will not be videotaped.

The pilot group will consist of a purposive sample [48] of former and current trainees from the NPM, Maternal-Fetal Medicine, Obstetrics, Pediatric Palliative Care, Genetics, and General Pediatrics residency programs (n=30). Our recruitment strategy intends to obtain a wide distribution of performances to ensure that raters will use the full range of the scale. We will recruit and train academic staff (n=15) to review the videos of the pilot group. Standardized patients (n=2) will be recruited to play-act the patient in the OSCE.

Ethical Considerations

Academic staff members from a variety of medical fields including collaborators who have participated in the creation of ACT4Ethics will also be piloting the tool. A research assistant or one of the authors (ALR) will approach academic staff, trainees, and standardized patients for consent. We will inform trainee participants that their choice will not affect their residency training assessment. Each academic staff participant will receive an incentive. The standardized patients, from the University of Ottawa Health Science Faculty, will be reimbursed for administration, training, and parking fees. The CHEO and Ottawa Health Science Network research ethics boards have approved this study.

Study Design

This pilot cohort study is part of a multiphase project designed according to modern validity principles—content (phases 1 and 2: creation of a blueprint), response process (phase 3: pilot test), internal structure, relation to other variables, and consequences (phase 4: implementation) [46]—where evidence that supports a particular interpretation of the results is collected [49,50]. The design also considers threats to validity such as construct underrepresentation and factors exerting nonrandom influence on scores.

Part A: Train the Raters

We will invite 15 academic staff members for training on ACT4Ethics and the GKCAS clinician/faculty form. A 45- to 60-minute video-based session developed by members of the project team (TD, EF, JS, and ALR) will be used to train academic staff/future raters on ACT4Ethics and help them achieve and promote consistency in assessment [51]. Future raters will test the tool on three short videos depicting a typical clinical scenario.

The short videos (approximatively 15 minutes in length) will be developed with the participation of one of the coauthors (ALR) acting out three levels of competence: transition to discipline, core of discipline, and advanced expertise [11], while interacting with a parent played by a standardized patient. A half-day session will involve at least three coauthors (TD, EF, ALR) or collaborators defining and standardizing the script for the videos. Each video will correspond to different levels of competence. They will include a number of key decision points where future raters should be able to identify and address specific features of a situation concerning communication (eg, ambiguous statements that require closing the loop), decision-making (eg, essential diagnostic information not presented), and respect/empathy (eg, family behavior or personal/religious beliefs affecting interaction).

Part B: Obtain Validity Evidence

The pilot-test will be conducted using 30 distinct 15-minute videotaped interactions between a trainee and a standardized patient. We will ask the trainees to provide information on their previous experience in ethical encounters in their practice and ethical training, level of confidence in navigating an ethically charged situation, and basic demographics (eg, gender, age, year, and subspecialty of training). We will also ask them to complete an evaluation survey of the session.

Each trained rater will receive the videos of 6 different encounters, and they will use ACT4Ethics and GKCAS clinician/faculty form [27] to score trainee performance. The scores and debriefing will not be included in current trainees’ final subspecialty training evaluation. After providing their ratings, trained raters will provide feedback on a standardized satisfaction survey. Their feedback will allow comparison of rater views between ACT4Ethics and the GKCAS clinician/faculty form.

Data Analysis

The research assistant or coauthor will enter all data into SPSS Statistics (IBM Corp). Descriptive statistics (mean and standard deviation or median and interquartile range) will be used to characterize participant demographics, responses to the survey, and the scores obtained with ACT4Ethics and compare them to the GKCAS clinician/faculty form.

Trainees’ prior experience with regard to ethical situations will allow us to measure its effect on their performance and interaction with the standardized patient, as well as on their overall scores from the ACT4Ethics and GKCAS tools.

To compare the scales, we will examine (1) the order of individual participants in terms of their aggregate performance for each scale and (2) the correlation between items from each subscale that should show a high degree of correspondence (eg, items that assess empathy in the civility and respect subscale). We anticipate that the trained raters will use the complete range of response scale and have their ratings correspond to the issues presented in the training videos. Each rater will assess a total of 6 videos to determine the interrater reliability of ACT4Ethics. This process will result in 90 independent evaluations assessed by 15 trained raters with each encounter being evaluated three times—three nested scores within participants. We will conduct a generalizability analysis with rater nested within video and crossed with item. This analysis will allow us to determine the proportion of variance accounted for by each variable as well as allowing us to generate reliability coefficients related to both interrater reliability and the internal consistency of the items on the scale. We will use the conventional level of reliability of .8 or higher for the overall subscale to analyze our results. We will additionally obtain correlations of ratings for items between subscales and within a subscale. The scores from raters using both assessment instruments will allow an evaluation of the correlation between ratings.

Responses from the satisfaction survey completed by the trained raters will be used to assess the acceptance and usability of ACT4Ethics and explore facilitators and barriers for future implementation. We will review feedback on the experience and on the tool to adapt ACT4Ethics as appropriate.

Implementation Phase

The team will seek funding first to support the translation of ACT4Ethics into an electronic format ready for the one45 software platform, which supports the performance assessment of learners [52], and second, to facilitate the implementation of the tool in NPM, Pediatric, Maternal-Fetal Medicine, Medical Genetics, and Pediatric Palliative Care residency training programs. During this phase, the coauthors will continuously examine the construct validity of the assessment instrument.

We anticipate this project will take a total of 18 months to complete and expect the results to be available by September 2019.

Assessment of Communication Tool for Ethics to Maximize Learning of Communication Skills

Engaging in conversations with patients and families facing ethically challenging situations requires well-trained staff with advanced communication skills to support them. These skills also facilitate shared decision-making about the provision of care that is in the patient’s best interest [53,54]. Physicians need to be taught these skills during their training. Without a well-designed assessment instrument specifically adapted to ethically sensitive situations, assessment of communication relies on subjectivity [55]. Such assessments will not maximize the learning of communication skills adapted to these encounters, leaving future physicians with underdeveloped skills potentially increasing the risk of conflict between patients, families, and the physician [56]. Although we are using an OSCE in the context of neonatology for the validation of our assessment instrument, the principles included in ACT4Ethics are also applicable in many other areas in health care professions. Our tool considers the specific affective, interconnected elements of verbal and nonverbal communication and cognitive features of ethically challenging clinical sets of circumstances encountered in medicine [24].

Assessment Scale Construct Validity

ACT4Ethics was created through a multiphase process to develop a valid and reliable assessment tool [57]. Using an integrated knowledge translation model [58] to facilitate the implementation phase of ACT4Ethics into different medicine subspecialty residency programs, the project engaged key collaborators from different medical fields [49] with expertise in physician-patient communication, clinical ethics, and/or medical education. They participated in focus groups, dyadic interviews, and Web-based surveys to define the goals, use, and content of the assessment tool.

The instrument was developed to be (1) comprehensive, (2) easily applied within the clinical context, and (3) capable of adapting to many subspecialties across the health care professions while providing expedient assessment. The overall goal of ACT4Ethics is to guide specific, meaningful feedback to facilitate trainee identification of the ethical affordances of communication regarding complex clinical situations that arise in NPM and many medical subspecialties.

During scale development, collaborators identified components of unique competencies that would enable clinicians to balance the demands of rigorous clinical decision-making with the need to ensure that they address the ethically sensitive features of these scenarios. While related, we think that each of the 12 competencies used within the scale should allow us to examine how they might independently contribute to a successful clinical encounter. As an important determinant of the construct validity of an assessment instrument [46], each of the three subscales should assess distinct features of a clinical scenario and show minimal overlap in terms of the ratings. For instance, items on ACT4Ethics that assess a learner’s civility and respectful behavior should only show a weak relationship with items related to clinical judgment and communication. Ratings for items within individual subscales (eg, all communication items) should correlate with each other. Differentiating responses for items within one subscale from those in a different subscale will validate the internal structure of ACT4Ethics. Ratings should increase following a training session in a pre- and posttest design. Such a pattern would suggest that the scale captures the response processes associated with trainee acquisition of these skills [46].

Our study will attempt to demonstrate how ACT4Ethics is an improvement over other communication assessment scales. For instance, while ACT4Ethics and GKCAS [28] both assess empathy, our scale breaks down empathy in cognitive (eg, perspective-taking) [39] and affective (eg, feelings) [40] components, whereas GKCAS does not make this distinction. These constructs reflect an important distinction that is likely to be relevant in clinical practice: while a clinician might be able to understand a family member’s concerns, they might not be adequately emotionally responsive. Similarly, while GKCAS addresses relationship building, this construct is somewhat ambiguous. We instead assume that relationship building should be assessed by looking at the clinician’s ability to adapt, demonstrate sensitivity to differences in perspective by different individuals within a clinical encounter, engage family members in the decision, and manage disagreement and ethical conflicts. Our analyses will examine how ACT4Ethics and GKCAS relate to one another.

The logic and thoroughness behind the creation of ACT4Ethics support our belief that it will improve the assessment of trainees and positively contribute to providing objective, meaningful feedback on communication skills during challenging ethical situations. Nonspecific in-training rotation evaluations and other tools lack this type of feedback. By assessing competencies associated with communication skills, civility and empathy, along with clinical and ethical judgment and decision-making, we should be able to identify specific strengths and weaknesses of a particular trainee. Feedback from the ACT4Ethics scale can then be used by learners to promote self-reflection and strengthen their competencies in specific domains of practice. Overall, trends in ACT4Ethics scores of multiple trainees could allow program directors to readily identify teaching deficiencies and allow them to adjust their curriculum accordingly.

Curriculum and Accreditation Needs

ACT4Ethics will assist with local curriculum and program development. We are currently evaluating the effect of our local neonatal ethics teaching program. Our evaluation follows Guskey’s levels of training program evaluation [13]. We have already evaluated reactions (Guskey level 1) and demonstrated that the neonatal ethics teaching program sessions were well-received by trainees and participants, with an overall satisfaction score of 5.8 out of 7 [2]. Currently, we are running our pre- and posttraining knowledge test with our ongoing cohorts to evaluate the acquisition of knowledge related to ethics (Guskey level 2) [18]. The support received from the University of Ottawa to implement the program and develop our tools and the expansion to other residency programs indicates that the organization supports our model (Guskey level 3: organization support and change). ACT4Ethics will allow our research team to evaluate learning and use of skills (Guskey level 4). For example, this tool may help to evaluate the efficacy of neonatal ethics teaching programs and other specific teaching strategies to improve learning and use of both knowledge and skills to navigate ethically challenging clinical situations. We believe that this tool can also provide other education and research teams with a means to evaluate the success of their educational interventions and programs.

The ACT4Ethics scale addresses a need identified by accreditation bodies. With an overarching goal of contributing to and improving the means through which the milestones included in the Royal College of Physicians and Surgeons Canada 2015 CanMEDS roles of communicator, professional, and collaborator are assessed, ACT4Ethics will allow mentors and supervisors to assess and monitor competency levels of trainees. Although the validation evidence will be obtained using the context of neonatology, the creation of videos depicting typical ethically sensitive scenarios encountered in other areas in health care professions can be used to demonstrate its generalizability. We anticipate that ACT4Ethics will guide clinician-teacher supervisors, making explicit which communication skills to assess and how to rate them according to a clear range of competence. The tool can provide trainees with a clear sense of what the expectations are for their communication skills during difficult and ethically sensitive conversations. Consistent and relevant feedback is more likely to affect trainees’ learning of skillful behaviors and communication skills [59], potentially improving patients’ and parents’ satisfaction relationship with the physician-trainee.


We thank the Department of Innovation in Medical Education of the University of Ottawa and Dr Tim Wood for his help in reviewing the research protocol.

Authors' Contributions

We thank the following collaborators who participated in the development of the communication assessment tool scale: Nadya Ben Fadel, MD, Neonatology, Children’s Hospital of Eastern Ontario; Karen Fung Kee Fung, MD, Maternal-Fetal Medicine, The Ottawa Hospital; Sarah Lawrence, MD, Neonatology, Children’s Hospital of Eastern Ontario; Anna-Theresa Lobos, MD, Pediatric Intensive Care, Children’s Hospital of Eastern Ontario; Gillian MacLean, MD, Fellow, Neonatology, Children’s Hospital of Eastern Ontario; Paul Muirhead, LLM, LLB; Julie Richer, MD, Pediatric Genetics, Children’s Hospital of Eastern Ontario; and Chris Vadeboncoeur, MD, Palliative Care Medicine, Children’s Hospital of Eastern Ontario.

Conflicts of Interest

None declared.

  1. Stokes TA, Watson KL, Boss RD. Teaching antenatal counseling skills to neonatal providers. Semin Perinatol 2014 Feb;38(1):47-51. [CrossRef] [Medline]
  2. Daboval T, Moore G, Ferretti E. How we teach ethics and communication during a Canadian neonatal perinatal medicine residency: an interactive experience. Med Teach 2013;35(3):194-200. [CrossRef] [Medline]
  3. Boss RD, Urban A, Barnett MD, Arnold RM. Neonatal Critical Care Communication (NC3): training NICU physicians and nurse practitioners. J Perinatol 2013 Aug;33(8):642-646. [CrossRef] [Medline]
  4. Cuttano A, Scaramuzzo RT, Gentile M, Ciantelli M, Sigali E, Boldrini A. Education in neonatology by simulation: between reality and declaration of intent. J Matern Fetal Neonatal Med 2011 Oct;24 Suppl 1:97-98. [CrossRef] [Medline]
  5. Tongue JR, Epps HR, Forese LL. Communication skills. Instr Course Lect 2005;54:3-9. [Medline]
  6. Emanuel LL, Alpert HR, Baldwin DC, Emanuel EJ. What terminally ill patients care about: toward a validated construct of patients' perspectives. J Palliat Med 2000;3(4):419-431. [CrossRef] [Medline]
  7. Papageorgiou A. Medical & ethical issues related to extreme prematurity. J Perinat Med 2013;41(s1):121. [CrossRef]
  8. Cummings CL. Teaching and assessing ethics in the newborn ICU. Semin Perinatol 2016 Dec;40(4):261-269. [CrossRef] [Medline]
  9. Daboval T, Ferretti E, Moore G. Innovative holistic teaching in a Canadian Neonatal Perinatal Residency Program. Hastings Cent Rep 2014;44(6):21-25. [CrossRef] [Medline]
  10. Ferretti E, Rohde K, Moore G, Daboval T. Catch the moment: the power of turning mistakes into "precious" learning opportunities. Paediatr Child Heal 2018:4. [CrossRef]
  11. Frank J, Snell L, Sherbino J. Royal College of Physicians and Surgeons of Canada. 2015. CanMEDs 2015: Physician Competency Framework   URL: [accessed 2019-04-18] [WebCite Cache]
  12. Accreditation Council for Graduate Medical Education. 2016. ACGME program requirements for graduate medical education in the subspecialties of pediatrics   URL: https:/​/www.​​Portals/​0/​PFAssets/​ProgramRequirements/​329_neonatal-perinatal_medicine_2017-07-01.​pdf [accessed 2019-04-18] [WebCite Cache]
  13. Guskey T. Evaluating Professional Development. Thousand Oaks: Sage Publishing; 2000.
  14. Daboval T, Ferretti E, Moussa A, van Manen M, Moore G, Srinivasan G, et al. Needs assessment of ethics and communication teaching for neonatal perinatal medicine programs in Canada. Paediatr Child Health 2018 Aug 28:1-9. [CrossRef]
  15. Sulmasy DP, Geller G, Levine DM, Faden R. Medical house officers' knowledge, attitudes, and confidence regarding medical ethics. Arch Intern Med 1990 Dec;150(12):2509-2513. [Medline]
  16. Kesselheim JC, McMahon GT, Joffe S. Development of a Test of Residents' Ethics Knowledge for Pediatrics (TREK-P). J Grad Med Educ 2012 Jun;4(2):242-245 [FREE Full text] [CrossRef] [Medline]
  17. Cummings CL, Geis GM, Feldman HA, Berson ER, Kesselheim JC. Assessing ethics knowledge: development of a test of ethics knowledge in neonatology. J Pediatr 2018 Dec;199:57-64. [CrossRef] [Medline]
  18. Moore GP, Ferretti E, Daboval T. Developing a knowledge test for a neonatal ethics teaching program. Cureus 2017 Dec 20;9(12):e1971 [FREE Full text] [CrossRef] [Medline]
  19. Browning DM, Solomon MZ. Relational learning in pediatric palliative care: transformative education and the culture of medicine. Child Adolesc Psychiatr Clin N Am 2006 Jul;15(3):795-815. [CrossRef] [Medline]
  20. Ellman MS, Fortin AH. Benefits of teaching medical students how to communicate with patients having serious illness: comparison of two approaches to experiential, skill-based, and self-reflective learning. Yale J Biol Med 2012 Jun;85(2):261-270 [FREE Full text] [Medline]
  21. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990 Sep;65(9 Suppl):S63-S67. [Medline]
  22. Makoul G. Essential elements of communication in medical encounters: the Kalamazoo consensus statement. Acad Med 2001 Apr;76(4):390-393. [Medline]
  23. Kurtz S, Silverman J, Draper J. Teaching and Learning Communication Skills in Medicine. Oxford: Radcliffe Medical Press; 1998.
  24. Singer PA, Robb A, Cohen R, Norman G, Turnbull J. Performance-based assessment of clinical ethics using an objective structured clinical examination. Acad Med 1996 May;71(5):495-498. [Medline]
  25. Lynch DC, Surdyk PM, Eiser AR. Assessing professionalism: a review of the literature. Med Teach 2004 Jun;26(4):366-373. [CrossRef] [Medline]
  26. Baile WF, Buckman R, Lenzi R, Glober G, Beale EA, Kudelka AP. SPIKES—a six-step protocol for delivering bad news: application to the patient with cancer. Oncologist 2000;5(4):302-311 [FREE Full text] [Medline]
  27. Peterson EB, Calhoun AW, Rider EA. The reliability of a modified Kalamazoo Consensus Statement Checklist for assessing the communication skills of multidisciplinary clinicians in the simulated environment. Patient Educ Couns 2014 Sep;96(3):411-418. [CrossRef] [Medline]
  28. Calhoun AW, Rider EA, Meyer EC, Lamiani G, Truog RD. Assessment of communication skills and self-appraisal in the simulated environment: feasibility of multirater feedback with gap analysis. Simul Healthc 2009;4(1):22-29. [CrossRef] [Medline]
  29. Daboval T, Shidler S. Ethical framework for shared decision making in the neonatal intensive care unit: communicative ethics. Paediatr Child Health 2014 Jun;19(6):302-304 [FREE Full text] [Medline]
  30. Rest J, Thoma S, Edwards L. Designing and validating a measure of moral judgment: stage preference and stage consistency approaches. J Educ Psychol 1997;89(1):5-28. [CrossRef]
  31. Daboval T, Shidler S, Thomas D. Shared decision making at the limit of viability: a blueprint for physician action. PLoS One 2016;11(11):e0166151 [FREE Full text] [CrossRef] [Medline]
  32. Kurtz S, Silverman J, Benson J, Draper J. Marrying content and process in clinical method teaching: enhancing the Calgary-Cambridge guides. Acad Med 2003 Aug;78(8):802-809. [Medline]
  33. Brindley PG, Reynolds SF. Improving verbal communication in critical care medicine. J Crit Care 2011 Apr;26(2):155-159. [CrossRef] [Medline]
  34. Andersson L, Pearson C. Tit for tat? The spiraling effect of incivility in the workplace. Acad Manag Rev 1999 Jul;24(3):452-455. [CrossRef]
  35. Cortina LM, Magley VJ, Williams JH, Langhout RD. Incivility in the workplace: incidence and impact. J Occup Health Psychol 2001 Jan;6(1):64-80. [Medline]
  36. Guidroz AM, Burnfield-Geimer JL, Clark O, Schwetschenau HM, Jex SM. The nursing incivility scale: development and validation of an occupation-specific measure. J Nurs Meas 2010;18(3):176-200. [Medline]
  37. Hutton S, Gates D. Workplace incivility and productivity losses among direct care staff. AAOHN J 2008 Apr;56(4):168-175. [Medline]
  38. Papastavrou E, Efstathiou G, Tsangari H, Suhonen R, Leino-Kilpi H, Patiraki E, et al. Patients' and nurses' perceptions of respect and human presence through caring behaviours: a comparative study. Nurs Ethics 2012 May;19(3):369-379. [CrossRef] [Medline]
  39. Eslinger PJ. Neurological and neuropsychological bases of empathy. Eur Neurol 1998;39(4):193-199. [CrossRef] [Medline]
  40. Davis M. Empathy: A Social Psychological Approach. New York: Taylor & Francis; 1994.
  41. Shamay-Tsoory SG, Aharon-Peretz J. Dissociable prefrontal networks for cognitive and affective theory of mind: a lesion study. Neuropsychologia 2007 Oct 01;45(13):3054-3067. [CrossRef] [Medline]
  42. Shamay-Tsoory SG, Aharon-Peretz J, Perry D. Two systems for empathy: a double dissociation between emotional and cognitive empathy in inferior frontal gyrus versus ventromedial prefrontal lesions. Brain 2009 Mar;132(Pt 3):617-627. [CrossRef] [Medline]
  43. Endsley MR. Toward a theory of situation awareness in dynamic systems. Hum Factors 1995 Mar 01;37(1):32-64. [CrossRef]
  44. Kahneman D. Thinking, Fast and Slow. New York: Anchor Canada; 2011.
  45. Klein G. Sources of Power: How People Make Decisions. Cambridge: The MIT Press; 1999.
  46. Messick S. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am Psychol 1995;50(9):741-749. [CrossRef]
  47. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. Br Med J 1975 Feb 22;1(5955):447-451 [FREE Full text] [Medline]
  48. Creswell J, Clark V. Designing and Conducting Mixed-Methods Research. 2nd Edition. Thousand Oaks: Sage Publications; 2011.
  49. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006 Feb;119(2):166.e7-166.e16. [CrossRef] [Medline]
  50. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ 2003 Sep;37(9):830-837. [Medline]
  51. Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence: a randomized trial. Ann Intern Med 2004 Jun 01;140(11):874-881. [Medline]
  52. one45.   URL: [accessed 2019-04-18] [WebCite Cache]
  53. Stirrat GM. Education in ethics. Clin Perinatol 2003 Mar;30(1):1-15. [Medline]
  54. Janvier A, Barrington K, Farlow B. Communication with parents concerning withholding or withdrawing of life-sustaining interventions in neonatology. Semin Perinatol 2014 Feb;38(1):38-46. [CrossRef] [Medline]
  55. O'Donnell JA, Oakley M, Haney S, O'Neill PN, Taylor D. Rubrics 101: a primer for rubric development in dental education. J Dent Educ 2011 Sep;75(9):1163-1175 [FREE Full text] [Medline]
  56. Levinson W, Roter DL, Mullooly JP, Dull VT, Frankel RM. Physician-patient communication. The relationship with malpractice claims among primary care physicians and surgeons. JAMA 1997 Feb 19;277(7):553-559. [Medline]
  57. Hamstra S. Keynote address: the focus on competencies and individual learner assessment as emerging themes in medical education research. Acad Emerg Med 2012 Dec;19(12):1336-1343 [FREE Full text] [CrossRef] [Medline]
  58. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J, Knowledge Transfer Study Group. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81(2):221-248 [FREE Full text] [Medline]
  59. Allen D, Tanner K. Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE Life Sci Educ 2006;5(3):197-203 [FREE Full text] [CrossRef] [Medline]

NPM: neonatal-perinatal medicine
GKCAS: Gap-Kalamazoo Communication Skills Assessment Form
ACT4Ethics: Assessment of Communication Tool for Ethics
CHEO: Children’s Hospital of Eastern Ontario
OSCE: Objective Structured Clinical Examination

Edited by C Hoving; submitted 03.09.18; peer-reviewed by K Aikin, T Risling; comments to author 18.12.18; revised version received 02.03.19; accepted 29.03.19; published 08.05.19


©Thierry Daboval, Natalie Ward, Jordan R Schoenherr, Gregory P Moore, Caitlin Carew, Alicia Lambrinakos-Raymond, Emanuela Ferretti. Originally published in JMIR Research Protocols (, 08.05.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.