Does LEADS Canada have guidelines for selecting responders?
Yes – Responder selection guidelines (the responder categories approved for your organization) are sent to each 360 participant. These guidelines also appear in the “PLEASE ENTER YOUR RESPONDERS” LEADS 360 system email that participants receive asking them to go online to enter their responders.
Does LEADS Canada enter the responders into the online system?
No – LEADS 360 participants (subjects of the assessment) are given complete instructions through the automated system regarding how to enter their responders into the LEADS 360 online system. They have one week in which to do so, followed by another week in which your organization will follow up with those who have not completed their responder entry by the stated deadline. A responder entry status report will be provided after the stated responder entry completion deadline, showing which participants have, and have not, entered their responders.
How and why does the online system merge some responder categories in the final report?
To protect confidentiality of the responders, when there are fewer than 3 completed responses in any responder category (other than “Supervisor“), the responses will be merged with another responder category in the final report.
It is not always the same responder categories that are merged. For each Subject different categories could be merged based on the number of responses in each responder category.
Here is how the merging happens in general:
- When there are only 1-2 responders in a responder category, the system will look for another category to merge it with. The system will look for a category with the lowest number of responses, and merge these two categories together.
- If, after merging of two categories, there are still not enough responses, the system will look for a third category to merge with.
- If after all the merging there still aren’t enough responses, the system will omit this category with 1-2 responses from the report.
For example:
At the end of the 360 process, “Fred” has 3 responses in the DIRECT REPORTS category and only 2 in the PEERS category. In this case, PEERS will merge with DIRECT REPORTS to maintain confidentiality.
“Nancy” has 5 completed PEER responses but only 1 OTHER and 2 DIRECT REPORTS. In this case, the OTHER and DIRECT REPORT categories will merge, and PEERS will stay unmerged.
How long should an organization wait before repeating a 360 project with the same subjects?
If you look at the literature, repeat 360s can be conducted from 10 – 18 months after the first one was completed. What seems to be the key deciding factor is the length of the program the 360 assesses are engaged in. If they are engaged in a leadership development program where they will be receiving training over 12 months, then it would not be advisable to the repeat 360 until another 6 months after that. People need time to integrate their learning, own it and have it start showing in their behaviours – which is what the LEADS 360 is measuring ‘observed behaviours’. LEADS 360 participants will need time to grow from their training and experience. Another element that needs to be considered is their learning plan. When it was set up, what were the expectations? Is it expected that the individual will have achieved measurable results in 6, 8, 10, 12, 18 … months? This is a key aspect of the learning plan and the accountability contract. The 360 repeat will be of little value if the individual has not completed the goals they set for themselves as part of their learning plan and accountability contract.
Responder selection for repeat assessments:
The raters should only be the same as those in the previous assessment if they are still in a position to see the individual being assessed in action. If the individual being assessed has been promoted, is now in a different position etc., then the raters should be new ones. The key is the raters must know the individual and have worked with him or her for some time – otherwise they cannot assess observable behaviours.
Can you run a report that will compare the results of our current 360 project to prior year results for the same cohort?
There are a number of things to consider here. While it is possible to run such a report, it is not recommended for the following reasons:
- A 360 is a snapshot in time and the results will need to be considered in the context of what is going on in the organization, how a role has changed and what is going on for the participants and respondents. It is about how leadership behaviour is interpreted by others at that point in time.
- When the respondents are different from one year to the other, responses might be different. Differences in the report might reflect a difference in respondents as well as development differences.
- The discussion is most fruitful when the debrief takes place in the context of leadership development goals, the contextual changes (often a lot can shift in the organization in a year) and individual growth and awareness.
- Sometimes, when the 360 is taken again after a big emphasis on leader development in the organization, the participants’ understanding and expectation of leaders grows as well. This might be seen in the 360 and can lead to very rich observations and feedback.
- Given the exposure to Leadership practices and the LEADS Framework may even set up higher expectations (for those contributing to the assessment).
- Additionally, at the individual level, the participants need to realize that the contributors may or may not be the same as in the earlier year and that even if they were the same people; these people have changed themselves as well as the organizational (context).
- The nature of the coaching debriefs may be affected, as well the coach’s interpretation of the two reports separately and together. Might be hard to have this type of conversation in 30 minutes. It would be important for the selected coaches to have a mini meeting to review the nature of the reports and the processes to have an effective debrief.
Can the LEADS 360 assessment be utilized as a performance management tool?
Best practices with the use of 360s make it clear that 360 assessments should be used for future development planning, not for performance assessment. A number of studies have shown the risks of using 360s for performance assessment, and there are few, if any, 360s designed for performance review purposes.
Further, the LEADS 360 questions measure frequency, not effectiveness, of behaviour, so interpreting results as performance assessment would be incorrect.
The LEADS 360 assessment can certainly be considered as a valuable part of performance management, however the tool is not designed for that purpose.
Supervisor 1, 2 and 3 categories – in the report that is generated are all three supervisors scoring reported separately and if so – couldn’t that set up the supervisors up to be at odds should they not agree with that employee’s capabilities?
In any 360 leadership feedback assessment, the value of the feedback is in understanding how behaviour is interpreted and observed by others. There may indeed be differences between raters, and the interpretation of these differences is what brings out the depth and richness of a 360. Different supervisors may observe different aspects of leadership behaviour more frequently and may have different interpretations of that behaviour. Interpreting the different frequency scores (which is different from quality scores) and comments and thinking about the context of an individual’s work and personal development, knowledge and skills will lead to insights that might be very different than what a single (agreed upon) score would give you.
360’s are also a great opportunity to discuss differing expectations between supervisors (should they exist). They might also show areas of agreement and can acknowledge strengths.
One of the participants received a 360 report with disrespectful language, should we take disciplinary action?
Participant Wellness, Preparing Respondents and Confidentiality:
Our first concern is always with the 360 participant, and making sure that the coach helps him/her interpret the results. This is why we recommend that coaches always check for language before sending the report, this helps them interpret the report. Coaches may decide to have a conversation with the participant prior to sending the individual the report.
Our second concern is about protecting the integrity of the 360 process and ensuring that the feedback provided is confidential. Disciplining a respondent for providing confidential feedback would break this principle. It is likely in a situation like this, that the use of disrespectful language is but one expression of the responder’s attitude and behaviour, and it is important to protect the integrity and confidentiality of the 360 process. The organization may want to focus instead on discipline in other areas of the responder’s behaviour.
It is critical to ensure that responders understand that the nature of feedback in the 360 is for leadership development, and that the expectation is that respondents be honest, direct and respectful. Instructions to responders for all LEADS 360’s have been updated to reflect this expectation.
Are there any IT issues that may interfere with our participants’ and responders’ ability to complete the assessments?
In January of 2016 MicroSoft stopped supporting older versions of Internet Explorer. If a participant or responder has an older version of Internet Explorer (before IE 11) and/or does not have an updated operating system on their computer (Windows 7, Windows 8.1 or Windows 10) it can cause problems with the system interface to the LEADS 360 platform.
The Canadian College of Health Leaders (CCHL) has some support documents that can be provided to your IT department(s) that will assist in ensuring that LEADS 360 emails are not caught in your organization’s spam filters. Please provide CCHL with contact information for your IT department.
If no other solution can be found for an individual responder, CCHL can provide a pdf of the questionnaire for the responder to complete manually. The responder would then scan and return the completed questionnaire to CCHL for input into the online system.
Who receives the ‘undeliverable’ emails? And who provides the corrected email addresses?
The subject who enters the responder email address into the online system will receive any ‘bounce back’ emails, returned as ‘undeliverable’ because the email address was entered incorrectly. It is up to the LEADS 360 participant (subject) to correct the incorrectly entered responder email address.
How is the confidentiality of subjects and responders protected?
The raw data of the LEADS 360 assessment remains on the secure (Canadian) server of our IT partner Panoramic Feedback indefinitely, allowing the Canadian College of Health Leaders (CCHL) to re-generate and send the 360 final report upon request should the subject require a copy at a later date. The raw data may be used to pull trends and statistics.
Access to this secure database is limited to a small number of CCHL staff who have each signed a confidentiality agreement.
CCHL administrative staff may support activities such as the mailing of links and the Final Report to the subject or the certified debriefer. No hard copies or pdfs of the LEADS 360 Final Reports are stored by CCHL.
The responders’ numerical responses are anonymous because they are averaged with the responses of other people into a final report, which is provided in pdf format to the subject and the subject’s debriefing coach only. A ‘Learning Plan’ outlining one or more areas of focus for development may be shared with the organization (the debriefing coach will assist the subject with creation of the Learning Plan during the debriefing process).
Text comments entered by responders will appear verbatim in the final report, however the responder will not be identified as the source of these comments (unless he or she is the subject’s SUPERVISOR – Supervisor feedback is the only feedback that, by virtue of the reporting relationship, is not anonymous).
Subjects are cautioned not to self-identify by providing details of specific interactions with the subject, or stating their title or the capacity in which they work with the subject of the assessment. The responses of the subject (for his or her self-assessment) and those of all responders are confidential, as is the final LEADS 360 Report.
Why is a debriefing necessary (and a requirement for the purchase of LEADS 360 licenses)?
While administering the inventory and collecting the data is often seen as the major part of the 360 assessment project, it’s the meaning that participants make of the data that is most significant. The 360 report can be up to 40 pages long, and includes more than 500 discrete quantitative data points, and often 30 or more qualitative responses. This is a daunting collection of information to analyze, understand and transform into a meaningful learning plan.
The purpose of the debrief is to help the participant make sense of all the data in a coherent way, comparing the data with their experience and translating their understanding into a learning plan that captures the essence of the feedback, plus the learning needs and aspirations of the participant. Our experience is that this debriefing process is best supported by a Certified Executive Coach who is knowledgeable about the LEADS in a Caring Environment leadership capabilities framework and experienced in fairly sophisticated analysis of 360 data and translation of all this data into a doable plan. Oftentimes the coach must also support the participant to make sense of data that is a surprise to them or that they do not agree with, and turn this data into a meaningful forward plan.
During the debriefing, coaches discuss communication of the learning plan and any follow up clarification of data that might be necessary. Again, best practice literature shows that the actual data of the report is best kept confidential to the participant. What is recommended, however, is wide sharing of the learning plan with the participant’s supervisor, peers and direct reports. This engagement of respondents in knowing the plan and inviting further feedback on activation of the plan achieves the best results for the entire process. CCHL agrees with best practice literature that the actual report belongs to the participant and is not shared with anyone else. While managers may feel their own need to see the data, the value in the process is in the conversation that the manager has with the participant about their plan, the needs of the unit and how the individual and organizational development needs will be met by the plan.
We have people with coaching experience within our organization. Can they do the debriefings?
Individuals doing LEADS 360 assessment debriefings must be ICF certified as Executive Coaches, and have completed the two LEADS 360 Coach Certification webinars. If the organization has internal coaches who have completed the LEADS 360 coach certification webinars, they are qualified to do the debriefings. This should be outlined in the LEADS 360 contract.
