Uses Analysis

Introduction

This article presents how the uses of assessment techniques were analysed. The analysis follows the determination of the project's general and specific questions and the operationalization of the concepts used in the specific question.

As the specific question points out, elements beyond the techniques themselves are considered, including a broader understanding of what performance entails, the users who employ those techniques, and exactly what those techniques measure. Here, the focus is on identifying the uses, their chronology, antecedents, effects, and causes. Other articles and notes are mentioned for more details on each aspect being presented here.

Methodology

The analysis of the “Use of assessment technique” process was conducted following the methodology prescribed by Miles and Huberman. Observations of uses, their various and presumed effects, and causes were made across the large exploration field. The observed contrasts were identified, coded, and transcribed into matrices ordered by concepts or roles using propositions. Propositions were subsumed and grouped. Four cycles of analysis can be identified:

• Nature of uses
• Chronology of uses
• Antecedents and effects of uses
• Users’ roles
• Causal analysis

These five cycles are intertwined. The first two cycles in the chronology and nature of use allowed a first grouping into 16 themes of use. The analysis of the antecedents and effects, on the one hand, and the users’ roles, on the other, enables the examination of the causal effects.

The hypotheses and theoretical model were developed based on the last cycle of the causality analysis. In this analytical process, particularly in the last cycle of causal effects, and during phase 1, observations from the large and small fields were combined with theories. This process was continued and refined with new users, techniques, and companies during phases 2 and 3.

To conduct the analyses, the elementary propositions entered into the matrices were expressly removed from their context. The anchoring points of the observations were not detailed to ensure that the framework encompasses uses, antecedents, and effects that are possible across all types of organizational and individual situations, even if some uses, antecedents, and effects cannot be directly identified in a particular context.

Nature of Uses

The nature of uses was identified in the large field of exploration, and a coding plan was devised. The elementary propositions were listed and then transcribed into a thematic conceptual matrix. The propositions were then regrouped in themes. To help with the grouping, the effects of the uses were analyzed along three axes:

• Medium or long-term effects
• Effects on people or the organization
• Effects on oneself or others

16 themes of use were identified on the large field at the beginning of phase 1. They are the following:

Theme	Description	Code
Selection	The assessment technique is used in hiring, one-on-one recruitment, mass recruitment, executive search, and for internal mobility and promotion within the organization.	SELE
Interviewing	The assessment technique is used in various interview scenarios: annual or semi-annual performance reviews, recruitment interviews, and other interviews or informal conversations.	ENTR
Management	The assessment technique is used in all situations of individual interaction with the team, to understand team members, and to set realistic objectives. It helps identify effective solutions to support team members in developing their skills and growth.	MANA
Learning	The assessment technique is used to experience its benefits and use across numerous areas, along with the techniques, knowledge, examples, and support that accompany it. The first phase of learning helps initiate subsequent ongoing learning.	FORM
Succession planning	The assessment technique is used to evaluate a medium- or long-term need to replace someone within the organization. This person might be the company's owner or an employee approaching retirement, or a board director nearing the end of their term. It is also used to develop the individual who will replace the current person. Identifying and developing high-potential employees also falls into this category.	PLAN
Leading	The assessment technique is used to develop leadership by increasing self-awareness, adaptability, and emotional self-regulation. The information helps to rally people around a vision, optimize talent, empower others to act, and build the social skills necessary to lead people and organizations.	LEAD
Career development	The assessment technique is used to help advise on career development and assist with support, and training to achieve vocational or career goals.	CONS
Teamwork	The assessment technique is used for a team dynamic exercise, typically involving about a dozen people. The team could be the executive team, the board of directors, or a project team. This use also includes onboarding a new employee into the team.	EQUI
Conflict Resolution	The assessment technique is used to identify, resolve, and prevent conflicts. Since conflicts can be either constructive or destructive, the assessment helps understand their nature, trace their origins in how people behave, and find creative solutions.	CONF
Organizational development	The assessment technique is used for organizational applications and strategic long-term decisions at a company level or within one of its departments or subsidiaries. The decision being made usually becomes part of the organization’s rules and policies.	ORGA
Reorganization	The assessment technique is employed in mergers, acquisitions, de-mergers, spin-offs, re-engineering, and organizational transformation.	REOR
Communication	The assessment technique improves communication among organization members, as well as supporting decision-making and its use in different areas.	COMM
Self and Social Awareness	The assessment technique helps people understand themselves and others, providing advice for practical use. Individuals reflect on themselves and uncover differences they weren’t aware of, improving their relationships.	SELF
Coaching	The assessment technique is used for one-on-one sessions and team coaching. It can be applied in life or business coaching. In mentoring, it may involve not only the leader being mentored but also their team.	COAC
Clinical	The assessment technique is used to identify and address individual, relational, and organizational challenges of various kinds, including psychological disorders.	CLIN
Medium Effect	The assessment technique helps mediate, facilitate, and entertain a conversation while establishing trust with the person. It is employed in group presentations, which can include small or large audiences of several hundred people.	ENTR

At this stage, other uses of assessment techniques for purposes not directly connected with the specific question and organizational performance were dropped. They include the following: understanding consumer purchasing habits, approaches to medical treatments, an individual's political beliefs, or their willingness to join a group, such as but not limited to religious organizations. Other uses include matchmaking, parenting, social work, family therapy, marriage counseling, and psychiatric treatments.

Chronological Analysis

The chronological analysis has focused on two processes: first, the use process in general, as it appears across all assessment techniques, including private techniques, statistics-based techniques, and GRI’s adaptive profiles; and second, the learning process.

The initial training varies depending on the techniques and users. With techniques based on typologies and traits, consultants, coaches, and HR experts usually undergo training that provides additional knowledge of the assessment technique. Clinicians receive formal training in clinical psychology and assessment techniques. Leaders and Managers usually only get briefed on those techniques. The skills acquired during the initial training are enabled and dependent on the assessment capabilities.

While learning assessments from reports and trait measures is immediate, especially when they use plain language, training becomes increasingly important when assessment results employ language and symbols that convey strategic implications. During training, working on real cases and facilitation are crucial for acquiring new social skills. Learning continues after training through the regular application of the assessment technique.

Antecedent Analysis

Once the major themes of use were identified, the antecedents and effects of use were analyzed and reported in matrices. A first coding plan was defined. This analysis began to provide explanations for the specific question: Why did the uses appear? What were the causes of use, either general or specific? What are the impacts on people and the environment? As with the analysis of uses, the propositions were identified by contrasting and comparing them in the large field of exploration. This process was refined in phases two and three of the project, with new techniques, users, and companies being considered.

The main qualities of the antecedents and effects were emphasized in all the cases studied. Similar qualities were evident in several companies to varying degrees. With companies using and being trained on adaptive profiles, observations began with people's first contact with them and continued during their use at irregular intervals. This process was refined over the three phases of the project. The four antecedent variables are the following:

• Assessment technique. 21 different types of assessment techniques were identified, regrouped in four categories: parallel techniques, semi-formal techniques, formal techniques, and statistics-based techniques. 158 statistics-based techniques were analyzed, which allowed the extraction of 29 characteristics that can apply in large part to other techniques, too. Those characteristics were subsumed into four groups:
  • Upfront characteristics or how the technique functions when participants take the assessment.
  • Intrinsic qualities of the measures being assessed.
  • Assessment results, and quality of the results for use in various applications.
  • Theory and manual, and the rules of inference produced by using and sharing about the assessment technique.

• Users. 14 characteristics have been identified, including: attitude toward answering the assessment, attitude toward people’s behavior, attitude toward the assessment technique, attitude toward the facilitator, interest in human aspects, operational vs functional role, management experience, adaptive profile of the decision maker, experience with either assessment techniques, problems encountered in previous management situations, directly concerned or not by the effects, confidence in using the personality assessment, recommendation provided for using the assessment, experience in HR, OD or Coaching.

• Environment. 12 characteristics were identified, with 10 from the organization level using the assessment and 2 from the environment at large. For the organization, the characteristics evidenced were the size of the company, its maturity, its industry sector, the cultural significance of the personality test in the organization, the politicization of people aspects, the use of other techniques, prior use in the organization, the role of human resources when the service exists, the relationship between the organization and outside consultants and the internationalization of the organization. The characteristics of the environment at large were highlighted in nine countries considered: the environmental competitiveness and the more or less strong development of assessments at the national or state level.

• Publisher and Consultant. 9 characteristics. The publisher, facilitator, and consultant play important roles in how the assessment technique is deployed in an organization. Aspects such as the support provided, the training delivered, the availability of a survey and analytic platform, and the way assessment and services are packaged are important factors that determine how the technique will be used.

Effects Analysis

The analysis of the effects made it possible to consider 10 distinct effects in addition to the first three already mentioned above for the first phase of analyzing the uses:

Effect	Description	Code 7
Improvement in the 16 themes of use	Users of the assessment technique and others who benefit from it can demonstrate improvements across the 16 themes of use.	AMTH
Sensitivity to people's differences	The use of the assessment technique increases the sensitivity and interest in better understanding people and their differences.	SEAH
Self-confidence and trust	Self-confidence, psychological safety, and trust among people improve.	COAU
Decision-making, prescriptions, and communication	The measures positively affect the decision-making, prescriptions, and communication.	DEPR
Confidence in the assessment	The more people use the assessment technique, the more their confidence in their capabilities increases.	COTE
Interest from other leaders and managers	The use of the assessment technique by some prompts others to become literate in its use and language as well.	FRAM
Recommendations to other people	Users recommend the use of the assessment technique to others.	REAP
Increased meaning of the symbols coming from the assessment	The symbols resulting from the assessment techniques are increasingly charged with meaning by their users.	ACMS
Development in the organization	The use of the assessment technique is growing across other services, at various hierarchical levels, and in other applications.	DEOR
Performance impact	Over time, performance indicators (KPR, KPI) are positively affected by using the assessment technique.	IMPE
Fit between jobs and the occupant's effective behaviors	Over time, the Effective behaviors of the adaptive profiles get closer to the PBI profiles of the jobs.	HOPS
Engagement and minimal behavior adaptation to jobs	Over time, engagement levels from the adaptive profiles increase. The Role profiles get closer to the Natural profiles.	HOCS
Fit between the characteristics of jobs and the company's general policy.	Compensation policies, internal promotion plans, career paths, and training are aligned with the job profiles, as can be seen in the occupants' Natural profiles.	COPG

Role analysis

Five user roles were identified during the first phase of the project: leaders and managers (analyzed separately as trained and untrained), human resources executives, recruiters, consultants, and facilitators. Four additional roles were added later in phases 2 and 3 as observations progressed and new users and applications emerged: career counselor, coach, clinician, and individual contributor.

Seven other roles were identified but not included in the analysis, even though they were important, because they lacked direct relevance to organizational performance. These roles are the following: clinician - other, researcher, medical doctor, matchmaker, marketer, parent-educator, and economist. The main user characteristics and their impact on performance were summarized in a matrix.

Causal Analysis

The specific research question guides the analysis of the reasons underlying the link between the use of the assessment technique and performance. This work was carried out alongside the development of the hypotheses. It is possible to clearly order the antecedent variables, the independent variables of use, the variables related to their effects, and to envision cause-and-effect relationships over time.

The 16 uses were initially reduced to nine using the role, antecedent, and effect analyses. The Self and Social Awareness was extracted from Leadership. Leadership and Management variables were later merged. The Coach, Clinical, and Medium Effect variables were included in phases 2 and 3 of the project. The nine intermediate variables were grouped into two categories.

• The first category comprises the uses in “Organizational development”, “Recruitment”, “Leadership”, “Coaching”, “Clinical”, and “Medium Effect”, which are of practical use.
• The second category includes the uses in “Self and social awareness”, “Learning”, and “Language and Signs”, which are more abstract and theoretical and transversal to the other six practical uses.

The antecedent variable “Organization” was merged with “Environment”. There are, therefore, four antecedent variables: “Assessment Technique,” “Individual user,” “Environment,” and “Publisher and consultant”. In parallel with the work to reduce the 16 uses, causal diagrams were constructed for the “Learning” and “Language and Signs” variables as well as for the role analysis.

The following two aspects of the adaptive profiles apply to measuring individual and organizational performance:

• Disengagement: A strong disengagement, as measured in the Natural and Role of the adaptive profiles, will result in poor performance.
• Lack of Job fit: A disconnect between the social behaviors expected in jobs and how people actually perform in them, as measured by the job profile (or PBI) and the Effective of the adaptive profile, will ultimately create inefficiency

Those effects need to be considered as “effects of effects”. The impact of engagement and role fit is more distant in time than that of the other effects. All these effects are inevitably connected, as they result from the implementation of the assessment technique, whether it be the assessment and the adaptive profiles, or any other assessment technique, including parallel techniques.

Those specific effects on performance are therefore at the end of the causal chain, as variables dependent on the uses and linked by the causal chain to the antecedent variables of the assessment technique, individual users, the environment, and the publisher and consultant’s business model. The other types of performance measured by KPR, KPIs, production, and financial indicators also occur in parallel at the end of the causal chain. They result from individual and organizational performance.

The techniques for constructing a logical chain and achieving conceptual and theoretical coherence, advocated by Miles and Huberman were applied to develop the hypotheses and the theoretical framework. This work was gradually refined by going back and forth between the large exploration field and the matrices, by the two testing sites in phase 1, and later by enlarging the exploration field and including new users and uses during phases 2 and 3.