Uses Analysis

Introduction

The analysis of the “Use of assessment technique” process was conducted following Miles and Huberman’s recommendations. Observations of uses, their various and presumed effects, and causes were made across the large exploration field. The observed contrasts were identified, coded, and transcribed into matrices ordered by concepts or roles using propositions. Propositions were subsumed and grouped. Four cycles of analysis can be identified:

• Nature and chronology of the uses
• Antecedents and effects of the uses
• Users’ roles
• Causalities

These four cycles are intertwined. The first cycle on the uses chronology and nature allowed a first grouping into 16 themes of use. The analysis of the antecedents and effects on the one hand and the users’ roles on the other hand makes it possible to analyze the causalities.

The hypotheses and the theoretical model were built based on the last cycle of causality analysis. In this analytical process, and particularly in the last cycle of causalities, the observations made in the large and small fields were mixed with theories.

To conduct the analyses, the elementary propositions entered into the matrices were expressly removed from their context. The anchoring points of the observed phenomena were not detailed to ensure that the analytical framework comprises uses, antecedents, and effects that are possible in all types of organizational and individual situations, even if some of the uses, antecedents, and effects cannot be directly identified in a particular context.

Chronological Analysis and the Nature of Uses

The chronological analysis focused on two processes: on the one hand, the process of use in general, as it appears with all assessment techniques, including private techniques, statistics-based techniques, and GRI’s adaptive profiles, and on the other hand, the learning process. It emerges from observations that although the learning from reports and trait measures is immediate, especially when those use lay language, the training is all the more important when the assessment results use language and symbols that concentrate expert meaning in them. During the training, the interactions between participants and the facilitator are critical for the acquisition of the new skills. The learning continues after the training, during implementation with regular use of the assessment technique.

The initial training varies depending on the techniques and users. Consultants and coaches trained in typologies usually go through trainings that provide additional knowledge of the assessment. Clinicians go through formal training in clinical psychology and clinical assessment techniques as well. In any case, the skills acquired during the initial training are enabled and dependent on the assessment capabilities. The same rule applies to leaders and managers being trained.

The natures of the uses were identified in the large field of exploration, and a coding plan was devised. The elementary propositions were listed and then transcribed into a thematic conceptual matrix. The propositions were then regrouped in themes. To help with the grouping, the effects of the uses were analyzed along three axes:

• Medium or long-term effects
• Effects on people or the organization
• Effects on oneself or others

16 themes of use were identified at this first stage, which are the following:

Theme	Meaning	Code
Selection	The assessment technique is used in hiring, one-on-one recruitment, mass recruitment, executive search, and for internal mobility and promotion within the organization.	SELE
Interviewing	The assessment technique is used in various interview scenarios: annual or semi-annual performance reviews, recruitment interviews, and other interviews or informal conversations.	ENTR
Management	The assessment technique is used in all situations of individual interaction with the team, to understand team members, and to set realistic objectives. It helps identify effective solutions to support team members in developing their skills and growth.	MANA
Learning	The assessment technique is used to experience its benefits and use across numerous areas, along with the techniques, knowledge, examples, and support that accompany it. The first phase of learning helps initiate subsequent ongoing learning.	FORM
Succession planning	The assessment technique is used to evaluate a medium- or long-term need to replace someone within the organization. This person might be the company's owner or an employee approaching retirement, or a board director nearing the end of their term. It is also used to develop the individual who will replace the current person. Identifying and developing high-potential employees also falls into this category.	PLAN
Leading	The assessment technique is used to develop leadership by increasing self-awareness, adaptability, and emotional self-regulation. The information helps to rally people around a vision, optimize talent, empower others to act, and build the social skills necessary to lead people and organizations.	LEAD
Career development	The assessment technique is used to help advise on career development and assist with support, and training to achieve vocational or career goals.	CONS
Teamwork	The assessment technique is used for a team dynamic exercise, typically involving about a dozen people. The team could be the executive team, the board of directors, or a project team. This use also includes onboarding a new employee into the team.	EQUI
Conflict Resolution	The assessment technique is used to identify, resolve, and prevent conflicts. Since conflicts can be either constructive or destructive, the assessment helps understand their nature, trace their origins in how people behave, and find creative solutions.	CONF
Organizational development	The assessment technique is used for organizational applications and strategic long-term decisions at a company level or within one of its departments or subsidiaries. The decision being made usually becomes part of the organization’s rules and policies.	ORGA
Reorganization	The assessment technique is employed in mergers, acquisitions, de-mergers, spin-offs, re-engineering, and organizational transformation.	REOR
Communication	The assessment technique improves communication among organization members, as well as supporting decision-making and its use in different areas.	COMM
Self and Social Awareness	The assessment technique helps people understand themselves and others, providing advice for practical use. Individuals reflect on themselves and uncover differences they weren’t aware of, improving their relationships.	SELF
Coaching	The assessment technique is used for one-on-one sessions and team coaching. It can be applied in life or business coaching. In mentoring, it may involve not only the leader being mentored but also their team.	COAC
Clinical	The assessment technique is used to identify and address individual, relational, and organizational challenges of various kinds, including psychological disorders.	CLIN
Medium Effect	The assessment technique helps mediate, facilitate, and entertain a conversation while establishing trust with the person. It is employed in group presentations, which can include small or large audiences of several hundred people.	ENTR

At this stage, other uses of assessment techniques for purposes not directly connected with the specific question and organizational performance were dropped. They include the following: understanding consumer purchasing habits, approaches to medical treatments, an individual's political beliefs, or their willingness to join a group, such as but not limited to religious organizations. Other uses include matchmaking, parenting, social work, family therapy, marriage counseling, and psychiatric treatments.

Antecedent Analysis

In a second cycle of analysis, after the major themes of use were identified, the antecedents and effects of use were analyzed and reported in matrices. A coding plan was adopted. This analysis starts to provide explanations to the questions: Why did the uses appear? What was the cause, general or specific? What are the impacts on people and the environment? As with the analysis of uses, the propositions could be identified by contrasts and comparisons in the large field of exploration.

The salient characteristics of the antecedents and effects were highlighted in all the cases studied. Some characteristics are found on several sites, in a more or less pronounced way. Observations conducted in the field began with people's first contact with the assessment technique and continued throughout the use of the technique at irregularly spaced time intervals. The four antecedents to the uses identified at this stage are the following:

• Assessment technique. 21 different types of assessment techniques were identified, regrouped in four categories: parallel techniques, semi-formal techniques, formal techniques, and statistics-based techniques. 158 statistics-based techniques were analyzed, which allowed the extraction of 29 characteristics that can apply in large part to other techniques, too. Those characteristics were subsumed into four groups:
  • Upfront Characteristics: (6 characteristics) Assessment techniques function differently from a user standpoint, upfront, when participants are being assessed. For instance, some techniques are easy to answer and instantaneous, while others are more complex and take time.
  • Intrinsic Qualities: (12 characteristics) They are the qualities of the measures being assessed. They include the nature of the concept(s), what the assessment says it assesses, and how the assessment works internally, eventually, for more advanced techniques, by using the statistics to refine the quality of the measures being produced.
  • Assessment Results: (5 characteristics) These are the quality of the results being produced for their various applications. The potential use of these qualities varies depending on users and the practical benefits shared among them.
  • Theory and Manual: (6 characteristics) These are the rules of inference produced by using and sharing about the assessment technique, its use, added values, and benefits. All the rules developed by experiencing the technique constitute the assessment’s theory. They vary in depth and quantity between techniques. They may differ from the theory that the instrument originated from.

• Individuals. 11 characteristics have been identified, including: attitude toward answering the assessment, attitude toward people’s behavior, attitude toward the assessment technique, attitude toward the facilitator, interest in human aspects, operational vs functional role, management experience, adaptive profile of the decision maker, experience with either assessment techniques, problems encountered in previous management situations, directly concerned or not by the effects.

• Environment. 12 characteristics were evidenced, with 10 of them from the organization level using the assessment and two from the environment at large. For the organization, the characteristics evidenced were the size of the company, its maturity, its industry sector, the cultural significance of the personality test in the organization, the politicization of people aspects, the use of other techniques, prior use in the organization, the role of human resources when the service exists, the relationship between the organization and outside consultants and the internationalization of the organization. The characteristics of the environment at large were highlighted in nine countries considered: the environmental competitiveness and the more or less strong development of assessments at the national or state level.

• Publisher. This variable was isolated because of the particular role played by the consultant in starting and supporting the use. The importance of the impetus given for use in human resources or management, the way of packaging the services or even of accompanying the users or not plays a determining role for the uses of the personality test.

Effects Analysis

The analysis of the effects made it possible to consider 10 distinct effects in addition to the first three already mentioned above for the first phase of analyzing the uses:

Characteristic	Effect	Code 7
Improvement in the 16 themes of use	Users of the assessment technique and others who benefit from it can observe the improvements on the 16 themes of use.	AMTH
Sensitivity to people's differences	The use of the assessment technique increases the sensitivity and interest in better understanding people and their differences.	SEAH
Self-confidence and trust	Self-confidence, psychological safety and trust between people are improved.	COAU
Decisions, prescriptions and communication	The measures positively affect the decisions, prescriptions, and communication.	DEPR
Confidence in the assessment	The more people use the assessment technique, the more their confidence in their capabilities increases.	COTE
Frustration of other managers	The use of the assessment technique by some stimulates others to be literate in its use and language too.	FRAM
Recommendations to other people	Users recommend the use of the assessment technique to others.	REAP
Increased meaning of the symbols coming from the assessment	The symbols resulting from the assessment techniques are increasingly charged with meaning by their users.	ACMS
Development in the organization	The use of the assessment technique grows in other services, at various hierarchical levels, or in other applications.	DEOR
Performance impact	Over time, performance indicators (KPR, KPI) are positively affected by using the assessment technique.	IMPE
Fit between jobs and the occupant's effective behaviors	Over time, the Effective behaviors of the adaptive profiles get closer to the PBI profiles of the jobs.	HOPS
Engagement and minimal behavior adaptation to jobs	Over time, engagement levels of the adaptive profiles get higher. The Role profiles get closer to the Natural profiles.	HOCS
Fit between the characteristics of jobs and the company's general policy.	Compensation policies, internal promotion plans, career paths, and training are in line with what the job profiles require, and as can be read in occupants' Natural profiles too.	COPG

Role analysis

Nine groups of users were identified: trained leaders, untrained leaders, human resources directors, recruiters, employees, consultants, facilitators, coaches, and clinicians. The roles of each were analyzed in the cycle of causal analysis discussed below.

Causal Analysis

The specific research question directs the analysis of the reasons that can explain the link between the use of the assessment technique and performance. This work was carried out alongside the development of the hypotheses. It is possible to clearly order the antecedent variables, the independent variables of use, the variables related to their effects, and to envision cause-and-effect relationships over time.

The 16 uses were initially reduced to nine using the role, antecedent, and effect analyses. The Self and Social Awareness was extracted from Leadership. Leadership and Management variables were later merged. The Coach, Clinical, and Medium effect variables came with the new phase of the research. The nine intermediate variables were grouped into two categories.

• The first group comprises the uses in “Organizational development”, “Recruitment”, “Leadership”, “Coaching”, “Clinical”, and “Medium Effect”, which are of practical use.
• The second group includes the uses in “Self and social awareness”, “Learning”, and “Language”, which are more abstract and theoretical and transversal to the other six practical uses.

The antecedent variable “Organization” was merged with “Environment”. There are therefore four antecedent variables: “Assessment Technique,” “Individual user,” “Environment,” and "Publisher and consultant”.

In parallel with the work of reducing the 16 uses, causal diagrams were constructed for the “Learning” and “Language” variables as well as for the role analysis.

The following two aspects of GRI’s adaptive profiles apply to measuring individual and organizational performance:

• Disengagement: A strong adaptation and disengagement, as measured in the Natural and Role of the adaptive profiles, will result in inefficiency and poor performance.
• Lack of Job fit: A disconnect between the behaviors expected in jobs and how people indeed perform in them, as measured with the job profile (or PBI) and the Effective of the adaptive profile, will create disengagement, and consequently inefficiency and poor performance.

The two above effects need to be considered as effects of effects. The impact of the disengagement and lack of role fit is more distant in time than the other effects. All these effects are inevitably connected since they all result from the implementation of the assessment technique, whether it be GRI’s assessment and the adaptive profiles, or any other assessment technique, including parallel techniques. Those specific effects on performance are therefore at the end of the causal chain, as a variable dependent on the uses, and linked by the causal chain to the antecedent variables of the assessment technique, individual users, the environment, and the publisher and consultant’s business model.

The other types of performances measured by KPR, KPIs, production, and financial indicators come in parallel at the end of the causal chain as well. They result from individual and organizational performance.

The techniques of constructing a logical chain and achieving conceptual and theoretical coherence, advocated by Miles and Huberman were applied to build the hypotheses and the theoretical framework. This work was gradually refined by going back and forth between the large exploration field and the matrices, but also later with the testing sites that include the case studies.