Operationalization of Concepts

Introduction

This article discusses the concepts that need to be operationalized for answering the specific question:

'How does the use of assessment techniques by their users impact an organization’s performance?'

“Use”, “Assessment techniques”, “Users,” and performance need to be defined before being analyzed, and before a framework can be drawn. The methodology used to narrow down a general question to a specific question is explained in another article. The specific question used during the project’s Phase 1 focused on the use of personality assessment by leaders and managers and its impact on performance. The new specific question extends to other techniques, concepts, and users. The dependent variable: organizational performance, remains the same. It is operationalized similarly to Phase 1, but with the new adaptive profiles being refined and coming from the GRI survey.

Performance

The general framework was originally driven by discussions on performance at an individual and organizational level. Typical performance assessments rely on economic performance, that is, financial and production performance. But years of research on the topic suggested otherwise: human efficiency from an emotional standpoint, preferences, and values need to be taken into account. Social behavior plays a central role.

Organizational Performance is discussed at large from different perspectives in this article here, followed by how individual and organizational performance are conceptualized and measured with the adaptive profiles. By measuring and representing the behaviors expected in jobs and the adaptations needed by individuals, we can evidence three groups of performance indicators: economic, strategic, and social.

• The strategic performance indicator characterizes the ability of the organization to achieve its objectives in the manner expected by its leadership.
• The social performance indicator measures employees’ adaptation efforts to their position in the organization.
• The economic indicators cover financial, production, sales, and other indicators not covered by the other two indicators.

Assessments Techniques

The assessment techniques used in organizations are numerous. Although Phase 1 of the project focused on personality assessments, or more precisely—since personality is a broad concept—on social behavior assessments, other assessments were progressively studied that needed to be contrasted with each other. The assessment techniques are listed below. Code 6a is used to code assessment techniques when analyzing case studies.

Category	Technique	Definition	Code 6a
Parallel Technique	Private technique	Anyone makes inferences and constructs judgments without any formal process or detailed statistics. These techniques may eventually be shared.	T-PRIV
	Common technique		T-COMS
	Esoteric technique		T-ESOT
Semi-formal Technique	Non-directive interview	Organizations often use semi-formal techniques. They don't require a specific process or statistics, although they might benefit from both when the amount of results being gathered is large enough.	T-NDIN
	Semi-directive interview		T-SDIN
	Reference check		T-REFC
	On-the-job-trial		T-OTJT
	Resume		T-RESU
	Biodata		T-BIOD
Formal Technique	Directive interview	These techniques are formally implemented in organizations, often involving a job analysis, a thorough process, and trained assessors.They may also use statistics to validate the results . Assessment centers combine other techniques, including those from the other three groups.	T-DINT
	Structured interview		T-SINT
	Work-sample test		T-WORT
	Job simulation		T-JOBS
	In-basket exercise		T-IBSK
	Assessment Center		T-ASSC
Statistics-based Technique	360-degree assessment	These techniques rely on statistics to analyze and verify the results. They focus on specific individual characteristics and are often combined with other assessment techniques.	T-360D
	Personality assessment		T-PERS
	Intelligence assessment		T-INTL
	Skills assessment		T-SKIL
	Technical test		T-TEST
	Social Media Scan		T-SOCS

Parallel techniques are probably the most used techniques and need to be taken into account as well in the framework for that reason. Their uses have an impact on performance, too. Statistic-based techniques compete with them. The various techniques are detailed and discussed in this other article here.

Most techniques have been researched by numerous studies. They can be classified according to characteristics such as the time it takes to assess, the quality of what is measured, the means of administration (phone, laptop, etc.), the availability of manuals, etc. Assessment techniques can all be subjected to statistical and qualitative analyses. Since Phase 1, this project has allowed us to unveil the characteristics that differentiate the assessment techniques with regard to their use, users, and effect on performance. They are presented in this other article here.

Concepts Measured by the Techniques

Although the specific question doesn’t mention it, techniques always assess something. The project initially focused on social behaviors and their assessment by statistics-based techniques, but needed to expand for the same reason as for the techniques, to be able to contrast and better include in the framework concepts, whose content and scopes compete with each other.

The concept of social behavior served as a common thread in our framework for the analysis and conceptualization of strategic and social performance. It is represented and measured by the adaptive profiles representing relatively stable behaviors, adaptation efforts, and engagement, allowing for understanding people’s performance in context. Other concepts offer important insights that are eventually more easily accessible, measurable, and less controversial and competitive than those of social behaviors.

Users

The nine considered roles are the following. They are detailed below: Leader, HR Executive, Recruiter, Facilitator, Consultant, Career Counselor, Coach, Clinician, and Individual Contributor. Code 7 is used to code users in the large exploration field and in case studies. Mentor was regrouped under Coach, Search under Recruitment, and Career Counseling under Consulting. Without ignoring the differences between those roles, their use and impact on people and the organization are nevertheless similar.

User	Highlights	Code 7
Leader	Executive, director, business owner, entrepreneur, team leader, scrum master, in charge of a team or organization. It is important to differentiate trained, not trained, or half-trained leaders from the assessment technique (see below).	LEDR
HR Executive	HR Director, HR and OD experts. Provide advice to leaders about people and their organization. Assist with conflicts. Develop more expert skills in using the assessment technique. Use their skills on their HR team, too.	HREX
Recruiter	Includes external and internal recruiters and executive search. Source, interview, recruit candidates, provide recommendations, and advice.	RECR
Facilitator	Transfers know-how during training and follow-up support.	FACL
Consultant	Provides services on an hourly or daily basis, including organizational development, reorganizations, mergers, and acquisitions. It includes career counselling.	COLT
Career Counselor	Assists in identifying and raising career interests and vocations.	CARE
Coach	Provides individual and team coaching. It includes mentors.	COCH
Clinician	Clinical or Industrial-organizational psychologists who use a therapeutic approach to improve employee well-being and psychological safety.	CLNI
Individual Contributors	Employees with no team to manage and use results for their own development.	INDC

The term leader is used as a generic term to designate the person whom the literature, depending on the situation, also calls manager, chief, boss, executive, team-lead, front-line manager, director, or leader. Leaders in this project designate people who are responsible for other people; Otherwise, they are considered in our project as individual contributors, even while having influence and leadership. A business executive with advisory responsibility but without supervising anyone is not included in our definition as a leader, but individual contributor.

In addition to this distinction and in conjunction with the learning of the assessment technique, leaders are differentiated based on how they learn and use the language conveyed by the assessment technique. This aspect is further discussed below when starting to use the technique. As Phase 1’s specific question focused on leaders’ uses, the general framework needed to include other potential users with other roles in the organization than leadership, and who use assessment techniques too.

Start of Use

The training of the assessment technique is the critical event that characterizes the start of its use. The techniques, what they measure, how they deploy, and who uses them vary considerably. So does their training, too, their content, and the techniques used for the training. The more potential applications of the technique, use at an organizational level, and for long-term impact, the more time the training may take, much like learning a new language from scratch.

Phase 1 revealed the importance of training for leaders by applying what they learned to applications that are relevant to their leadership role, including making strategic decisions, adjusting attitudes, and communicating effectively. Those uses cannot be made by anyone else but them. This is the case of adaptive profiles and the factor-based approach, which provides measures in context with long-term implications on the organization and retrospectively on them. With the adaptive profiles, a leader who is not trained or half-trained is not considered as using the technique. Other assessment techniques than the adaptive profiles are deployed differently, with short training or no training at all.