When faced with the prospect of using an assessment, business professionals understand that the assessment needs to be valid. However, the seemingly simple question, “Is it valid?” can lead to a lot of confusion. Much of this confusion can be attributed to the fact that validity is a broad, complex concept and people often mean different things when they use the term.
There are three main components of validity: Technical, ROI and Legal.
Technical – When testing technical validity the main factor being addressed is whether or not an assessment is well designed and whether or not it measures what it is supposed to measure. Well-developed assessments have technical manuals that detail how the assessment was built. These manuals include a range of statistical data regarding types of validity evidence. Without proper evidence of validity, the quality of the assessment is in question. The challenge in understanding validity in this context is that technical expertise and training is required to accurately interpret the information.
ROI – Not surprisingly, organizations interested in using assessments want to know if they add value. In this context, the question, “Is it valid?” really means, “Will it work and have a positive impact?” Past success of the assessment or analytics conducted for individual organizations can be used to demonstrate ROI.
Legal – The question, “Is it valid?” can also refer to the legality of an assessment’s use and the defensibility of the assessment process. If assessment use does not indicate adverse impact, there is little concern. This does not eliminate potential challenges, however, so being prepared with evidence that demonstrates job relevance can help mitigate risk. Different methods can be used to provide evidence of job relevance, including client- specific research showing the relationship between the assessment and relevant job performance metrics. If an organization has concerns about legality, it is always recommended to engage the services of a law firm familiar with use of assessments in the workplace.
As you can see, validity is a multi-faceted concept. Adding to the difficulty of communicating about validity are the variety of terms used in relation to it. Construct, content, criterion, divergent, convergent, concurrent, and predictive are descriptors for some of the different techniques utilized to demonstrate evidence of validity. It is important to understand that not all methods are appropriate in all instances. A thorough understanding of each method, their relationships with each other, and appropriate applications requires education in this area. A qualified professional must interpret and evaluate the information. Clarifying which “Is it valid?” question to analyze is critical for building buy-in with both clients and employees. Further, it’s essential for the successful use of assessments.