Performance-Based Test Design: Putting Competence In Context

Competence is what performance-based learning and performance-based assessments are all about.

Performance-based assessments aim to determine whether students, candidates, or employees can be thrown into real-world situations and real-life scenarios that require them to do meaningful work to a suitable standard under normal conditions—i.e., to determine whether they are competent.

A wrong answer given in performance assessments can help us determine learning needs far better than traditional testing methods—but those performance assessments need to be expertly designed.

What is normal?

In terms of designing performance-based assessments, agreeing on what is “normal” is where the problem lies. “Normal” assumes agreement on both the uniformity and diversity of work tasks and the workplace environment. 

People with the same job title may or may not do the same tasks, even in the same company. They certainly do not share the same workplace environment if they work in different locations or for different bosses. 

What good is a standardized performance assessment for roles of the same name but that are, in fact, entirely different?

Likewise, in most real-world situations, different people feel vastly different in the same environments and under the same conditions, based on a range of unique personal factors. Normal is subjective.

Rob Foshay, Ph.D. et.al, in their research on emerging best practices in the private sector, discusses the rising interest in requiring employees and candidates to complete performance-based assessments to measure competence. 

Part of that interest is fueled by the evidence that performance-based assessments have higher fidelity (they more fully reflect the workplace environment) and; therefore, can demonstrate greater external integrity (passing correlates with something important to the organization). 

For example, is there a correlation between the number of people who pass the test and those who do well on the job, stay in the job longer, or have fewer cost overruns or missed deadlines?

Perrformance-based assessment results mean nothing and will not correlate with anything meaningful unless those performance assessments are designed with the full and true measure of competence in mind—as it exists for that specific workplace. 

It’s a complex problem, but by designing performance assessments in such a way that they target important workplace-specific variables instead of imagining that “normal” really exists among individual people and unique work environments, we can start to see meaningful results.

The design challenge for performance-based assessments

The challenge is to design a performance-based assessment that accommodates workplace variables so that test results correlate with measures that the organization cares about. 

Consider the following real-life situations. Assume you want to measure the skills of:

• Mobile software developers who are writing code for mobile computing platforms for a vendor, such as Google,
• Network management professionals who use software to manage corporate networks for a vendor, such as CISCO, or
• System administrators who manage servers for a vendor, such as SUSE.

Read more about optimizing performance assessments in these fields.

In each of these examples, you want to identify what is different and similar about the tasks and the work environment, as these will provide contextually relevant criteria for measures of competence in performance-based assessments. 

For example:

• Are tasks done as part of a team or individually?
• Is there a difference in the maturity of the technology used and the efficiency of the procedures?
• Do bosses have the same expectations?

How do I identify performance-based assessment criteria?

You do this by asking 1) the right questions and 2) the right people. 

The process is called a practice analysis and it is a necessary preparation phase in the design process for performance-based assessments—at least if you want the results of your performance-based assessments to be meaningful. 

Typically, the task analysis depends on the input of a few people, and the questions are limited to what you do, and what do you have to know to do it.

A practice analysis solicits opinions from diverse perspectives, such as the people upstream and downstream who are impacted by what is done and how it is done in a given role. 

It allows us to develop contextually relevant criteria for performance-based assessments for complex roles because it’s based on diverse perspectives of the same thing.

The practice analysis questions are about: 

• the role’s key characteristics
• what information, systems, and support people in the role can access
• how people in the role work
• what level of performance is expected of people in the role
• what performance feedback is available to those in the role 
• examples of performance that are considered adequate for the role
• the maturity of the systems
• and what outputs and behaviors really get rewarded. 

This information allows the performance-based assessment designer to build in constraints that more accurately measure and predict on-the-job performance.

A knowledge test can answer the question, “Do people know relevant rules, terms, and concepts?” 

However, performance-based tests or performance assessments can answer questions about competence only if they reflect the context in which the work is done, that is, emulating the environment. 

Read more to dispel common myths about performance assessments.

Frequently asked questions

In performance-based tests and assessments, what is competence?

Competence is a measure of actionable learning, higher education, on and off-the-job training, critical thinking skills, problem-solving skills, higher-order thinking skills, student, candidate, or employee achievement, student, candidate, or employee role-related knowledge and understanding, student, candidate, or employee performance, and performance task outcomes.

Why are performance-based tests better?

Academia is one thing, but in the professional world, book-smart is not as useful as real-world smart—not by a long shot! Performance-based assessments provide measures of actionable knowledge and knowledge-backed abilities by testing real-world performance and behavioral outcomes instead of theoretical knowledge alone.

In closing

Traditional assessments, even traditional performance assessments, have no place in complex modern roles with complex variables.

The performance-based assessment was once only accessible to large organizations and only used for high-risk scenarios. Today, many of the economic and technical barriers to adoption have been largely removed. 

TrueAbility, for example, has shown that it can cost-effectively emulate software platforms used at the worksite and deliver performance-based assessment globally and at scale.

The challenges that remain relate to organizational alignment and the tried-and-true practice of performance-based assessment design. And fortunately for many practitioners, the resources available to help you on this journey are growing every day.

Read more about context in performance assessments.