Analyzing the Five Questions Vol 3
Welcome back once more to my series on analyzing the five questions contained within ARC-PA Standard C1.01.
In our previous posts, we explored how programs can develop a defensible process for answering the C1.01 questions and how Areas Needing Improvement (ANIs) can be identified through thoughtful analysis of benchmarks, trends, triangulation, and context. Those concepts provide the foundation for effective self-assessment under the 6th Edition Standards.
Today, I'd like to turn our attention to what may be the most challenging of the five questions:
Are program faculty effective in operating the program outside of teaching?
At first glance, the question appears straightforward. Yet many program leaders quickly discover that answering it can be surprisingly difficult. Unlike admissions effectiveness, didactic curriculum effectiveness, or clinical curriculum effectiveness, there are relatively few obvious data sources available. In many cases, the challenge is not interpreting the data—it is identifying meaningful evidence in the first place.
That reality came up repeatedly during the webinar and again during the question-and-answer session that followed. Programs are not struggling because they don't value faculty effectiveness. They are struggling because they are trying to determine how best to measure it.
What Is ARC-PA Actually Asking?
One of the first steps in answering any C1.01 question is making sure we understand what is being evaluated.
When many people hear the phrase "faculty effectiveness," their minds naturally go to teaching performance. They think about course evaluations, student satisfaction surveys, examination results, and classroom instruction.
However, that is not what this particular question is asking.
C1.01 specifically focuses on faculty effectiveness in operating the program outside of teaching responsibilities, shifting away from classroom performance and toward the many other activities required to maintain a successful PA program.
Those responsibilities may include:
Admissions activities
Assessment and evaluation processes
Committee participation
Accreditation support
Student advising
Remediation efforts
Administrative responsibilities
Program operations
This distinction is important because it changes the type of evidence programs should be seeking.
The Data Challenge
This question often generates more discussion than any of the other four C1.01 questions. The reason is simple: There is no universally accepted data set that neatly answers it.
The 6th Edition Standards are still relatively new, and institutions across the country are working through how best to evaluate faculty effectiveness outside the classroom. While there may not yet be a perfect solution, there are reasonable approaches that can help programs build a defensible process.
The goal is finding the most meaningful evidence available.
Start with the People Who Observe the Work
One of the simplest approaches involves Program Director evaluations.
At first glance, this solution can seem almost too simple. However, when we consider who directly observes faculty participation in program operations, the rationale becomes clear.
Program Directors often have firsthand knowledge of faculty involvement in:
Admissions decisions
Committee work
Assessment activities
Student support
Remediation processes
Accreditation preparation
General program operations
Because they observe these activities directly, their evaluations can provide meaningful evidence regarding faculty effectiveness outside of teaching responsibilities.
This does not mean a Program Director evaluation should stand alone. However, it may serve as an important component of a larger assessment strategy.
Building Multiple Sources of Evidence
Throughout the webinar, I emphasized the value of triangulation whenever possible. Although Question One presents unique challenges, programs can still seek multiple perspectives on faculty performance. Potential sources of evidence may include:
Faculty Self-Assessments: Self-reflection can provide insight into faculty contributions, responsibilities, and professional engagement. While self-assessments should not be viewed as objective measures, they can add useful context when combined with other evidence.
Administrative Evaluations: Department Chairs, Deans, and other administrators may have perspectives on faculty participation in institutional activities, service commitments, and program operations.
Committee and Service Participation: Documentation of committee work, assessment responsibilities, accreditation involvement, and program initiatives can help demonstrate faculty engagement beyond instructional duties.
Advising and Student Support Activities: Programs may also consider evidence related to advising, mentoring, remediation, and student support services when evaluating faculty contributions.
No single measure is likely to answer the question completely. However, together these sources can help create a more comprehensive picture of faculty effectiveness.
Defensible Is Better Than Perfect
One of the themes running throughout this entire webinar series has been the importance of building a defensible process. That principle applies here as much as anywhere else.
Programs sometimes become frustrated when they cannot identify a perfect metric. Yet perfection is rarely the goal of program assessment. Instead, the goal is to gather reasonable evidence, apply it consistently, analyze it thoughtfully, and arrive at conclusions that can be explained and supported.
That is what makes an evaluation defensible.
As the profession continues to adapt to the 6th Edition Standards, approaches to Question One will undoubtedly continue to evolve. Additional best practices will develop. Institutions will learn from one another. For now, however, the most practical advice may also be the simplest:
Do the best you can with the evidence available, and make sure your reasoning is clear.
Next time…
In our next post, we'll examine the next two C1.01 questions and explore how programs can evaluate admissions effectiveness and didactic curriculum effectiveness.
Unlike Question One, these areas typically offer a much larger collection of available data. The challenge becomes less about finding evidence and more about determining what that evidence means.
Please join me then as we continue the conversation.
