What is “Complex Content?”

Welcome back to my blog series, where we’ve been busy discussing the most frequent citations found in SSRs. As we look at frequent Standard B citations, the phrase “complex content” appears many times. With all the work that goes into an SSR, it doesn’t sound like something that should warrant a citation, does it? If anything, the ARC-PA standards demand complexity in response to their many requirements. 

Once again, this is about definitions. ARC-PA citations use the term “complex content” to mean something very specific: your evaluations don’t match your learning outcomes, and thus, learning outcomes cannot be measured.

The citations will contain language like this:

“The evaluation areas of assessment within the rotation specific preceptor evaluation forms included complex content leaving the program unable to determine if the student met all aspects of the learning outcomes . . .”

Let’s talk about precisely what they mean.

Examples Displaying Complex Content 

Looking at examples makes this much more straightforward. Below are sections from two preceptor evaluations for clinical rotations.

In this first example, preceptors must rate their agreement that the student has demonstrated sufficient working knowledge of diseases commonly seen in emergency medicine, both in adult patients and in elderly patients.

However, the medical knowledge outcome lists four aspects of this skill: epidemiology, pathophysiology, presentation, and diagnosis. So when the preceptor rates students, what exactly are they rating? The evaluation does not distinguish any of those aspects, so your program cannot measure whether students achieved their learning outcomes. “Epidemiology, pathophysiology, presentation, and diagnosis” become complex content—that is, content that your assessment misses.

In the assessment above, what does a “3” mean? Did the student show excellent demonstration but with only three of the four factors, or did they perform passably well with all four factors, or did something entirely different happen?

Vague assessment challenges your preceptors. How can they rate students fairly? How can they express that the students did not have a chance to demonstrate everything? Here is another example. The preceptor must rate a student’s participation in creating and following acute and emergent care patient management plans.

Once again, the evaluation fails to take into consideration the complex content, which states that students must demonstrate their incorporation of several factors into these management plans (patient assessment, evidence-based practice, clinical judgment, patient’s personal situation), plus take into consideration “pharmacologic and non-pharmacologic modalities,” which adds yet another dimension. And once again, the preceptor has to estimate across the board, while you cannot accurately measure the overall results.

Other examples

• In another instance, a Family Medicine preceptor evaluation form included a general statement in the headings of knowledge; clinical/technical skills; clinical reasoning/problem solving; and interpersonal skills that added age groups: “In the practice of Family Medicine for infant, child, adolescent, adult, and elderly patients in the outpatient setting.” The assessment items did not delineate the learning outcome for each age category. 

• Another citation stated, “At the time of the site visit, the program stated the common technical skills and procedures will be assessed on the Appendix 12G Skills Checkoff sheet. The skills identified in the Checkoff sheet did not align with the learning outcomes or instructional objectives within the clinical rotation syllabi.” A skills check-off sheet is common, but alignment is necessary between those two elements.

The ARC-PA’s requirements may feel downright fussy, but they have a point. How do you know if your students get the necessary exposure and training? If students receive poor assessments, what do they need to compensate? What have they missed? You don’t know, and neither does ARC-PA.

Remember that the learning outcomes must also match the course syllabi. Inconsistencies will get you in trouble. 

The solution is relatively simple. ARC-PA wants aligned outcomes and assessments. Let’s look at the surgery preceptor evaluation form that was deemed out of compliance.

This compiles too many factors (the stages of surgery) into a single evaluation point. Here is a revised preceptor evaluation form.

Eliminating Complex Content

Your first instinct might be to create a formidably lengthy assessment form listing every aspect of the complex content separately. I don’t know if that’s always the best choice, however. Assessment forms are already longer than ever. How do we avoid this?

Instead, take a hard look at the complexity of your learning outcomes. Before creating aligned syllabi, outcomes, and assessments, consider what a course is reasonably supposed to teach. There is no point in including extra dimensions, age groups, skills, or other complicating assessment factors if they are unnecessary to a PA education. 

In my next blog…

We’ll move on to observations and citations for Standard B3.03. You will see that alignment in outcomes and evaluations continues to be an essential factor in meeting ARC-PA Standards.