Benchmarks and Gap Analysis

Welcome back! In last week’s blog, we looked at the B3.03c citation, how to construct learning outcomes for women’s care, and how to create preceptor evaluations that eliminate concerns about complex content. 

Today, we’ll examine the last step, which involves establishing benchmarks for learning and instructional outcomes and conducting a gap analysis when a program fails to meet those benchmarks.

Benchmarks

Here is an example of a chart on learning outcome benchmarks. You’ll see that preceptor evaluations are only half the assessment; we also use test scores and diagnostic scripts.

Instructional objectives could be more precise in terms of what the ARC-PA requires. Some programs crosswalk their instructional objectives with their learning objectives and then use the same process to establish benchmarks. The chart below shows instructional objectives categorized with a Bloom’s Level, a domain, and an assessment method. At the very least, you must have a process that describes how you achieve your instructional objective, but the ARC-PA is somewhat less stringent about these than it is about the specificity of your learning outcomes. 

Every program has its requirements. If one requires that a student get a score of three or above within the domains, what happens if they get a one or a two? Think about your policies because if the preceptor doesn’t feel that the student meets the minimum requirements, either giving them a score of two or saying “not observed” or “not applicable,” this is one of the most significant areas of trouble that programs get into. Why? Because they don’t have a process to address this. 

The most common deficiencies are: 

(a) procedures that students have yet to do, and 

(b) when a preceptor determines that the student did not achieve the level of proficiency required.

Everything must be evaluated. So, for that reason, we’ll discuss the gap analysis. A gap analysis shows that a deficiency has been noted and that a clinical team is monitoring it.

Gap Analysis 

Here’s how to conduct a gap analysis for submission to ARC-PA.

1. Record any elements that could not be performed, received less than a satisfactory grade, or did not meet program expectations.

2. Include the program’s plan for future clinical exposure to fulfill missing elements.

3. Track missing gaps longitudinally for completion and future rotations, using a spreadsheet, for example, or whatever method you prefer.

4. If necessary, use simulation to fulfill missing learning outcomes. Practice this sparingly because they made it clear that extensive simulations are educationally questionable at the last ARC-PA meeting at PAEA. As things are now, I don’t think you would get a straight answer about how much you can do through simulation. 

5. Include Gap analysis documentation. Below is a form I developed, designed for when a student finishes the rotation. The form is completed and placed in their electronic file. That way, it can be longitudinally followed up. 

Summary and conclusions

As we complete our discussion of Standard B, here are a few essential things to remember:

• The formerly common practice of using generic preceptor evaluation forms no longer meets the Standards.

• Logging demonstrates exposure but not competency. Logging used to be sufficient; it no longer is. 

• Learning outcomes must be perfectly aligned with assessment tools, such as preceptor evaluations.

• Learning outcomes must be measurable. If there is complex content within a learning outcome, the preceptor evaluation form needs a breakout menu to assess each component separately.

• Any event that a student does not achieve a minimum score in a specific learning outcome or did not have the opportunity to complete said procedure there must have a gap analysis process to ensure completion prior to graduation.

• Learning Outcomes for SCPE’s have to meet the glossary definitions for ARC-PA, otherwise there is a citation for failing to meet them. You can have fewer learning outcomes, but they must hit each of those elements. I think that making them simple is something you want to aspire to, especially not having complex content. Having too many learning outcomes just sets you up to having a nightmare with tracking and achieving all those. 

• Have a sufficient preceptor making the ratings. A general practitioner is insufficient as a preceptor for women’s health, even in areas where it seems like the medicine can take care of both. Nothing states that there must be five weeks of any particular topic, but as long as students have experience with an acceptable preceptor, I think you’re probably in good shape. This issue is causing problems among programs nationwide because many have had very effective ways of doing this, including not using a board-certified provider in that area, but that will no longer be sufficient. However, I’ll add that if you have a PA practicing in that setting and designate the PA as the preceptor in that setting, that is sufficient. 

Next time…

We’re steadily going down the list of the most common citations. In my next blog, I’ll move on to Standard A. 

Before concluding, I’d like to remind readers that if they need clarification on any of these matters, they can get more information at my website: https://drscottmassey.com/. The website includes information on my free webinars, as well as how to contact our consultants with questions.