Are We Cheating Ourselves Out of Our Competency?

Last October, I attended the CAP Residents Forum where I heard the President of the American Board of Pathology speak about the use of remembrances for boards studying and how it was considered cheating and a breach of the honor code. I was a bit surprised when we took a live vote via text that the majority of residents felt that the use of remembrances was not cheating. But this all goes back to the medical school culture (and maybe even before that) in which we were trained.

During my 1st year of medical school, I remember feeling that it was inequitable that some students got old exams from their assigned 2nd year “big siblings” and that others got nothing. So I did something to neutralize the playing field – before every test, I scanned the old exams I had and emailed them to the class listserv. For the few exams that I didn’t have an old exam, other classmates stepped up and scanned and emailed them out. And so we built a culture of sharing. I was very proud that my classmates did not put having a personal advantage over the concept of equity.

But unexpected and unintentional shenanigans ensued. Once a classmate emailed out what was thought to be an old exam but was later found to be a “stolen” exam as that professor purposely did not give back his exams. This required our waiting while he re-wrote parts of the exam as this was discovered only on the morning of our actual exam. And during my 1st year of PhD (I was initially DO/PhD), when my original classmates were in the midst of their 3rd year rotations, a classmate emailed some study materials they had obtained from a friend at another DO school that unbeknownst to them were remembrances from previous NBOME shelf exams.

Eventually as the year went on, my classmates began to realize that what they had were remembrances and one classmate actually stepped up and turned them into one of our deans. Since I was no longer in this class, I only heard after the fact, but our dean had called the NBOME to warn them to retire the questions in these remembrances and had also called the school from which they originated to let them know what their students had done. At that time, I was not very invested in what was unfolding as I was in the graduate school portion of my dual degree program. But I do understand the fear of others having advantages that you may not and having that thought cloud one’s thinking.

These memories flooded back into my mind as I listened to this talk at the Residents Forum and voted. I didn’t realize it then when I watched from the sidelines but do now, that the use of remembrances—whether it is considered cheating or not—is in fact cheating oneself. A Machiavellian “ends justify the means” mentality is often used to justify such actions. But in the end, what have we truly accomplished? Yes, maybe passing our boards. But what happens when we become practicing pathologists if we didn’t understand what it was that we so diligently memorized off of remembrances?

It may be slower and more difficult, but I’ve pledged to myself that I won’t take the easy road, no matter how tempting it may be (I admit I’m just as vulnerable to temptation as everyone else). I tell my M2 students when I TA their pathology small group discussion lab that if they don’t know even one word when they are reading to learn a concept…to stop immediately and look it up, even if it just means to Google. And I’ve tried to practice this as well. I still may not pick up on surgical pathology concepts (especially grossing) as quickly as I do clinical pathology concepts, but I have noticed a difference. Concepts seem to stick better in my aging brain because I have shifted how I focus my efforts. I try not to waste as much time on worrying (but I still sometimes do) that others may have an advantage over me because that is wasted effort.

Gaining competency means to take ownership and responsibility for one’s learning. Figure out how you learn best and make a plan to do it. And if you fail, keep trying until you make it. Really try to understand the mechanisms and not just memorize minutiae or facts. I find that I can figure out the answers this way if I had a solid foundation. And don’t take too many shortcuts, which invariably, remembrances may turn out to be. But I’d like to hear your take on the topic…is the use of remembrances cheating or not? And do you think that using remembrances cheats you out of obtaining competency as fast and as well as you might have gotten there without them?

 

Chung

Betty Chung, DO, MPH, MA is a second year resident physician at the University of Illinois Hospital and Health Sciences System in Chicago, IL.

What Type of “Graduated Responsibility” Do We Need in Order to Gain Competency?

Building on the brief historical piece I wrote last week about the progression toward “competency-based” resident training and the ultimate outcome of evaluation by the Milestones, I’d like to ask, “How can residents achieve competency in training?”

A resident recently stated at the most recent CAP Residents Forum that true graduated responsibility means to be able to verify a case, whether surgical pathology or frozen section, and I would guess clinical pathology results were implied, without the oversight of an attending. As a PGY-2, I am not sure I can agree with that statement. But who knows, maybe closer to graduation, I might. How many of us would be a Milestones rating of “4” by the end of our funded 3 or 4 years, and therefore, competent enough to theoretically verify a case on our own?

I would like to repeat some statistics from the 2013 ASCP Fellowship and Job Market Surveys: only 16% of residents felt they were ready to sign out general pathology cases upon graduation and 95% intended to seek fellowship positions. 59% of PGY-3 and PGY-4 felt that they needed fellowship training to feel confident in general pathology and 17% to address a perceived educational deficiency. So, if this is a true, then the majority of us may not feel comfortable verifying general pathology cases, with the possibility of malpractice, even as a PGY-4.

Even though I cannot verify cases, I still feel that my program has given me the opportunity for graduated responsibility. For instance, I began to enter my diagnoses into our electronic surgical pathology reports before the end of my first month of training. Then during sign out, my attending would teach me as well as correct my diagnosis prior to verification. As I mentioned before, I learn by doing and even more so by being wrong, than by reading or being lectured to so this works for me. I have the safety net of being allowed to be a trainee while applying and improving my developing surgical pathology skills. And for me, being able to verify or not, would not change how I approach my cases or my diagnoses.

Our PGY-2 and above, are encouraged to work up a case (eg, order stains, etc) prior to sign out at our academic and VA hospitals (not sure about the community hospitals since I haven’t rotated at them yet), even if it is not our names that go on the final “dotted line.” We don’t all reach this point at the same time – it is about trust in our knowledge and skills by our attending and our initiative to broach the subject before we can do so. At the start of my PGY-2, I was ordering flow cytometry panels on cases because my attending trusted my hematopathology skills.

So, for me, at least for now, “graduated” responsibility is in the eye of the beholder and I have been nurtured to be where I need to be at this point in my training. I feel that I have the freedom to grow under our system. So, what does “graduated responsibility” mean at your institution? Let me know by leaving a comment.

-Betty Chung

ACGME Competencies and Milestones: What Does It Take to Be a Good Pathologist?

In 1998, the Accreditation Council for Graduate Medical Education (ACGME) advocated the Outcome Project, a competency-based paradigm for resident training. Previously, completion of a residency was based on a fixed number of years of training specific for each specialty. “Competency” to practice was based on the passage of a certifying board exam. Notable changes due to this initiative were the increased use of objective structured clinical exams (OSCE), increased resident engagement in quality improvement and evidence-based medicine projects, and the incorporation of additional didactics and approaches in GME curricula. Given all this, did the requirements for residency completion truly change?

In 2012, the ACGME introduced their standards for the evolution of the Outcome Project to the Next Accreditation System (NAS). Each specialty developed their own outcomes-based milestones within the 6 clinical competency domains. Residents are graded on a scale from 1-5 in each domain; level 4 represents the “graduation target.” Level 5 is the equivalent to the performance of a pathologist who has been in practice for several years.

The Milestones should be applied without regard to the trainee’s specific year. But would exceptional trainees then graduate earlier, and those who fall behind, later? Currently, the federal government funds residents based on a specific number of years for a given specialty. Can we apply the Milestones in a standardized manner for each AP/CP subspecialty? And what about a resident who excels in some rotations but doesn’t meet the Milestones in another? Anecdotally, based on a few pilot beta sites, each interpreted the Milestones and conducted their evaluation process, differently…so is there a best way to implement them? Or are there multiple, equally acceptable, ways?

In terms of the resident perspective, the 2013 ASCP Fellowship and Job Market Surveys, indicated that only 16% of residents felt that they were ready to sign out general pathology cases upon graduation and that 95% would seek fellowship positions. 59% of PGY-3 and PGY-4 felt that they needed fellowship training to feel confident in general pathology and 17% to address a perceived educational deficiency.

When I started residency, I was evaluated on a scale of 1-5 with 3 being “usually meets expectations,” (a permutation of the A-F scale we’ve all known since elementary school)in the 6 ACGME core competencies: 1) patient care, 2) medical knowledge, 3) practice-based learning and improvement, 4) interpersonal and communication skills, 5) professionalism, and 6) systems based practice. And for my next to final rotation of PGY-1,the same as my first, I was evaluated by my rotation director and performed a self-evaluation with the Milestones. We then met and discussed our respective evaluations. We pretty much agreed in scoring and with respect to those that we disagreed, I usually graded myself harsher.I’m not sure if one method of evaluation was better than the other – what I found most helpful in both processes were always the comments, not the numerical score.

So are the Milestones an improvement in terms of how we evaluate competency of residents for practice? Only time will tell.

You can find the Pathology Milestone Project, published in September 2012 at http://www.acgme-nas.org/assets/pdf/Milestones/PathologyMilestones.pdf.

Let me know how you feel about the Milestones and resident competency in our comments section.

-Betty Chung