Total Recall; Practice, Practice, Practice; and the Power of Misinterpretation

Reading "Total Recall" by Gary Marcus, a cognitive psychologist, in Wired magazine reminded me of the power of practice and how easily examining can go astray. The subject of the article is Jill Price, a woman with a supposedly "perfect" memory. If you'll click on the link to the Wired article, you'll also see Diane Sawyer's interview of Ms. Price.

After interviewing (please note all fans of the blog: interviewing!) Ms. Price over 5 years, James McGaugh, a neuroscientist, and his team at UC Irvine concluded that she had a near perfect memory in this article first published in the journal Neurocase. An editor at the Free Press went even further, entitling her story The Woman Who Can't Forget: The Extraordinary Story of Living With the Most Remarkable Memory Known to Science.

Ms. Price does indeed have an amazing memory, but Dr. Marcus, upon examining her further, discovered that her memory isn't all-inclusive. Rather her forte is autobiographical memory. Dr. Marcus writes that the source of this autobiographic memory is rumination that appears obsessive. She didn't seek out this memory and she didn't know its source; it was an unsought consequence of her journaling and other daily habits.

OK, OK, so what does this have to do with ministerial examining?

• Examinees can mislead examiners even when they are not trying to do so.
• Interviewing is a limited tool at best even when in the hands of scientists who should know better. (Being a professional doesn't assure that you are a skilled interviewer, and even skilled interviewers can be misled or can deceive themselves.)
• Examining and assessment should be based on a multiplicity of instruments, and there should be appeal/review mechanisms because of the possibility of error.
• Interviewers are likely to be tempted, and may even consciously or unconsciously fall to the temptation, to seek information that affirms the conclusions they've already reached.
• "Ordinary" people can achieve extraordinary results with practice, practice, practice. (The thesis of Outliers by Malcolm Gladwell, which is discussed elsewhere in this blog.)

So let's acknowledge the inherent worth and dignity of each individual involved in developing and implementing the current UU ministerial examining process. I am confident that they are doing their best with the best of intentions. I'm grateful for their professional and their volunteer efforts. Yet, I will continue to advocate for evolution in UU ministerial examining.

The Quest for Certainty

Currently, I'm reading The Age of American Unreason by Susan Jacoby, author of Freethinkers: A History of American Secularism. On page 157 of the former, she writes that in the 1960s:

The appeal of the Christian fundamentalists was similar to that of strict Hasidic sects in the Jewish community: They offered rules and certainty to some young men and women who had found only unhappiness where others had found personal freedom.

Back in the day, I was one of the youth who was revelling in the freedom of the time. Brought up UU in a diverse Washington, DC, congregation, when I went to the University of Wisconsin (Madison), during the turmoil of the late 1960s, I felt like the rest of the world was joining our free, liberal faith. In contrast to my felt sense at the time, Jacoby points out that the turmoil of the 1960s was the seedbed for neoconservatism, fundamentalism, and the Reagan Revolution.

In Outliers: The Story of Success and this wonderful video from the 2008 New Yorker Conference, Malcolm Gladwell points to a "mismeasurement" problem in many fields that is due to a misplaced quest for certainty in examining. While it's true that the best predictor of future behavior is past behavior (the mantra of performance-based interviewing), the value of past behavior as a predictor varies with its relevance to the expected future behavior. Dividing a job into its component parts and testing each of these may not provide good results. Please see the Gladwell video for specific examples.

I wonder whether the complexity and the nature of the current UU ministerial examining process is not in part a consequence of the difficulties of some ministers and ministries. How has our UU history served as a seedbed for our conservatism in examining?

Temporary Autism

Autism is defined in Merriam-Webster's Online Dictionary as:

A variable developmental disorder that appears by age three and is characterized by impairment of the ability to form normal social relationships, by impairment of the ability to communicate with others, and by stereotyped behavior patterns.

In Blink, Malcolm Gladwell lauds the merits of "rapid cognition," something we sometimes know as "intuition." However, he also points out that it can often go astray. One example he gives is what he calls "temporary autism."

Gladwell is focusing on our ability to assess the emotions of others by "reading" their faces, an ability that people suffering from autism appear to lack. Gladwell says that when those of us who are not suffering from autism becomes sufficiently stressed, we become temporarily autistic, losing our capacity to read in the emotions of others.

I went to the MFC presentation at the General Assembly in 2007 in Portland. A then member of the MFC said that he knew that ministerial candidates were stressed by the MFC interview, but he didn't think that the MFC members were all that scary a bunch and that the candidates would face much greater stresses when they became ministers.

I would have to agree that the MFC members are not all that scary a bunch. In fact I find them to be hard working, talented, dedicated, and well-intentioned. But the former MFC member's assessment did not take into account the situation in which the candidates find themselves when they are facing the MFC.

When working as a chaplain in medical center emergency rooms and critical care, medical, surgical, and psychiatric units, there are people and situations that are more frightening than the MFC and its members. Yet no single interaction has the foreseen capacity to postpone or even derail one's call to ministry.

In the parallel situation of a job interview, you know that if you don't get this job, you are likely to get another. You may also know that there are discrimination complaint and other review procedures if you believe you have been treated unfairly.

None of these options really apply to the MFC interview. If you're UU down to your bones and you're not fellowshipped as a UU, it's not likely that you're to become a candidate for fellowshipping in another religion. Furthermore, there's no review process outside of the MFC for its preliminary fellowshipping decisions.

The pressure therefore is enormous. When I listen to the stories of aspirants and candidates who "froze" in their interviews, it sounds an awful lot like temporary autism.

What do I recommend?

• Put less emphasis on the interview in the fellowshipping process;
• Create a review process outside the MFC; and
• Increase interview training for aspirants and candidates.
  

Malcolmtarian Gladwellversalist

Okay, okay, I'm exaggerating. I have no intention of starting a new religion based on Malcolm Gladwell, nor do I think his books, such as Outliers and Blink, are holy writ. However, I'm beginning to suspect that if he didn't exist I would have had to invent him.

I've referred to Outliers in these 2 posts. Two main themes of this book are the "mismatch" problem of using faulty measures for examining and the importance of practice in the development of expertise.

Though Blink was written before Outliers, I'm just now getting around to reading it. In it, Gladwell points to the many benefits of "thin slicing" and gives examples of "intuition" being superior to analysis. However, he also points to risks and errors of intuition in the story of Warren G. Harding, a man who most looked like but was least capable of being an effective president.

Gladwell's writing provides an excellent explanation of the phenomena I named in the post "15 Seconds." Intuitions can be a critical source of knowledge not immediately available to conscious thought. They can also mislead and support discrimination.

Though Gladwell is a journalist, not a scientist, but he's done a great job of writing about the science and the experiences that support his theses. After you read his books, I doubt that you'll ever again think about examining and thinking the same way again.

Best Predictors

The mantra for performance based interviewing was the following: The best predictor of future performance is past performance. Therefore, we designed interview questions that allowed candidates to describe how they had performed when facing tasks and challenges similar to those of the positions for which they were applying.

However, there is an even better predictor of future performance on a job than past performance in other jobs and life situations: Past performance on the same job. Christine Robinson, aka iMinister, well makes that point in her comment below on my last post. Her thinking is affirmed by Malcolm Gladwell in his latest book, Outliers: The Story of Success, in which he criticizes many evaluation techniques as being poor predictors of future performance. He calls this "the mismatch problem," and encourages the use of on-the-job evaluation.

This is a justice issue. One of Gladwell's examples was preference for minorities at the University of Michigan law school. Twenty years after the preferences were granted, minority attorneys who had been granted preferences were performing as well as non-minority candidates who had "higher" qualifications.

In "Minorities Will Strive to Cross a Higher Bar," David L. Kirp, professor of public policy at UC Berkeley, cites a recent study that shows that undergraduate minority students at elite private universities who receive preferential treatment perform better. People do respond to rising expectations. Kirp shows the fallacies in claims that this preferencial treatment creates another kind of mismatch problem.

How does all this translate into recommendations for examining for ministers?

• Use the best measures available and recognize their limitations.
• Take iMinister's advice and put more emphasis on on-the-job performance rather than academics.
• Provide preferential treatment when warranted and publicize your processes and rationales. This whole-person approach can consider class and other distinctions and expections that have impacted past performance.

15 Seconds

When I was involved in promoting performance-based interviewing (AKA behavioral-based interviewing) in the Federal government, someone told me that there was research showing that most interview decisions were made within the first 15 seconds of the interview. While I'm not sure which research he was referring to, there is some information at this link about the validity and reliability of such decisions.

As I have reflected upon this phenomena, I have wondered about the impact of reviewing application material before an interview. Does the application material help preprogram the interviewers in their decision-making?

My deeper ethical concern is about the possibility of discrimination in the absence of structured interviews and trained interviewers. There is an excellent video called, "More Than a Gut Feeling," about performance-based interviewing. While the above link points to the surprising accuracy of 15 second evaluations, it also sounds a cautionary note, especially regarding culturally variable assessments. And, arguably, all assessments are culturally variable.

Right Path

Twenty years ago (1988), Bill Moyers interviewed Joseph Campbell on The Power of Myth. During those interviews, Campbell famously said "follow your bliss," indicating that when you were on the right path doors would open where you didn't know doors existed.

A little over five years ago at an informal gathering with members of the Ministerial Fellowshipping Committee (MFC), their answers to questions about their examining process--such as, "How did you validate the process? What do you do to assure reliability? Who is your psychometrician? Are the interviews performance-based?"--raised concerns. However, as a first-year student and an aspirant for UU ministry, I judged that I lacked the standing (and/or the courage?) to challenge their process beyond asking my indelicate questions.

Earlier this month I was welcomed into preliminary fellowship as a UU minister by the MFC. After they announced their decision, they asked me whether I had any comments on their process. I made several, and they asked that I provide further information to David Pettee.

Research on this subject has demonstrated Campbell's principle or possibly the more common one that "timing is everything."

• The Unitarian Universalist Association (UUA) conducted a Summit on Excellence in Ministry. (The Reverend Christine Robinson covered the conference live via her blog, "iMinister,"an inspiration for this blog.)

• Malcolm Gladwell, author of The Tipping Point and Blink, has been commenting on what he calls the "mismatch" problem between examining procedures and occupational performance in this excellent and interesting video and his new book Outliers. While Gladwell doesn't mention ministers, many of his comments about preparation, qualification, and selection procedures for professional sports, teachers, and attorneys apply to UU ministers as well.
  

Many may argue for the uniqueness of the ministry, that it's not comparable to other professions. Others may question whether a newly minted minister has the qualifications to be commenting on examining for ministry. However, this is precisely the point: the qualifications for ministry should the clear, public, and readily understood. They should be developed and revised with input from multiple sources. To do so would be a service for ministers and candidates for ministry, as well as the people and communities they serve.

The next post will speak the process of developing examining guides.