Blind Spots

Happy New Year!

Maybe you've had this experience. You get a notice from an airline frequent flyer program that you'll lose your miles unless you either fly or extend their life by using some for a magazine subscription. Since you're not a real frequent flyer, but someone who's accumulated a few miles from the occasional trip, you go for the magazine subscription. Thus I became a Wired subscriber.

I'm loving it. Tho I don't have the quals to be a card-carrying geek, I have the nerd's love of things of the mind and have found many wonderful articles in Wired, including this one about the neuroscience of illusion, staring Teller of "Penn & Teller" fame.

An aside: It's a great article, and I still have a fondness for reading paper rather than a computer screen, but there are clear advantages of the net as is illustrated by the embedding in the online version of the article of YouTube videos showing Teller performing the illusions/magic which are only described/pictured in the paper version.

Teller is now a co-author of an article in a neuroscience journal on the new field of "magicology, the mining of stage illusions (magic) for insights into brain function." Not only is the hand quicker than the eye; the brain is so enamored of its perceptual heuristics that it still can't perceive/understand what's happening when the secrets of some illusions are revealed. As Jonah Learner, the author of the article and the books How We Decide, and Proust Was a Neuroscientist, writes:

What's surprising is just how limited the repertoire of magical illusions actually is. The Nature Reviews Neuroscience paper lists nine fundamental "conjuring effects" of modern magic, from the vanish and the restoration to telekinesis and ESP. While these basic tricks have been varied endlessly—you can "restore" a cut rope, a sawed-in-half assistant, a shredded piece of paper—each of the effects relies on a specific perceptual phenomenon. This may be why exposing the "secret" of a magic trick is so often deflating. Most of the time, the secret is that we're gullible and our brains are riddled with blind spots.

OK, you who have been reading this blog for a while know what's coming. I read the above and immediately thought about the implications for interviewing and examining. Good design is critical for examinations precisely because "our brains are riddled with blind spots" and cognitive frames that are usually functional but sometimes misleading. We interview in groups partially in hope of reducing our blind spots and confirmation biases (ignoring evidence contrary to our beliefs), but then we risk group polarization, the tendency of people with shared beliefs to become more certain of those beliefs when they congregate.

Knowing these limitations to human perception and understanding encourages the spiritual practice of humility. In this new year, may we wear our beliefs like comfortable old garments, soft to the touch and with room for growth.

Resistance is Futile (and Unwise): Excellence & Oppression

Often people are wisely concerned about the relationship between examining and oppression. A powerful historical example of oppressive examining is the "literacy tests" that were used to deny the vote to African Americans in the South from the 1890s through the 1960s.

The other side of the examining coin is presented in this history of the relationship between barbers and surgeons. Suffice it to say that we all may be thankful that the examining of surgeons is much more rigorous than it once was.

A third interesting example comes from the film Jazz. Listening to Wynton Marsalis wax poetic about Louis Armstrong, we are reminded of the roles that genius can play.

Incompetence in the professions is dangerous. People do not die on the operating table because of an incompetent chaplain, but incompetent and unethical ministers have done much damage.

Ministers have also inspired. While not every minister may have the genius of the Rev. Dr. Martin Luther King, Jr., we may all--ministers and laity--be inspired by his example.

Resistance to examining is futile and unwise. It's important to make judgments about competency, excellence, and genius. However, it's also important that such judgments and the processes by which they are reached are congruent with the first (the inherent worth and dignity of every person) and second (justice, equity, and compassion in human relations) principles of UUism.

Competencies & Performance

"The best predictor of future performance is past performance."

The above was our mantra for the Performance-Based Interviewing project at the Department of Veterans Affairs (VA). Jobs at VA were described in terms of duties and responsibilities, the competencies required to perform the duties and exercise the responsibilities were identified, and interview questions were developed to assess whether candidates possessed the required competencies.

In Hire With Your Head: Using Performance-Based Hiring to Build Great Teams, Lou Adler suggests a more direct approach: figure out what you want done and ask the candidates to describe or demonstrate the capacity to do it.

In one way, this is a very old idea. When I entered the workforce as a typist, I took a typing test. (AKA keyboarding for the post-typewriter generations. :-) )

The idea of performance testing gets much more complicated when dealing with duties and responsibilities that are more difficult to demonstrate in an interview. Adler gives the example of an interview for a position involving the development of marketing plans. The interviewer and interviewee outlined a plan during the interview. The interviewee was hired and used that outline to complete and implement the plan.

If we try to apply this concept to the MFC (Ministerial Fellowshipping Committee) interview, we can see that the first challenge is deciding what we want ministers to do. The tasks of healthcare chaplaincy, community organizing, and parish ministry overlap, but are far from identical.

The second challenge is evaluation. During an interview, you can demonstrate skill outlining a marketing plan or delivering a 10-minute sermon. Neither assures is that you will be successful at implementing a marketing plan or regularly leading worship that inspires a congregation/community.

That these challenges are daunting should not be discouraging. The reflection and dialogue required to address them will be strengthen and deepen our religion.

Structured Interviews: What They Are; Why You Should Use Them

Ministerial Fellowshipping Committee (MFC) interviews should be structured to increase the likelihood of accuracy and fairness. However, when I've made this recommendation, I've received some rather strong reactions that appear to be based on differing understandings of what is meant by "structured" interviews.

Interviews may be structured in many ways. Here's a way to do so for professional positions that increases the likelihood of fairness and accuracy while giving interviewers latitude to improvise and to inquire in depth.

The consistent questions (i.e., those asked of all candidates who are expected to possess a competency) in a structured interview tend to be broad and serve as jumping off points for more detailed, individualized questions. For example, a broad question might be the following: "What has been a significant challenge in your anti-oppression work and how did you overcome it?"

We can imagine that answers to this question will vary greatly depending upon the social location and life experience of the interviewee. Follow-up questions, tailored to the interviewee's response, might be one or more of the following:

• Were there other consequences to your actions?
• After your this initial success (failure), were other actions undertaken at a later date?
• What role did collaboration play in your success?
• What, if any, networking and research did you do before acting?

Contrasting this strategy for structuring interviews with other types of examining may help put it more in focus. Interviews structured in this way are not:

• Standardized tests.
• Foolishly consistent.

Structured Interviews vs. Standardized Tests
Someone expressed the concern that structured interviews were standardized tests and would be subject to cultural bias. As noted below, while some authorities equate "structured interviews" with "standardized interviews," that's not how I'm using the terms here. Furthermore, when most of us thing of standardized tests, we think of multiple choice tests.

Standardized testing risks cultural bias in both questions and the answers. The risk is often greater in the answers than the questions because in many forms of standardized testing, such as multiple choice tests, there is only one right answer.

No form of examining based on content validity (i.e., the test is designed to see whether candidates have the knowledge required for the position) is immune from the risk of cultural bias. However, there is less risk with structured interviewing than with multiple choice tests because in the former interviewers can ask follow-up questions designed to more thoroughly elicit the candidate's competencies and because there is no single right answer to a structured interview question.

Structured Interviews vs. Standardized Interviews
Another interlocutor was concerned that a structured interview would tie the hands of the interviewers. This concern is quite understandable because some authorities use "standardized interviews" and "structured interviews" synonymously.

For some kinds of research, standardized interviews in which all the interviewees are asked the same questions in the same sequence is a strategy for increasing validity and reliability. It also permits the use of interviewers with minimal training.

This strategy is not likely to be effective in interviewing for professional positions. In such interviews, you do not wish to assure accuracy and reliability in a manner that does not permit you to draw upon the experience and expertise of your interview panel.

A structured interview is a wonderful midpoint between free-form interviewing and standardized interviewing. It provides a degree of consistency while allowing the interview panel to delve more deeply in to the qualifications of the interviewees.