hi chauncey,

was glad to meet at the workshop. i have some thoughts about your musings:

despite unique learning histories and consequent idiographic response
biases (sources of error variance with respect to what is replicable),
science searches for robust cause-and-effect relationships that are
invariant across instances of therapy. so, we must separate the active
ingredients of therapy (e.g., contingent responding to CRBs) from
therapist variables (e.g., stimulus properties of the therapist or
therapist response stereotypies). a prerequisite for establishing the
reliability of measurements is to first demonstrate that the
observables to be measured are valid (i.e., significant). only when
measurements correspond to important and orderly events in the world
does it make sense to determine how well separate measuring events agree.

validity, however, does not establish reliability, but only makes it
possible. once the constructs of interest are defined in terms of
observables, then we can determine the coherence of the data measured.

although inconsistent reporting may be a functional class of CRB1s,
and although clients may report different topographies of one specific
functional class of behavior, the primary methodological challenge is
to agree on valid observables for each meaningful class of CRBs.

while anybody familiar with respondent and operant conditioning may
recognize patterns of behavioral change that result from contingencies
... after-the-fact (and thus can agree on coding contingent
responses), there is danger of seduction by the allure of consequently
reliable post-hoc analysis; the fundamental questions about which
behaviors should be changed, and how the behaviors will be shaped when
targets are met, remain unanswered. the face validity and construct
validity of CRBs are important to establish before gauging
reliability. the questions arising during the workshop, regarding the
polysemy of raw behaviors, may owe their origins to "putting the cart
before the horse" in this way.

i believe that glenn's research is an important preliminary attempt to
identify CRBs with valid relevance to clinical goals.

i plan to respond to jonathan's email in a bit, to clarify some of
these points.

thanks for the prompts (Sd's).

keith sonnanburg, ph.d.

--- In functionalanalyticpsychotherapy@yahoogroups.com,
"chaunceyparkeriv" <chaunceyparkeriv@y...> wrote:
> Thinking about your comments regarding reliable assessment. I think
> that is a good goal and it occurred to me that therapists/assessors
> would present different stimulus functions and would, due to their
> unique histories, elicit/evoke and be sensitive to different stimulus
> functions of the client. That is, the differing stimulus properties
> and sensitivities would itself confound reliability. I know, thanks
> for simplifying things, right?
>
> On the one hand, I like the idea of reliability of rating/assessment
> (although I have traumatic flashbacks of actually establishing and
> measuring it). On the other hand, I think I was starting to see Bill
> Follette's point about determining variables (CRB 1s & 2s) that we
> can demonstrable push around (show contingent responding). That is,
> the reliability given validity argument. Did that make any sense?
>
> Also, you mentioned that clients are not consistently reporting
> problems (I think you mean that one session they have one complaint,
> next week something else is their problem). I wonder if that is is a
> functional class in itself and do these problems differ mainly in
> topography?
>
> enough, I think shorter messages are more digestable than lengthy.
> >
>
> > I am becoming more and more strident about the necessity of
> remaining
> > behavior analytic in this research. To me, the key is idiographic
> > assessment, conceptualization, and measurement of CRBs and how they
> > change over the course of therapy. I am slowly seeing a model for
> what
> > this research should involve. This of course is what Glenn has been
> > working on for several years now, so what I write below will be
> quite
> > redundant with Glenns work. Im summarizing it here because Im
> > really interested in a program of research that systematically
> > addresses various themes important to FAP, and Im just now
> > starting to see how all the pieces that Glenn has been working on
> > fit together.
>
> > 1. Case conceptualization and FAP assessment to determine CRBs.
> > There are several research questions here.
> >
> > a. First, are there classes of CRBs? Glenn has made an attempt to
> > come up with these with the FIAT interview. The FIAT ended up
> > classifying CRBs into five potential categories (assertiveness, bi-
> > directional communication, conflict, disclosure, and emotional
> > expression).
> > More loosely, I know in some FAP writings by Bobs lab CRBs have
> > been classified into three categories (intimacy, avoidance, and
> cognitive).
> >
> > b. Second, can some sort of interview be developed to reliably
> identify
> > such classes? Again, Glenn has already worked on this, but
> unfortunately
> > my labs brief, initial attempts to use his system in a reliable way
> > failed rather badly, so work has to be done in this area. But it
> seems
> > doable. For example, once an interview is created, two people could
> > independently interview the same person to see if the same CRBs
> result,
> > or two people could watch the same interview and we would see if
> > these two raters would come up with the same CRBs.
> >
> > 2. Once CRBs can be reliably identified, can they be
> operationalized in
> > a way that allows for weekly measurement? Measurement would have to
> > be of both in-session CRBs and corresponding outside problems.
> >
> > a. Glenns FAPRS system is meant to measure the occurrence of the
> > CRBs in session. Unlike the FIAT, where I see room for some
> improvement, I
> > think the FAPRS is pretty close to perfect, as long as it can be
> > established that raters can use it reliably. I know Glenn has
> already
> > studied this and right now we are trying to learn the system and
> > test out reliability between us and Glenn.
> >
> > b. The FIAT is designed to measure corresponding out of session
> > problems. In our lab, weve been trying to use a form of the FIAT
> to
> > track problems that we identified idiographically with two current
> > clients, and it has been very difficult. Weve been hoping to
> establish
> > a stable baseline in these problems over several weeks before
> > implementing the interventions, in classic A-B design fashion, but
> it
> > has been extremely frustrating because the clients arent reporting
> > the problems reliably, so the baselines are unstable.
> >
> > 3. Once CRBs can be reliably identified, and both in session CRBs
> and
> > corresponding out of session problems can be reliably measured, we
> can
> > put the whole thing together, and try to show a relationship
> between the
> > occurrences of in session CRBs (identified with a reliable
> interview and
> > measured with a reliable observer-based system), contingent
> therapist
> > responding to CRBs (measured with reliable observer-based system)
> and
> > corresponding out of session problems (reported reliably by the
> clients
> > using the FIAT or a similar diary card system). Glenn has already
> worked
> > on half of this equation  using lag analysis to show a
> relationship
> > between in session CRBs and in session therapist responding, but I
> dont
> > know if he has also added in the statistical relation to changes in
> out
> > of session problems.
> >
> > Thats the program. Any comments or thoughts are welcome.
> >
> > --
> > Jonathan W. Kanter, Ph.D.
> > Assistant Professor and Clinic Coordinator
> > Department of Psychology
> > University of Wisconsin-Milwaukee
> > P.O. Box 413
> > Milwaukee, WI 53201
> > Office: Garland Hall 238C
> > (414) 229-3834