Research Support

DBT AC-I Psychometric Properties

Observer-Rated

When the DBT AC-I was used by observers who had been trained to reliability in the original DBT Adherence Coding Scale (DBT ACS) it was found to have:

• Excellent inter-rater reliability (ICC = 0.93).

• High convergent validity with the DBT ACS computed global score (r = 0.90, p < .001).

• Strong ability to identify adherent sessions (sensitivity = 91%) and non-adherent sessions (specificity = 80%) as defined by the DBT ACS using a cut-off score of 23.

Take-Home Message: The DBT AC-I offers an efficient and effective option for evaluating adherence to DBT when used by trained observers.

Therapist Self-Report

When the DBT AC-I was used by therapists who read the training manual and then rated their own adherence it was found to have:

• Low inter-rater reliability between therapists and trained observers (ICC = 0.09).

• Poor convergent validity with the observer-rated DBT ACS computed global score (r = 0.05, p = .63).

• No ability to distinguish between adherent and non-adherent sessions as defined by the DBT ACS.

The degree to which therapists were able to accurately rate their own sessions varied widely (68-100% accuracy across therapists, mean = 84%). Therapists who were more accurate had:

• Higher observer-rated adherence to DBT.

• More knowledge of DBT.

Take-Home Message: Therapists vary considerably in their ability to accurately evaluate their own adherence to DBT. Therefore, therapists’ self-rated adherence on the DBT AC-I should not be assumed to reflect their actual adherence to DBT.