This article was downloaded by: [George Mason Univ] On: 9 February 2010 Access details: Access Details:

[subscription number 908159977] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK

Journal of Mental Health Research in Intellectual Disabilities

Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t780786798

Psychometric Evaluation of Romanian Translations of the Behavior Problems Inventory-01 and the Nisonger Child Behavior Rating Form

Camelia E. Mircea a; Johannes Rojahn a; Anna J. Esbensen b a Department of Psychology, George Mason University, b Division of Developmental and Behavioral Pediatrics Cincinnati Children's Hospital Medical Center, Online publication date: 07 January 2010

To cite this Article Mircea, Camelia E., Rojahn, Johannes and Esbensen, Anna J.(2010) 'Psychometric Evaluation of

Romanian Translations of the Behavior Problems Inventory-01 and the Nisonger Child Behavior Rating Form', Journal of Mental Health Research in Intellectual Disabilities, 3: 1, 51 65 To link to this Article: DOI: 10.1080/19315860903520515 URL: http://dx.doi.org/10.1080/19315860903520515

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Journal of Mental Health Research in Intellectual Disabilities, 3:5165, 2010 Copyright Taylor & Francis Group, LLC ISSN: 1931-5864 print / 1931-5872 online DOI: 10.1080/19315860903520515

Journal 1931-5872 1931-5864 UMID of Mental Health Research in Intellectual Disabilities Vol. 3, No. 1, December 2009: pp. 00 Disabilities,

Psychometric Evaluation of Romanian Translations of the Behavior Problems Inventory-01 and the Nisonger Child Behavior Rating Form
CAMELIA E. MIRCEA AND JOHANNES ROJAHN
Department of Psychology George Mason University

C. E. Mircea Evaluation Psychometricet al.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

ANNA J. ESBENSEN
Division of Developmental and Behavioral Pediatrics Cincinnati Childrens Hospital Medical Center

Psychometric properties of Romanian translations of the Behavior Problems Inventory-01 (BPI-01; Rojahn, Matson, Lott, Esbensen, & Smalls, 2001) and the Nisonger Child Behavior Rating Form (NCBRF; Aman, Tass, Rojahn, & Hammer, 1996) were explored. Respondents completed the instruments for 115 children and adolescents with intellectual disabilities. Internal consistency of both instruments total scales was good to excellent (Cronbachs a ranging from .88 to .95) and the subscale internal consistencies were fair to excellent (with a ranging from .74 to .94). The results based on bivariate Spearman correlations and multiple regression analyses provided strong convergent and discriminant validity of both instruments. Overall, the Romanian versions of the behavior rating scales had very good reliability (internal consistencies) and convergent and discriminant validity. It is hoped that this study will be a catalyst for Romanian clinicians and researchers as well as for transcultural researchers working with Romanian individuals with intellectual disabilities to continue the exploration of the utility and psychometric quality not only of the BPI-01 and the NCBRF but also of other English language psychopathology assessment instruments.
Address correspondence to Johannes Rojahn, George Mason University, 10340 Democracy Lane, Suite 202, Fairfax, VA 22030. E-mail: jrojahn@gmu.edu 51

52

C. E. Mircea et al.

KEYWORDS intellectual disabilities, Romania, Behavior Problems Inventory-01 (BPI-01), Nisonger Child Behavior Rating Form (NCBRF), convergent validity, discriminant validity, internal consistency

Individuals with intellectual disabilities are often at risk for a variety of comorbid conditions, which is often referred to as dual diagnosis (Lunsky & Bradley, 2007; Nezu, 1994). Not only is there an increased risk for neurological ailments (e.g., seizure disorders, cerebral palsy) but many individuals with intellectual disabilities also exhibit severe behavior problems such as aggressive, destructive, self-injurious, and stereotypic behaviors (Cooper, Smiley, Allan, et al., 2009). Such behavior problems can have far-reaching consequences as they tend to impede the opportunity to learn desirable behavior, diminish the chances for successful community integration, burden support resources (Mansell, Ashman, Macdonald, & Beadle-Brown, 2002), and negatively affect staff morale (Hartley & MacLean, 2007; Rose & Rose, 2005). Moreover, individuals with intellectual disabilities seem to be especially vulnerable to develop psychiatric disorders (e.g., depression, anxiety, schizophrenia). A recent review by Rojahn and Meier (2010) found that international prevalence estimates of mental illness across all intellectual disability levels, sex, and age ranged from approximately 16% to 54% as compared with a 20% prevalence in the general U.S. population (e.g., Robins, Locke, & Regier, 1991). Although prevalence estimates of psychiatric disorders in the population with intellectual disability from several countries have been published (Cooper, Smiley, Jackson, et al., 2009; Dekker & Koot, 2003; Einfeld et al., 2006; Linna et al., 1999; Molteno, Molteno, Finchilescu, & Dawes, 2001; Rojahn & Meier, 2010), no such data are available from Romania. One of the reasons is that there are essentially no behavior rating scales for individuals with intellectual and/or developmental disabilities in Romania, a country where psychometric research in the field of intellectual disabilities is almost nonexistent due to lack of appropriate funding and specially trained researchers. What complicates the matter is that the current Romanian mental health administration establishes the individuals degree of disability according to medical and social criteria, thus using a broad disability category that includes intellectual disabilities, physical disabilities, HIV/AIDS, rare diseases, and psychiatric disorders (Cra, 2004). Specialized services for these different disabilities are provided by several national institutional authorities (such as the National Authority for Persons with Handicap; Ministry of Work, Social Solidarity and FamilyDepartment of Social Work and Family Policy; National Authority for Childrens Rights Protection; Department of Social Work and Child Protection, etc.) as well as by local and international nongovernmental organizations. According to the most recent report by the National Authority for Persons with Handicap (2009), of 113,938 people with disabilities, 11,867 were children

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

Psychometric Evaluation

53

(28 institutionalized and 11,839 noninstitutionalized) and 102,071 were adults (7,999 institutionalized and 94,072 noninstitutionalized). Given the lack of psychometrically sound instruments, we can only guess from those numbers how many individuals with dual diagnosis there must be in Romania. Several English language psychopathology assessment instruments have been developed during the past 25 years for different age groups and varying levels of functioning of individuals with intellectual disabilities. Some instruments cover a broad spectrum of psychopathology (such as several versions of the Aberrant Behaviour Checklist [ABC; Aman, Burrow, & Wolford, 1995; Aman, Singh, Stewart, & Field, 1985; Brown, Aman, & Havercamp, 2002], the Developmental Behavior Checklist [DBC; Einfeld & Tonge, 1994, 2002; Mohr, Tonge, Einfeld, 2005], the Assessment of Dual Diagnosis [ADD; Matson & Bamburg, 1998], the Diagnostic Assessment of the Severely Handicapped-II [DASH-II; Matson, 1995], the PAS-ADD ChecklistRevised [Moss, 2002], the Reiss Screen for Maladaptive Behavior [RSMB; Reiss, 1988], and the Nisonger Child Behavior Rating Form [NCBRF; Aman, Tass, Rojahn, & Hammer, 1996]), whereas others focus on select phenomena (such as the Glasgow Depression Scale for People with Autism [GDS-LD; Cuthill, Espie, & Cooper, 2003], the Anxiety Depression and Mood Screen [ADAMS; Esbensen, Rojahn, Aman, & Ruedrich, 2003], the Repetitive Behavior Scale-Revised [RBS-R; Bodfish, Symons, & Lewis, 1999], the Adult Scale of Hostility and Aggression: Reactive/Proactive [A-SHARP; Matlock & Aman, under review], the Childrens Scale of Hostility and Aggression: Reactive/ Proactive [C-SHARP; Farmer & Aman, 2009], and the Behavior Problems Inventory-01 [BPI-01; Rojahn, Matson, Lott, Esbensen, & Smalls, 2001]). The purpose of this study was to examine the psychometric properties (internal consistency and convergent and discriminant validity) of the Romanian translations of one broad spectrum and one narrow spectrum English language assessment instruments that have shown promise and utility as clinical as well as applied research tools: the Nisonger Child Behavior Rating Form (NCBRF; Aman et al., 1996) and the Behavior Problems Inventory-01 (BPI-01; Rojahn et al., 2001). Both instruments had been translated into several other languages, however, not into Romanian.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

METHOD Participants
Participants were 115 children and adolescents between the ages of 3 and 23 years (M = 14.9, SD = 4.9) with mild to profound intellectual disabilities and 115 professionals in education and mental health care as well as relatives who acted as respondents. Demographic characteristics and other types of pertinent information regarding the individuals with intellectual disabilities and the respondents are provided in Table 1. The participants

54

C. E. Mircea et al. TABLE 1 Demographic Information (N = 115) n Participants Sex Male Female Not reported Living situation With parents Residential centers Other (group home, lives alone, etc.) Not reported Level of intellectual disability Mild Moderate Severe Profound Not reported Respondents Relationship with the client Mental health professionala Special education teacher Other (parent, guardian relative, etc.) Not reported Length of time rater has known client 3 to 6 months More than 6 months Not reported Time spent daily with the client Less than 1 hr 1 hr More than 1 hr Not reported
a

51 46 18 14 93 1 7 4 13 80 12 6

44.3 40.0 15.7 12.2 80.9 0.9 6.1 3.5 11.3 69.6 10.4 5.2

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

43 49 12 11 1 101 13 5 24 72 14

37.4 42.6 10.4 9.6 0.9 87.8 11.3 4.3 20.9 62.6 12.2

Psychologist, social worker, therapist, and so on.

were receiving services from the agency responsible for individuals with intellectual disabilities, the Department of Social Work and Child Protection (DSWCP) of Suceava County (Romania). At the time the data were collected, these individuals were living in alternative noninstitutional housing (daycare residential and nonresidential centers, supported living homes, centers for occupational therapy, family-type homes, etc.). Members of DSWCP had expressed interest to the first author (a native Romanian) in piloting translations of the instruments in order to determine their usefulness for service delivery and accountability and it was agreed that the authors would perform the data analysis.1 Direct care staff members,
1. The study protocol was reviewed and approved by George Mason University Human Subjects Review Board and the Department of Social Work and Child Protection, Suceava, gave us permission to analyze the data.

Psychometric Evaluation

55

teachers, and relatives completed the Romanian versions of the BPI-01 and NCBRF.

Instruments
The assessment instruments described here were translated into Romanian by the first author, who has extensive knowledge of the psychological jargon in Romanian, English, and French. Due to the lack of resources back-translations were not possible. Instead, after the initial translation from English to Romanian, the first author compared the Romanian translation with French translations of the instruments to ensure that the psychological constructs used in all three languages were conceptually and semantically congruent (Flaherty et al., 1998). To ascertain that the translated versions of the instruments were culturally appropriate (content equivalence; Flaherty et al., 1998), mental health professionals from DSWCP, Suceava, with satisfactory proficiency in English reviewed the English and the Romanian versions of the instruments and provided comments and suggestions. After reconciling semantic differences between the first author and her Romanian colleagues it was concluded that the translation was precise, understandable, and conceptually and semantically equivalent. Behavior Problems Inventory-01 (BPI-01). The BPI-01 (Rojahn et al., 2001) is a 49-item informant-based assessment instrument to assess selfinjurious, stereotyped, and aggressive/destructive behavior in individuals with intellectual disabilities. Items were assigned a priori to three subscales: Self-Injurious Behavior (SIB; 14 items), Stereotyped Behavior (23 items), and Aggressive/Destructive Behavior (11 items). Each item is rated on a 5point frequency scale (0 = never to 4 = hourly) and a 4-point severity scale (0 = never/no problem to 3 = severe problem). In this study only the frequency scale data were collected because the frequency-scale and the severity-scale scores were shown repeatedly to be highly correlated (Rojahn, Aman, Matson, & Mayville, 2003; Rojahn et al., 2001). Various aspects of the validity and reliability of the BPI-01 have been explored in previous studies (e.g., Gonzlez et al., 2009; Mansell et al., 2002; Mansell, Beadle-Brown, Macdonald, & Ashman, 2003; Matson & Rivet, 2008). The reliability of the BPI-01 (interrater agreement, test-retest, and internal consistency) has been shown to vary from good to excellent. Its criterion-related validity was generally found to be high and factorial validity adequate. Nisonger Child Behavior Rating Form (NCBRF). The NCBRF (Aman et al., 1996) is an informant-based rating scale designed to specifically analyze the behaviors of children with intellectual disabilities. The instrument has two versions (teacher and parent), but only the teacher version was used in this study. The NCBRF consists of two parts, Social Competence and Problem Behavior. The Social Competence part has two five-item subscales, Compliance and Social Adaptive Behavior, and each item is scored on

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

56

C. E. Mircea et al.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

4-point scales (0 = not true to 3 = completely or always true). Six Problem Behavior subscales consist of 62 items: Conduct Problem (13 items), Insecure/Anxious (15 items), Hyperactive (8 items), Self-Injury/Stereotypic (9 items), Self-Isolated/Ritualistic (11 items), and Irritable (6 items). Items are rated on 4-point scales combining the frequency and the severity of the behavior (e.g., behavior occurred a lot or was a severe problem). Tass, Aman, Hammer, and Rojahn (1996) provided age- and sex-based norms for the NCBRF on a sample of 369 children and adolescents between the ages of 3 and 16 years who had been referred to a developmental disabilities diagnostic clinic in an urban center in the midwestern United States. Several psychometric studies have been conducted on the NCBRF that examined various forms of reliability (Aman et al., 1996; Girouard, Morin, & Tass, 1998; Gray et al., 2009; Lecavalier, Leone, & Wiltz, 2006; Tass, Girouard, & Morin, 2000; Tass & Lecavalier, 2000) and validity (Lecavalier, Aman, Hammer, Stoica, & Mathews, 2004; Norris & Lecavalier, 2007; Tass et al., 2000; Tass & Lecavalier, 2000) of the scale with mixed results.

RESULTS
Tables 2 and 3 present descriptive statistics for the BPI-01 and the NCBRF, respectively, broken down by level of intellectual disability and sex.

Internal Consistency
Cronbachs a coefficients were calculated for the total scores and the subscale scores of the two instruments (see Table 4). Using the interpretative

TABLE 2 Descriptive Statistics for the BPI-01 Subscales by Sex and Level of Intellectual Disability SIB Level of ID Male Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total M 0.0 4.9 3.4 4.9 3.7 1.0 8.0 7.3 4.0 7.1 SD 0.0 5.9 5.7 5.8 5.6 2.5 7.9 1.4 7.3 n 2 7 31 8 48 1 5 37 2 45 Stereotyped behavior M 9.0 19.9 27.9 35.0 27.1 10.0 42.0 30.4 35.0 31.4 SD 12.7 19.1 19.7 25.8 20.7 18.1 24.2 42.4 24.0 n 2 7 31 8 48 1 5 37 2 45 Aggressive/ Destructive behavior M 6.0 7.4 6.0 5.8 6.2 8.0 13.4 7.5 2.0 7.9 SD 8.5 7.8 10.2 7.9 9.2 6.3 8.6 2.8 8.4 n 2 7 31 8 48 1 5 37 2 45

Female

Note. ID = Intellectual Disability; SIB = Self-Injurious Behavior.

Psychometric Evaluation

57

TABLE 3 Descriptive Statistics for the NCBRF Subscales by Sex and Level of Intellectual Disability Level of ID Male Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total Mild Moderate Severe Profound Total M 3.7 4.6 3.4 1.6 3.3 5.0 6.2 3.2 5.5 3.7 SD n M SD n

Female

Male
Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

Female

Male

Female

Male

Female

Compliant/Calm 3.2 3 4.0 7 2.4 31 1.6 8 2.7 49 1 2.2 5 2.6 37 6.4 2 2.8 45 Conduct Problem 2.0 2.0 3 7.4 10.0 7 5.0 6.9 31 3.3 6.5 8 4.9 7.1 49 15.0 1 16.2 7.3 5 9.9 9.8 37 1.5 2.1 2 10.4 9.5 45 Hyperactive 9.0 1.7 3 10.9 6.5 7 9.9 6.3 31 8.9 5.6 8 9.8 6.0 49 5.0 1 13.4 3.8 5 11.7 6.2 37 6.0 0.0 2 11.5 6.0 45 Self-Isolated/Ritualistic 5.3 3.8 3 7.3 3.5 7 8.8 5.2 31 11.3 5.5 8 8.8 5.0 49 10.0 1 12.4 9.0 5 11.6 7.1 37 18.0 12.7 2 11.9 7.3 45

Adaptive Social 3.7 3.5 3 3.3 3.4 7 2.6 2.5 31 0.4 0.7 8 2.4 2.6 49 6.0 1 4.0 1.9 5 2.8 3.3 37 3.5 4.9 2 3.0 3.1 45 Insecure/Anxious 5.3 2.1 3 9.0 6.7 7 3.0 2.6 31 5.1 4.6 8 4.4 4.2 49 23.0 1 19.6 9.9 5 8.6 6.1 37 6.5 0.7 2 10.1 7.5 45 Self-Injury/Stereotypic 1.3 1.5 3 5.6 4.0 7 6.5 4.3 31 6.8 4.3 8 6.1 4.2 49 6.0 1 12.2 4.6 5 10.6 8.0 37 10.0 7.1 2 10.6 7.5 45 Irritable 3.0 1.7 3 5.1 4.7 7 4.5 4.4 31 6.9 5.5 8 4.9 4.5 49 15.0 1 12.6 3.2 5 8.4 5.0 37 10.0 7.1 2 9.1 5.0 45

Note. NCBRF = Nisonger Child Behavior Rating Form; ID = Intellectual Disability.

58

C. E. Mircea et al.

TABLE 4 Descriptive Statistics for the Total Sample and Internal Consistency n BPI-01 Self-Injurious Behavior Stereotyped Behavior Aggressive/Destructive Behavior Total BPI-01 NCBRF Social Competence Compliant/Calm Adaptive Social Total Social Competence Problem Behavior Conduct Problem Insecure/Anxious Hyperactive Self-Injury/Stereotypic Self-Isolated/Ritualistic Irritable Total Problem Behavior 112 111 112 111 M 5.90 29.00 6.90 SD 7.20 22.60 8.70 Min 0 0 0 Max 31 85 40 Skew 1.6 0.6 1.5 a .82 .94 .91 .95

114 114 114 114 114 114 114 114 114 114

3.50 2.60

2.70 2.80

0 0

10 11

0.4 1.0

.76 .80 .88 .91 .83 .78 .84 .74 .84 .94

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

7.70 7.40 10.90 7.80 10.40 7.00

8.80 6.70 5.80 6.40 6.20 5.00

0 0 0 0 0 0

32 36 24 27 27 18

1.1 1.6 0.0 0.8 0.6 0.5

Note. BPI-01 = Behavior Problems Inventory-01; NCBRF = Nisonger Child Behavior Rating Form.

guidelines by Cicchetti and Sparrow (1990),2 internal consistency of the total scores of the instruments was good to excellent, ranging from .88 to .95. The BPI-01 subscales had good-to-excellent internal consistency. The NBCRF subscales (Social Competence and Problem Behavior) showed adequate-togood internal consistency (a coefficients ranging from .74 to .91).

Convergent and Discriminant Validity

As a sign of convergent validity of the instruments, it was hypothesized that the NCBRF subscale Self-Injury/Stereotypic would predict the BPI-01 Self-Injurious Behavior and Stereotyped Behavior subscales whereas the NCBRF Conduct Problem would predict BPI-01 Aggressive/Destructive Behavior. First, bivariate Spearman r correlations were computed between the NCBRF and the BPI-01 subscales and the demographic variables (sex, age, and level of intellectual disability; see Table 5). Table 5 shows that 18 of 33 bivariate correlations between the BPI-01, the three demographic variables, and the NCBRF subscales were statistically significant (p < .01). The BPI-01 subscale SelfInjurious Behavior was significantly correlated with sex. Table 2 shows that

2. Reliability interpretative guidelines: below .70 = unacceptable, between .70 and .79 = fair, between .80 and .89 = good, .90 and above = excellent.

Psychometric Evaluation

59

TABLE 5 Convergent and Discriminant Validity: Bivariate Spearman r Correlations Between Demographic Variables, NCBRF, and BPI-01 Subscales BPI-01a Aggressive/ Self-Injurious Stereotyped Destructive behavior behavior behavior Demographics Sex 0.30** Level of ID 0.05 Age 0.00 NCBRF Social Competence Compliant/Calm 0.07 Adaptive Social 0.05 NCBRF Problem Behavior Conduct Problem 0.49*** Insecure/Anxious 0.25** Hyperactive 0.37*** Self-Injury/ 0.64*** Stereotypic Self-Isolated/ 0.17 Ritualistic Irritable 0.51*** 0.08 0.14 0.07c 0.04 0.02 0.47*** 0.13 0.37*** 0.72** 0.26** 0.45*** 0.15 0.17b 0.07c 0.39*** 0.43*** 0.79*** 0.51*** 0.41*** 0.46*** 0.09 0.59*** 0.20* 0.21* 0.04 0.07 0.19d 0.26** 0.12 0.08 0.07 0.11 0.20*c 0.23*c Demographics Level of ID

Sex

Age

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

0.34** 0.12 0.44*** 0.25**c 0.15 0.03 0.34** 0.09 0.26** 0.18

0.42*** 0.01

Note. NCBRF = Nisonger Child Behavior Rating Form, BPI-01 = Behavior Problems Inventory-01; ID = Intellectual Disability. Bold and underlined font = expected significant correlations (convergent validity). a BPI-01 frequency scales. bA negative correlations with Level of ID means that a higher score on the BPI-01 Aggressive/Destructive Behavior was more highly associated with higher functioning individuals than with low-functioning individuals (nonsignificant). cNegative correlations with Age mean that higher scores on the BPI-01 or the NCBRF were more highly associated with younger age than older age. dThis negative correlation with Sex means that a higher NCBRF Hyperactivity score was more highly associated with male than female sex (nonsignificant). *p < .05. **p < .01. ***p < .001.

females had higher Self-Injurious Behavior scores than males. Sex was also significantly correlated with NCBRF Compliant/Calm and Adaptive Social subscales as well as with Self-Injury/Stereotypic. Age and level of intellectual disability were not correlated with the BPI-01 Stereotyped Behavior and Aggressive/Destructive Behavior. However, level of intellectual disability was positively correlated with the NCBRF subscales Conduct Problem, Insecure/Anxious, Self-Injury/Stereotypic, Self-Isolated/Ritualistic, and Irritable, and chronological age was negatively correlated with the Compliant/Calm, Adaptive Social, and Insecure/Anxious. As bivariate correlations were numerous and generally quite high, multiple regression analyses were computed to identify variables that independently predicted BPI-01 subscales. Three stepwise multiple regression analyses were computed, one for each BPI-01 subscale. NCBRF subscales and demographic variables that had significant bivariate correlations with the respective BPI-01 subscale but

60

C. E. Mircea et al.

TABLE 6 Predicting BPI-01 Subscales From NCBRF Problem Behavior Subscales by Hierarchical Multiple Regression Analysis Step 1 Outcome Self-Injurious Behavior Predictors Sex Conduct Problem Insecure/Anxious Hyperactive Irritable Self-Injury/Stereotypic 0.34*** Stereotyped Behavior Conduct Problem Hyperactive Self Isolated/Ritualistic Irritable Self-Injury/Stereotypic 0.30*** Aggressive/Destructive Behavior Compliant/Calm Adaptive Social Insecure/Anxious Hyperactive Self-Injury/Stereotypic Irritable Conduct Problem 0.39*** 0.20 0.08 0.10 0.14 0.28 0.16 0.32 0.12 0.19 0.09 * * 0.23*** 0.22 0.10 0.08 0.04 0.15 0.05 0.66 *** 0.14*** DR2 b 1.59 0.39 0.47 0.03 0.40 ** *** * 0.17*** 0.19 0.04 0.05 0.10 0.64 *** DR2 Step 2 b 0.06 0.25 0.24 * 0.03 0.02 0.61 ***

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

**

Note. BPI-01 = Behavior Problems Inventory-01; NCBRF = Nisonger Child Behavior Rating Form. *p < .05. **p < .01. ***p < .001.

were not hypothesized to be a confirmatory variable had to be controlled for and hence were entered into the regression equation in Step 1. NCBRF subscales that were hypothesized to predict the respective BPI-01 subscales were subsequently entered in Step 2. The outcome was by and large consistent with expectations providing support for convergent validity of the BPI-01 and the NCBRF Problem Behavior subscales. The BPI-01 SIB subscale was statistically significant related only to the NCBRF Self-Injurious/ Stereotypic subscale. When the NCBRF Self-Injurious/Stereotypic subscale was entered in the regression analysis in Step 2, NCBRF Self-Injurious/ Stereotypic and Insecure/Anxious were the only significant positive predictors that remained in the equation, whereas the other NCBRF subscales (Conduct Problem and Irritable) lost the significance they had in Step 1 (see Table 6). Insecure/Anxious had a negative predictive value for BPI-01 SIB. Consistent with our hypotheses, the NCBRF Self-Injurious/Stereotypic subscale was the only statistically significant predictor of the BPI-01 Stereotyped Behavior subscale and the NCBRF Conduct Problem subscale was the only statistically significant predictor of the BPI-01 Aggressive/Destructive Behavior subscale.

Psychometric Evaluation

61

DISCUSSION
This study explored the psychometric properties of Romanian translations of two behavior rating scales for individuals with intellectual disabilities, the BPI-01 and the NCBRF.

Internal Consistency
Internal consistency of the Romanian BPI-01 was found to be good and even somewhat better than some studies on the English version have shown so far (Gonzlez et al., 2009; Gray et al., 2009; Rojahn et al., 2001; van Ingen, Moore, Zaja, & Rojahn, 2010). For the NCBRF Social Competence subscales and for the six NCBRF Problem Behavior subscales internal consistency ranged between fair to excellent and generally fell in the ballpark found by other researchers working on the English version (Aman et al., 1996; Gray et al., 2009; Lecavalier et al., 2006; Tass & Lecavalier, 2000) and a French version (Girouard et al., 1998; Tass et al., 2000; Tass & Lecavalier, 2000).

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

Convergent and Discriminant Validity

Convergent validity was well supported for both instruments. As hypothesized, all three BPI-01 subscales were statistically significant and highly related to their clinically equivalent subscales of the NCBRF. The BPI-01 SIB and Stereotype Behavior subscales were related to the NCBRF Self-Injurious/Stereotypic subscale after we controlled for the four remaining NCBRF Problem Behavior subscales and sex. The BPI-01 Aggressive/Destructive Behavior subscale was associated only with the NCBRF Conduct Problem. The only deviation from our prediction was that the BPI-01 SIB subscale was not only predicted by NCBRF Self-Injurious/Stereotypic subscale but also by Insecure/Anxious. Interestingly, Insecure/Anxious showed a negative relationship to BPI-01 SIB, which is not consistent with previous convergent validity studies when the BPI-01 was compared with ABC (Rojahn et al., 2003) or the ADD (Rojahn, Matson, Naglieri, & Mayville, 2004). Future studies will show whether this was a sampling artifact or a relevant finding. However, it seems an implausible notion that insecurity and anxiousness might be protective factors when it comes to the frequency of SIB. Nonetheless, because Insecure/Anxious was the only variable that was not predicted to be significant in predicting any of the BPI-01 SIB, we can conclude that the discriminant validity of the two instruments was also substantiated.

Limitations of the Study

This study was based on a relatively small nonrepresentative sample of an administratively predefined group of individuals who received services from

62

C. E. Mircea et al.

a state agency, which led to considerable misrepresentations of the different levels of intellectual disability (i.e., underrepresentation of individuals with mild intellectual disabilities and overrepresentation of individuals with severe and profound intellectual disabilities). Another concern was the nonstandard translation method, which could have impacted the results. Typically, a back-translation should have been performed to double-check the semantic equivalence between the original and the translated versions of the instruments. Nonetheless, this is the first psychometric study on Romanian versions of the NCBRF and the BPI-01 and it is hoped that this will fill an important research gap in the developmental disabilities practice and applied research in that country.

Conclusion
Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

In summary, the Romanian version of these two behavior rating instruments had good internal consistency and convergent and discriminant validity. It is hoped that this study will provide an incentive to Romanian clinicians and researchers, as well as to cross-cultural researchers who include Romanian samples of individuals with intellectual disabilities, to use these instruments in their work and to researchers to consider refining them.

ACKNOWLEDGMENTS
We thank Florin Tarnauceanu, executive director of the Department of Social Work and Child Protection of Suceava County (Romania), and all the mental health professionals who administered the assessment instruments.

REFERENCES
Aman, M. G., Burrow, W. H., & Wolford, P. L. (1995). The Aberrant Behavior Checklist-Community: Factor validity and effect of subject variables for adults in group homes. American Journal on Mental Retardation, 100, 28392. Aman, M. G., Singh, N. N., Stewart, A. W., & Field, C. J. (1985). The Aberrant Behavior Checklist: A behavior rating scale for the assessment of treatment effects. American Journal of Mental Deficiency, 89, 485491. Aman, M. G., Tass, M. J., Rojahn, J., & Hammer, D. (1996). The Nisonger CBRF: A child behavior rating form for children with developmental disabilities. Research in Developmental Disabilities, 17, 4157. Bodfish, J. W., Symons, F. J., & Lewis, M. H. (1999). Repetitive Behavior Scales. Morganton, NC: Western Carolina Center Research Reports. Brown, E. C., Aman, M. G., & Havercamp, S. M. (2002). Factor analysis and norms for parent ratings on the Aberrant Behavior Checklist-Community for young people in special education. Research in Developmental Disabilities, 23, 4560.

Psychometric Evaluation

63

Cra, A. C. (2004). Annex VII: Romania. Pomona health indicators for people with intellectual disabilities. Report submitted to the European Commission and General Directorate for Health and Consumer Protection. Retrieved February 27, 2009, from http://ec.europa.eu/health/ph_projects/2004/action1/docs/ action1_2004_a08j_14_en.pdf Cicchetti, D. V., & Sparrow, S. S. (1990). Assessment of adaptive behavior in young children. In J. J. Johnson & J. Goldman (Eds.), Developmental assessment in clinical child psychology: A handbook (pp.173196). New York: Pergamon. Cooper, S. A., Smiley, E., Allan, L. M., Jackson, A., Finlayson, J., Mantry, D., et al. (2009). Adults with intellectual disabilities: Prevalence, incidence and remission of self-injurious behaviour, and related factors. Journal of Intellectual Disability Research, 53, 200216. Cooper, S. A., Smiley, E., Jackson, A., Finlayson, J., Allan, L., Mantry, D., et al. (2009). Adults with intellectual disabilities: Prevalence, incidence and remission of aggressive behaviour and related factors. Journal of Intellectual Disability Research, 53, 217232. Cuthill, F. M., Espie, C.A., & Cooper, S.A. (2003). Development and psychometric properties of the Glasgow Depression Scale for people with a Learning Disability. British Journal of Psychiatry, 182, 347353. Dekker, M., & Koot, H. M. (2003). DSM-IV disorders in children with borderline to moderate intellectual disability. I: Prevalence and impact. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 915922. Einfeld, S. L., Piccinin, A. M., Mackinnon, A., Hofer, S. M., Taffe, J., Gray, K. M., et al. (2006). Psychopathology in young people with intellectual disability. The Journal of the American Medical Association, 296, 19811989. Einfeld, S. L., & Tonge, B. J. (1994). Manual for the Developmental Behaviour Checklist: Primary Carer Version (DBC-P). Melbourne, Australia: Monash University Centre for Developmental Psychiatry. Einfeld, S., & Tonge, B. J. (2002). Manual for the Developmental Behaviour Checklist: Primary Carer Version (DBC-P) and Teacher Version (DBC-T) (2nd ed.). Clayton, Australia: University of New South Wales and Monash University Centre for Developmental Psychiatry and Psychology. Esbensen, A. J., Rojahn, J., Aman, M. G., & Ruedrich, S. (2003). The reliability and validity of an assessment instrument for anxiety, depression, and mood among individuals with mental retardation. Journal of Autism and Developmental Disorders, 33, 617629. Farmer, C. A., & Aman, M. G. (2009). Psychometric properties of the Childrens Scale of Hostility and Aggression: Reactive/Proactive (C-SHARP). Research in Developmental Disabilities, 31, 270280. Flaherty, J. A., Gaviria, F. M., Pathak, D., Mitchell, T., Wintrob, R., Richmond, J. A., et al. (1998). Developing instruments for cross-cultural psychiatric research. Journal of Nervous and Mental Disease, 176, 257263. Girouard, N., Morin, I. N., & Tasse, M. J. (1998). tude de fidlit test-retest et accord inter-judges de la Grille Dvaluation Comportmentale pour Enfants Nisonger (GCEN) [A study of test-retest and inter-rater reliability of the Nisonger Child Behavior Rating Form (NCBRF)]. Revue Francophone de la Dficience Intellectuelle, 9, 127135.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

64

C. E. Mircea et al.

Gonzlez, M. L., Dixon, D. R., Rojahn, J., Esbensen, A. J., Matson, J. L., Terlonge, C., et al. (2009). The Behavior Problems Inventory: Reliability and factor validity in institutionalized adults with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 22, 223235. Gray, A., Booth, A., Rojahn, J., Macken, J., DeLitta, D., Kimbrell, K., et al. (2009, August). Teacher ratings of the BPI and the NCBRF: Psychometric properties in youths with intellectual disabilities. Poster session presented at the annual meeting of the American Psychological Association, Toronto, Canada. Hartley, S. L., & MacLean, W. E. (2007). Staff-averse challenging behaviour in older adults with intellectual disabilities. Journal of Applied Research in Intellectual Disabilities, 20, 519528. Lecavalier, L., Aman, M. G., Hammer, D., Stoica, W., & Mathews, G. L. (2004). Factor analysis of the Nisonger Child Behavior Rating Form in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 34, 709721. Lecavalier, L., Leone, S., & Wiltz, J. (2006). The impact of behavior problems on caregiver stress in young people with autism spectrum disorders. Journal of Intellectual Disability Research, 50, 172183. Linna, S., Moilanen, I., Ebeling, H., Piha, J., Kumpulainen K., Tamminen, T., et al. (1999). Psychiatric symptoms in children with intellectual disability. European Child and Adolescent Psychiatry, 8, 7782. Lunsky, Y., & Bradley, E. (2007). Dual diagnosis or dual confusion: Limitations when utilizing non-specialist clinical data. Journal on Developmental Disabilities, 13, 185189. Mansell, J., Ashman, B., Macdonald, S., & Beadle-Brown, J. (2002). Residential care in the community for adults with intellectual disability: Needs, characteristics, and services. Journal of Intellectual Disability Research, 46, 625633. Mansell, J., Beadle-Brown, J., Macdonald, S., & Ashman, V. (2003). Resident involvement in activity in small community homes for people with learning disabilities. Journal of Applied Research in Intellectual Disabilities, 16, 6374. Matlock, S. T., & Aman, M. G. (under review). Development of the Adult Scale of Hostility and Aggression: Reactive/Proactive (A-SHARP). Manuscript submitted for publication. Matson, J. L. (1995). The Diagnostic Assessment of the Severely Handicapped-Revised (DASH-II). Baton Rouge, LA: Disability Consultants, LLC. Matson, J. L., & Bamburg, J. W. (1998). Reliability of the Assessment of Dual Diagnosis (ADD). Research in Developmental Disabilities, 19, 8995. Matson, J. L., & Rivet, T. T. (2008). Reliability and factor structure of the Autism Spectrum Disorders-Behavior Problems for Adults (ASD-BPA) with intellectual disabilities. Journal of Mental Health Research in Intellectual Disabilities, 1, 3447. Mohr, C., Tonge, B. J., & Einfeld, S. L. (2005). The development of a new measure for the assessment of psychopathology in adults with intellectual disability. Journal of Intellectual Disability Research, 49, 469480. Molteno, G., Molteno, C. D., Finchilescu, G., & Dawes, A. R. L. (2001). Behavioural and emotional problems in children with intellectual disability attending special schools in Cape Town, South Africa. Journal of Intellectual Disability Research, 45, 515520. Moss, S. (2002). The PAS-ADD Checklist-Revised. Brighton, UK: Pavilion Publishing.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010

Psychometric Evaluation

65

National Authority for Persons with Handicap. (2009, June). Quarterly Statistical Bulletin. Retrieved November 22, 2009, from http://www.anph.ro/eng/ news.php?ida=6 Nezu, A. M. (1994). Introduction to special section: Mental retardation and mental illness. Journal of Consulting and Clinical Psychology, 62, 45. Norris, M., & Lecavalier, L. (2007, August). Assessing the validity of the Nisonger Child Behavior Rating Form in a community sample of youth with intellectual disability. Poster session presented at the annual meeting of the American Psychological Association, San Francisco, CA. Reiss, S. (1988). Test manual for the Reiss Screen for Maladaptive Behavior. Orlando Park, IL: International Diagnostic Systems. Robins, L. N., Locke, B. Z., & Regier, D. A. (1991). An overview of psychiatric disorders in America. In L. N. Robins & D. A. Regier (Eds.), Psychiatric disorders in America: The epidemiologic catchment area study (pp. 329366). New York: The Free Press. Rojahn, J., Aman, M. G., Matson, J. L., & Mayville, E. (2003). The Aberrant Behavior Checklist and the Behavior Problems Inventory: Convergent and divergent validity. Research in Developmental Disabilities, 24, 391404. Rojahn, J., Matson, J. L., Lott, D., Esbensen, A. J., & Smalls, Y. (2001). The Behavior Problems Inventory: An instrument for the assessment of self-injury, stereotyped behavior, and aggression/destruction in individuals with developmental disabilities. Journal of Autism and Developmental Disorders, 31, 577588. Rojahn, J., Matson, J. L., Naglieri, J. A., & Mayville, E. (2004). Relationships between psychiatric conditions and behavioral problems among adults with mental retardation. American Journal on Mental Retardation, 109, 2133. Rojahn, J., & Meier, L. J. (2010). Psychopathology in intellectual disabilities: Epidemiology and assessment. In R. M. Hodapp (Ed.), International review of research in mental retardation (Vol. 38, pp. 239287). Amsterdam: Elsevier. Rose, D., & Rose, J. (2005). Staff in services for people with intellectual disabilities: The impact of stress on attributions of challenging behavior. Journal of Intellectual Disability Research, 49, 827848. Tass, M. J., Aman, M. G., Hammer, D., & Rojahn, J. (1996). The Nisonger Child Behavior Rating Form: Age and sex effects and norms. Research in Developmental Disabilities, 17, 5975. Tass, M. J., Girouard, N., & Morin, I. N. (2000). Traduction et validation canadiennefranaise du Nisonger Child Behavior Rating Scale [French Canadian translation and validation of the Nisonger Child Behavior Rating Form]. Canadian Psychology/ Psychologie Canadienne, 41, 116123. Tass, M. J., & Lecavalier (2000). Comparing parent and teacher ratings of social competence and problem behaviors. American Journal on Mental Retardation, 105, 252259. van Ingen, D. J., Moore, L. L., Zaja, R. H., & Rojahn, J. (2010). The Behavior Problems Inventory (BPI-01) in community-based adults with intellectual disabilities: Reliability and concurrent validity vis--vis the Inventory for Client and Agency Planning (ICAP). Research in Developmental Disabilities, 31, 98107.

Downloaded By: [George Mason Univ] At: 19:05 9 February 2010