Suggested citation: Sokoli E, Bodinaku B, Paço J, Gjergji O. Assessment of Albanian adults psychopathology through cross-informant agreement. Alban Med J 2016;2:34-43.
Assessment of Albanian adults’ psychopathology through cross-informant agreement
Elvisa Sokoli1, Blerta Bodinaku1, Joana Paço1, Ornela Gjergji1
1 Faculty of Social Sciences, University of Tirana, Albania.
Corresponding author: Elvisa Sokoli
Address: Faculty of Social Sciences, Tirana University, Tirana, Albania;
Telephone: +355696050060; E-mail: elvis.sokoli@gmail.com
Abstract
Aim: Adult psychopathology is principally assessed with self-report measures. The person assessed frequently offers a biased perspective of self. People knowing the person well (informants) usually view the person being assessed more realistically. The aim of this study was to compare respondents’ and informants’ perspectives in the Albanian adult population.
Methods: Adult psychopathology in this study was assessed using two widely known psychometric measures of Achenbach System of Empirically Based Assessment (ASEBA): Adult Self Report for Ages 18-59 (ASR) and Adult Behavior Checklist for Ages 18-59 (ABCL). Normative data were collected in an epidemiological cross-sectional design including a nationwide representative sample of 1500 individuals (750 respondents and 750 informants).
Results: We calculated correlations between adult respondents and informants’ reports of problems for the full sample. High correlations were obtained both for the global scale (r=0.83) and for the eight syndrome scales (range from r=0.58 to r=0.84) in the sample. Tests of mean level differences showed that respondents self-reported significantly more Internalizing Problems (P<0.05) compared to informants’ reports. In contrary, informants reported significantly higher levels of Externalizing Problems (P<0.001) compared to respondents’ self-reports.
Conclusion: Our findings indicate that both parties perceive more their own discomfort and suffering than that of the other one. These findings can be described, among other things, by the traditional behavioral rules in the Albanian society. This study also confirmed the ASEBA instruments as valid and suitable tools for practical use in Albania.
Keywords: ABCL, ASR, collateral report, cross-informant agreement, psychopathology, self-report.
Introduction
Assessment of adult psychopathology relies mostly on data obtained from the person being assessed. The data are obtained through different methods, but typically through self-report measures, questionnaire forms, or structured interviews (1). Assessment decisions using self-report are much more source dependent, because they rely exclusively on the perspective of the patient (2), which can be affected by memory problems, confusion, a distorted self-image and the denial or failure to recognize personal characteristics, or problematic manifestations (3,4).
While trying to assess for symptoms of psychopathology, clinicians note that there are cases when the respondent’s (defined as the person being assessed) answers, are an inadequate basis for making assessment decisions. Reports from people who know well the person being assessed (defined as informants) are likely to be sought when self-reports are questionable, owing to cognitive limitations, substance use or addressing other similar issues (5). In order to develop a comprehensive assessment, data from multiple informants are obtained with parallel self-report and collateral-reports (defined as reports from informants) and they can be compared, aggregated and used in many different ways (6). Therefore parallel instruments were developed for people who know well the person being assessed and can report on their behalf.
Agreements/disagreements between ratings obtained from the person being assessed and different informants who have different perspectives on the people being assessed are defined as cross-informant agreements/disagreements (5).
From an outside perspective, it seems like the problem of the lack or distortion of information reported from the person who is being assessed is solved by asking an informant’s report, but discrepancies are often found between self-reports and reports by others regarding symptoms of psychopathology (6,7). Even if we expect that self-reports and other ratings on the observed samples of behavior should be similar, usually discrepant parallel reports are gathered, which then makes it difficult to properly assess an individual’s psychological functioning in clinical settings (8).
There are a variety of factors related to discrepancies, and various theories have been proposed to explain the meaning behind discrepant reports (9). Each informant’s ratings are apt to be affected by such factors as the characteristics of the informant, the nature of the relationship with the respondent, the situation in which the respondent is observed. Another difference to be considered is also the difference between the roles of informants (e.g. spouse/partner, siblings, parents, etc.) because even informants who have nominally similar roles with the person being assessed may nevertheless differ in how they rate the person being assessed because of differences in personal characteristics or knowledge of the person’s behavior (7).
By comparing self-ratings with collateral-ratings, clinicians can identify similarities and differences between reports which could assist in identifying problems that are not recognized or are not reported by the respondent, but that are recognized and reported by informants (7). Cross-informant comparisons can also be used to determine the degree to which particular problems are reported by all informants versus only one or few of informants, in order to indicate that the problems are relatively specific to certain situations or to certain interaction partners (6,7).
Studies have indicated that discrepancies between respondents and informants reports may be of interests, because if analyzed properly they might reflect underlying problems that contribute to individual’s psychopathology (10). If discrepant reports are conceptualized in this way, they can prove useful information and may lead to more accurate assessments of individual psychological functioning (8). For example, if the informant chosen by the respondent is a sibling or a parent, the clinician might combine the information given by each, to create an idea of their relationship and how it would be helpful to be incorporated to the conceptualization of the individual’s symptomatology and treatment plan (6,7).
It is important to note that during assessment of adults who seek mental health services for themselves, it may be impractical to request assessment data from other informants, because adults might refuse to grant permission to obtain data from people who know them. This means that there might be practical obstacles to multi informant assessment of adults for clinical or research purposes (5).
Usually, agreements among informants tend to be modest, but nevertheless each informant may contribute useful information about different aspects of a person’s functioning (6). Often discrepancies may be found between self-reports and informant-reports, but if analyzed properly and conceptualized positively, they can shed light to different aspects of psychopathology. Achenbach stated (6,7), that it is strongly recommended to use data from multiple informants in order to advance clinical assessment and the search for causes and cures of psychopathology.
To date, Thomas Achenbach is among the few researchers who chose to rely constantly in cross-informant approach in assessing people psychopathological functioning. The Achenbach System of Empirically Based Assessment (ASEBA) constitutes an all-inclusive evidence-based assessment system assessing competencies, adaptive functioning, social, emotional, and behavior problems of individuals from an age of 1.5 years to 90 years. One of the most distinguishing features of ASEBA is that it provides comparisons of information obtained about people’s functioning from parallel questionnaires completed by different informants. ASEBA instruments are consistently used for clinical assessment, outcome evaluations, epidemiological survey and research (11). All the ASEBA forms are constructed on the same basic principle, an empirically based approach to taxonomy and assessment of adaptive and maladaptive functioning (12).
For the purpose of this study we will be using the ASEBA parallel form for assessing adults of age 18-59 years: The Adult Self-Report for Ages 18-59 (ASR) and Adult Behavior Checklist for ages 18-59 (ABCL). These parallel forms facilitate comparisons between adults’ perceptions of their own psychological functioning (completing ASR) and a close person’s report of the respondent psychological functioning (completes ABCL) (11,12).
The current study aimed to compare data obtained from respondent and close informants on adult’s emotional, social and behavioral problems using ASEBA adult forms in the Albanian general population. For the purpose of comparison, we sought data from parents, children, siblings, spouses/partners, friends and relatives of the person being assessed. Our analyses on cross-informant agreements were intended to examine the consistency between adult’s self-report and close informant’s ratings of adults’ emotional and behavior problems. It was expected that Albanian adults would have a moderate to high level of agreement with their close informants. It was also expected that levels of agreements would be higher for ratings of adults’ internalizing behavior problems relative to their externalizing behavior problems.
Methods
This was a cross-sectional study covering the time period 2013-2014.
Participants
Normative data were collected in an epidemiological cross-sectional design. A total of N=1500; 750 respondents and 750 informants from nationally demographically representative distribution participated in this study. The sample represents the population of Albania regarding three demographic variables: gender, age and counties based on Census 2011 data (13). The respondents’ sample participating in this study were stratified by two age groups (18-35 and 36-59 years old), gender, 12 Albanian counties and area (urban and rural) in order to guarantee the representativeness of the Albanian population. The sample of respondents’ aged 18-59 years (mean age M=37.32, SD=12.75), comprised n=377 men (50.3%) and n=373 women (49.7%). They were all invited to self-report on their psychological functioning by completing ASR. After completing the ASR, each respondent was asked to nominate a close person who knew them well (an informant) to complete a parallel form describing the psychological functioning of the respondent. If the respondent granted informed consent, an interviewer contacted the informant to complete the informant version of the questionnaire.
The informants’ sample aged 18-59 years (mean age M=35.09, SD=12.77) comprised n=299 males (39.9%) and n=451 females (60.1%). They were all invited to complete the ABCL about an 18-59 year old person they knew well who nominated the informant to give a collateral-report on the respondent’s psychological functioning. There were 750 matched pairs of answered questionnaires divided into groups by the kind of the relationship. Informant reports were taken from a spouse/partner (39%), friend or known person (21%), sibling (15%), child (13%), parent (6%) and relative (5%).
Instruments
The ASEBA for adults of age 18-59 years consists of two parallel questionnaires: The Adult Self-Report (ASR) and The Adult Behavior Checklist (ABCL) that facilitate comparisons between people’s perceptions of their own psychological functioning and other people’s perceptions of their psychological functioning (12).
The ASR and ABCL for adults were introduced in Albania in 2015 and will be currently used in mental health services, such as clinics, community mental health centers, substance abuse treatment settings, educational settings, children’s services, forensic settings such as courts, division programs, medical settings and outcome evaluations (11). ASR and ABCL have proven to be reliable and valid measures for gathering self-reported and collateral information regarding adaptive and maladaptive psychological functioning of adults (9,14,15). The ASR and ABCL for ages 18-59, assess behavioral, emotional, and social problems, including also adaptive functioning, substance use and personal strengths (12).
The ASR questionnaire has to be completed by the adult that is being assessed, while the ABCL has to be completed by the informant chosen by the respondent, to report on their behalf. Both forms are divided into two main sections and the norms for all the scales that they comprise are constructed for two age groups: 18-35 years and 36-59 years (12). The first part of the instruments assesses the adult’s level of adaptive functioning. The second part consists of 123 problem items for which the respondent and the informant rate the emotional, social and behavioral problems concerning each item on a 3-point Likert scale with “0” corresponding to “not true,” “1” corresponding to “somewhat or sometimes true,” and “2” corresponding to “very true or often true.” Via factor-analytic methods, eight empirically based syndrome scales are derived: Anxious/Depressed (18 items), Withdrawn (9 items), Somatic Complaints (12 items), Thought Problems (10 items), Attention Problems (15 items), Aggressive Behavior (15 items), Rule-Breaking Behavior (14 items), and Intrusive (6 items) (12). Furthermore, the syndromes are grouped into two main grouping of problems: (I) “Externalizing Problems” including: aggressive behavior, rule-breaking behavior, and intrusive and (II) “Internalizing Problems” including anxious/depressed, withdrawn, somatic complaints. The questionnaires also include a Critical Items scale (19 items) and an Other Problem scale (21 items). Summation of all the scales yields a “Total Problems” score (12).
Translation and back-translation procedures
In order for instruments to be used in another language, they must be forwardly translated, back translated and culturally adapted and validated in the culture of interest (16). These procedures are consistent with the Guidelines for Translating and Adapting Tests (17), and the Standards for Educational and Psychological Testing (18,19). The forward translation of ASEBA measures was performed by an interdisciplinary group composed of psychologists, public health professionals, education specialists, English professors, graduate students in psychology and a person having no background in any of the above mentioned fields. The translation focused on retaining the original wording and containing the meaning of the Albanian language. The group of professionals appraised the ASR and ABCL item-by-item and provided detailed feedback on each item’s wording and meanings. This feedback was then included in the ASR and ABCL questionnaires in order to produce their Albanian versions (16). Even though the Albanian language contains dialects that vary somewhat by geographic regions, the translation was based on the standardized Albanian language (20), which can be generalizable to all Albanian-speaking regions. A native English speaker translator who knew Albanian very well then performed a back-translation of the Albanian version of ASR and ABCL. The original English and the back translations were reviewed by the experts and researcher to revise ambiguous or misleading items. Finally, the group of experts revised item by item all forward translations and back-translation, leading to the final Albanian version of ASR and ABCL. This procedure provided a culturally sensitive version of two widely used instruments to measure adult’s psychological functioning. The standardized Albanian version of the instruments remained as close to the original English versions as possible which ensured validity and reliability when comparing English-speaking populations with Albanian-speaking populations (21).
Results
The data in this study were analyzed through the Statistical Package for Social Sciences (SPSS), version 21.0. A total of N=1500 participants aged 18-59 years completed ASR and ABCL. Response rates were excellent for both measures with no excluded forms from the analysis.
In order to check the reliability of the Albanian version of the ASR and ABCL, we computed the internal consistency, using the Cronbach’s Alpha coefficient as an indicator of internal consistency (22,23). For the Albanian version of ASR, alpha coefficient for Total Problems was 0.96, and alphas for Internalizing and Externalizing Problems were >0.91 and >0.93, showing an excellent internal consistency. The mean alpha scores for the other scales ranged from 0.50 to 0.89. Particularly, Aggressive Behavior, Anxiety/Depression, Attention Problems, Rule-Breaking Behavior and Somatic Complaints had very good internal consistency (alphas ranged between 0.80 and 0.89). Whereas Withdrawn, Intrusive, Social Desirability and Critical Items had acceptable internal consistency (alpha values between 0.72 and 0.79). Other Problems (alpha 0.68) showed questionable internal consistency while Thought Problems had the lowest internal consistency (alpha 0.50).
For the Albanian version of ABCL, alpha coefficient for Total Problems was 0.97, while the alphas for Internalizing and Externalizing Problems were >0.91 and >0.94, respectively. The mean alpha scores for the other scales ranged from 0.61 to 0.92. Particularly, Anxious/Depressed, Attention Problems and Critical Items had very good internal consistency (alphas ranged between 0.81 and 0.89). The Aggressive Behavior scale (α=0.92), showed an excellent internal consistency, whereas Withdrawn, Somatic Complaints, Rule-Breaking Behavior, Intrusive, Other Problems and Social Desirability had acceptable internal consistency (alpha values between 0.71 and 0.79). Thought Problems had the lowest internal consistency (alpha 0.61).
Confirmatory factor analysis (CFA), were used to test the fit of self-ratings and collateral-ratings in the Albanian sample to the eight syndrome model which derived from 123 items, 99 of which loaded significantly on the syndromes. We hypothesized that the ASR and ABCL syndrome model would be supported by our CFAs of self-ratings and collateral-ratings by adults in the Albanian sample. The primary fit (Index Root Mean Square Error of Approximation) showed good fit for the ASR (.026) and ABCL (.029) total sample. RMSEA were computed for men and women subsamples. With ASR values range from .024 for men and .022 for women, and ABCL values ranging from .024 for women and .027 for men. RMSEA indicated a good model fit for both genders in both forms in the Albanian sample.
Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) were also computed secondary to the RMSEA. We referred to the ranges presented by Marsh et al. (24), to estimate the CFI and TLI values. CFI values for ASR ranged from .914 for the total sample; .920 for men and .932 for women, while CFI values for ABCL ranged from .939 for the total sample; .953 for women and .947 for men. TLI values for ASR ranged from .911 for the total sample; .930 for women and 917 for men. For the ABCL, TLI values ranged from .938 for the total sample; .946 for men and .951 for women. CFI and TLI values indicated a good model fit for both genders in both forms in the Albanian sample and subsamples.
To address the cross-agreement indices between self-reports and collateral reports we calculated correlations between, and assessed mean differences in respondents and informants reports of problems for the full sample.
The Pearson correlation coefficient is the most commonly used statistic to assess agreement between self-reports and informants’ reports on psychological functioning. It provides information about the order or relative standing of scores from two informants (25). For these reason Pearson correlations between respondents and informants scores were computed to examine the cross-informant agreement. Respondents’ and informants’ reports were correlated significantly and positively for all scales (Table 1). Correlation between respondents ASR and informants ABCL scores were positive and ranged from moderately high to high. High correlation among respondents’ reports and informants’ reports was found for the Total Problems scale (r=0.83). Also high correlations were found for Internalizing Problems (r=0.84) including Anxiety/Depression scale (r=0.81), Withdrawn (r=0.76) and Somatic Complaints (r=0.80). Respondents’ and informants’ reports correlated moderately high for the Externalizing Problems (r=0.77) including Aggressive Behavior scale r=0.76, Rule-Breaking Behavior (r=0.73) and Intrusive (r=0.64) which had the lowest correlation. Moderately high correlations were also found for Attention Problems (r=0.79), Critical Items scale (r=0.76) and Other Problems (r=0.74). Thought Problems had the lowest correlation coefficient (r=0.58).
Because correlations do not provide information about the direction of the differences between self-reports and informants’ reports we examined mean differences between respondents’ and informants’ for all scales conducting paired samples t-tests to investigate the magnitude of the cross-informants agreement. Effect sizes were calculated using Cohen’s d (26) (Table 1). Paired samples t-tests showed that respondents’ reported more symptoms on: Somatic Complaints scale t(749)=2.389, P<0.05, d=.056; Thought Problems scale t(749) = 2.746, P<0.01, d=.092; Attention Problems t(749)=3.577, P<0.001, d=.088; Other Problems scale t(749)=9.416, P<0.001, d=.257; compared to their close persons. On the contrary paired sample t-tests showed that informants’ reported more symptoms on: Externalizing Problems t(749)= -10.922, P<0.001, d=.275 including Aggressive Behavior t(749)= -3.878, P<0.001, d=.088; Rule-Breaking Behavior t(749)= -18.435, P<0.001, d=.493; Intrusive t(749)= -8.737, P<0.001, d=.271. No significant differences among respondents’ and informants’ reports were found for the other scales.
Table 1. Descriptive statistics, correlations and comparisons between respondents’ and ASR and informants’ ABCL score for the total sample
d: Cohen’s effect size; t: paired samples t-statistic; r: Pearson’s correlation coefficient.
Discussion
ASEBA adults’ parallel forms are among the most widely used assessment instruments for adult emotional, social and behavioral problems and has a proven multicultural factor structure in many developed societies and a few developing societies (11). The aim of this study was to compare data obtained from respondent and close informants on adult’s emotional, social and behavioral problems using ASEBA adult forms.
One of the most important factors when evaluating the quality of the results from this study refers to the study population’s representativeness which was achieved. Our findings are based on data obtained from a population-based sample of nationally representative adults’ aged 18-59 years. To assess the reliability of the Albanian version of the ASR and ABCL, the internal consistency was conducted. Most of the scales of the original form for adults had adequate internal consistency. The mean Cronbach’s alpha was 0.80 for the ASR syndrome scales, and 0.81 for the ABCL syndrome scales.
This study tested the generalizability of the eight syndrome scales of ASR and ABCL for assessing adult psychopathology in Albania. In our nationally representative sample, the eight syndrome model converged, and RMSEA and secondary indices (CFI and TLI) indicated a good model fit. The results supported the eight syndrome model of ASR and ABCL scales in the Albanian sample and gender subsamples.
The current study contributes to the literature on self-reports and collateral reports agreement about psychological problems of Albanian adults. For the purpose of comparison we sought data from parents, children, siblings, spouses/partners, friends and relatives of the person being assessed. Our analyses on cross-informant agreements were intended to examine the consistency between adult’s self-report and close informant’s ratings of adults’ emotional and behavior problems.
We compared the respondent’ sample self-reports on their psychopathology with the informant’s sample collateral reports on the respondents’ psychopathology. Our findings on agreement between respondents’ and informants’ reports on various aspects of adults’ functioning showed cross-informant correlations ranging from 0.58 to 0.84, with a mean correlation of 0.75. As expected, the Albanian adults had a moderate to high level of agreement with their close informants. The good level of cross-informant agreement on the adult’s problems assessed in this study may be explained by the Albanian adults’ ability and willingness to communicate about their problems with their close persons which provides informants’ with a very good knowledge of the person being assessed (27). This attitude of Albanian being very much involved in with each-other can be explained with the collectivistic identity of Albanians (28). The high correlations between self-reports and informants’ reports in the Albanian sample also reflect the accuracy of the data obtained from both sources (6,7,12).
Although the mean correlation of agreement between self-reports and collateral reports is moderately high, the mean differences suggest differences between the assessment of adults obtained from self-reports and informant’s reports. Tests of mean level differences in problem scores showed that respondents’ self-reported significantly more problems than informants’ in Internalizing Problems, Thought Problems, Attention Problems and Other problems. In contrary informants reported significantly higher levels of Externalizing Problems (including Rule-Breaking Behavior and Intrusive) for the respondents being assessed compared to respondents self-reports. As we predicted, results showed that levels of agreements were higher for ratings of adults’ internalizing behavior problems vis-à-vis their externalizing behavior problems. By comparing self-reports with reports by informants, we identified similarities and differences among Albanian adults reports. The differences between self-reports and reports by the close persons on the above mentioned scales indicate that some problems that are not acknowledged or are not reported by the person being assessed but that are recognized and reported by others.
The finding of this study showed that the Albanian respondents and informants provided a different picture of psychopathology for the adult being assessed in scale level of the parallel forms of ASR and ABCL. Moreover, informants’ reports on item level added more value to different aspects of psychopathology assessment of Albanian adults beyond what it had in common with the self-reports (6,7).
It is important to emphasize that literature suggests that larger discrepancies are related to worst psychological adjustment (29,30) and it appears likely that larger discrepancies do not reflect a normative functioning level, but rather a pathological one. Consequently, we can assume that the small discrepancies found between respondents and informants ratings in the Albanian non-referred sample reflect a normative psychological functioning level of the Albanian adults assessed in this study and not a pathological one.
People often may not see themselves as other can see them, and thus the may provide different pictures of their problems than would be obtained from other persons or relatives, who know them (6,7). Comparisons between the self-ratings versus the one of informants’ ratings have shown that both these forms can assess independently different aspects of psychological functioning (31). Our data generated generally larger correlations between self-reports and informants’ reports compared to similar studies using the cross-informant approach in diverse cultures (6,7). Diverse factors might have affected correlations between self-reports and informant reports of psychopathology in this study such as the target variables of assessing psychopathology; type of population being assessed (general population vs. psychiatric one) and type and sample size of informants (close family members vs. relatives or friends). In addition, the correlations between reports on parallel versions of the same instrument are usually higher than correlations between reports on different instruments, because of method variance common to the parallel instruments (5). To conclude, our study results show that Albanian adults are aware, to an acceptable extent, of their close persons’ psychological problems.
This study shows some limitations. The non-referred sample participating in this study was representative of the Albanian adults’ general population. No referred sample (psychiatric sample) was used in this study in order to examine the potential differences of cross-informant approach among the referred and not-referred samples. Further studies using the cross-informant agreement approach on assessing adult psychopathology should take into consideration the comparison between referred and non-referred Albanian representative samples.
There is no doubt that self-reports are needed for the assessment of mental health. Anyhow, because reports of psychological problems certainly depend a lot on the informants’ perspectives, further studies should be conducted in different settings that vary on situation and person variables, in order to obtain a more comprehensive assessment. Cross-informant comparisons in further studies can also be used to determine the degree to which particular problems are reported by all informants versus only one or some informants, which may indicate that the problems are relatively specific to certain situations or to certain interaction informants. To summarize, follow-up research could explore these issues in larger, more diverse community samples and in clinical samples with additional informants to validate self-reports and collateral reports of Albanian adults psychological functioning (27).
Study results showed that clinical constructs of adult psychopathology were supported in the Albanian sample. Norm scores and cut-off scores have been generated separately for men and women of ages 18-35 years and 36-59 years. The questionnaires are now available to be used by the Albanian mental health professionals to assess emotional, social and behavioral problems of the Albanian adults using the cross-informant approach. We can conclude that the larger cross-informant correlations for parallel instruments and also the appropriateness of users to compare self-reports with informant reports for the same items, indicate the use of parallel instruments whenever possible (11). Finally, we can admit that the assessment of Albanian individuals might benefit from supplementing self-reports (using ASR) with informants’ reports (using ABCL) for both clinical and research purposes.
Conflicts of interest: None declared.
References
1. Klonsky ED, Oltmanns TF. Informant‐reports of personality disorder: Relation to self‐reports and future research directions. Clin Psychol Sci Pract 2002;9:300-11.
2. Meyer GJ. Implications of information gathering methods for a refined taxonomy of psychopathology. In Beutler LE, Malik ML (Eds). Rethinking the DSM: A psychological perspective. American Psychological Association, Washington, DC; 2002:69-105.
3. Meyer GJ. On the integration of personality assessment methods: The Rorschach and MMPI-2. J Pers Assess 1997;68:297-300.
4. Robins RW, John OP. Effects of visual perspective and narcissism on self-perception: Is seeing believing? Psychol Sci 1997;8:37-42.
5. Achenbach TM, Bernstein A, Dumenci L. DSM-Oriented Scales and Statistically Based Syndromes for Ages 18 to 59: Linking Taxonomic Paradigms to Facilitate Multi-taxonomic Approaches. J Pers Assess 2005;84:49-63.
6. Achenbach T. As Others See Us: Clinical and Research Implications of Cross-Informant Correlations for psychopathology. Curr Dir Psychol 2006;15:94-8.
7. Achenbach T, Krukowski AR, Dumenci L, Ivanova M. Assessment of adult psychopathology: Meta-Analyses and Implications of Cross-Informant Correlations. Psychol Bull 2005;131:361-82.
8. Guion K, Mrug S, Windle M. Predictive value of informant discrepancies in reports of parenting: Relation to early adolescents’ adjustment. JACP 2009;37:17-30.
9. Van der Ende J, Verhulst F, Tiemeier H. Agreement of informants on emotional and behavioral problems from childhood to adulthood. Psychol Assess 2012;24:293-300.
10. Grills AE, Ollendick TH. Issues in parent-child agreement: The case of structured diagnostic interviews. Clin Child Fam Psychol Rev 2002;5:57-83.
11. Achenbach TM, Rescorla LA. Multicultural Supplement to the Manual for the ASEBA Adult Forms & Profiles. Burlington, VT: University of Vermont; 2015.
12. Achenbach TM, Rescorla LA. Manual for the ASEBA adult forms & profiles. Burlington, VT: University of Vermont; 2003.
13. National Institute of Statistics. Population and housing census 2011. Tirana: Albania; 2011.
14. Buysse CM, Vermunt LC, Raat H, Hazelzet JA, Hop WC, Utens EM, et al. Surviving meningococcal septic shock in childhood: Long-term overall outcome and the effect on health-related quality of life. Crit Care 2010;14:R124.
15. Saavedra LM, Silverman WK, Morgan-Lopez AA, Kurtines WM. Cognitive behavioral treatment for childhood anxiety disorders: Long-term effects on anxiety and secondary disorders in young adulthood. J Child Psychol Psychiatry 2010;51:924-34.
16. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: Report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005;8:94-104.
17. International Test Commission. International Test Commission Guidelines for Translating and Adapting Tests; 2010. Available from: http://www.intestcom.org (Accessed: January 30, 2016).
18. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC; 1999.
19. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, (4th ed.) Washington, DC; 2000.
20. The Institute of Language and Literature of the State University of Tirana. Albanian language Congress; 1972 [In Albanian].
21. Ivanova MY, Achenbach TM, Rescorla LA, Turner LV, Árnadóttir HA, Au A, et al. Syndromes of collateral-reported psychopathology for ages 18-59 in 18 Societies. Int J Clin Health Psychol 2015;15:18-28.
22. Cronbach LJ. (1951). Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297-334.
23. Cronbach LJ. Test validity. In: Thorndike RL (Ed.). Educational measurement (2nd ed). Washington, DC: 1971:430-507.
24. Marsh HW, Hau KT, Wen Z. In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Struct Equ Modeling 2004;11:320-41.
25. Kazdin AE. Informant variability in the assessment of childhood depression. In: Reynolds WM, Johnston HF (Eds.). Handbook of depression in children and adolescents. Plenum, New York; 1994;249-71.
26. Rosenthal R, Rosnow RL. Essentials of behavioral research: Methods and data analysis (2nd ed.). New York: McGraw Hill; 1991.
27. Barker ET, Bornstein MH, Putnick DL, Hendricks C, Suwalsky JT. Adolescent Mother Agreement about Adolescent Problem Behaviors: Direction and Predictors of Disagreement. J Youth Adolesc 2007;36:950-62.
28. Bodinaku B. Translation, validation and standardization of the Albanian version of the SCL-90-R (Symptom Checklist-90-Revised) and CORE-OM (Clinical Outcomes in Routine Evaluations – Outcome Measure). Unpublished doctoral dissertation. Sigmund Freud Private University, Vienna; 2014.
29. Feinberg ME, Howe GW, Reiss D, Hetherington EM. Relationship between perceptual differences of parenting and adolescent antisocial behavior and depressive symptoms. J Fam Psychol 2000;14:531-55.
30. Ferdinand RF, van der Ende J, Verhulst FC. Prognostic value of parent-adolescent disagreement in a referred sample. Eur Child Adolesc Psychiatry 2006;15:156-62.
31. Lara, M. E., Ferro, T., & Klein, D.N. Family history assessment of personality disorders: II associations with measures of psychosocial functioning. J Pers Disord 1997;11:137-45.