A multilevel Multitrait–Multimethod Analysis of Self– and Peer

Abstract

We examined the psychometric properties of an experience–sampling measure of affect (PANAS) using data from self– and peer reports. A multivariate multilevel model was used to assess the reliability of the latent PANAS scales at the within– and between–person level. Findings suggest satisfying internal consistencies for self– and peer reports of affective experiences at both levels of analysis. Convergent and discriminant validity of the two affect scales were examined by means of a multilevel multitrait–multimethod approach (MLM–MTMM) indicating distinct findings at the within– and between–person level. These findings provide further insights into the structural relations between the two PANAS scales: Whereas positive and negative affect were unrelated at the between–person level; they were negatively correlated at the within–person level. Copyright © 2010 John Wiley & Sons, Ltd.

Keywords

experience-sampling method daily assessments PANAS peer reports multilevel modelling MTMM

Get full access to this article

View all access options for this article.

References

Baker

L. A.

, Cesa

I. L.

, Gatz

, & Mellins

(1992).Genetic and environmental influences on Positive and Negative Affect: Support for a two–factor theory.Psychology and Aging,7,158–163.

Bleidorn

(2009).Linking personality states, current social roles and major life goals.European Journal of Personality,23,509–530. DOI:10.1002/per.731

Bolger

, Davis

, & Rafaeli

(2003).Diary methods: Capturing life as it is lived.Annual Review of Psychology,54,579–616. DOI:0.1146/annurev.psych.54.101601.145030

Campbell

D. T.

, & Fiske

D. W.

(1959).Convergent and discriminant validation by the multitrait–multimethod matrix.Psychological Bulletin,56,81–105.

Conner

, Feldman Barrett

, Tugade

M. M.

, & Tennen

(2007).Idiographic personality: The theory and practice of experience sampling. In Robins

R. W.

, Fraley

R. C.

, & Krueger

R. F.

(Eds.),Handbook of research methods in personality psychology (pp.79–96).New York: NY:Guilford Press.

Csikszentmihalyi

, & Larson

(1987).Validity and reliability of the experience–sampling method.Journal of Nervous and Mental Disease,175,526–536.

Denissen

J. J. A.

, Penke

, Butalid

, & van Aken

M. A. G.

(2008).The effects of weather on daily mood: A multilevel approach.Emotion,8,662–667. DOI:10.1037/a0013497

Diener

, & Emmons

R. A.

(1984).The independence of positive and negative affect.Journal of Personality and Social Psychology,47,1105–1117.

Fleeson

(2001).Toward a structure– and process–integrated view of personality: Traits as density distributions of states.Journal of Personality and Social Psychology,80,1011–1027. DOI:10.1037//0022–3514.80.6.1011

10.

Flügel

J. C.

(1925).A quantitative study of feeling and emotion in everyday life.British Journal of Psychology,15,318–355.

11.

Funder

D. C.

(1989).Accuracy in personality judgment and the dancing bear. In Buss

D. M.

, & Cantor

(Eds.),Personality psychology: Recent trends and emerging directions (pp.210–223).New York:Springer–Verlag.

12.

Furr

R. M.

(2009).The study of behaviour in personality psychology: Meaning, importance and measurement.European Journal of Personality.23,437–453. DOI:10.1002/per.726

13.

Green

D. P.

, Goldman

S. L.

, & Salovey

(1993).Measurement error masks bipolarity in affect ratings.Journal of Personality and Social Psychology,64,1029–1041.

14.

Hektner

J. M.

, Schmidt

J. A.

, & Csikszentmihalyi

(2007).Experience sampling method: Measuring the quality of everyday life.Thousand Oaks, CA, US:Sage Publications.

15.

Hox

J. J.

(2002).Multilevel analysis: Techniques and applications. Quantitative methodology series.Mahwah, NJ:Erlbaum.

16.

Hox

J. J.

, Maas

C. J. M.

, & Brinkhuis

M. J. S.

(2010).The effect of estimation method and sample size in multilevel structural equation modeling.Statistica Neerlandica,64,157–170. DOI:10.1111/j.1467–9574.2009.00445.x

17.

Ilies

, Dimotakis

, & Watson

(2010).Mood, blood pressure, and heart rate at work: An experience–sampling study.Journal of Occupational Health Psychology,15,120–130. DOI:10.1037/a0018350

18.

Kaplan

, Bradley

J. C.

, Luchman

J. N.

, & Haynes

(2009).On the role of positive and negative affectivity in job performance: A meta–analytic investigation.Journal of Applied Psychology,94,162–176. DOI:10.1037/a0013115

19.

Kraan

, Meertens

, Hilwig

, Volovics

, Dijkman–Caes

C. I. M.

, & Portegijs

(1992).Selecting measures, diagnostic validity and scaling in the study of depression. In de Vries

M. W.

(Ed.),The experience of psychopathology: Investigating mental disorders in their natural settings (pp.324–338).Cambridge:Cambridge University Press.

20.

Krohne

H. W.

, Egloff

, Kohlmann

C. W.

, & Tausch

(1996).Untersuchungen mit einer deutschen Version der ‘Positive and Negative Affect Schedule’ (PANAS) [Investigations with a German version of the ‘Positive and Negative Affect Schedule’ (PANAS)].Diagnostica,42,139–156.

21.

Laenen

, Alonso

, Molenberghs

, & Vangeneugden

(2009).Reliability of a longitudinal sequence of scale ratings.Psychometrika,74,49–64. DOI:10.1007/S11336–008–9079–7

22.

Laenen

, Vangeneugden

, Geys

, & Molenberghs

(2006).Generalized reliability estimation using repeated measurements.British Journal of Mathematical and Statistical Psychology,59,113–131. DOI:10.1348/000711005X66068

23.

Larsen

R. J.

, Diener

, & Lucas

(2002).Emotion: Models, measures, and individual differences. In Lord

, Klimoski

, & Kanfer

(Eds.),Emotions at work (pp.64–106).San Francisco:Jossey–Bass.

24.

Larson

R. W.

, Moneta

, Richards

M. H.

, & Wilson

(2002).Continuity, stability, and change in daily emotional experience across adolescence.Child Development,73,1151–1165. DOI:10.1111/1467–8624.00464

25.

Lucas

R. E.

, Diener

, & Suh

(1996).Discriminant validity of well–being measures.Journal of Personality and Social Psychology,71,616–628.

26.

Miner

A. G.

, Glomb

T. M.

, & Hulin

(2005).Experience sampling mood and its correlates at work.Journal of Occupational and Organizational Psychology,78,171–193. DOI:10.1348/096317905X40105

27.

Muthén

B. O.

(1994).Multilevel covariance structure analysis.Sociological Methods & Research,22,376–398.

28.

Nezlek

J. B.

(2007).A multilevel framework for understanding relationships among traits, states, situations and behaviours.European Journal of Personality,21,789–810. DOI:10.1002/per.640

29.

Nezlek

J. B.

, & Gable

S. L.

(2001).Depression as a moderator of relationships between positive daily events and day–to–day psychological adjustment.Personality and Social Psychology Bulletin,27,1692–1704. DOI:10.1177/01461672012712012

30.

Osburn

H. G.

(2000).Coefficient alpha and related internal consistency reliability coefficients.Psychological Methods,5,343–355. DOI:10.1037//1082–989X.5.3.343

31.

Rafaeli

, & Revelle

(2006).A premature consensus: Are happiness and sadness truly opposite affects?Motivation and Emotion,30,1–2, DOI:10.1007/s11031–006–9004–2

32.

Rafaeli

, Rogers

G. M.

, & Revelle

(2007).Affective synchrony: Individual differences in mixed emotions.Personality and Social Psychology Bulletin,33,915–932. DOI:10.1177/0146167207301009

33.

Raudenbush

S. W.

, & Bryk

A. S.

(2002).Hierarchical linear models: Applications and data analysis methods (2nd edition).Thousand Oaks, CA:Sage Publications.

34.

Raudenbush

S. W.

, Bryk

A. S.

, Cheong

Y. F.

, & Congdon

(2000). HLM (Version 6). Chicago, IL: Scientific Software (Computer software).

35.

Raudenbush

S. W.

, Rowan

, & Kang

S. J.

(1991).A multilevel, multivariate model for studying school climate with estimation via the EM algorithm and application to U.S. high–school data.Journal of Educational and Behavioral Statistics,16,295–330. DOI:10.3102/10769986016004295

36.

Reis

H. T.

, & Gable

S. L.

(2000).Event–sampling and other methods for studying everyday experience. In Reis

H. T.

, & Judd

C. M.

(Eds.),Handbook of research methods in social and personality psychology (pp.190–222).Cambridge:Cambridge University Press.

37.

Russell

J. A.

, & Carroll

J. M.

(1999).On the bipolarity of positive and negative affect.Psychological Bulletin,125,3–30.

38.

Schimmack

(2001).Pleasure, displeasure, and mixed feelings? Are semantic opposites mutually exclusive?Cognition and Emotion,15,81–97. DOI:10.1080/0269993004200123

39.

Schmuckle

S. C.

, Egloff

, & Burns

L. R.

(2002).The relationship between positive and negative affect in the Positive and Negative Affect Schedule.Journal of Research in Personality,36,463–475. DOI:10.1016/S0092–6566(02)00007–7

40.

Scollon

C. N.

, Diener

, Oishi

, & Biswas–Diener

(2005).An experience sampling and cross–cultural investigation of the relation between pleasant and unpleasant affect.Cognition & Emotion,19,27–52. DOI:10.1080/02699930441000076

41.

Scollon

C. N.

, Kim–Prieto

, & Diener

(2003).Experience sampling: Promises and pitfalls, strengths and weaknesses.Journal of Happiness Studies,4, 5–34. DOI:10.1023/A:1023605205115

42.

Shadish

W. R.

, Cook

T. D.

, & Campbell

D. T.

(2002).Experimental and quasi–experimental designs for generalized causal inference.Boston, Mass:Houghton Mifflin.

43.

Shrout

P. E.

, & Fleiss

J. L.

(1979).Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin,86,420–428.

44.

Shrout

P. E.

, & Fiske

S. T.

(1995).Personality research, methods, and theory: A festschrift honoring Donald W. Fiske.Hillsdale, NJ:Erlbaum Associates.

45.

Sijtsma

(2009).On the use, the misuse, and the very limited usefulness of Cronbach's alpha.Psychometrika,74,107–120. DOI:10.1007/S11336–008–9101–0

46.

Snijders

T. A. B.

, & Bosker

R. J.

(1999).Multilevel analysis: An introduction to basic and advanced multilevel modeling.London:Sage.

47.

Steyer

, Schwenkmezger

, Notz

, & Eid

(1997).Der Mehrdimensionale Befindlichkeitsfragebogen (MDBF) [The Multidimensional Mood Questionnaire (MMQ)].Göttingen:Hogrefe.

48.

Vansteelandt

, van Mechelen

, & Nezlek

J. B.

(2005).The co–occurrence of emotions in daily life: A multilevel approach.Journal of Research in Personality,39,325–335. DOI:10.1016/j.jrp.2004.05.006

49.

Watson

(2000).Mood and temperament.New York:Guilford Press.

50.

Watson

, & Clark

L. A.

(1991).Self– versus peer ratings of specific emotional traits: Evidence of convergent and discriminant validity.Journal of Personality and Social Psychology,60,927–940.

51.

Watson

, Clark

L. A.

, & Tellegen

(1988).Development and validation of brief measures of positive and negative affect: The PANAS scales.Journal of Personality and Social Psychology,54,1063–1070.

52.

Watson

, & Tellegen

(1985).Toward a consensual structure of mood.Psychological Bulletin,98,219–235.

53.

Watson

, & Tellegen

(1999).Issues in the dimensional structure of affect: Effects of descriptors, measurement error, and response formats: Comment on Russell and Carroll (1999).Psychological Bulletin,125,601–610.

54.

Watson

, & Vaidya

(2003).Mood measurement: Current status and future directions. In Schinka

J. A.

, & Velicer

(Eds.),Comprehensive handbook of psychology Vol. 2 Research methods (pp.351–375).New York:Wiley.

55.

Watson

, Wiese

, Vaidya

, & Tellegen

(1999).The two general activation systems of affect: Structural findings, evolutionary considerations and psychobiological evidence.Journal of Personality and Social Psychology,76,820–838.

56.

Wilhelm

, & Schoebi

(2007).Assessing mood in daily life: Structural validity, sensitivity to change, and reliability of a short–scale to measure three basic dimensions of mood.European Journal of Psychological Assessment,23,258–267. DOI:10.1027/1015–5759.23.4.258

57.

Wirtz

, & Caspar

(2002).Beurteilerübereinstimmung und Beurteilerreliabilität [Interrater agreement and interrater reliability].Göttingen:Hogrefe.

58.

Zelenski

J. M.

, & Larsen

R. J.

(2000).The distributions of basic emotions in everyday life: A state and trait perspective from experience sampling data.Journal of Research in Personality,34,178–197. DOI:10.1006/jrpe.1999.2275

A multilevel Multitrait–Multimethod Analysis of Self– and Peer–Reported Daily Affective Experiences

Abstract

Keywords

Get full access to this article

References