Online Friendship,Romance,and Sex: Properties and Associations of the Online Relationship Initiation Scale

Abstract

Online relationships are increasingly central to many people's lives. As a result, there is a growing need to scientifically examine their psychosocial implications. This study developed and tested the Online Relationship Initiation Scale (ORIS) through classical and item response theory analyses to address this need. An anonymous online survey included 713 adults, aged 18–71 years. The ORIS was tested on psychometric properties and examined for associations with gender and several standardized psychosocial measures. Results demonstrated unidimensionality of nine items, strong factor loadings, and high internal consistency (α = 0.90, ω_t = 0.94). All items captured significant information on the latent trait and none showed differential item functioning by sex, age group, or ethnicity. General linear modeling confirmed hypotheses that men were more likely than women to initiate online relationships. Online relationship initiation was not strongly associated with perceived social support, but was positively related to financial distress, and willingness to engage in infidelity or unprotected sex. The ORIS was negatively associated with age and satisfaction with life and showed modest interactions with ethnicity and hours online. This study provided empirical evidence for an interpersonal relationship initiation construct. The ORIS was shown to be a psychometrically sound instrument for evaluating online interpersonal behaviors and their associations with psychosocial and demographic factors. Such psychometrically sound instruments can be useful in exploring online interpersonal behaviors and their significance.

Introduction

Online dating sites, computer-mediated communications, and other social networking opportunities are becoming increasingly common and have fundamentally changed how people relate to one another.^1–4 More than a third of US marriages now originate online⁵; countless friendships are initiated on social networks; and some relationships end because of the perceived ease of finding alternate partners online.⁶ On an experiential level, it appears obvious that how we interact—socially, professionally, romantically, and sexually—has dramatically changed with the advent of the internet and related technologies. Yet, too little data are available about what motivates or predisposes someone to search for or initiate relationships online. To the extent that online relationships are a cornerstone of contemporary personal happiness and social cohesion, there is a clear need to better understand these behaviors and when online relationship seeking may be a step that enhances well-being or a symptom of distress that should be addressed.

Close interpersonal relationships have been identified as a fundamental human need.^7,8 For understanding online relationship formation, we might look to Murray, Maslow, and McClelland, who theorized that people have basic needs for love, affection, and affiliation.^8–10 Numerous theories may inform online relationship formation, but the need for love and affection may best fit available evidence. It may be that those who have difficulties fulfilling basic interpersonal needs offline turn to information technologies to develop more satisfying relationships.^2,11

A growing body of evidence suggests that online interpersonal relationship development is related to psychiatric symptoms or personality types. For example, a study of youth found that both girls and boys who formed close online relationships were more likely to have problems with parents than those without close online relationships.¹² Seward and Harris¹³ found that younger and more distressed individuals reported greater attraction to online interpersonal support than their healthier cohorts. Other studies have shown that online relationship behaviors were positively associated with loneliness, social anxiety, and willingness to engage in infidelity and unprotected sex.^14–18 Finally, the personality traits, extraversion and neuroticism, were predictors of sexting (sending sexually explicit electronic messages) in a study of undergraduate students.¹⁹

Sex differences have also been seen in relationship behaviors.¹⁹ Early research found women were more likely than men to form personal relationships online, regardless of age or marital status.²⁰ More recently, a US study revealed that online dating was more common in men than women and among younger than older adults,¹ while another study found men engaged in more relationship initiation behaviors than women.²¹

Interpersonal Relationship Assessment

The importance of interpersonal relationships has led to a plethora of instruments measuring related constructs. Some researchers calculate initiation-type behaviors or ask people where they met a relationship partner, which may include online sites as one option.^22,23 In the Handbook of Relationship Initiation,²⁴ romantic relationships were covered from numerous psychological perspectives. However, one aspect that was lacking was the measurement of relationship initiation efforts. The Online Relationship Building scale (ORB)²⁵ assessed initiation of online relationships and showed positive associations with psychiatric symptoms.²⁵ However, it combined romantic with sexual relationship development. Yet, differentiating friend, romantic, and sexual relationships may be informative. The ORB also did not explore perceptions of differences between online and face-to-face relationship initiation, a potentially helpful data point.

Study Aims

This study tested the feasibility of assessing online interpersonal relationship initiation, including perceptions of differences between online and offline relationship development. The validity of the Online Relationship Initiation Scale (ORIS) was tested through both classical test theory (CTT) and item response theory (IRT) analyses. We hypothesized that there would be a general factor related to relationship initiation. We also hypothesized that men would report higher online relationship initiation behaviors than women.^1,21 We also explored ORIS associations with psychosocial variables, hypothesizing that online relationship initiation would be negatively associated with extraversion and positively associated with psychiatric symptoms and willingness to engage in unprotected sex and infidelity.^14–18

Methods

Participants

An anonymous online survey included 713 participants, aged 18–71 years (M = 31.48, SD = 13.53); 77.1% were women; 78.5% identified as Caucasian/white, 13.9% as Asian, and 7.6% as other ethnicities. Education ranged from less than high school graduate (1.5%) to university degree holders (56.5%). About half (55.8%) were from Australia and New Zealand, 23.5% from other English-speaking countries, and the rest from other regions. Most were from urban (42.2%) or other developed regions (36.3%) and 21.4% from rural or remote areas.²⁶ Participants reported 1–100 weekly hours online (M = 23.18, SD = 20.64). No participation incentives were offered.

Measures

Online Relationship Initiation Scale

Items were taken from the ORB²⁵ and expanded and revised through focus group discussions. The resulting ORIS (Table 1) included nine questions. The first six assessed engagement and perceived success in online relationship initiation and were scored as 1 = never to 5 = frequently. The last three questions asked participants to compare online with offline relationship development and were scored as 1 = not at all to 5 = much easier. Higher ORIS scores indicate greater likelihood of searching for and developing new interpersonal relationships in an online environment.

Table 1.

Characteristics of the Online Relationship Initiation Scale

					Factor
Item	M	SD	Interitem r	Item-total r	Loading	h²
1. Were you ever looking/hoping to make new friends online?	2.36	1.41	0.34–0.65	0.67	0.66	0.59
2. Did you make any new friends online?	2.49	1.43	0.27–0.65	0.55	0.54	0.52
3. Were you ever looking/hoping to make new romantic relationships online?	1.86	1.28	0.37–0.70	0.72	0.80	0.71
4. Did you make any new romantic relationships online?	1.68	1.17	0.36–0.68	0.68	0.75	0.65
5. Were you ever looking/hoping to make new sexual relationships online?	1.58	1.13	0.27–0.74	0.64	0.73	0.71
6. Did you make any new sexual relationships online?	1.47	1.01	0.25–0.65	0.61	0.69	0.69
7. Do you find it easier to make new friends online compared with offline?	2.30	1.45	0.25–0.69	0.61	0.61	0.61
8. Do you find it easier to make new romantic relationships online compared with offline?	1.95	1.32	0.35–0.75	0.73	0.76	0.73
9. Do you find it easier to make new sexual relationships online compared with offline?	1.82	1.29	0.30–0.75	0.64	0.73	0.64

Items 1–6 were scored: 1 = never, 5 = frequently; Items 7–9 were scored: 1 = not at all, 5 = much easier.

The survey included the Multidimensional Scale of Perceived Social Support subscales Special Person, Family, and Friends²⁷; the International Personality Item Pool subscale Extraversion²⁸; the Satisfaction With Life scale,²⁹ and the Depression, Anxiety, and Stress Scales.³⁰ All measures demonstrated adequate reliability in the current study, α ≥ 0.80. Individual items assessed demographics; psychiatric history (diagnosis, hospitalization, and psychopharmacological treatment); financial distress; weekly hours online; and willingness to engage in unprotected sex and infidelity.

Procedure

The host institutional review board granted ethics approval. An online survey was chosen as it is appropriate for the target population—those going online to develop new interpersonal relationships. No IP addresses or other identifying data were collected. Over 4 months, participants were recruited through English-language Google and Facebook advertisements. Other than Facebook, the survey was not promoted on sites specific to online relationship development, such as dating sites, to avoid site-specific bias (e.g., sites oriented toward extramarital affairs or ethnicity). The exit page included links to free and anonymous support.

Analyses

Data cleansing included treatment of univariate and multivariate outliers. Missing values were missing completely at random and were replaced through the expectation maximization method. We followed Harris et al.'s³¹ procedures for developing a psychometrically sound and validated measure, including a construct definition based on theory and empirical evidence^32,33; only labeling anchor points³⁴; a four- to seven-point response range^35,36; and using both CTT and IRT to test item and scale validity.^32,37,38 Differential item functioning (DIF, item bias) was checked to ensure items function equivalently across demographic groups.³⁹ The graded response model was used as items may vary in their ability to capture information on the latent trait.⁴⁰ ORIS associations with other variables were exploratory, therefore significance was set at p < 0.05. Analyses used SPSS v.22 (IBM Corp.); EasyDIF⁴¹; and R v.3.3.0⁴² packages: psych,⁴³ ltm,⁴⁴ and compute.es.⁴⁵

Reliability

We recognized α ≥0.80 as adequate for research and 0.90–0.95 as preferable for clinically relevant purposes.^32,46,47 McDonald's omega (total) was also calculated as experts consider it a more valid internal reliability statistic.^48,49

Validity

Exploratory factor analysis (EFA) and McDonald's omega hierarchical analyses examined item loadings on common factors, construct validity, and unidimensionality.^43,50,51 Regression modeling tested convergent and divergent validity through associations between ORIS scores with psychosocial factors.^52,53 To better ensure findings might be replicated in other samples, only correlations ≥|0.20| were considered meaningful. General linear modeling examined construct validity and tested hypotheses comparing ORIS by sex.

Results

Most participants (59.2%) indicated some intentions of making new friends online, 38.8% attempted new romantic relationships online, and 27.2% attempted new sexual relationships online. Fewer than 10% of participants indicated above-median responses (frequently or near-frequently) for sexual relationship items, and only 14.3% indicated that they found it easier to develop new sexual relationships online. As these items may not be relevant to some groups, a nonsexual ORIS (ORIS-NS) was formed by excluding the three sexual relationship items (items 5, 6, and 9 in Table 1).

There were 79 cases per item and ORIS items were suitable for EFA (KMO >0.80). Costello and Osborne⁵⁰ defined a strong factor as including at least five items loading ≥0.50 and recommend only retaining items with communalities (h²) ≥0.40. EFA (maximum likelihood, oblimin rotation) revealed first eigenvalue = 4.92 and second eigenvalue = 1.26. Parallel analysis indicated that two factors might be acceptable, while the scree plot and McDonald's ω_h showed one-factor solutions. McDonald's ω_h = 0.68 indicated that the single factor explained 68% of general factor variance. Observation of loadings for the two-factor solution revealed patterns that did not meet theoretical or face validity. Therefore, the single-factor model was accepted. For ORIS-NS, ω_h = 0.70.

Table 1 shows all items loaded strongly on the single factor (>0.50). In addition, five items met criteria of excellent factor loadings (≥0.71).⁵⁴ Interitem correlations ranged from 0.26 to 0.75, and item-total correlations ranged from 0.54 to 0.72. Communalities were moderately high (mean h² > 0.65), providing some confidence that the observed structure is likely to be a good representation of population factors.^55,56 Internal reliability was high, Cronbach's alpha = 0.90 and McDonald's ω_t = 0.94. For ORIS-NS, α = 0.86 and ω_t = 0.93.

IRT analyses

Item discrimination levels, a (slopes), indicate the peak level of theta (the latent trait) an item discriminates on; b parameters indicate the item's ability to discriminate from low to high levels of theta.³⁸ All ORIS items had relatively high slopes (a ≥ 1.5) and showed fairly wide ranges on b parameters (Table 2). The information function (IF, the total information an item captures on theta) depends on the size of a and the spread of the b thresholds.³⁸ Item 3 captured the most information on theta (15.0%), while item 2 captured the least (6.3%). Although these IFs should be considered preliminary, a simple item weighting method was applied to calculate ORIS and ORIS-NS scores, which were used in the following analyses.

Table 2.

Graded Response Model Analyses of the Online Relationship Initiation Scale

Item	b₁	b₂	b₃	b₄	a	Pct. (%)	IF	Weight
1. Look friends	−0.27	0.34	1.02	1.54	2.06	9.2	4.48	1.00
2. Make friends	−0.47	0.20	0.94	1.85	1.47	6.3	3.08	0.50
3. Look romance	0.37	0.84	1.22	1.69	3.16	15.0	7.29	1.50
4. Make romance	0.63	0.98	1.38	2.01	2.86	13.1	6.40	1.25
5. Look sexual	0.75	1.17	1.49	1.89	2.84	11.9	5.79	1.00
6. Make sexual	0.93	1.31	1.64	2.15	2.86	12.4	6.04	1.00
7. Friend easier	−0.08	0.40	1.04	1.59	1.82	7.4	3.59	0.50
8. Romance easier	0.31	0.68	1.18	1.68	2.89	13.3	6.48	1.25
9. Sexual easier	0.53	0.82	1.30	1.76	2.69	11.4	5.54	1.00
Total test						100.0	48.69

b, item difficulty threshold; a, item discrimination level; IF, information function; Pct., percentage of total scale information.

ORIS and demographics

DIF analyses tested whether any items functioned differently by age groups (18–29, 30–49, and 50–76 years), ethnicity (Caucasian/white, Asian, and other), or sex. Kumagai⁴¹ showed that K Index significance indicates a large DIF.⁵⁷ Analyses revealed no DIF, demonstrating that all items functioned similarly on the latent trait for these groupings. ORIS items and totals showed no meaningful associations (r ≥ 0.20) with participant's size of community, education level, or weekly hours online. However, Pearson correlations showed that age was negatively associated with ORIS, r = −0.27, p < 0.001, and ORIS-NS, r = −0.29, p < 0.001, indicating that younger participants were more likely to initiate online relationships. Point-biserial correlations also revealed statistically significant associations between ORIS scores and ethnicity (white or other) and hours online, showing that nonwhite participants and those online more often were more likely to have higher ORIS scores. However, correlations were low (r < 0.20).

ORIS comparisons by sex

We next tested the hypothesis that sex differences exist in online interpersonal behaviors. Table 3 shows support for the hypothesis as ANCOVAs revealed that men reported higher likelihood of all assessed aspects of online interpersonal behaviors (ps < 0.01). A followup logistic regression tested which ORIS items best explain sex variance on online relationship development. Results showed that ORIS items accounted for significant variance on sex, Nagelkerke R² = 0.23, χ²(7) = 20.26, p = 0.005. Wald criteria revealed that looking for sexual relationships (Wald = 19.88, p < 0.001), making romantic relationships (Wald = 12.71, p < 0.001), finding online friendship formation easier (Wald = 5.59, p = 0.018), and looking for friends online (Wald = 4.60, p = 0.032) accounted for statistically significant sex variance when all items were included in the model.

Table 3.

ANCOVA Comparisons of Women and Men on the Online Relationship Initiation Scale

	Women (n = 550)		Men (n = 163)
Item	M	SE	M	SE	F	d	95% CI
1. Look friends	2.19	0.06	2.92	0.10	37.67^***	0.53	0.36	0.71
2. Make friends	2.39	0.06	2.84	0.11	13.62^***	0.32	0.15	0.49
3. Look romance	1.66	0.05	2.52	0.10	61.36^***	0.69	0.51	0.87
4. Make romance	1.58	0.05	2.02	0.09	17.78^***	0.37	0.20	0.55
5. Look sexual	1.35	0.04	2.37	0.08	118.59^***	0.96	0.78	1.15
6. Make sexual	1.32	0.04	1.97	0.08	56.60^***	0.67	0.49	0.85
7. Friend easier	2.22	0.06	2.57	0.11	8.05^**	0.24	0.07	0.41
8. Romance easier	1.83	0.05	2.34	0.10	20.54^***	0.39	0.22	0.56
9. Sexual easier	1.67	0.05	2.30	0.10	31.86^***	0.49	0.32	0.67
ORIS-NS	11.25	0.25	14.86	0.45	48.57^***	0.60	0.43	0.77
ORIS total	15.60	0.34	21.49	0.62	69.15^***	0.72	0.54	0.89

Controlling for age, ORIS-NS = nonsexual ORIS.

p < 0.01, ^***p < 0.001.

ORIS, Online Relationship Initiation Scale.

We next examined participants' beliefs regarding the success of their online relationship initiation efforts. Within-subjects t-tests compared going online to develop new relationships with participants' perceived success at forming each relationship type. Table 4 shows women were likely to report that they formed more friendships online than they attempted, but by a small effect size. In contrast, men were likely to report that they failed to form as many online romances and sexual relationships as they hoped.

Table 4.

Within-Subjects t-tests Comparing Looking/Hoping for Online Relationships with Perceived Relationship Initiation Success, by Sex

	Look/hope		Developed
Sex/relationship	M	SD	M	SD	t	d	95% CI
Women (n = 550)
Friendship	2.21	1.36	2.40	1.42	−3.95^***	0.24	0.12	0.36
Romance	1.67	1.15	1.59	1.12	2.09^*	0.13	0.01	0.24
Sexual	1.36	0.90	1.33	0.85	1.33	0.08	−0.04	0.20
Men (n = 163)
Friendship	2.87	1.46	2.79	1.42	0.84	0.09	−0.12	0.31
Romance	2.48	1.50	1.98	1.30	5.25^***	0.58	0.36	0.80
Sexual	2.34	1.46	1.95	1.30	4.55^***	0.50	0.28	0.73

p < 0.05, ^***p < 0.001.

Psychosocial factors

Hierarchical linear regression modeling next tested assessed psychosocial variables as predictors of ORIS scores after accounting for effects of age, sex, ethnicity, and weekly hours online. Table 5 shows that the demographic variables of Step 1 accounted for 20% of the variance in ORIS scores. The psychosocial variables of Step 2 accounted for an additional 8% of variance in ORIS scores. That variance was mostly explained by low satisfaction with life, higher financial distress, willingness to engage in unprotected sex, and infidelity.

Table 5.

Hierarchical Linear Regression Model Predicting Online Relationship Initiation Scale Scores

				95% CI
Step/predictor	B	SEB	β	LL	UL
Step 1
Male	6.92	0.66	0.31^***	4.63	7.22
Age	−0.14	0.02	−0.24^***	−0.18	−0.10
Hours online	0.05	0.01	0.13^***	0.03	0.08
White/Caucasian	−2.49	0.68	−0.12^***	−3.84	−1.15
Constant	20.54	0.96		18.66	22.43
Step 2
SWL	−0.15	0.05	−0.17^**	−0.25	−0.06
Financial distress	0.43	0.14	0.11^**	0.15	0.71
Unprotected sex	0.39	0.12	0.10^**	0.14	0.63
Infidelity	0.34	0.13	0.09^*	0.08	0.59
Friend support	0.08	0.05	0.07	−0.03	0.18
Depression	0.03	0.04	0.04	−0.06	0.11
Stress	0.03	0.05	0.04	−0.06	0.12
Special person support	0.04	0.05	0.04	−0.05	0.13
Extraversion	−0.39	0.48	−0.03	−1.33	0.55
Family support	0.02	0.05	0.02	−0.07	0.12
Anxiety	0.01	0.04	0.01	−0.08	0.09
Constant	15.61	2.04		11.60	19.61

Step 1, R² = 0.20, F(702, 4) = 44.71, p < 0.001; Step 2, ΔR² = 0.08, F(693, 11) = 18.53, p < 0.001.

p < 0.05, ^**p < 0.01, ^***p < 0.001.

SWL, satisfaction with life.

Discussion

This study established two principal points, that relationship initiation is a definable construct that can be assessed as such, and that the ORIS is a valid instrument for evaluating online relationship development. The ORIS demonstrated strong psychometric properties and revealed meaningful associations with demographic and psychosocial factors.

Of particular interest were the strong sex differences in online relationship development. As hypothesized, and consistent with previous research,^1,21 results showed that men were more likely than women to attempt new relationships online. However, women were somewhat more likely to actually form online friendships than they attempted. This may relate to supply and demand factors, where more men are pursuing relationships and women are more often the target of relationship initiations.

Results also showed that people struggling offline may find online technologies of some benefit to their personal lives. For both women and men, low satisfaction with life and financial distress predicted online relationship initiation. As demonstrated previously,^14,15 those experiencing some distress in their personal life may go online to improve their interpersonal relationships. It is likely that at least some of these new relationships are beneficial in meeting basic human needs for love and affection. Also of interest, and supporting hypotheses, willingness to engage in both unprotected sex and infidelity predicted higher ORIS scores. In contrast to hypotheses and previous findings, extraversion showed no significant association with ORIS scores.^14,58 It is also noteworthy that ORIS scores were not strongly associated with perceived social support.

Strengths and limitations

The present study examined online interpersonal relationship development, but was not promoted at sites where people might look for new relationships, other than Facebook. That strategy helped limit biases associated with such sites (e.g., sexual preference, ethnicity). However, this study may have missed a significant number of individuals who use the internet most for relationship initiation. Further study could clarify associations between use of specific types of sites (e.g., dating sites) for relationship development, with demographic and psychosocial factors.

The use of IRT and CTT analyses was a strength of the current study. However, item weights can only be established through very large samples.⁵⁹ It is important to emphasize that no DIF was found for ORIS items, indicating that they functioned equivalently on assessing the latent trait across the demographic groupings tested. An earlier study found positive associations with nonheterosexual orientation and online relationship development.⁶⁰ Further examination of younger ages and generational effects, additional cultures, as well as those who may feel stigmatized in the face-to-face world, could reveal additional ORIS associations of interest to mental health professionals and internet site developers.

Conclusions

It is hard to imagine a more universal human need than companionship, whether with a close friend, a loving partner, or a sexual mate. While this need may be timeless, the internet and related technologies have dramatically reconfigured how these relationships form. Yet, too little data exist explaining how this shift to the online world has occurred or what is motivating it. This study provided initial empirical validation of the ORIS as a tool for evaluating online relationship development. Results showed strong psychometric factors and significant associations with psychosocial factors and strong sex differences. Further research is required to understand the relevance of these factors and whether, by contributing to a transition of affective life to the online world, they are contributing to a net gain in overall personal and societal well-being. Still, affective attributes cannot be divorced from other transitions to the internet that have occurred in practically every other domain—entertainment, information seeking, economic transactions, etc. Whether similar triggers are at work in those other crucial fields also deserves consideration and research attention.

Footnotes

Author Disclosure Statement

The ORIS is available through this publication as a public domain instrument. The authors have no financial gain through its use. No competing financial interests exist.

References

Smith

, Duggan

. (2013) Online dating and relationships. Washington, DC: Pew Research Center's Internet & American Life Project

Finkel

, Eastwick

, Karney

, et al. Online dating: a critical analysis from the perspective of psychological science. Psychological Science in the Public Interest, Supplement, 2012; 13:3–66.

Amichai-Hamburger

, Kingsbury

, Schneider

. Friendship: an old concept with a new meaning?. Computers in Human Behavior, 2013; 29:33–39.

Aboujaoude

. Problematic internet use: an overview. World Psychiatry, 2010; 9:85–90.

Cacioppo

, Cacioppo

, Gonzaga

, et al. Marital satisfaction and break-ups differ across on-line and off-line meeting venues. Proceedings of the National Academy of Sciences of the United States of America, 2013; 110:10135–10140.

Aboujaoude

. (2011) Virtually you: The dangerous powers of the e-personality. New York: W. W. Norton.

Baumeister

, Leary

. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 1995; 117:497–529.

Murray

. (1938) Explorations in personality. New York: Oxford University Press.

Maslow

. A theory of human motivation. Psychological Review, 1943; 50:370–396.

10.

McClelland

. (1987) Human motivation. Cambridge, MA: Cambridge University Press.

11.

Teppers

, Luyckx

, Klimstra

, et al. Loneliness and Facebook motives in adolescence: a longitudinal inquiry into directionality of effect. Journal of Adolescence, 2014; 37:691–699.

12.

Wolak

, Mitchell

, Finkelhor

. Escaping or connecting? Characteristics of youth who form close online relationships. Journal of Adolescence, 2003; 26:105–119.

13.

Seward

A-L

, Harris

. Offline vs. online suicide-related help-seeking: changing domains, changing paradigms. Journal of Clinical Psychology, 2016; 72:606–620.

14.

Skues

, Williams

, Wise

. The effects of personality traits, self-esteem, loneliness, and narcissism on Facebook use among university students. Computers in Human Behavior, 2012; 28:2414–2419.

15.

Lee

, Stapinski

. Seeking safety on the internet: relationship between social anxiety and problematic internet use. Journal of Anxiety Disorders, 2012; 26:197–205.

16.

Clayton

. The third wheel: the impact of twitter use on relationship infidelity and divorce. Cyberpsychology Behavior and Social Networking, 2014; 17:425–430.

17.

Crimmins

, Seigfried-Spellar

. Peer attachment, sexual experiences, and risky online behaviors as predictors of sexting behaviors among undergraduate students. Computers in Human Behavior, 2014; 32:268–275.

18.

Wieland

. Computer addiction: implications for nursing psychotherapy practice. Perspectives in Psychiatric Care, 2005; 41:153–161.

19.

Delevi

, Weisskirch

. Personality factors as predictors of sexting. Computers in Human Behavior, 2013; 29:2589–2594.

20.

Parks

, Floyd

. Making friends in cyberspace. Journal of Communication, 1996; 46:80–97.

21.

MacGregor

JCD

, Cavallo

. Breaking the rules: personal control increases women's direct relationship initiation. Journal of Social and Personal Relationships, 2011; 28:848–867.

22.

Korchmaros

, Ybarra

, Mitchell

. Adolescent online romantic relationship initiation: differences by sexual and gender identification. Journal of Adolescence, 2015; 40:54–64.

23.

Paul

. Is online better than offline for meeting partners? depends: are you looking to marry or to date?. Cyberpsychology Behavior and Social Networking, 2014; 17:664–667.

24.

Sprecher

, Wenzel

, Harvey

. (2008) Handbook of relationship initiation. New York, NY: Psychology Press.

25.

Harris

, McLean

, Sheffield

. Examining suicide-risk individuals who go online for suicide-related purposes. Archives of Suicide Research, 2009; 13:264–276.

26.

Australian Bureau of Statistics. (2003) ASGC remoteness classification: purpose and use. Canberra, ACT: Commonwealth of Australia.

27.

Zimet

, Powell

, Farley

, et al. Psychometric characteristics of the multidimensional scale of perceived social support. Journal of Personality Assessment, 1990; 55:610–617.

28.

Goldberg

. The development of markers for the big-five factor structure. Psychological Assessment, 1992; 4:26–42.

29.

Diener

, Emmons

, Larsen

, et al. The satisfaction with life scale. Journal of Personality Assessment, 1985; 49:71–75.

30.

Antony

, Bieling

, Cox

, et al. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychological Assessment, 1998; 10:176–181.

31.

Harris

, Syu

J-J

, Lello

, et al. The ABC's of suicide risk assessment: applying a tripartite approach to individual evaluations. PLoS One, 2015; 10:e0127442.

32.

DeVellis

. (2012) Scale development: theory and applications. Los Angeles, CA: Sage.

33.

Streiner

, Norman

, Cairney

. (2015) Health measurement scales: a practical guide to their development and use. Oxford, UK: Oxford University Press.

34.

Ware

, Gander

. The SF-36 health survey: development and use in mental health research and the IQOLA project. International Journal of Mental Health, 1994; 23:49–73.

35.

Lozano

, García-Cueto

, Muñiz

. Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 2008; 4:73–79.

36.

Hofmans

, Theuns

, Mairesse

. Impact of the number of response categories on linearity and sensitivity of self-anchoring scales: a functional measurement approach. Methodology, 2007; 3:160–169.

37.

Ostini

, Nering

. (2006) Polytomous item response theory models. Thousand Oaks, CA: Sage.

38.

Embretson

, Reise

. (2000) Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Assoc.

39.

Osterlin

, Everson

. (2009) Differential item functioning. London: SAGE Publications.

40.

de Ayala

. (2009) The theory and practice of item response theory. New York: Guilford Press.

41.

Kumagai

. A new method for estimating differential item functioning (DIF) for multiple groups and polytomous items: development of index K and the computer program “EasyDIF”. Japanese Journal of Psychology, 2012; 83:35–43.

42.

R Development Core Team. (2016) R: a language and environment for statistical computing. 3.2.4 ed. Viena, Austria: R Foundation for Statistical Computing.

43.

Revelle

. (2016) Psych: procedures for personality and psychological research. 1.6.3 ed. Evanston, IL: Northwestern University.

44.

Rizopoulos

. ltm: an R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 2006; 17:1–25.

45.

Del Re

(2013) compute.es: compute effect sizes. R package version 0.2-2. http://CRAN.R-project.org/package=compute.es

46.

Nunnally

, Bernstein

. (1994) Psychometric theory. New York, NY: McGraw-Hill.

47.

Groth-Marnat

. (2009) Handbook of psychological assessment. Hoboken, NJ: John Wiley & Sons.

48.

Revelle

, Zinbarg

. Coefficients alpha, beta, omega, and the glb: comments on sijtsma. Psychometrika, 2009; 74:145–154.

49.

Zinbarg

, Revelle

, Yovel

, et al. Cronbach's α, Revelle's β and McDonald's ω H: their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 2005; 70:123–133.

50.

Costello

, Osborne

. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 2005; 10:173–178.

51.

Henson

, Roberts

. Use of exploratory factor analysis in published research: common errors and some comment on improved practice. Educational and Psychological Measurement, 2006; 66:393–416.

52.

Campbell

, Fiske

. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 1959; 56:81–105.

53.

Strauss

, Smith

. Construct validity: Advances in theory and methodology. Annual Review of Clinical Psychology, 2009; 5:1–25.

54.

Comrey

, Lee

. (1992) A first course in factor analysis. Hillsdale, NJ: Erlbaum.

55.

MacCallum

, Widaman

, Zhang

, et al. Sample size in factor analysis. Psychological Methods, 1999; 4:84–99.

56.

MacCallum

, Widaman

, Preacher

, et al. Sample size in factor analysis: the role of model error. Multivariate Behavioral Research, 2001; 36:611–637.

57.

Roussos

, Stout

. Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. J Educ Meas, 1996; 33:215–230.

58.

Diener

, Seligman

MEP

. Very happy people. Psychological Science, 2002; 13:81–84.

59.

Reise

, Yu

. Parameter recovery in the graded response model using MULTILOG. J Educ Meas, 1990; 27:133–144.

60.

Harris

. Sexuality and suicidality: matched-pairs analyses reveal unique characteristics in non-heterosexual suicidal behaviors. Archives of Sexual Behavior, 2013; 42:729–737.