Graduate Admissions in Psychology: Using Quantitative and Qualitative Measures to Understand the Frequency and Fatality of Applicant Errors

Abstract

Background

Graduate admissions in psychology continue to be a popular and competitive venture, with the demand for new graduate student opportunities exceeding the annual supply.

Objective

Our present work was a partial replication and extension of Appleby and Appleby (2006). We added closed- and open-ended questions regarding social media to gauge how graduate admissions committees utilize social media to evaluate applicants.

Method

We asked U.S. graduate admissions directors to answer six open-ended questions and then rate the frequency and fatality/harmfulness of 17 potential applicant errors. From the population of 467 graduate admissions directors, 56 provided complete responses (12.0% response rate).

Results

We examine the closed-ended quantitative results presenting descriptive data and combining the frequency and fatality scales into a scatterplot; outcomes from the open-ended qualitative results provide rich and nuanced advice about graduate admissions errors.

Conclusion

Poorly written application materials are to be avoided (obviously), but the evidence-informed advice offered here is much more nuanced and complex.

Teaching Implications

Mentors and faculty advisors can use information from this study to provide data-informed advice to students interested in improving their chances for admission to graduate programs in psychology, offering specific tips on the most harmful/fatal mistakes to avoid.

Keywords

graduate admissions applicant errors frequency fatality/harmfulness kisses of death

The popularity and competitiveness for admission to graduate programs in psychology in the United States remains high. In the American Psychological Association’s 2020 survey of 475 graduate-level Departments of Psychology (Michalski et al., 2019), graduate applications and acceptances data were compiled for master’s and doctoral level programs. The aggregate acceptance rate for master’s degree programs was 49.3% (11,897 acceptances out of 24,113 applications); the aggregate acceptance rate for doctoral programs was 13.7% (9,580 acceptances out of 69,711 applications). Given the popularity (relatively high numbers of applications) and competitiveness (relatively low acceptance percentages), providing promising undergraduate students with helpful and relevant advice about graduate admissions in psychology is critical. The processes used within psychology admissions are not deemed perfect and should be reviewed from time to time to think about how the graduate admissions process could be improved (Roberts & Ostreko, 2018).

In part because demand outweighs supply, distributing pertinent admissions information and advice is highly valued because faculty members (especially mentors and advisors) desire to provide graduate school data to their students that is current, relevant, and advantageous to students’ career planning strategy (Schatz & Ansburg, 2020). This goal continues to be relevant with the onset of worldwide pandemic conditions in March 2020. Students may have received a great deal of assistance in considering their undergraduate college experience, that is, applying to college during high school for many traditionally-aged college students. This transition is often supported and promoted in communities by high school guidance counselors, family discussions, and so on. However, some younger students often know less about the process in transitioning from an undergraduate institution to graduate-level work; this can be especially true for first-generation students (Strapp et al., 2020). Understanding how undergraduate psychology majors think about the graduate admissions process is valuable (Sanders & Landrum, 2012), and understanding how graduate admissions committees operate so that students applying for admission can optimize their chances for successful entrance are important goals. For instance, Strunk and Bailey (2015) reported from their empirical research that one-word changes (wife, partner, or husband in referring to one’s “significant other”) in the admission essay for graduate school changed the perception of the applicant. Clark and colleagues (2020) described a multiple mini-interview process that their graduate program implemented as part of the recruitment and selection process, and as they collected data about the efficacy of the multiple mini-interview process while using it, found strengths and weaknesses to the approach.

Multiple books and book chapters exist in this genre that can be used to assist graduate school applicants; these authors provide advice, summarize existing research, and often provide exemplar materials that can be quite helpful to graduate school applicants. The advice for successful graduate school admission can be provided from the perspective of faculty members (Neimeyer & Stevenson, 2008; Prinstein, 2013), students and psychologists (Kracen & Wallace, 2008), or organizations (American Psychological Association, 2007).

Because of the importance of this topic and the importance of the talent pipeline in our field, numerous researchers continue to design and conduct original studies that advance our knowledge about the graduate admissions process in psychology, both from the perspective of applicant materials preparation to admission committee selections. In an effort to be more diagnostically helpful, researchers are often specific with their research questions and hypothesis testing. To provide insights about admissions into graduate clinical neuropsychology programs, Karazsia et al. (2013) surveyed graduate faculty and clinical neuropsychology trainees about successful graduate school preparation; faculty expectations were highest for applicant training in research methods, statistical methods, and assessment. Karazsia and Smith (2016) focused specifically on PhD clinical, PsyD clinical, and PhD counseling admissions processes in later empirical work, with general interpersonal skills, intellect, and knowledge of scientific methods emerging most often as most important to graduate faculty. Using an even more specialized approach, Davis et al. (2018) surveyed faculty members in counselor education, clinical, and counseling doctoral programs about the importance and key characteristics of personal statements as part of the graduate admissions process, with personal statements emerging as the most valued part of the application in this sample.

There has been a long tradition of researchers and scholars conducting work in this area to help students know what to do and what not to do regarding the application process for graduate schools in the U.S. Appleby and Appleby (2006) received 156 responses from 88 graduate admissions committee members asking about kisses of death they had encountered while performing their admissions committee work. Appleby and Appleby provided this definition to respondents that “kisses of death” could be thought of as “aberrant types of information that cause graduate admissions committees to reject otherwise strong applicants” (p. 19). From the open-ended prompt and the 156 responses received, the authors conducted a detailed qualitative analysis, identifying five major types of kisses of death, and provided multiple examples for each of the major types. Other researchers have continued to explore potential kisses of death in graduate admissions, such as Devendorf (2020) and the potential negative stigma effects of disclosing personal mental health problems.

The “kisses of death” work (Appleby & Appleby, 2006) was published 15 years ago. It would be valuable to know if the applicant mistakes identified by graduate admissions committee members in 2006 are also identified and similarly harmful currently. Furthermore, it seems that the opportunities to be visible on various forms of social media provide chances for self-promotion as well as chances for self-defeat. Although not a salient advising concern for undergraduates considering graduate school in 2006, it would be helpful today to know what type of advice to offer graduate school-bound applicants about the potential impact of their social media profiles and activities on graduate school admissions. For instance, Wester et al. (2013) recommended that applied training programs in psychology establish overt policies defining an admissions committee’s use of online, social media information because the potential riskiness of the situation for students and faculty was high. The issues that these authors considered included the information potential applicants make public, expectations of professionalism and reasonable rights to privacy, and the work of a graduate admissions committee to identify the overt evaluative criteria that can be reliably gleaned from online data sources. To preview, the goal of the present study was to partially replicate, to extend, and to update the work of Appleby and Appleby (2006). We purposely chose not to adopt the language of “kisses of death” but we asked about applicants’ most harmful mistakes and we framed those as potentially “fatal flaws.”

Given that our research goal was to update, replicate, and extend the work of Appleby and Appleby (2006), we did not generate specific hypotheses or research questions a priori. Our analytical approaches—both quantitative and qualitative—are framed to be more descriptive and to be utilized by faculty advisors and students applying to graduate school alike to better understand both the frequency and relative harm that application errors can incur.

Method

Participants

We invited the entire population of individuals serving as directors of graduate admissions committees of departments of psychology in the U.S. to participate in our research (as presented in the online Graduate Study in Psychology database – APA, 2020); we did not originally intend to study a representative sample per se. We did not include standard demographic questions for respondents such as age, sex, and ethnic and racial group; in retrospect we should have asked these questions to provide a better description of those individuals who comprised our respondents. We did ask, however, for the number of years of experience serving on graduate admissions committees, with the average = 19.07 (SD =10.81), ranging from 2 years to 40 years.

To be fair, we have no idea if the 12% response rate achieved here (56 responses from a survey frame of 467 delivered surveys) is a representative sample of graduate admissions directors from the U.S.—that type of conclusion is not possible without more robust demographic data about the study participants. However, we believe that there is still value in the data and potential insights provided here. In such an environment where high demand for a resource leads to such competition for access, any meaningful advice that we can provide that comes directly from the source—as it does in this study from 56 graduate admissions directors—can certainly be helpful, even noting the limitations and cautions about interpreting the data.

Measures

We asked six open-ended questions at the beginning of the online Qualtrics survey:

• Based on a general sense of your graduate admissions committee, what is the most common mistake individuals make when applying to your graduate program?

• Based on a general sense of your graduate admissions committee, what is the most irritating mistake (i.e., a potentially fatal flaw) individuals make when applying to your graduate program?

• Based on a general sense of your graduate admissions committee and considering social media profiles, what would be considered damaging or detrimental online content to a candidate’s application?

• In the past 5 years, are there any trends that you and your colleagues are noting in your review of graduate school applications, either positive or negative, that you wish you could communicate to potential applicants?

• In the past 5 years, please describe any instances of immediate disqualification (i.e., fatal flaws) discovered through the review of applicants' social media profiles.

• In the past 5 years (not revealing specific identities), please describe any other/additional instances of immediate disqualification (i.e., fatal flaws) of an applicant to your graduate program.

We also converted the categories and examples from Appleby and Appleby (2006; Table 1) into 12 declarative statements, and each was rated on the frequency of its occurrence and the fatality (harmfulness) of its effect on graduate school admission. Similarly, we generated five social media declarative statements; all 17 items are presented in Table 1. We also asked respondents “How important is an applicant’s social media presence when considering their admission to your graduate program,” using a scale from 1 = not at all important to 4 = very important. Lastly, we asked respondents how many years they had served on graduate admissions committees.

Table 1.

Descriptive Statistical Outcomes for Frequency and Fatality Scores for Individual Survey Items.

	Frequency^a Mean (SD)	Fatality^b Mean (SD)
1. Applicant lacks understanding about the graduate program	2.73 (0.69)	3.27 (0.67)
2. Applicant believes they are a good fit for program, but they are not	2.56 (0.75)	3.27 (0.78)
3. Excessive altruism in a personal statement	1.79 (0.78)	1.90 (0.71)
4. Excessive self-disclosure in a personal statement	2.35 (0.71)	2.80 (0.80)
5. Inappropriate source (writer) for a letter of recommendation	2.18 (0.87)	2.80 (0.69)
6. Name-dropping famous psychologists or public figures	1.46 (0.70)	1.76 (0.82)
7. Poorly written application materials	2.75 (0.76)	3.65 (0.52)
8. Professionally inappropriate in a personal statement	1.85 (0.83)	3.61 (0.60)
9. Revealing personal mental health details in a personal statement	2.19 (0.82)	2.50 (0.81)
10. Self-praise in spite of being poorly prepared by their undergraduate program	1.83 (0.79)	3.10 (0.81)
11. Spelling and grammar errors in the application	2.53 (0.73)	3.10 (0.64)
12. Undesirable applicant characteristics mentioned in a letter of recommendation	1.83 (0.72)	3.63 (0.56)
13. Applicants social media posts contain vulgarity, swearing, or otherwise unprofessionally written content	1.13 (0.43)	3.21 (0.96)
14. Social media posts containing photos of applicants in revealing clothing of sexually suggestive nature (e.g., photos in bikinis, speedos)	1.17 (0.47)	2.63 (1.04)
15. Applicants social media posts displaying alcohol and/or substance use/abuse	1.24 (0.44)	2.96 (0.94)
16. A social media presence of an applicant that is generally inconsistent with information given on the application	1.04 (0.19)	2.81 (1.04)
17. Applicants over-sharing personal life details via social media posts	1.25 (0.44)	2.67 (0.96)

^aNotes. When answering using the frequency scale, respondents used 1 = not very often, 2 = occasionally, 3 = sometimes, and 4 = often.

^bWhen answering using the fatality scale, respondents used 1 = not at all harmful, 2 = slightly harmful, 3 = moderately harmful, and 4 = very harmful.

Procedure

Using the online commercial product Graduate Study in Psychology from the American Psychological Association (2020) we manually extracted the name and email contact information for every director of a master’s or doctoral program housed in a Department of Psychology (or Department of Psychological Science, or similarly named department) in the United States. Because some campus systems within a state utilize the same Director of Graduate Admissions, in those cases we only contacted that person once. In cases where the email was addressed to a generic inbox, we visited these departmental sites individually/manually, found wherever possible the Director of Graduate Admissions personal email address, and updated our database with their specific contact details.

After data cleaning and data reduction, this yielded an initial population of 473 graduate admissions directors in our contact database. Using Qualtrics, we emailed each potential respondent, providing a URL to our survey, and asking for a completed response in one months’ time. Two reminders were sent to non-respondents. Removing six bounced emails, the ultimate sample frame was 467, the number of useable responses received was 56, resulting in a response rate of 12.0%.

After reading an email, learning about the study, and clicking on a link that indicated consent to participate, respondents were presented with six open-ended response items (see above), followed by the 17 closed-ended response items (see Table 1) that were answered with respect to frequency of occurrence (not very often, occasionally, sometimes, often) and with respect to the “fatality” of the event occurring in the graduate admissions process (not at all harmful, slightly harmful, moderately harmful, very harmful). Two more items followed, and we asked about the importance of social media and the participants' number of nears experience on graduate admissions committees. The first 12 items presented in the frequency/fatality section were directly adapted from the Appleby and Appleby (2006) results.

Results and Discussion

We organize the outcomes of this work into two sections, the quantitative outcomes and the qualitative outcomes, followed by a general discussion and conclusions.

Quantitative Outcomes

Twelve statements were adapted directly the results from Appleby and Appleby (2006) and converted into declarative statements; these are presented in Table 1. The graduate admissions committee chairs rated each item on its frequency of occurrence in graduate admissions materials and in its harmfulness/“fatality” to the candidate’s chances for successful application to their program. We generated five new statements about social media situations and how they could relate to the graduate admissions process; these items are also available in Table 1 (with means and standard deviations for all 17 items).

With regard to frequency, the five most frequently occurring events (with these five means ranging from 2.35 to 2.75 on a scale from 1 = not very often to 4 = often; highest first) are

• poorly written application materials

• applicant lacks understanding about the graduate program

• applicant believes they are good fit for the program, but they are not

• spelling and grammar errors in the application

• excessive self-disclosure in a personal statement.

Fortunately, these are all errors or situations that can be solved or avoided with the assistance of mentors and faculty members. Understanding match and fit for a program involves advisors and mentors providing good advice; undergraduates applying to graduate school have to seek out that advice, however. The same is true for avoiding excessive self-disclosure on one’s personal statement. Graduate school applicants should be encouraged to consult with multiple faculty members in addition to mentors or order to ensure that application materials are appropriate and that the match and fit discussions have taken place.

With regard to fatality/harm to graduate admissions, the five most fatal flaws (with these five means ranging from 3.27 to 3.65 on a scale from 1 = not at all harmful to 4 = very harmful; highest first) are

• poorly written application materials

• undesirable applicant characteristics mentioned in a letter of recommendation

• professionally inappropriate in a personal statement

• applicant lacks understanding about the graduate program

• applicant believes they are a good fit for program, but they are not.

The important of mentoring and faculty support at a student’s undergraduate institution become clear in considering these fatality data in addition to the frequency data. Poorly written application materials and mismatched program applications can be avoiding to some extent with quality mentoring and advising. If a graduate school applicant seeks out advice about their personal statement, they can avoid pitfalls as well. Some pitfalls may be unavoidable for some applicants. If a student does not have three strong letters of recommendation—and one of the letter writers composes a generally positive letter but also mentions an incident or characteristic that may reflect poorly on the applicant, the student may be “stuck.” Occasionally students underestimate that unflattering or unkind interactions with a faculty member over a grade in a class likely disqualifies that faculty member as a letter writer, or worse yet, that faculty member writes for the student but feels compelled to mention the incident in the letter.

One additional closed-ended survey item was asked of respondents: “How important is an applicant’s social media presence when considering their admission to your graduate program?” When provided with the scale 1 = not at all important, 2 = slightly important, 3 = moderately important, and 4 = very important, M = 1.36, SD = 0.53.

Although the descriptive outcomes can be useful as is, they do have limited value. We decided to combine these data graphically (see Figure 1) so that we could observe items that were rated as high frequency/high fatality, high frequency/low fatality, low frequency/low fatality, and low frequency/high fatality. By depicting the variables together in this fashion, we believe there is added value because the reader can understand likelihood and severity at-a-glance. The practical application of this study would be for graduate applicants (and their advisors) to pay attention to survey items in the upper right quadrant of Figure 1.

Figure 1.

Scatterplot of the quantitative survey outcomes using the frequency and fatality means for each item. Notes. Items depicted in blue were derived from Appleby and Appleby (2006); items depicted in yellow are social media survey questions original to this study. See Table 1 for complete wording of survey items. Bold-faced abbreviations used in the figure are defined below: app: application, appl: applicant, pers state: personal statement, LoR: letter of recommendation, prog: program, soc med: social media.

The quantitative data presented here partially replicate, extend, and expand the findings of Appleby and Appleby (2006). At the beginning of our survey, however, we asked six open-ended questions. The outcomes from those responses are reported in the next section.

Qualitative Outcomes

Because we framed our open-ended survey items to explore similar themes to Appleby and Appleby (2006) and that 12 of the 17 closed-ended directly mapped onto the themes revealed in the 2006 article, we hypothesized that those similar themes would be prevalent in our open-ended responses too. Additionally, as our research was conducted some 15 years later, we expected some changes over time and some novel results due to the inclusion of question about the influence of social media, that is, how one’s Internet presence might be considered in the admission process (and potentially work against) a prospective graduate student. Critically, these open-ended questions were completed before the 17 closed-ended survey items were presented so that graduate admissions committee respondents were not primed.

Two raters analyzed the open-ended responses using both inductive and deductive processes. We coded responses based on the 17 quantitative survey items and each rater independently noted emergent themes as they reviewed responses to each of the six open-ended questions. The six open-ended items were coded one at a time and followed this process: (1) raters individually identified responses that aligned with survey items; (2) raters individually noted emergent novel themes; (3) raters met to discuss and define novel emergent themes; (4) raters individually reviewed responses a second time, coding for alignment with survey items and novel emergent themes; (5) codes were compared and initial interrater agreement was calculated (see Table 2); (6) each area of disagreement was discussed to attain 100% agreement; and (7) any novel emergent themes were retained as a coding category for subsequent open-ended items.

Table 2.

Initial Interrater Reliabilities for the Six Open-Ended Survey Items.

Open-Ended Survey Questions	Interrater Agreement
Based on a general sense of your graduate admissions committee, what is the most common mistake individuals make when applying to your program?	.74
Based on a general sense of your graduate admissions committee, what is the most irritating mistake (i.e., a potentially fatal flaw) individuals make when applying to your graduate program?	.78
In the past 5 years, are there any trends that you and your colleagues are noting in your review of graduate school applications, either positive or negative, that you wish you could communicate to potential applicants?	.90
In the past 5 years (not revealing any specific identities), please describe any other/additional instances of immediate disqualification (i.e., fatal flaws) of an applicant to your graduate program?	.90
Based on a general sense of your graduate admissions committee and considering social media profiles, what would be considered damaging or detrimental online content to a candidate’s application?	.91
In the past 5 years, please describe any instances of immediate disqualification (i.e., fatal flaws) discovered through the review of applicants’ social media profiles.	.86
Total Interrater Reliability	.85

Notes. This table depicts the initial percent agreement between the raters. After discussion on areas of disagreement, the raters attained 100% agreement across all codes.

This process was repeated for each of the six open-ended items. Then, raters reviewed all open-ended responses again to ensure emergent themes were accurately captured.

General “Fatal Flaws”

Four open-ended survey items addressed general “fatal flaws” and two additional open-ended survey items specifically addressed social media. We examine those groups of questions separately. The four general questions were

1. Based on a general sense of your graduate admissions committee, what is the most common mistake individuals make when applying to your program?

2. Based on a general sense of your graduate admissions committee, what is the most irritating mistake (i.e., a potentially fatal flaw) individuals make when applying to your graduate program?

3. In the past 5 years, are there any trends that you and your colleagues are noting in your review of graduate school applications, either positive or negative, that you wish you could communicate to potential applicants?

4. In the past 5 years (not revealing any specific identities), please describe any other/additional instances of immediate disqualification (i.e., fatal flaws) of an applicant to your graduate program?

A total of 348 codes were identified from participant responses across the four open-ended questions. A small number of statements were coded twice; this coding anomaly occurred due to the overlap in survey items related to writing (Items 7, 11), and the overlap in survey items related to self-disclosure in the personal statement (Items 4, 8). In a number of the open-ended responses here, we identified fatal flaw response themes consistent with the kisses of death themes reported by Appleby and Appleby (2006). Said another way, when we asked open-ended questions about graduate student applicant mistakes before presenting the closed-ended survey questions based on Appleby and Appleby, graduate admissions committee members responded with text in a similar fashion on the topics on which they were about to answer the first 12 of 17 survey questions.

Open-Ended Themes Aligned with Closed-Ended Survey Items

A total of 148 codes (42.2%) represented themes captured by the survey. This indicates that while many of the same concerns noted some 15 years ago still cause problems for graduate school applicants, there are new concerns currently being expressed by graduate admissions committees. See Table 3 for a summary of the survey questions and representative sample participant statements and Table 4 for frequencies of codes by question.

Table 3.

Closed-Ended Survey Items and Sample Participant Statements.

Themes Aligned with Survey Items	Sample Participant Statement
Applicant lacks understanding about the graduate program (Item #1)	We would like for applicants to be invested enough in getting to know the program that they could articulate their own connection with our program mission and training goals better
Applicant believes the are a good fit for program, but they are not (Item #2)	Read program materials carefully and make sure applicants interest match the goals of the training program
Excessive altruism in a personal statement (Item #3)	Personal statements that are overly flowery or general—“I’m applying to graduate school because I want to help people,” etc.
Excessive self-disclosure in a personal statement (Item #4)	Incorporating personal experiences is fine but don’t revolve your whole letter around personal/family experiences. In other words, sometimes there is simply too much information (TMI).
Inappropriate source (writer)for a letter of recommendation (Item #5)	We have found more and more students sending letters of recommendation that are not from purely professional sources.
Poorly written application materials (Item #7)	Decreased ability to expresses themselves—both a function of poor writing and lack of direction
Professionally inappropriate in a personal statement (Item #8)	…making the PS [personal statement] too personal—talking too much about personal problems and issues and not addressing research interests
Revealing personal mental health details in a personal statement (Item #9)	...don’t address psychopathology in statements.
Spelling and grammar errors in the application (Item #11)	Lack of checking for grammatical errors beyond a Word spell check
Undesirable applicant characteristics mentioned in a letter of recommendation (Item #12)	Negative recommendation letter indicating the individual was a bully

Note. Items 6 and 10 were not represented in participants’ responses.

Table 4.

Frequency of Themes Aligned with Survey Items Among General “Fatal Flaw” Questions.

	Most Common	Most Irritating	Instances of Immediate Disqualification	Trends Good or Bad	Total Codes by Survey Theme	Percent of Total Codes (N =348)
%, Applicant lacks understanding about the graduate program (Item #1)	14	16	2	2	34	9.8
Applicant believes the are a good fit for program, but they are not (Item #2)	9	6	2	3	20	5.7
Excessive altruism in a personal statement (Item #3)	—	—	1	—	1	0.3
Excessive self-disclosure in a personal statement (Item #4)	—	3	1	1	5	1.1
Inappropriate source (writer) for a letter of recommendation (Item #5)	1	4	2	2	9	2.6
Poorly written application materials (Item #7)	20	15	4	6	45	12.9
Professionally inappropriate in a personal statement (Item #8)	4	3	1	1	9	2.6
Revealing personal mental health details in a personal statement (Item #9)	—	—	—	1	1	0.3
Spelling and grammar errors in the application (Item #11)	5	9	1	2	17	4.9
Undesirable applicant characteristics mentioned in a letter of recommendation (item #12)	—	1	6	—	7	2.0
Total codes aligned with survey	53	57	20	18	148	42.2

Note. Items 6 and 10 were not represented in participants’ responses.

Of the survey items based on Appleby and Appleby’s (2006) themes, the most commonly cited problems included poorly written application materials, n = 45 (12.9%), applicant lacks understanding about the graduate program, n = 34 (9.8%), applicant believes they are a good fit for the program, but they are not, n = 20 (5.7%), and spelling and grammar errors in the application, n = 17 (4.9%).

Notably, the quantitative data mirror these findings with lacking understanding of the program, believing they are a good fit and they are not, poorly written application materials, and spelling and grammar errors receiving the highest means in terms of frequency. Three of these survey items were among the top five with regard to fatality means (lacks understanding, believes they are a good fit, and poorly written application materials).

Novel Themes

Beyond the most common conclusions drawn by Appleby and Appleby (2006), novel themes emerged as expected. Of the 348 codes, 200 (57.3%) were categorized under our novel emergent themes. In other words, more than half of the coded responses to questions about recent trends and common, irritating, and potentially disqualifying application mistakes—were represented by novel concerns (the data analysis presented here does not yet include our entirely new questions about the influence of social media on graduate school admissions). These themes may reflect changes to the application process or behaviors that have emerged in graduate student applicants not captured previously. Novel themes and frequency by question are summarized in Table 5. Definitions and sample statements of novel themes are summarized below.

Table 5.

Frequency of Novel Emergent Themes Among General “Fatal Flaw” Questions.

	Most Common	Most Irritating	Instances of Immediate Disqualification	Trends Good or Bad	Total Codes by Survey Theme	Percent of Total Codes (N = 348)
%, Preparation	6	2	20	22	50	14.4
Lack of professionalism	1	14	17	5	37	10.6
Incomplete applications	13	4	7	—	24	6.9
Wrong university/program	3	9	1	—	13	3.7
Poor undergraduate advising	4	5	2	1	12	3.4
Mentor solicitation/identification	6	4	—	—	10	2.8
Illegal activity	—	—	3	—	3	0.9
Explicit bias	—	—	3	—	3	0.9
None	—	—	4	15	19	5.4
Uncodeable/Did not answer	2	4	11	12	29	8.3
Totals	42	35	68	55	200	57.3

Preparation

Fifty of the statements coded (14.4% overall) addressed applicant preparation. Although it is arguable that some of the survey items addressed preparation, for example, applicant thinks they are a good fit for the program, but they are not, and self-praise in spite of being poorly prepared by their undergraduate program, the raters felt these items did not fully capture the concerns noted and adopted a novel code. The responses categorized into this code addressed issues such as GRE scores, undergraduate courses, and general undergraduate experiences including participation in undergraduate research, being published, and experiences related to diversity. It should be noted that due to the nature of open-ended Survey Question #4 which asked for positive and negative trends, sample statements include examples of applicants who are both well- and ill-prepared. Regardless of valence, this theme reiterates the importance of high-impact undergraduate experiences in addition to more traditional metrics of preparation including GRE scores.

-Nothing I can think of, other than that successful applicants are much more likely to have co-authored journal publications than in the past.

-Higher GRE scores, more interest in working with diverse backgrounds.

-Sometimes I see what appears to be weak UG courses, such as bowling, etc.,

-Scary low GRE scores.

-Having a strong record of undergraduate research experience, (e.g., working in one or more professors’ labs, presenting papers at conferences, designing and executing their own study is a clear strength and appears to be getting better among competitive applicants).

-Many are seeking additional work experience in related jobs or master’s degrees in relevant fields prior to application.

-Applicants need to realize the importance of research—either working in a lab, completing an honors’ (or independent) project, conducting presentations. This is heavily weighted in our application process—more so than GRE and GPA.

-GPAs below 3.0 do not lead to immediate disqualification, but no applicant in the past years made it past initial screening with an average point of less than B.

Lack of Professionalism

The raters also identified this as a separate theme although, like “preparation,” it was captured more granularly on the open-ended survey. For example, survey responses related to inappropriate sources of recommendation letters or inappropriate self-disclosure, may be reasonably considered adjacent to “professionalism,” broadly defined. However, respondents identified concerns that could better be defined as unprofessional behavioral manifestations in addition to specific problems with application materials. For instance, responses which communicated that prospective students were repeatedly emailing or haranguing the department, asking questions of the faculty or department that were clearly answered on the website, or yelling or having outbursts during interviews were coded as “professionalism.” A total of 37 statements (10.6%) were coded this way.

-We noticed many candidates seem very anxious and bombard the office or director with questions that are addressed in our handbook. I think candidates think that frequent emails or requests to meet with faculty and staff is the way to show interest. For us, it shows a lack of initiative and insight to investigate their questions on their own.

-There seems to be an increasing trend in applications along the lines of the applicant saying tell me why I should attend your institution. This does not come off well as the applicant has applied to the program and is not yet selected for an interview.

-I would much rather have a student that didn't have amazing GRE scores or grades, but who is a hard worker and a nice person than to have a smart jerk in our program. This is best demonstrated in the interview, and it is astounding what kinds of terrible behaviors students share (screaming at people when they disagree, stalking ex-girlfriends).

-Students who have really inappropriate email addresses (violent ones like “killeveryone@email.com” or unprofessional ones like “ugly_loser@email.com”)

Incomplete Applications

Respondents explicitly identified incomplete applications frequently as a “most common” mistake. A total of 24 codes (6.9%) were recorded for this theme.

- Failing to submit all required materials.

- The most common mistake is that some students do not read the instructions for applying to our particular program, they only fill out the generic university application and do not know that they are supposed to supply us with letters of recommendation or a personal statement.

Mentioning the Wrong University Name or Program

Whereas this theme may be considered part of poorly written materials, it was initially identified as the raters reviewed “most irritating” mistakes. Strikingly, it was the fourth most frequently mentioned theme for that question (n = 9). Across all general “fatal flaw” questions it was recorded 13 times (3.7%). In some cases, respondents noted this specifically as a failure to properly edit and some attributed the error to poor copy/paste performance.

- While it is understood that applicants apply to many programs, one of the most irritating mistakes is when their letter of intent/cover letter has incorrect information related to the program they are currently applying to (e.g., forgetting to remove information/names from other sites they applied to).

- Some students are not careful and write a different school name and/or graduate program type in their application. It makes it seem like they are sloppy in their work habits and/or do not know what they want to study.

- Mentioning that they are applying to a program other than the one they chose in their application. In our department, this mostly means that the application mentions clinical—we don’t have a clinical program!

Poor Undergraduate Advising

Statements pertaining to applicants’ lack of understanding about a domain of psychology or what it takes to be a graduate student more generally were coded under this theme. In some ways, statements that fit this category could be interpreted as “lacks understanding of the graduate program” (Closed-Ended Survey Item #1) or “believes they are a good fit for the program, but they are not” (Closed-Ended Survey Item #2). Yet, as we examined the responses something seemed qualitatively different from this lack of understanding or perhaps an overinflated ego (as Item #2 may indicate). Rather, it had more to do with students applying to programs for subdisciplines of psychology and demonstrating that they did not know what that subdiscipline consisted of. Respondents noted more predictable confusions such as counseling and clinical, and more substantive confusion such as applying to Industrial/Organizational and not demonstrating knowledge of the field, or applying to a program and noting that they are looking forward to a future in a work environment that the program does not prepare graduates for. In some cases, respondents explicitly attributed this lack of understanding to poor undergraduate advising. 12 statements (3.4%) were coded for this theme.

- Revealing a fundamental misunderstanding about graduate training by focusing on goals like “earning my graduate degree” or “becoming a professor” rather than on substantive areas of research.

- Not understanding the difference between clinical and counseling psychology.

- As a school psychology, program, we have had a few people who clearly are unfamiliar with what a school psychologist is.

- Students don’t know what the field actually is. Students regularly apply to I-O programs saying they want to do “counseling” of people at work...that is not what we do!

- ...it is the undergraduate faculty advisor’s role to educate (themselves first) their advisees as they begin their graduate program application process. Graduate programs represent (and misrepresent) their graduate opportunities. Simply perusing a website does not provide a complete picture of program requirements, credits, practicum experiences, whether it is a full- or part-time program, and most importantly how long it takes to complete a particular program.

Mentoring Solicitation/Identification

Another novel theme arose around identifying or soliciting a research mentor or faculty member the prospective student hoped to work with. Respondents noted that students failed to do this when they were required, or, did so clumsily or superficially. This was first identified as the raters reviewed “most common” mistakes. In some cases, this was characterized as the student not truly knowing the person’s area, identifying people who were not accepting new students, or not tailoring their application to the mentor’s interests that they’ve identified. This theme was identified 10 times (2.8%) exclusively as a “common” or “irritating” mistake. Notably, this was not identified in Appleby and Appleby (2006), likely because it is a relatively new practice and represents a shift in the landscape in applying to graduate and invites additional opportunities for students to be rejected.

-Failure to reach out to potential mentors to determine if their application is competitive.

-Not identifying a match with one or more faculty members.

-They don’t give specific information in regards to whom they want to be mentored by/apprentice to and why.

-Writing that they want to work with faculty member X on research topic A, when research topic A is not at all what faculty member X does. Similarly, I advise students to avoid putting too much emphasis on a particular faculty member or research area when applying. First, early graduate students have the right, and often do, change what they are interested in while they are in a program. Second, if faculty member X does not accept students that year, they may be shut out. Instead, I advise students to write something like “I am interested in A, B, and C, and thus could see myself working with Drs. X, Y, and Z.”

Illegal Activity and Explicit Bias

Even though this only accounted for 6 (1.8%) of the codes (3 indicating illegal behavior, three indicating explicit bias) this is a surprising and remarkably novel theme because it was exclusively mentioned in the context of the question about instances of immediate disqualification. And, this did not materialize in the Appleby and Appleby (2006) outcomes. Respondents mentioned applicants disclosing felonies and criminal misconduct as well as discriminatory language.

- …falsifying an application on the basis of past behavioral misconduct (e.g., criminal misconduct), would be fatal.

- Recent felonies.

- Clear bias in working with individuals from diverse backgrounds and/or LGBTQ populations.

- Language in essays that reflects anger, superiority, entitlement, or discriminatory beliefs.

No Instances of “Immediate Disqualification” or Identifiable Trends. A total of 19 (5.4%) explicitly noted that there had been no instances of immediate disqualification or any identifiable trends, good or bad, within the past 5 years. It is worthy of note that the data collection for this study was completed prior to the onset of the COVID-19 pandemic, and that the “identifiable trends” outcome could be different now following the grand pivot in higher education.

Social Media “Fatal Flaws”

Because Appleby and Appleby (2006) collected data at a time when social media was not broadly used, we asked two open-ended questions about the potential influence of applicants’ social media presence on graduate admissions committees’ decision making. These questions were

1. Based on a general sense of your graduate admissions committee and considering social media profiles, what would be considered damaging or detrimental online content to a candidate’s application?

2. In the past 5 years, please describe any instances of immediate disqualification (i.e., fatal flaws) discovered through the review of applicants’ social media profiles.

Together, a total of 148 statements were coded for themes aligning with the survey and emergent novel themes. Table 6 displays frequencies by question and theme and shows combined percentage of theme frequency across both social media questions. However, because these questions differ in terms of what would be damaging, and what has been damaging, we examine these questions separately here. As before, responses were coded for alignment with survey items and novel emergent themes. See Table 7 for a summary of sample responses by question and theme.

Table 6.

Frequency and Percentages of Themes from Social Media Questions.

Theme	Potentially Damaging	Resulted in Immediate Disqualification	Total	Percent of Total Social Media Codes (N = 148)
Social media posts containing photos of applicants in revealing clothing of sexually suggestive nature (e.g., photos in bikinis, speedos) (Item #14)	3	1	4	2.7%
Applicant’s social media posts displaying alcohol and/or substance use/abuse (Item #15)	9	1	10	6.8%
A social media presence of an applicant that is generally inconsistent with information given on the application (Item #16)	1	—	1	0.7%
Don’t check social media	30	25	55	37.2%
None	1	17	18	12.2%
Lack of professionalism	19	1	20	13.5%
Bias	19	3	22	14.9%
Crime	3	—	3	2.0%
Uncodeable/Didn’t answer	3	12	15	10.0%
Totals	88	60	148	100

Note. Item 17 was not represented in participants’ responses.

Table 7.

Sample Participant Statements by Social Media Question and Theme.

Theme	Sample Statements: Potentially Damaging	Sample Statements: Immediate Disqualification
Social media posts containing photos of applicants in revealing clothing of sexually suggestive nature (e.g., photos in bikinis, speedos) (Item #14)	[I]mages/cover photos of questionable content suggesting questionable decision making including nude images that are readily available.	I recall one person talking about their social media involvement which ended up having pictures of her in a bikini.
Applicant’s social media posts displaying alcohol and/or substance use/abuse (item #15)	Excessive partying and obvious drug use would likely lead the committee to view an application negatively—as well as question the professional judgment of the candidate. [H]aving pictures with alcohol is fine, but images showing obvious drunkenness are not great.	Just having public access and content is about “partying.”
A social media presence of an applicant that is generally inconsistent with information given on the application (Item #16)	It would be damaging if an applicant indicated a duplicitous stance toward our program on any social media platform. Content which depicts behavior incongruent with our institution’s community covenant would also be detrimental.
Lack of professionalism	Anything that shows poor judgment or lack of maturity. Negative comments about undergraduate program/faculty or anti-scientific comments.	One person had a blog and some of their ideas were concerning to the committee.
Bias	Racist, sexist, extremist posts.Posting of inappropriate comments and photos suggesting prejudice or things done in poor taste. Negative or insensitive comments about diverse groups; postings that suggest the person is not open to considering other perspectives.	Expression of racial bias or inappropriate views of mental disorders.
Crime	Evidence of illegal activity.

Note. Item 17 was not represented in participants’ responses.

Potentially Damaging Social Media Content

A total of 88 codes were assigned to participant responses for this question. Overwhelmingly, respondents explicitly reported that they “don’t do this” (n = 30, 34.1%). The other most frequently recorded themes were lack of professionalism and explicit bias, each generating 19 codes (separately accounting for 22.0% of responses, together, 44.0%). The only other novel codes included crime (n = 3, 3.4%) or explicitly noting “none” (n = 1, 1.1%). Otherwise, responses to this question aligned with the closed-ended survey items (#13 through #17) that probed committees’ consideration of social media content.

A total of 13 codes aligned with three survey items about social media, applicant’s social media posts displaying alcohol and/or substance use/abuse (n = 9, 10.0%), social media posts containing photos of applicants in revealing clothing of sexually suggestive nature (e.g., photos in bikinis, speedos) (n = 3, 3.4%), and a social media presence of an applicant that is generally inconsistent with the information given on the application (n = 1, 1.1%). If we compare this to the quantitative survey data, these survey items were rated by respondents to be low in frequency but moderate in terms of potential fatality with means ranging from 2.63 to 2.96. However, it seems that low frequency may be due to the fact that so few examine this content.

Instances of Immediate Disqualification Due to Social Media Content

When respondents were asked to identify instances in which applicants were immediately disqualified due to social media content, a total of 60 statements were coded. Once again, the most common code was we “don’t do this” (n = 25, 41.7%) followed by “none” or “n/a” indicating that this had not occurred (n = 17, 28.3%), a full 20% were instances where the respondent didn’t answer or was otherwise judged as uncodeable. Only six codes were assigned for responses indicating immediate disqualification had occurred. The reasons identified were explicit bias (n = 3, 1.7%), lack of professionalism (n = 1, 1.7%), and two that aligned with survey items: social media post containing photos of applicants in revealing clothing of sexually suggestive nature (n = 1, 1.7), and social media posts displaying alcohol and/or substance use/abuse (n = 1, 1.7%).

By and large, it appears that graduate admissions committee members are not actively looking at applicants’ social media content and are rarely disqualifying students for their social media presence. As presented in Table 6, not checking social media accounted for 37.2% of the coded themes across both social media questions.

General Discussion/Conclusions

Graduate admissions in psychology continue to be a popular and competitive venture, with the demand for new graduate student opportunities exceeding the supply each year. Researchers who contribute to the graduate admissions literature—especially contributions like Appleby and Appleby (2006)—are influential and instrumental in helping potential graduate applicants (and their advisors) make the best decisions possible. Our present work was a partial replication, extension, and expansion of the Appleby and Appleby work whereby after asking six open-ended questions we asked graduate admissions directors from the U.S. to rate 17 declarative statements about graduate admissions errors, rating each statement about its frequency of occurrence and its fatality/harmfulness to a successful graduate school application. When we examine the quantitative results, we combine the frequency and fatality scales into a scatterplot (see Figure 1). Outcomes from the qualitative results provide rich and detailed nuances about graduate admissions errors, with specific instances providing elucidation and validation/confirmation of the quantitative outcomes.

The most frequent errors reported in the quantitative portion of this work involved (starting with the most frequent) poorly written application materials, lacking an understanding about the graduate program, a mismatched belief about program fit, spelling and grammatical errors in the application, and too much self-disclosure. Although those were the most frequently reported errors, the most fatal/most harmful errors (starting with the most harmful) were poorly written application materials, letter writers mentioning undesirable characteristics, personal statements that were inappropriate, not understanding the graduate program, and the applicant’s lack of fit with graduate program. The theme that emerges here is that “poorly written application materials” is the most dangerous mistake of all—it is reported as both the most frequent and the most fatal in quantitative responses.

Considering the frequency of an event and its fatality simultaneously appears to be a fruitful approach, so we provide a graphic depiction in Figure 1. “Poorly written application materials” is Item #7 from Table 1, and when graphed in two dimensions in Figure 1, it appears in the upper right quadrant. If we were giving advice to our undergraduates about graduate school admissions based on these data alone, we would warn them about these four errors: poorly written application materials (Item 7), applicant believes they are a good fit for the program, but they are not (Item 2), applicant lacks understanding about the graduate program (Item 1), and spelling and grammar errors in the application (Item 11). When considering items in the lower left quadrant, given that they happen less often or they are less fatal (or both), we would emphasize these events less to our students.

As presented in the Results section, many of the open-ended themes aligned well with these closed-ended survey items (see Table 1), even though the open-ended items were presented first in the sequence in the Qualtrics survey. This pattern of convergence provides some validation of the universality of these themes and that they persist even from 15 years ago (i.e., Appleby & Appleby, 2006). Although it is comforting to replicate prior outcomes, we also extend the reach of previous researchers by the novel results identified in the qualitative data. Given the nature of qualitative data analyses, these themes are not necessarily completely separate ideas from the quantitative items or from one another, but by our interpretation, we deemed these themes as novel or providing a more nuanced details that previously identified themes:

1. Preparation: participation in high-impact undergraduate experiences (e.g., research assistantship), adequacy of GRE scores.

2. Lack of professionalism: inappropriate writers of letters of recommendation, inappropriate self-disclosures, repeated emails to department, asking questions of department or faculty whose answers are clearly available on website.

3. Incomplete applications: not following instructions.

4. Mentioning the wrong university or wrong program: often attributed by respondents as an error in copying/pasting when applicants are applying to multiple programs.

5. Poor undergraduate advising: a lack of understanding by the applicant regarding the specialty area they are applying to, confusion between counseling and clinical psychology programs.

6. Mentoring solicitation/identification: in the graduate application process, this error occurred when the program required the applicant to identify a graduate faculty member as a potential mentor. Or this type of error occurred when applicants picked a faculty member as a mentor but there was a mismatch with their application specialty field.

7. Illegal activity/explicit bias: although rare, some applicants mentioned felonies, criminal misconduct, and used discriminatory language.

Two open-ended questions were asked about social media fatal flaws. Regarding potentially damaging social media content, graduate admissions committees seem not to monitor this according to our respondents. In those instances where social media was considered to have a potential impact, professionalism and explicit bias were identified as the key areas that would be concerning to committee members. When asked about what type of social media content would lead to the immediate disqualification of a graduate program applicant, the most common responses “we don't do this” or “this has never occurred.” In the rare instances where disqualification had occurred, the themes were in the realm of revealing, sexually suggestive photos and posts regarding alcohol and substance use/abuse.

These findings provide some useful direction for faculty who are mentoring students through the graduate application process. First, from both the current study and Appleby and Appleby (2006), it is clear that carefully written and edited application materials is critical. Students should be advised to cautiously recycle application materials so that they are both tailored to the individual program and do not contain references to the incorrect university. Both of these are easily-fixed errors. Second, assisting students in navigating the nuances of the application process is another important mentoring takeaway. Applicants should be encouraged to research their programs of interest, potential faculty, and get feedback on their requests for graduate mentors. Moreover, the current study suggests that it is also important to discuss how to interact with support staff, when to ask questions, and the expectations for interview etiquette. These unspoken social conventions are likely to disproportionately negatively affect first generation and students of color. We used the word “professionalism” as a category of responses because it was used explicitly by respondents and yet we understand this word is fraught and can be code for white normativity. Similarly, the notion that the candidate was not a good “fit” for the program could also be laden with unintentional bias against a student of color or first-generation student. Both constructs are highlighted as potential roadblocks for these students; thus, advisors should take extra care to mentor these students to avoid these potential barriers and assist them in navigating the unwritten rules of the graduate application process. Helping these students to articulate how their interests align with their programs of choice can also help them identify what they truly want out of a graduate program, and whether or not that matches up with a prospective program. Third, social media does not seem to weigh as heavily as many faculty might imagine. Even so, there are instances when undesirable online content could negatively affect an applicant’s prospects. Thus, students should still be urged to consider their online footprint. Lastly, meaningful, high-impact educational experiences do matter. When asked about trends both good and bad, undergraduate research was frequently noted.

There are limitations to this study. We did not select a sample of all entries from the Graduate Study in Psychology (APA, 2020) resource, but the entire population of master’s level and doctoral level programs housed in Departments of Psychology (or similarly named units). Additionally, there are doctoral and master’s programs in psychology in the U.S. that do not maintain a listing in the APA volume or database; thus, this resource does not reflect the entirety of opportunities available for psychology graduate education in the U.S. When asking a specific respondent to reply from the graduate program, the only demographic question we asked was number of years' experience serving on graduate admissions committees (M = 19.07, SD = 10.81); we did not ask about age or gender. Given that our response rate was 12.0%, we cannot make any assumptions about our respondents being representative of the entire population of graduate admissions directors.

Graduate Admissions in Psychology is Competitive

Given the current state, faculty advisors and mentors should provide detailed guidance to potential graduate school applicants about what mistakes to avoid. Based on the results of our research, we can temper our advice based on the frequency of certain errors occurring and the severity (i.e., fatality) of those errors occurring.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

R. Eric Landrum

Leslie D. Cramblet Alvarez

K. Nicole Jones

References

American Psychological Association (2007). Getting in: A step-by-step plan for gaining admission to graduate school in psychology (2nd ed.). American Psychological Association.

American Psychological Association . (2020). Graduate study in psychology: Find a psychology graduate program. https://gradstudy.apa.org/

Appleby

D. C.

Appleby

K. M.

(2006). Kisses of death in the graduate school application process. Teaching of Psychology, 33(1), 19–24. https://doi.org/10.1207/s15328023top3301_5

Clark

J. R.

Miller

C. A.

Garwood

E. L.

(2020). Rethinking the admissions interview: Piloting multiple mini-interviews in a graduate psychology program. Psychological Reports, 123(35), 1869–1886. https://doi.org/10.1177/0033294119896062

Davis

K. M.

Doll

J. F.

Sterner

W. R.

(2018). The importance of personal statements in counselor education and psychology doctoral program applications. Teaching of Psychology, 45(3), 256–263. https://doi.org/10.1177/0098628318779273

Devendorf

A. R.

(2020). Is ‘me-search’ a kiss of death in mental health research? Psychological Services. Advance online publication. https://doi.org/10.1037/ser0000507

Karazsia

B. T.

Smith

(2016). Preparing for graduate-level training in professional psychology: Comparisons across clinical PhD, counseling PhD, and clinical PsyD programs. Teaching of Psychology, 43(4), 305–313. https://doi.org/10.1177/0098628316662760

Karazsia

B. T.

Stavnezer

A. J.

Reeves

J. W.

(2013). Graduate admissions in clinical neuropsychology: The importance of undergraduate training. Archives of Clinical Neuropsychology, 28(7), 711–720. https://doi.org/10.1093/arclin/act056

Kracen

A. C.

Wallace

J. J.

(2008). Applying to graduate school in psychology: Advice from successful students and prominent psychologists. American Psychological Association.

10.

Michalski

D. S.

Cope

Fowler

G. A.

(2019, March). Graduate study in psychology summary report: Admissions, applications, and acceptances. Office of Graduate and Postgraduate Education and Training, Education Directorate, American Psychological Association. https://www.apa.org/education/grad/survey-data/2019-admissions-applications.pdf

11.

Neimeyer

G. J.

Stevenson

(2008). The virtual advisor: Successful strategies for getting into graduate school in psychology. SAGE.

12.

Prinstein

M. J.

(2013). The portable mentor: Expert guide to a successful career in psychology (2nd ed.). Springer Science + Business Media.

13.

Roberts

M. C.

Ostreko

(2018). GREs, public posting, and holistic admissions for diversity in professional psychology: Commentary on Callahan et al. Training and Education in Professional Psychology, 12(4), 286–290. https://doi.org/10.1037/tep0000219

14.

Sanders

C. E.

Landrum

R. E.

(2012). The graduate school application: What our students report they know. Teaching of Psychology, 39(2), 128–132. https://doi.org/10.1177/0098628312437697

15.

Schatz

R. T.

Ansburg

P. I.

(2020). Advising psychology majors about graduate school in psychology: Current practices and challenges. Scholarship of Teaching and Learning in Psychology, 6(1), 36–45. https://doi.org/10.1037/stl0000165

16.

Strapp

C. M.

Bredimus

Wright

Cochrane

Fields

(2020). Entering the workforce or going to graduate school: Themes in psychology alumni decision making. Teaching of Psychology, 48(2), 144–154. https://doi.org/10.1177/0098628320977770

17.

Strunk

K. K.

Bailey

L. E.

(2015). The difference one word makes: Imagining sexual orientation in graduate school application essays. Psychology of Sexual Orientation and Gender Diversity, 2(4), 456–462. https://doi.org/10.1037/sgd0000136

18.

Wester

S. R.

Danforth

Olle

(2013). Social networking sites and the evaluation of applicants and students in applied training programs in psychology. Training and Education in Professional Psychology, 7(3), 145–154. https://doi.org/10.1037/a0032376