The purpose,content and development phases of a graduate entry ePortfolio in occupational therapy

Abstract

BACKGROUND:

There is a trend in higher education towards the use of digital or electronic portfolios (ePortfolios) to collect evidence that demonstrates learning and skill development. There are very few papers that examine the key features and what to include in an occupational therapy ePortfolio.

OBJECTIVE:

This study presents an approach to developing a graduate entry ePortfolio to prepare occupational therapy students for transition to work.

METHODS:

An e-Delphi approach was used to gather the opinions of eight categories of experts. Three rounds of questionnaires were used to explore the purpose, terminology, content, structure, and development phases of a graduate entry ePortfolio.

RESULTS:

Key stakeholders indicated that the purpose of a professional portfolio is to create a professional profile, record experiences, skills and behaviours, and promote recording of lifelong learning and achievements. Delphi expert panel members emphasised the importance of personal choice in selecting evidence recorded in each collection, which must be guided by ethical decision-making.

CONCLUSIONS:

The findings of the study are important to students who wish to build an ePortfolio to organise and display evidence of competence prior to graduation in readiness to enter the workforce. The recommendations also will be of value to occupational therapy educators in curriculum development.

Keywords

Transition Delphi digital

1 Introduction

Paper-based portfolios have been commonplace in the profession of occupational therapy for many years [1, 2]. However, this practice is fast becoming outmoded as online electronic portfolios (ePortfolios) have emerged as an alternative [3 –5]. Increased use of ePortfolios is reported in the higher education literature including that dealing with medicine and nursing [6 –8]. ePortfolio is an essential tool in one’s work as an occupational therapy student and future occupational therapist, nonetheless, there is little information available to guide educators and students about what to include in an occupational therapy ePortfolio [9, 10]. This study proposes an approach to ePortfolio development to overcome this deficit.

Portfolios originated as a collection of paper-based resources that represent an individual’s experiences, knowledge and skills [1, 2], plans for professional growth [1, 10] and they can be used in the job seeking process [2, 7]. Due to the continued expansion in size and complexity of data coupled with difficulties in storing, managing and sharing materials [11], paper-based portfolios become cumbersome and unmanageable after a few years [7]. ePortfolios provide an online alternative to paper-based portfolios [12] as they provide a portable document reservoir that can be edited, updated, and shared when building an online professional identity [1 , 13]. These benefits are not restricted to the use of ePortfolios in occupational therapy as ePortfolios are reported to be in use in other areas of health education including medicine [6], nursing [7, 11], and dentistry [8, 17].

For students undertaking degrees in the professions, documenting evidence of the learning process is important [1, 14]. Through accurate and timely recording of professional development experiences and reflections, an ePortfolio assists occupational therapists to meet the requirements of ongoing registration and auditing [15]. An ePortfolio also provides a platform on which the student and early career therapist can record the experiences, ideas and recommendations necessary for improved clinical practice [8 , 17], and can be designed to include a combination of publicly accessible and confidential information. The construction, components and content of an ePortfolio must take into consideration the specific needs of each of the stakeholders and include students, teachers, the education institution, and future employers [3].

Our research adds to the developing body of literature for ePortfolio use in health professions and is one of the first studies to focus on occupational therapy. We developed an expert-informed strategic approach for the development of an occupational therapy ePortfolio that will be of value to students as they prepare to enter practice.

2 Methods

2.1 Design

The Delphi technique was selected to seek the opinions of a range of stakeholders from the occupational therapy profession. The Delphi technique is a process that is designed to collect and distil the anonymous judgments of experts using a series of data collection and analysis techniques interspersed with feedback [18]. According to Linstone and Turoff [19] the technique emerged from defence research and Project Delphi was the name given to the research approach developed by The Rand Corporation [20, 21]. The Delphi technique uses a survey involving multiple rounds of questionnaires [18]. Each round utilises a panel of experts that together bring a range of perspectives.

The Delphi technique is compatible with the constructivist paradigm [22] which proposes that researchers do not begin with a theory but develop one as the research proceeds [23, 24]. Researchers use the opinions gathered from experts to formulate theories, understand the socially constructed reality, and deepen their understanding of the human experience of a particular issue [24]. This approach is appropriate to exploring the research associated with the development and use of ePortfolios in occupational therapy in Australia and also globally.

2.2 Procedure

We implemented a three-round e-Delphi (online Delphi). This approach gave panelists the opportunity to offer individual feedback and responses privately using online questionnaires. The questionnaires were distributed via email and hosted on an online survey tool, SurveyMonkey^TM. This approach minimised some of the shortcomings of focus group research, where a dominant individual and group pressures can influence responses by panelists [25].

Ethics approval was received from the Human Research Ethics Committee at University of the Sunshine Coast with the proviso that each questionnaire was sent to the ethics review panel for approval before dissemination. To facilitate timely completion of the research, the questionnaire links sent to panelists were active for one week. Two reminders were sent to the panelists during the seven days and on two occasions response time was extended by two days.

The trustworthiness of the data collection process and interpretation of responses was examined in terms of credibility, dependability, confirmability and transferability [26]. The credibility of the research was enhanced through the inclusion of a clear audit trail describing the decision making processes that determined the structure and membership of the expert panel, questionnaire design, and decision making [18]. The dependability of the research was enhanced through the recruitment of expert panelists and recurrent member checking, whereby experts were able to review and amend their previous responses [27]. Confirmability and transferability was enhanced through clear documentation of the recruitment criteria, questionnaire design, and triangulation of the process with previous research [18].

2.3 Sampling

There is no one sample size recommended for Delphi studies [28] and the choice of each panelist was determined after the development of expert categories. The development of expert categories was iterative through group discussion and examination of the literature. A range of expert categories was identified and possible criteria were presented to the expert stakeholder groups. Eight categories emerged from this process (Table 1). The primary work location of all the expert panelists was in Australia.

Table 1
Stakeholder categories, expert qualification criteria and rationale

Stakeholder Groups Expert Criteria Contribution

1. Occupational therapy competency expert Qualified occupational therapist, with expert knowledge in the Australian occupational therapy competencies What is needed to satisfy Australian occupational therapy competencies?

2. EPortfolio expert Expert in ePortfolios, not limited to occupational therapists What is the current or potential use of an ePortfolio, and its strengths and limitations?

3. Experienced occupational therapist Occupational therapist currently practicing, with 5 + years of experience How is ongoing competence demonstrated currently in the workforce?

4. New graduate occupational therapist Occupational therapy graduate (class of 2013), who has used an ePortfolio in the last 12 months Recent experience in using ePortfolio, and whether it has influenced employment or job applications.

5. Occupational therapy registration expert Current or previous member of an occupational therapy registration board (e.g. AHPRA or state registration board), with expert knowledge in occupational therapy registration What is required to satisfy ongoing Australian occupational therapy registration?

6. Occupational therapy academic Current occupational therapy academic using or who has used ePortfolios How can ePortfolios be used and applied to improve the quality of undergraduate occupational therapy education and professional competencies in relation to clinical placement?

7. Professional development expert Member of Occupational Therapy Australia, or Queensland Health with expertise in professional development What is required to show continuing professional development for occupational therapists?

8. Employer Manager, employer, or interviewer of occupational therapists What do employers look for in successful applicant’s portfolios?

Stakeholder Groups	Expert Criteria	Contribution
1. Occupational therapy competency expert	Qualified occupational therapist, with expert knowledge in the Australian occupational therapy competencies	What is needed to satisfy Australian occupational therapy competencies?
2. EPortfolio expert	Expert in ePortfolios, not limited to occupational therapists	What is the current or potential use of an ePortfolio, and its strengths and limitations?
3. Experienced occupational therapist	Occupational therapist currently practicing, with 5 + years of experience	How is ongoing competence demonstrated currently in the workforce?
4. New graduate occupational therapist	Occupational therapy graduate (class of 2013), who has used an ePortfolio in the last 12 months	Recent experience in using ePortfolio, and whether it has influenced employment or job applications.
5. Occupational therapy registration expert	Current or previous member of an occupational therapy registration board (e.g. AHPRA or state registration board), with expert knowledge in occupational therapy registration	What is required to satisfy ongoing Australian occupational therapy registration?
6. Occupational therapy academic	Current occupational therapy academic using or who has used ePortfolios	How can ePortfolios be used and applied to improve the quality of undergraduate occupational therapy education and professional competencies in relation to clinical placement?
7. Professional development expert	Member of Occupational Therapy Australia, or Queensland Health with expertise in professional development	What is required to show continuing professional development for occupational therapists?
8. Employer	Manager, employer, or interviewer of occupational therapists	What do employers look for in successful applicant’s portfolios?

Panelists were purposively selected to contribute their expertise to the research. Current student perspectives were not sought but recent graduates (previous two years) were included to capture current and recent use and application of ePortfolios. Real life experience in the transition from student to practitioner was considered integral to establishing the credibility of the study. Individuals who were known to meet the criteria of a category were invited to participate in the research. The researchers recruited two or more expert panelists to each category to ensure all the stakeholder categories were represented. After purposive recruitment was exhausted some expert categories were filled using snowball sampling [27] ensuring that each of the eight categories had at least two representatives.

To confirm that panelist members were appropriate for their category, they were asked to indicate how their experience and knowledge satisfied the proposed criteria and to provide a summary of their qualifications relevant to the particular category. This was checked against the criteria in Table 1 and each panelist was found to be eligible to represent at least one expert category.

2.4 Development and administration of questionnaires

2.4.1 Round 1

Questionnaire one required informed consent and included questions regarding demographics and expert quality control measures to ensure credibility and transferability of the research. The Round 1 questionnaire consisted of two open-ended questions regarding the purpose of an ePortfolio and the components for inclusion or exclusion in the Australian occupational therapy context. Panelists were asked to give their consent to continue to Round 2.

2.4.2 Round 2

The Round 2 questionnaire opened with a summary of the themes identified in round one. The themes also structured the development of the Round 2 questions and direct the experts’ attention towards the key aim of the project, which was to develop an approach to building an ePortfolio. Questions were structured so that panelists could select preferences from multiple options and provide comments. The questions explored the following areas: the purpose of an ePortfolio in occupational therapy; ePortfolio terminology; the key items for inclusion in an ePortfolio and the phases of ePortfolio development. Expert panelists were asked to agree, agree and comment on, or disagree and comment on the proposed items in order to verify the accuracy of the themes identified from round one.

To focus on developing an approach to ePortfolio development, expert panelists were asked to rank the key items for inclusion in order of importance and were given the opportunity to select the most preferred items for each key phase. Expert panelists were then asked to prioritise each item for inclusion in the ePortfolio and select which items should be public, private, or optional. At the conclusion of Round 2, panelists were asked to give their consent to participate in the third round.

2.4.3 Round 3

The aim of the third questionnaire was to confirm what should be included in an ePortfolio to make it appropriate for use in occupational therapy education and practice in Australia. Panelists reviewed the revised versions of the ePortfolio purpose, terminology and development diagram and gave feedback on any necessary re-categorisation and re-naming of key items for inclusion. The modified ePortfolio development approach was presented to the panelists for review and feedback.

2.5 Data analysis

Panelist responses were de-identified. At the conclusion of each round, responses were thematically analysed, two members of the research team crosschecked themes and triangulated these using previous research [e.g., 3, 8, 14, 17]. Consistent with current practice in Delphi research, panelist consensus was measured by agreement or disagreement with each theme, question or category presented using a consensus of 75% agreement [28, 29].

Each new questionnaire was used to develop and clarify the themes that emerged in the previous round. Member checking was completed in each subsequent questionnaire to confirm the themes were an accurate representation of panelists’responses.

3 Results

There were 16 expert panelists in the first round, 13 in the second round, and 15 in the third round of the e-Delphi research. Seven of the eight stakeholder groups were represented throughout the three rounds but the occupational therapy registration expert was absent for Round 2. The demographic information and involvement of panelists is shown in Table 2.

Table 2
Panelist demographic information and participation

Round 1 Round 2 Round 3

PRIMARY WORK LOCATION

Queensland 10 10 9

New South Wales 2 1 2

Victoria 3 2 3

South Australia 1 0 1

PRIMARY STAKEHOLDER CATEGORY

Occupational therapy competency expert 2 2 3

EPortfolio expert 3 3 3

Experienced occupational therapist 2 2 2

New graduate occupational therapist 2 2 1

Occupational therapy registration expert 2 0 2

Occupational therapy academic 1 1 1

Professional development expert 2 1 2

Employer 2 2 1

TOTAL NUMBER OF PANELISTS

Invited to participate 24 17 16

Expressed interest to participate 19 17 16

Completed questionnaire 16 13 15

Formally declined to participate in next round 0 1 N/A

	Round 1	Round 2	Round 3
PRIMARY WORK LOCATION
Queensland	10	10	9
New South Wales	2	1	2
Victoria	3	2	3
South Australia	1	0	1
PRIMARY STAKEHOLDER CATEGORY
Occupational therapy competency expert	2	2	3
EPortfolio expert	3	3	3
Experienced occupational therapist	2	2	2
New graduate occupational therapist	2	2	1
Occupational therapy registration expert	2	0	2
Occupational therapy academic	1	1	1
Professional development expert	2	1	2
Employer	2	2	1
TOTAL NUMBER OF PANELISTS
Invited to participate	24	17	16
Expressed interest to participate	19	17	16
Completed questionnaire	16	13	15
Formally declined to participate in next round	0	1	N/A

Panelist responses are reported by question grouping (ePortfolio purpose, terminology, content and development phases) and the themes identified during analysis. The discussion section provides explanations for the development of each theme across the three rounds of the research.

3.1 ePortfolio purpose

3.1.1 Round 1

Panelists described an ePortfolio as useful for registration, employment, education, and personal development and its three major practical functions were identified as: filing and storage; ease of editing and updating; the capacity to share and control the availability of information. Two specific comments stood out during data analysis: ePortfolios are “the way of the future” and a way to “keep ahead of the pack”.

3.1.2 Round 2

In this round responses were analysed with a view to developing an ePortfolio purpose statement for occupational therapy in Australia. Although, no new themes were identified, panelists suggested examining and expanding the following themes to clarify the purposes:

meeting professional requirements;

demonstrating evidence of professional skill development;

developing self-concept and professional online identity;

“head hunting”/recruitment tool;

building a web presence that is viewable by members of the public, including clients and employers; and

the cyclical nature of an ePortfolio – it changes and develops in accordance with the user’s needs across time (phases)

Alterations to the list of ePortfolio purposes were made in accordance with the feedback received. Panelists were presented with the revised purpose in round three.

3.1.3 Round 3

Panelists agreed (100%) that an ePortfolio should achieve the following intentions.

Provide an online portfolio that is easily shared, edited and published, and can be changed and developed in accordance with the user’s needs.

Be used to demonstrate professional behaviours, ethical conduct and appropriate personal choice in the development of a publicly available professional online identity.

Provide a digital ‘thinking space’ where individuals can record and develop reflective skills.

Illustrate relevant knowledge and experiences, to demonstrate professional skill development and competence as an occupational therapist.

Help to build good habits such as record keeping and organisational skills.

Demonstrate how registration requirements are met and facilitate ease of response in case of professional audit.

Demonstrate graduate-level competence and readiness for practice in line with the industry and registration requirements through a capstone assessment (completion task) in an occupational therapy.

Demonstrate university-specific ‘graduate attributes’.

Aid in recruitment, by showcasing suitability for particular roles, and enable employers to discern an applicant’s suitability during recruitment by providing additional information on career goals.

Facilitate improved digital literacy.

Provide an online repository that starts early in the student’s degree and evolves as the student completes their studies, enters practice and gains experience.

Is “the way of the future” and can help people “keep ahead of the pack”.

These intentions have been synthesised/combined with research literature and are presented in the Discussion.

3.2 ePortfolio terminology

3.2.1 Round 1

Panelists did not use consistent or clearly defined terminology when discussing ePortfolios, which reduced the clarity of the responses. For example, the terminology used by expert panelists to describe components of an ePortfolio included evidence, example and category but there was no mention of how these terms might relate to each other or be categorised into sub-sections. During analysis the terminology used by the panelists was noted and matched with terminology in the literature. Terminology was categorised using a three-tiered system: component (heading), element (sub-section), or artefact (examples of evidence) and presented to the panelists in Round 2.

3.2.2 Round 2

The proposed three-tiered system of ePortfolio terminology resulted in mixed responses from the panelists. In an attempt to clarify terminology that would place one term as a sub-category of another, several sets of terms were proposed. There was confusion about the distinction between the terms ‘component’ and ‘element’, which is not surprising as these terms are synonymous. Although ‘artefact’ is identified as a common term amongst ePortfolio experts [30], three panelists strongly disagreed with the use of this term. The terms collection (heading), element (sub-heading) and evidence (examples) were suggested by panelists as alternatives.

3.2.3 Round 3

The terminology proposed to categorise the content of an ePortfolio for occupational therapy emerged from panelists’ responses. When given the choice to either agree or disagree with the proposed terminology, most (94%) panelists agreed that collection, element and evidence would be terms used for the first time for ePortfolios within occupational therapy in Australia where program-specific terminology does not currently exist.

3.3 ePortfolio content

3.3.1 Round 1

Panelists identified the need to consider key stakeholder requirements when deciding what to include in an ePortfolio for occupational therapy students entering practice. Key stakeholders were represented by the panelists, which included employers, university educators, Occupational Therapy Australia (OTA), the Occupational Therapy Board (OTB) of Australia and Australian Health Practitioner Regulation Agency (AHPRA) and their comments and guidance were critical to decisions regarding the content of the ePortfolio.

Panelists highlighted the importance of meeting professional guidelines which included The Australian Minimum Competency Standards for New Graduate Occupational Therapists [31]. The Occupational Therapy Board of Australia oversees the Registration Standards which include continuing professional development [15]. Panelists also identified university graduate attributes, such as creative and critical thinking and ethical decisions and activities contribute to professional skills and behaviours, but noted that their emphasis differs across universities.

The Round 1 analysis identified that there were nine components that could be included in an ePortfolio to be used in occupational therapy. The components are listed here and ranked by most suggested:

Professional Skills and Behaviours

Critical Reflections

Personal Profile

Resume/Curriculum Vitae

Experience

External Accounts of Competence

Documentation

Lifelong Learning (continuing professional development)

Key Achievements

Panelists expressed that personal choice should be exercised when deciding what to include in an ePortfolio and careful consideration of each collection, in the context of professionalism and alignment with the profession’s code of ethics, was important. Panelists’ opinions on revealing details of personal demographics and hobbies were divided. Those who supported the inclusion of hobbies (69%) reasoned that this information could highlight other relevant skills and enhance professional identity. The inclusion of demographic details, including gender and religious and political views, was opposed by 25% of the panelists (and the other 75% of panelists did not comment).

3.3.2 Round 2

In Round 2 the panelists were asked to use the categories necessary, optional, not relevant, or should be excluded to refine the list of nine potential ePortfolio components proposed in Round 1. Consensus was achieved for the following components as necessary items to include; Personal Profile (76.9%), Resume/CV (85%), Experience (100%), Professional Skills (100%), and Continuing Professional Development (CPD) (100%). While the panelists did not achieve consensus of agreement that the following components were necessary to include, there was more than 66% agreement that Critical Reflections (69.2%), Key Achievements (69.2%), Documentation (69.2%) could be included in an ePortfolio.

To differentiate between necessity for inclusion in the portfolio, and the need to display collections publicly, the panelists were asked to categorise each collection’s visibility as private (stored in the ePortfolio platform and not visible to others), password protected (able to share with individuals via the Internet at user’s discretion), and public (available to a public audience via the Internet). The importance of privacy and self-protection was evident in many of the responses especially in regard to protecting oneself from identity theft or bias.

The findings from Round 2 are included in Table 3. ePortfolio content and visibility.

Table 3
ePortfolio content and visibility

Collection Elements Necessity Visibility

Necessary Optional Not relevant/Exclude Password protected Private Public Not Applicable

Profile Personal details 76.9% 23.1% 0% 38.5% 15.4% 46.1% 0%

Education background 69.2% 30.8% 0% 30.8% 15.4% 53.8% 0%

Professional Social Media 15.4% 69.2% 15.4% 46.1% 7.7% 30.8% 15.4%

Career goals 30.8% 69.2% 0% 46.1% 15.4% 38.5% 0%

Professional interests 38.5% 61.5% 0% 15.4% 38.5% 46.1% 0%

Vision/ Personal statement 38.5% 61.5% 0% 46.1% 15.4% 38.5% 0%

Resume/CV Professional qualifications 84.6% 15.4% 0% 23.1% 7.7% 69.2% 0%

Professional membership 76.9% 23.1% 0% 23.1% 7.7% 69.2% 0%

Professional registration 69.2% 30.8% 0% 38.5% 7.7% 53.8% 0%

Practice Education 69.2% 30.8% 0% 30.8% 7.7% 53.8% 7.7%

Paid work 30.8% 69.2% 0% 30.8% 15.4% 53.8% 0%

Volunteering 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Hobbies 0% 69.2% 30.8% 30.75% 23.1% 30.75% 15.4%

Coursework 46.15% 46.15% 7.7% 30.8% 7.7% 53.8% 7.7%

Units of study 38.5% 38.5 % 23.1% 46.1% 0% 30.8% 23.1%

Other qualifications/licenses 53.85% 46.15% 0% 38.5% 7.7% 53.8% 0%

Experience Research 23.1% 76.9% 0% 23.1% 7.7% 69.2% 0%

Presentations 23.1% 76.9% 0% 38.5% 7.7% 53.8% 0%

Lifelong Learning CPD Record 69.2% 30.8% 0% 38.5% 15.4% 46.1% 0%

Formal learning 69.2% 30.8% 0% 46.10% 15.40% 38.50% 0%

Engagement with the profession 61.5% 38.5% 0% 53.8% 15.4% 30.8% 0%

Informal learning 53.85% 46.15% 0% 53.8% 15.4% 30.8% 0%

Learning Goals / CPD Plan 46.15% 46.15% 7.7% 61.5% 7.7% 7.7% 23.1%

Professional Skills and Behaviours Service evaluation 15.4% 84.6% 0% 53.8% 7.7% 38.5% 0%

Flexibility for changing circumstances 15.4% 76.9% 7.7% 46.1% 7.7% 38.5% 7.7%

Applying technologies 23.1% 76.9% 0% 46.15% 7.7% 46.15% 0%

Independence in work 23.1% 76.9% 0% 46.15% 7.7% 46.15% 0%

Interventions and service implementation 23.1% 76.9% 0% 53.8% 7.7% 38.5% 0%

Professional attitudes 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Professional behaviour 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Information gathering and collaborative goal setting 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Professional communication 30.8% 69.2% 0% 38.5% 7.7% 53.8% 0%

Organization skills 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Information literacy 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Professional attitudes 30.8% 69.2% 0% 46.15% 7.7% 46.15% 0%

Ethical conduct 38.5% 61.5% 0% 38.5% 7.7% 53.8% 0%

Key Achievements Projects, awards 30.8% 69.2% 0% 38.5% 7.7% 53.8% 0%

Competence Assessment, external appraisals 53.85% 46.15% 0% 69.2% 7.7% 23.1% 0%

Critical Reflections Reflective journal, CPD reflections 46.1% 38.5% 15.4% 46.1% 15.4% 23.1% 15.4%

Documentation Working with children certification 61.5% 38.5% 0% 30.8% 15.4% 53.8% 0%

Police check 61.5% 23.1% 15.4% 46.1% 15.4% 30.8% 7.7%

Collection	Elements	Necessity	Visibility
Profile	Personal details	76.9%	23.1%	0%	38.5%	15.4%	46.1%	0%
	Education background	69.2%	30.8%	0%	30.8%	15.4%	53.8%	0%
	Professional Social Media	15.4%	69.2%	15.4%	46.1%	7.7%	30.8%	15.4%
	Career goals	30.8%	69.2%	0%	46.1%	15.4%	38.5%	0%
	Professional interests	38.5%	61.5%	0%	15.4%	38.5%	46.1%	0%
	Vision/ Personal statement	38.5%	61.5%	0%	46.1%	15.4%	38.5%	0%
Resume/CV	Professional qualifications	84.6%	15.4%	0%	23.1%	7.7%	69.2%	0%
	Professional membership	76.9%	23.1%	0%	23.1%	7.7%	69.2%	0%
	Professional registration	69.2%	30.8%	0%	38.5%	7.7%	53.8%	0%
	Practice Education	69.2%	30.8%	0%	30.8%	7.7%	53.8%	7.7%
	Paid work	30.8%	69.2%	0%	30.8%	15.4%	53.8%	0%
	Volunteering	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Hobbies	0%	69.2%	30.8%	30.75%	23.1%	30.75%	15.4%
	Coursework	46.15%	46.15%	7.7%	30.8%	7.7%	53.8%	7.7%
	Units of study	38.5%	38.5 %	23.1%	46.1%	0%	30.8%	23.1%
	Other qualifications/licenses	53.85%	46.15%	0%	38.5%	7.7%	53.8%	0%
Experience	Research	23.1%	76.9%	0%	23.1%	7.7%	69.2%	0%
	Presentations	23.1%	76.9%	0%	38.5%	7.7%	53.8%	0%
Lifelong Learning	CPD Record	69.2%	30.8%	0%	38.5%	15.4%	46.1%	0%
	Formal learning	69.2%	30.8%	0%	46.10%	15.40%	38.50%	0%
	Engagement with the profession	61.5%	38.5%	0%	53.8%	15.4%	30.8%	0%
	Informal learning	53.85%	46.15%	0%	53.8%	15.4%	30.8%	0%
	Learning Goals / CPD Plan	46.15%	46.15%	7.7%	61.5%	7.7%	7.7%	23.1%
Professional Skills and Behaviours	Service evaluation	15.4%	84.6%	0%	53.8%	7.7%	38.5%	0%
	Flexibility for changing circumstances	15.4%	76.9%	7.7%	46.1%	7.7%	38.5%	7.7%
	Applying technologies	23.1%	76.9%	0%	46.15%	7.7%	46.15%	0%
	Independence in work	23.1%	76.9%	0%	46.15%	7.7%	46.15%	0%
	Interventions and service implementation	23.1%	76.9%	0%	53.8%	7.7%	38.5%	0%
	Professional attitudes	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Professional behaviour	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Information gathering and collaborative goal setting	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Professional communication	30.8%	69.2%	0%	38.5%	7.7%	53.8%	0%
	Organization skills	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Information literacy	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Professional attitudes	30.8%	69.2%	0%	46.15%	7.7%	46.15%	0%
	Ethical conduct	38.5%	61.5%	0%	38.5%	7.7%	53.8%	0%
Key Achievements	Projects, awards	30.8%	69.2%	0%	38.5%	7.7%	53.8%	0%
Competence	Assessment, external appraisals	53.85%	46.15%	0%	69.2%	7.7%	23.1%	0%
Critical Reflections	Reflective journal, CPD reflections	46.1%	38.5%	15.4%	46.1%	15.4%	23.1%	15.4%
Documentation	Working with children certification	61.5%	38.5%	0%	30.8%	15.4%	53.8%	0%
	Police check	61.5%	23.1%	15.4%	46.1%	15.4%	30.8%	7.7%

Overall, there was no consensus in round 2 recommendations on which elements are necessary and which should be public, which again highlighted the importance of ePortfolio users exercising personal choice. Comments and responses by panelists reflected the importance of considering the protection of the users’ online identity and ideas and work. The overarching theme placed emphasis on the considerations undertaken when choosing elements to include as examples in an ePortfolio, to ensure they reflect a positive, ethical and professional self-image. The Round 2 findings were used to create a preliminary proposal for development of a graduate entry ePortfolio, which was presented to the panelists in Round 3.

3.3.3 Round 3

Components were re-named collections after Round 2. In Round 3 the panelists’ responses focused on approaches to providing examples for evidence of learning, development and competence. Panelists proposed integrating a structured approach to self-reflection as a way of providing evidence of how new learning could be implemented in future practice.

3.4 ePortfolio development process and phases

3.4.1 Round 1

Responses to Round 1 informed the subsequent phases of development of an ePortfolio. Experts agreed that an ePortfolio is not static but grows and adapts, over time, to changing demands and purposes. In our research it became clear that ePortfolio development occurs over three phases: i) early education; ii) education/entry to practice; and iii) career development. A diagram that highlighted the phases of an ePortfolio was developed to depict the complex interaction between stakeholder influences and the ePortfolio development. The diagram was presented to the panelists in Round 2.

3.4.2 Round 2

Feedback on the diagram developed in Round 1 was mixed. Many (62%) found the first diagram complex or/and confusing although 30% reported that it was a good depiction of ePortfolio development (8% choosing to provide no comment). Comments relating to the layout indicated that 46% of the experts disliked the bottom-up arrangement. Due to the Western cultural norm of reading left to right, it was considered appropriate to change the diagram to that format. Some panelists (46%) reported that the overall purpose of an ePortfolio was unclear, that there was difficulty in interpreting acronyms, and clearer definitions of the ePortfolio phases were required. In response to the analysis of Round 2 a revised diagram was created which included changes to the names and time ranges for each phase, and was accompanied by a statement of purpose and a list ofacronyms.

Further questions were designed to investigate the relevance of the content in each phase. The items in the entry to practice phase were the same as in the career development phase except that lifelong learning was excluded as irrelevant. This is because entry-level education is seen as an entry point to the profession and lifelong learning depicts ongoing education and development.

3.4.3 Round 3

There was positive feedback about the changes made to the ePortfolio development diagram with 64% of panelists noting increased clarity. Comments for improvement included positioning the names of each of the stakeholders carefully and locating these with regard to where they are more or less important in the development of an ePortfolio. Another suggestion was to increase the prominence of the reflective cycles and to greater highlight that an ePortfolio is developed over time, through a process, and is not easily delineated into clear stages. These suggestions were integrated into the final revision of ePortfolio development process (Fig. 1).

Fig.1

The ePortfolio development process The purpose of this diagram was to a) clearly delineate the phases of ePortfolio development, b) display the collections for inclusion in an ePortfolio across each phase, and c) highlight the influence of key stakeholders in the development of an occupational therapy ePortfolio in Australia. Acronyms: Australian Minimum Competency Standards for New Graduate Occupational Therapists (ACSOT); Australian Health Practitioner Regulation Agency (AHPRA); Australian Qualifications Framework (AQF); Continuing Professional Development (CPD); Occupational Therapy Australia. (OTA); Occupational Therapy Board of Australia (OTBA); Occupational Therapy Council (OTC); World Federation of Occupational Therapists (WFOT).

Panelists were also asked to comment on which phase they believed ePortfolios were developed, choosing from Education Phase, Transition to Practice Phase, Career Development Phase and all phases. A summary of their responses is depicted in Fig. 2. The ePortfolio development phases.

Fig.2

The ePortfolio development phases.

4 Discussion

There is an increase in the use of ePortfolios in higher education [3 , 16] and specifically in courses concerning medical, nursing, allied health [7 , 13] and occupational therapy [14]. Our iterative research established guidelines that facilitate the development of an ePortfolio to be used by graduates in their transition to occupational therapy practice in Australia. This research has detailed the purpose, terminology, the structure and content of the graduate entry ePortfolio, and phases ofdevelopment.

4.1 ePortfolio purpose

This research highlights that a graduate entry ePortfolio is both a process and a product and that personal choice is a necessary component during its development. Expert panelists proposed that a graduate entry ePortfolio should be designed to demonstrate the process of achieving graduate-level competence and readiness for practice, and structured to capture and showcase subsequent developments during a career [4]. The research showed that ePortfolio use aids development of digital literacy and organisation skills, builds professional identity and develops professional accountability. These findings were consistent with those of Buckley et al. [32] and Domac et al. [5] who found that ePortfolios encourage graduates to develop skills in documentation, organisation and reflective skills as they prepare to enter practice. Our research supports findings by Balaban et al. [3] and Peacock et al. [10] who stated that an ePortfolio should be kept in a flexible, user-driven and secure digital repository that can be readily accessed by its author for editing. The expert panel proposed that an ePortfolio should provide both private and public spaces. The private space should provide a digital “thinking space” for the individual’s reflection, planning, learning, and record keeping. The experts also recommended that the public space could be used to build a professional identity by showcasing skills, attributes, experience and competence, and to demonstrate how they are keeping ahead of the competition.

4.2 Terminology

Agreement on a set of terms to be used in relation to the design, development and use of ePortfolios in occupational therapy would facilitate clear and consistent communication and integrate terms already being used by the profession. We proposed clear terminology for the graduate entry ePortfolio for use in occupational therapy practice. The terms collection, element and evidence provide a three-tiered structure to a portfolio. Element and evidence were proposed as they are used in the Australian Minimum Standards for Occupational Therapy Graduates (ACSOT) [31] (see Table 3).

4.3 Content

The expert panel agreed that membership of professional associations, such as Occupational Therapy Australia and the World Federation of Occupational Therapists, is a necessary component of a graduate entry ePortfolio because it demonstrates qualification to practice and commitment to the profession [33]. Our research did not identify a definitive list of topics and content for an ePortfolio and the expert panelists identified the importance of personal choice and that graduates need to carefully consider each item entered in the ePortfolio. We determined that what is included in an ePortfolio should be professional but ultimately the user’s personal choice.

4.4 ePortfolio development process and phases

The development of a graduate ePortfolio begins on entry to a degree course and continues to the final year and while preparing to enter practice (Fig. 1). This process reflects the expected transition from externally driven tuition to self-directed learning and building evidence of competence [1, 11]. This sequence was depicted by Fig. 2, which shows the differing phases of ePortfolio development and highlights the importance of progressively building and maintaining an ePortfolio across the education phase and into a career.

4.4.1 Limitations and areas for further research

The value of any expert panel is determined by the member’s qualifications and experience as well as the time given to analysing the responses made. The expert panel consisted of 16 members in the first round, 13 in the second round, and 15 in the third round. To overcome potential gaps in information due to panelist non-response and dropout, the remaining panel members were asked to respond to the questionnaire from multiple perspectives.

After analysing three rounds of questionnaire responses, the researchers reached agreement on what should be included in a graduate ePortfolio. Panelists were not in unanimous agreement with the structure and content of the ePortfolio in the undergraduate phase. Future research should investigate the other phases including user motivation to develop and maintain an ePortfolio post-graduation.

This research focused on the development of a graduate entry ePortfolio in the context of occupational therapy practice in Australia. There is relevance and transferability of these findings to other professions and this should be investigated.

5 Conclusion

The aim of our study was to better understand what to include in an ePortfolio for Australian occupational therapy students entering practice. We found that a graduate entry ePortfolio is both a process and a product and its use aids development of digital literacy and organisation skills, builds professional identity and instils professional accountability. A graduate entry ePortfolio should be designed to demonstrate the process of achieving graduate-level competence in readiness for practice, and structured to capture and showcase subsequent development during a career. An ePortfolio should be housed in a flexible and secure digital repository that can be readily accessed by the author for editing and saving and should provide both private and public spaces. The private spaces provide a place for reflection, planning, learning, and record keeping whereas the public space is designed to showcase professional identity, skills, attributes, experience and competence. The public space can also be used to provide additional information about career goals that can aid potential employers in the recruitment process.

This study provided insight into the evolution of a graduate entry ePortfolio across three development phases; early education; transition to practice; and career development. During transition to practice, students should include a professional profile in their ePortfolio as well as documenting professional skills and lifelong learning experiences. The content of an ePortfolio for occupational therapy students must meet professional expectations of being ethical.

The findings of this research provides students, practitioners, and educators with expert-based guidance on the process of designing and creating a comprehensive ePortfolio to meet the needs of stakeholders in the occupational therapy profession in Australia. Other health care professions could adopt the findings.

All occupational therapists working within Australia are required to be registered with AHPRA [34] and our research will be of value to occupational therapy students, educators, practitioners and employers. Annual renewal of registration necessitates that occupational therapists maintain and store an auditable log of continuing professional development plans and activities for five years [15]. A well-designed and constructed ePortfolio will assist occupational therapy students manage the process of transition from student to graduate, maintaining the ePortfolio thereafter will assist graduates to meet the requirements of registration across their career.

Conflict of interest

None to report.

Footnotes

Acknowledgments

We wish to acknowledge the assistance of all the expert panel members who participated in each of the three rounds of questionnaires and the University of the Sunshine Coast who provided financial and scholarly support for this research. Thank you to Professor Richard Burns, who provided editorial support in the final development of the manuscript.

References

Alsop

. Portfolios: Portraits of Our Professional Lives. British Journal of Occupational Therapy. 2002;65:201–6. doi:10.1177/030802260206500503

Nagayada

, Richardson

, Schindehette

. The professional portfolio in occupational therapy: Career development and continuing competence. Thorofare, NJ: SLACK; 2005.

Balaban

, Divjak

, Mu

, Balaban

, Divjak

, Mu

. Meta-model of EPortfolio Usage in Different Environments. International Journal of Emerging Technologies in Learning (IJET). 2011;6.

Barrie

, Hughes

, Crisp

, Bennison

. Assessing and assuring Australian graduate learning outcomes: Principles and practices within and across disciplines. Sydney, NSW: The University of Sydney; 2014.

Domac

, Anderson

, Smith

. Learning to be interprofessional through the use of reflective portfolios? Social Work Education. 2016;35:530–46. doi:10.1080/02615479.2016.1178717

Buckley

, Coleman

, Davison

, Khan

, Zamora

, Malick

, et al. The Educational Effects of Portfolios on Undergraduate Student Learning: A Best Evidence Medical Education Systematic Review. BEME Guide No. 11. Medical Teacher. 2009;31:282–98. doi:10.1080/01421590902889897

Hawks

. The Use of Electronic Portfolios in Nurse Anesthesia Education and Practice. AANA Journal; Park Ridge. 2012;80:89–93.

Walton

, Gardner

, Aleksejuniene

. Student ePortfolios to develop reflective skills and demonstrate competency development: Evaluation of a curriculum pilot project. European Journal of Dental Education. 2016;20:120–8. doi:10.1111/eje.12154

Cordier

, McAuliffe

, Wilson

, Totino

, Dender

, Smith

, et al. The appropriateness and feasibility of an online e-Portfolio for assessment of undergraduate allied health students. Australian Occupational Therapy Journal. 2016;63:154–63. doi:10.1111/1440-1630.12226

10.

Peacock

, Murray

, Scott

, Kelly

. The Transformative Role of ePortfolios: Feedback in Healthcare Learning. International Journal of EPortfolio. 2011;1:33–48.

11.

Lammintakanen

, Saranto

, Kivinen

, Kinnunen

. The digital portfolio: A tool for human resource management in health care? Journal of Nursing Management. 2002;10:321–8. doi:10.1046/j.1365-2834.2002.00346.x

12.

Slade

, Murfin

, Readman

. Evaluating Processes and Platforms for Potential ePortfolio Use: The Role of the Middle Agent. International Journal of EPortfolio. 2013;3:177–88.

13.

Lawson

, Kiegaldie

, Jolly

. Developing an ePortfolio for Health Profssional Educators: A Case Study. In: Jafari

, Kaufman

, editors. Handbook of Research on ePortfolios, Hershey PA: Idea Group Inc (IGI); 2006, pp. 273–82.

14.

Moores

, Parks

. Twelve tips for introducing E-Portfolios with undergraduate students. Medical Teacher. 2010;32:46–9. doi:10.3109/01421590903434151

15.

AHPRA. Occupational Therapy Board of Australia - Audit 2017. http://www.occupationaltherapyboard.gov.au/Registration/Audit.aspx.

16.

Fullana

, Pallisera

, Colomer

, Fernández

Peña R

, Pérez-Burriel

. Reflective learning in higher education: A qualitative study on students’ perceptions. Studies in Higher Education. 2016;41:1008–22. doi:10.1080/03075079.2014.950563

17.

Koole

, Vanobbergen

, De Visschere

, Aper

, Dornan

, Derese

. The influence of reflection on portfolio learning in undergraduate dental education. European Journal of Dental Education. 2013;17:e93–9. doi:10.1111/j.1600-0579.2012.00766.x

18.

Powell

. The Delphi technique: Myths and realities. Methodological Issues in Nursing Research. 2003;41:376–82.

19.

Linstone

, Turoff

. Introduction. In: Linstone

, Turoff

, editors. The Delphi Method: Techniques and Applications, Boston, MA: Addison-Wesley Publishing; 1975.

20.

Custer

, Scarcella

, Stewart

. The Modified Delphi Technique: A Rotational Modification. Journal of Vocational and Technical Education 1999;15:np.

21.

Dalkey

, Helmer

. An Experimental Application of the Delphi Method to the use of experts. Management Science. Santa Monica - California: The Rand Corporation; 1962.

22.

Aguilar

, Stupans

, Scutter

, King

. Towards a definition of professionalism in Australian occupational therapy: Using the Delphi technique to obtain consensus on essential values and behaviours. Aust Occup Ther J. 2013;60:206–16. doi:10.1111/1440-1630.12017

23.

Keeney

, Hasson

, McKenna

. Conducting the Research Using the Delphi Technique. The Delphi Technique in Nursing and Health Research, Wiley-Blackwell; 2011, pp. 69–83.

24.

Mackenzie

, Knipe

. Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research. 2006;16:193–205.

25.

Dalkey

. The Delphi Method: An Experimental Study of Group Opinion. Rand Corporation; 1969.

26.

Krefting

. Rigor in Qualitative Research: The Assessment of Trustworthiness. American Journal of Occupational Therapy. 1991;45:214–22. doi:10.5014/ajot.45.3.214

27.

Liamputtong

, editor. Research methods in health: Foundations for evidence-based practice. 2nd edition. South Melbourne, VIC: Oxford University Press; 2013.

28.

Timmer

, Unsworth

, Taylor

. Occupational therapy inpatient rehabilitation interventions with deconditioned older adults following an acute hospital admission: A Delphi study. Australian Occupational Therapy Journal. 2015;62:41–9. doi:10.1111/1440-1630.12169

29.

Diamond

, Grant

, Feldman

, Pencharz

, Ling

, Moore

, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67:401–9. doi:10.1016/j.jclinepi.2013.12.002

30.

Jafari

, Kaufman

. Handbook of Research on EPortfolios. Idea Group Inc (IGI); 2006.

31.

Occupational Therapy Australia. Australian Minimum Competency Standards for New Graduate Occupational Therapists 2010.

32.

Buckley

, Bain

, Luginbuhl

, Dyer

. Adding an active learning component to a large lecture course. Journal of Geography. 2004;103:231–7.

33.

Rivard Magnan

. Effects of professional commitment and organizational context on the professional development of Canadian occupational therapists. Doctoral Thesis. University of Alberta, 2010.

34.

Occupational Therapy Australia. National Registration for occupational therapists - Occupational Therapy Australia. Occupational Therapy Australia 2017. https://www.otaus.com.au/about/national-registration-association-membership.