Abstract
Valuable lessons learned and best practices gleaned from construction projects often do not transfer to future generations because of the lack of a formalized process. This ongoing issue gives rise to the need to impart fresh training to new North Carolina Department of Transportation (NCDOT) employees once the aging workforce retires or in the event of turnover. In addition, a platform for personnel to record pertinent project information about successes and failures in projects is needed. Such information can help solve problems and avoid repeated mistakes. The aim of this research project is to create a new program called Communicate Lessons, Exchange Advice, Record (CLEAR) to reposit knowledge gained by personnel. Integral to this program is an internal-only web-based database on NCDOT’s Connect SharePoint portal with MS Access as its backend. The North Carolina State University researchers used a Design for Six Sigma approach to identify, define, develop, optimize, and verify lessons learned/best practices to create the CLEAR database. The database fields were selected based on end-user input as well as a review of existing data, such as claims and supplemental agreements, within NCDOT data repositories. Training materials, including videos and standard operating procedures, were created to disseminate information about this new program. The CLEAR program will help the NCDOT to institutionalize knowledge and is expected to improve project cost variability and scheduling.
Construction organizations run the risk of repeat mistakes with financial implications because of the want of a formalized process to document and institutionalize knowledge gleaned by personnel. Construction projects are dynamic and keep changing in scope, personnel, or both, for various projects and over a range of time. Construction project management involves coordinating teams from all phases of the project’s lifecycle (i.e., planning, design, construction, and maintenance). If not documented properly, project participants may not have access to previous wisdom and contextual information which can lead to unwanted project performance ( 1 ). External uncontrolled factors such as utility coordination, right-of-way acquisition, project funding, and interagency communication, can lead to delays and claims ( 2 ). Failing to document experiential knowledge is one of the major reasons construction organizations repeat mistakes according to a survey performed by Anderson and Tucker ( 3 ). However, learning occurs on every project and having a formalized process to record and institutionalize such experiential knowledge can prove beneficial for organizations to be proactive in mitigating possible schedule and cost overrun issues. Lessons-learned databases have been effective in identifying improvements and innovations ( 4 – 7 ).
The Project Management Institute (PMI) defines lessons learned as the learning gained from the process of performing the project ( 6 ). Lessons learned is also one of the 17 best practices recognized by the Construction Industry Institute (CII) for enhanced project performance. The CII report on lessons learned ( 7 ) is a worthy resource in this field. It highlights the three main phases of a lessons-learned exercise as collection, analysis, and implementation. The CII report also notes that, in any organizational structure, knowing what information to document and where to document it can have an impact on the effectiveness of a designed lessons-learned tool. Therefore, lessons-learned databases are an effective means for recording and retrieving appropriate information to apprise users about past experiences, both good and bad. Establishing the right culture and upper management support is also essential to establishing a successful lessons-learned program, thereby driving innovation within the organization ( 8 ). Recent research has recommended recording knowledge to better navigate complex regulatory landscapes ( 9 ). Most organizations have now started to realize the full potential of a lessons-learned program within their organizations.
Background
The Value Management Office (VMO) at the North Carolina Department of Transportation (NCDOT) performed a study in 2014 as an initial step toward building a lessons-learned database. The intention was to create a meaningful interface between preconstruction units and field personnel and to document useful information about previous projects to act as a reference for future project planning. The study was originally referred to as the Post Construction Assessment Program (PCAP) because its primary aim at that time was to capture information about issues that arose after construction and to convey this information to the planning and design teams. Based on interviews with personnel, the identified need was to have a simple yet robust tool to institutionalize knowledge gained during and after construction. However, the VMO felt there was a need to have a broader focus encompassing all NCDOT work and not just the post-construction phase.
For this reason, the North Carolina State University (NCSU) research team was contracted to develop a new robust tool to institutionalize knowledge within the NCDOT in consultation with the North Carolina Department of Information Technology (DIT). This paper describes the effort to design and implement a lessons-learned/best-practices database named CLEAR (Communicate Lessons, Exchange Advice, Record) for the NCDOT. CLEAR is a Connect NCDOT SharePoint-based internal-only database that is intended for use mainly by personnel who are associated with any project life cycle stage within the NCDOT. Personnel from various project phases can now record information related to issues (both good and bad) that emerged in a particular project and avoid repeating mistakes. As an example of this need, during the data gathering phase of this project, the NCSU research team learned that no formal process was available for the design team to know if any issues or problems related to their designs had arisen during construction or whether any delays had occurred, additional monies were involved, or both. CLEAR is intended to communicate experiences among personnel so that successes and failures can be shared, recorded, and ultimately addressed.
The research approach here is to use the rich knowledge and experiences of NCDOT personnel that can be harnessed effectively in the form of an efficient knowledge repository program ( 10 ). The NCSU research team employed a Design for Six Sigma (DFSS) approach to accomplish this goal which is explained below. The concept that underlies CLEAR is to improve coordination among all divisions and units, and act as a best-practices guide and readiness indicator for future NCDOT projects. CLEAR thus provides a platform for interagency communication and for personnel to revisit past experiences that are rich in data.
Figure 1 shows how CLEAR aims to collect lessons learned and best management practices and encourage internal innovations within NCDOT ( 11 ). The submissions are vetted by an Expert Review Panel (ERP) that is composed of NCDOT personnel who are leaders in their respective fields and have the ability to inform and make policy changes relevant to their units or offices. To initiate a submission, a NCDOT employee would go to the Connect NCDOT CLEAR SharePoint site and fill out the necessary and relevant information online. The program can autofill some project information, and attachments such as photos or documents can be included in the submission. Once the information is complete, the submission goes to the gatekeeper in the VMO. The gatekeeper reviews the submission to ensure that it is complete and relevant and then forwards the submission to the ERP for thorough review and vetting. Once the lesson learned/best practice is approved, the ERP populates the database. The database is searchable by keywords and other functions such as filtering by county, division, project type, and so forth. CLEAR aims to create feedback loops within the department for all project phases, disciplines, modes of transportation, units, offices, and locations. This program is also expected to bring organizational changes to improve processes within the NCDOT. The NCDOT will greatly benefit from this knowledge repository, thereby aiding in achieving better project performance.
Communicate Lessons, Exchange Advice, Record (CLEAR): “What you need to know” ( 11 ).
Literature Review
Previous Work Regarding Lessons Learned at Other Organizations and Departments of Transportation
Numerous organizations have benefited from lessons-learned tools and programs to tap past experiences and make informed decisions. For example, the National Aeronautics and Space Administration has both public and internal lessons-learned systems. The United States Army’s Construction Engineering Research Laboratories uses DrChecks, which uses client-server architecture for online comment sharing among various parties for discussions that pertain to design documents. In addition, “the CROSS-US (Confidential Reporting on Structural Safety – United States) is a confidential reporting system to capture and share lessons learned from structural safety issues which might not otherwise have had public recognition, with the aim of preventing future failures” ( 12 ). The CROSS-US database is open access to the public and includes a search feature that is based on a construction taxonomy that has not been shared hitherto as public knowledge.
With regard to transportation organizations, the Indiana Department of Transportation (INDOT) was an early adopter of a lessons-learned database. McCullouch and Patty ( 13 ), researchers at Purdue University, conducted a series of interviews with INDOT personnel to improve coordination between the design and construction teams with the ultimate aim of achieving a better constructability review program. The team developed a Windows-based constructability lessons-learned software application using Visual Basic. Folio Views was used to store, index, and retrieve the constructability lessons learned in text form ( 13 ).
The Kentucky Transportation Center at the University of Kentucky conducted a similar research activity to develop a web-based lessons-learned database that could accept files both in text and image formats. Goodrum et al. ( 14 ) surveyed resident engineers, contractors, and consultants to obtain an initial understanding of their vision of a perfect lessons-learned database. Each user associated with the database was classified into three categories, that is, end user, gatekeeper, or administrator, with each of their functions clearly stated. The database was structured in two parts: one for users to enter new lessons learned, and the other for storing and retrieving cleaned-up lessons. Microsoft Access was implemented for data storage and retrieval and Microsoft FrontPage was used to accept lessons-learned input from users. The database also had provisions to search for specific terms within the database fields to yield specific results that would be helpful for design teams during a constructability review. However, this effort did not fulfill its intended purpose as the lessons learned database became defunct once its 2,000-row limit was reached. The main failure to ensure proper functioning of this database was caused by not mitigating the risk of running out of space beyond the permissible 2,000-row limit.
Retaining existing personnel and retraining new personnel has been a challenging issue that most U.S. Departments of Transportation (DOT) have been facing on a regular basis for quite a few decades. Between 1992 and 2003, the Virginia Department of Transport (VDOT) witnessed the departure of over 30% of its personnel, many of them with more than 30 years of experience, which led to increased departure of experiential knowledge ( 15 ). As a result, VDOT’s Knowledge Management (KM) division was created in 2003 to help retain knowledge and maintain the work culture despite the organizational changes in the complex environments. This division was also in charge of facilitating a smooth transition for VDOT personnel in finding the right balance between bureaucratic ways of doing things (what had worked well in the past) and a more business-oriented approach (focused on time and monetary savings). In April 2007, VDOT implemented the agency-wide construction lessons learned initiative to capture lessons from previous experiences to bring about changes in processes, procedures, and contract language. To do this, VDOT commissioned the Communities of Practice (CoPs) as a primary tool to encourage knowledge sharing and the creation of networks between individuals and across silos throughout the agency. A CoP is a group of people “who share a concern, set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” ( 16 ). VDOT’s CoPs were people-oriented with teams of at most 15 people from varied backgrounds such as work expertise or geographic locations. These teams convened on a schedule convenient for all team members, including the KM division team member, to share stories about successes and failures, and identify opportunities to improve workflow processes. The meeting notes, decisions, and actions were documented in a team site for common documents within the agency’s portal. The CoPs acted as a connection between personnel and upper management to articulate information to be used for reviewing and amending policy. Although this effort is no longer active, it does provide useful insights into developing strategies for effective knowledge retention within the organization.
More recently, other transportation organizations and DOTs also have developed knowledge repositories in the form of databases. For example, the Kentucky Transportation Cabinet funded a study to develop a constructability lessons-learned tool for use during the design phase to improve project outcomes ( 17 ). The Federal Highway Administration (FHWA) has compiled a list of lessons learned from various transportation-related projects from departments of transportation throughout the United States ( 18 ). This database is open access for the public and contains lessons learned in text format from various projects as well as project phases. The U.S. DOT has a lessons-learned database for its Intelligent Transportation Systems (ITS) called ITS Lessons Learned Knowledge Resource (LLKR) ( 19 ). The LLKR database captures knowledge from users who are involved in planning, deployment, operations, maintenance, and evaluation of ITS throughout the United States. This database is heavily reliant on gathering information from other related databases, such as ITS case studies, the ITS Electronic Document Library, the Transportation Research Board (TRB) Transportation Research Information Services, international transportation literature databases, and TRB conference proceedings. The LLKR is open access to the public and can be searched for lessons learned using keywords or by filtering based on location, categories, or both.
The Colorado Department of Transport (CDOT) created a program called Lean Ideas Everyday to encourage users to upload innovative suggestions and adopted practices to improve existing methodologies by clicking on “I fixed It!” and “I Suggest!” respectively. Although this database accepts information entry by authorized personnel only, the public has open access to Idea Cards that provide details about a few select innovations and how their use has helped the CDOT to improve its workflow processes. The Lean Ideas Everyday database was developed primarily using Google products, such as Google sheets and slides ( 20 ). Fong and Yip ( 21 ) assessed the level of readiness of construction professionals in Hong Kong to implement lessons-learned systems within their organizations. One of their key research findings was that construction personnel preferred not to record lessons learned while the project was ongoing, which could lead to the loss of important knowledge.
Lessons Learned from Previous Lessons-Learned Database Designs
Goodrum et al. ( 14 ) devised a list of suggestions for successfully designing and implementing a lessons-learned database, summarized as follows:
Lessons learned systems require a champion who is knowledgeable about the organizational work processes, with exceptional people and communication skills.
Lessons learned systems should be simplistic, easy to maintain, and not be used to criticize mistakes.
The success of lessons-learned systems depends a lot on the end-users and their willingness to embrace the process.
Methodology
A preliminary study was performed by the NCDOT’s VMO in 2014 to collect lessons learned from various NCDOT units and staff. The study identified the need for a formal medium to communicate information about projects within the NCDOT. The results indicated the lack of a medium to store knowledge that was gained on project sites and led to the conception of the PCAP in 2017, which in turn led to NCSU researchers being contracted to help develop a new lessons-learned database for the NCDOT. During the ongoing research efforts, PCAP was rechristened as CLEAR in 2018 as it entailed capturing knowledge throughout the NCDOT and not just the post-construction personnel as initially envisioned. Thus, the scope of the CLEAR program is to accept submissions throughout the organization related to any work within the organization; it includes but is not limited to new telecommuting practices, program enhancements, and operational improvements and, moreover, is not just limited to ongoing projects as initially envisioned in PCAP. The transition from PCAP to CLEAR did not warrant any special measures with respect to the database designs and the Identify, Define, Develop, Optimize, and Verify (IDDOV) model since it involved minor modifications in the database fields specific to the three new forms with no major changes necessary. The NCSU research team consulted the literature that focused on earlier lessons-learned databases to ensure that CLEAR was user friendly to ensure its longevity. The research team took precautions to avoid the snags that had been experienced in earlier research efforts. To this end, the NCSU researchers employed a DFSS approach to design and create the new and robust lessons-learned database. The five stages of the DFSS methodology, that is, IDDOV, form the basis of the final research outcomes.
Design for Six Sigma (DFSS)
The DFSS methodology is a systematic and disciplined problem prevention approach that is widely used to design new robust engineering systems. In addition to the IDDOV model, a few other models that use DFSS for generic technology development are I 2 DOV (invent, innovate, develop, optimize, verify), CDOV (concept, design, optimize, verify), and DMADV (define, measure, analyze, design and verify) ( 22 ). Although these models have their own benefits and drawbacks, the NCSU research team decided to use the concepts of the closed-loop IDDOV model that starts and ends with the customers. For this project, customers would refer to NCDOT end-users. Additionally, the IDDOV model fits well within the scope of the software development lifecycle to minimize defects ( 23 ), though this is out of scope for this paper. The research team thus finalized on using the concepts of IDDOV model, which appeared to be the most suitable of the various DFSS options, to design and build an error-free robust lessons learned database ( 24 ). Appendix A describes the five steps of the IDDOV model as applied to the CLEAR database.
Findings
The following section provides insights into the findings obtained using the IDDOV approach mentioned above. Each subsection within this heading pertains to how the IDDOV stages tie-in with the CLEAR program and the findings at each of these five stages.
Phase 1: Identifying Trends and Database Fields
The NCDOT VMO provided contact information for potential interview respondents to the NCSU research team. The research team then sent interview requests to 66 potential respondents at the NCDOT. During this phase of information gathering, 32 interviews were conducted with 46 personnel who had a total of 813 years of work experience. Figure 2 shows the respondent participation by the project phase they were then associated with. The interviews were conducted both in person and by phone with personnel from multiple project phases, such as preconstruction, design (e.g., safety and structures), construction, and maintenance. NCDOT personnel in areas of materials, design-build, and facilities management were also interviewed. This afforded the research team opportunities to explore diverse perspectives from within the NCDOT. Overall, the interview process helped the research team to obtain in-depth feedback about extant processes of information exchange and to determine the fields to be included in the new lessons-learned database.
Interview details by project phase.
The NCSU research team carefully documented the inputs from these interviews to not miss any important piece of information. For each interview, the research team prepared at least two sets of notes and entered the information into an MS Word file for each interview, whether obtained in person or by phone. Following each interview, the notes from all the research team members were combined to prepare a comprehensive list of responses. By the end of this phase, the research team had gained a good sense of current organizational practices to communicate lessons learned within the NCDOT and determine the proper direction to proceed with designing the lessons-learned database fields. The database design considerations can be found in Appendix B.
Phase 2: Defining the Database Fields
The initial version of the developed database had a single lesson-learned/best-practice form that was based on inputs gathered during the first phase and was divided into three segments. The first segment recorded basic user information such as name and division, and office information including email and telephone number. This information will not be displayed while displaying appropriate search results, but is there for the gatekeeper to be able to contact the end-user in case any missing/additional information is needed. The second segment input information about the issue and its solutions. Users could include attachments such as pictures, PDFs, revised contract language, and other relevant files to make it contextually easy to understand. The third segment recorded project information that pertained to the lesson learned or best practice. A few fields in this segment were intended to be populated from other internally linked databases to expedite data entry and encourage participation. However, based on VMO’s input and other studies such as the Risk Assessment study (described in a later section), this initial common form for lessons learned/best practices formed the basis for the now existing three forms for lessons learned, best practices, and solutions needed.
Principal Stakeholders
The workflow that follows shows where in the process each principal stakeholder participates. The principal stakeholders involved with the CLEAR database are as follows:
End-users: End-users are NCDOT personnel who are responsible for entering useful lessons learned and best practices based on knowledge gained at project sites. They are also responsible for searching for relevant knowledge to understand previous circumstances to avoid the repetition of issues. These end-users must have an NCDOT email; this portal is not currently accessible for those outside NCDOT but may be part of a future iteration.
Gatekeeper: The VMO at the NCDOT serves as the gatekeeper for CLEAR and is responsible for checking for the completeness of the submissions, forwarding the submissions to taskforce members, and subsequently approving the submissions after receiving the go-ahead from taskforce members. The gatekeeper is responsible for tracking the submission through acceptance into the portal and identifying next steps for implementation. The gatekeeper identifies key players, helps establish goals, and guides the outputs toward appropriate integration into the Department. The gatekeeper can identify if Lean Six Sigma is needed on a particular submission, a risk assessment, or help bringing similar efforts together to eliminate redundancies and improve communication on similar projects.
Task Force: The task force is composed of experts within various disciplines who are responsible for ensuring the quality of the content that is uploaded to the database. Based on its review of each submission, the task force will inform the gatekeeper of its decision to accept or reject the submission. It is worth noting that the task force consists of experts who cover all disciplines of work while the gatekeeper selects the ERP from this pool of experts based on their relevance and expertise to the submission.
Innovation Coordinators: These coordinators are highly motivated to encourage their units or offices to participate in the CLEAR program, thereby supporting innovation. There is an Innovation Coordinator in each of the Highway Divisions, Units within the Division of Highways, and Modes of Transportation. The coordinators have a shared workspace to exchange ideas on how to best engage the members of their units and give feedback to the Gatekeeper on what is needed to improve the program for their units.
Technical Advisory Group (TAG): The TAG is composed of taskforce/ERP members who focus on specific topics or areas and collectively review lesson-learned/best-practice submissions through the NCDOT and establish goals for solutions. The TAGs are created as needed to tackle these topics with a goal of implementing submissions from the CLEAR program. They are multi-discipline and multi-modal as needed to enhance the communication and cross collaboration. They can also solicit requests for submissions from end-users as needed. The TAGs are only convened when necessary and there is no standing committee. This keeps the TAGs fresh, active, and committed to a goal. The TAGs also work to make sure that the new ideas are being implemented in conjunction with ongoing improvement efforts and to eliminate the likelihood of redundancy or completing initiatives. This is currently being used at NCDOT.
Technical Coordination Committee (TCC): The TCC is composed of upper management, multi-disciplinary and multi-modal representatives, and external partners who provide guidance and review from a high-level/industry perspective. The TCC is needed if external partnerships are needed to implement an idea or to pilot an initiative. The TCC would also be tapped to search for external funding if needed. This is an iteration of the STIC committee familiar to most DOTs.
CLEAR Workflow
Figure 3 shows the detailed technical view of how a submission is completed in the workflow by the submitter (end-user), gatekeeper, and ERP (Task Force). The star icons show the stages at which an email is sent to the submitter. Figure 4 presents the steps followed in the CLEAR system starting with a user-initiated submission and ultimately leading to an implemented innovation. The star icons in Figure 4 indicate the stakeholders involved in the process of submitting and approving entries in CLEAR. Once a NCDOT employee submits an entry, the gatekeeper checks for the completeness of the data and forwards the submission to the appropriate ERP/taskforce member. The taskforce member then decides to accept, reject, or solicit additional relevant information on the entry. The stakeholders are kept informed at each pertinent stage by email so that they can keep track of the submission. One of the end-goals of the CLEAR program is to encourage organizational innovation among all units and divisions. If an idea could not be implemented at first-line expert level, but requires continuous communication and review efforts, a TAG would be formed to work toward the goal of implementation. Should external partnership or support be needed, the TCC would be engaged to assist in the implementation effort at this level.
Communicate Lessons, Exchange Advice, Record (CLEAR) workflow from the inception till the end.
Communicate Lessons, Exchange Advice, Record (CLEAR) steps for a lesson learned/best practice.
Phase 3: Database Development and Respondent Feedback
In the initial effort, the NCSU research team gathered 42 lessons learned and best practices from 19 end-users and from two pilot projects. The research team contacted NCDOT personnel who had participated during the data gathering phase to build the lessons learned repository. The 19 respondents provided their lessons learned/best practices over the telephone. In addition to gathering these telephone data, the research team also visited two pilot projects and gathered lessons learned and best practices by observing the project sites and talking with site personnel. The two pilot projects were the East End Connector project in Durham, NC and the Pitt County’s Resident Engineers office in Division 2. The effort to solicit as many high-quality lessons learned as possible from end-users, such as site engineers, inspectors, resident engineers, and other project personnel, is ongoing enabling users to enter information in the CLEAR database within the Connect SharePoint portal.
Phase 4: Optimize for Best Results
CLEAR is envisioned to be “a program to support internal communication, knowledge sharing, creativity, and innovation” ( 11 ). The success of this program hinges on the end-user’s willingness to embrace and enter useful knowledge into the database in the form of lessons learned, best practices, or solutions needed. To achieve this goal, the NCSU research team devised a strategy to promote the use of the CLEAR program among NCDOT personnel. This strategy was intended to develop the best possible ways to encourage participation by incorporating user preferences and possible incentives. With this strategy in mind, the NCSU team developed a survey with a list of questions that asked them about their preferences on the CLEAR program. The survey results indicated that the end-users were mainly looking for a robust yet simple and user-friendly database.
Phase 5: Verify Database Based on User Feedback
Risk Assessment Study
Once the North Carolina DIT had set up the CLEAR database in the SharePoint portal, the database needed to be validated by the end-users. For this task, the VMO at the NCDOT conducted a one-day risk assessment study of the CLEAR program in November 2019. The aim of this study was to understand the possible risks that could arise out of the program and possible mitigation measures. Figure 5 provides a breakdown of the severity of the risks. The study had 21 participants who identified 65 risks, of which 51 were deemed to require mitigation.
Risk classifications identified from risk assessment study.
All the identified risks were categorized based on topics such as search, collection, integration, sharing, and recognition. The VMO in consultation with the study participants devised proposed mitigation strategies for these risks and subsequently delegated them to the appropriate authorities to implement proper mitigation measures. More information on this study and the identified risks can be found on the NCDOT website ( 10 ). The majority of the risks were related to the end-users. We were able to break out the risks into two main categories: the willingness of end-users to submit information and use the portal; and the accessibility and ease with which the portal could be used. By making the portal easier to use, we already started tackling the willingness concern.
Based on the risks identified from the risk assessment study and work done in other DOTs such as the CDOT, the NCSU research team developed three forms to replace the existing single form in CLEAR albeit doing their best to preserve the database fields already created. These three forms were used to input information about: (1) lessons learned; (2) best practices/ideas; and (3) solutions (or control measures) that are needed to address obstacles/challenges faced at project sites. Figure 6 shows the latest snapshot of the landing page where the user can select an appropriate form to share their knowledge. The research team also developed a set of standard operating procedures (SOPs) for end-users and the ERP to provide information about how to use the appropriate functionality. For end-users, the SOPs explain how to enter information in the Lessons Learned, Best Practices, and Solution Needed forms and to search for content in the database. For the ERP, these SOPs detail the best practices to review and provide suitable disposition for the entries. In addition to these forms, the research team also developed “how to” videos that describe the steps to enter information in the CLEAR database. These videos are intended to act primarily as a training resource for first-time users of this database although users can also use them as reference material when using CLEAR. The NCSU research team, in consultation with the VMO also prepared a list of definitions and frequently asked questions (FAQ) which are currently uploaded in the CLEAR website. All stakeholders related to the CLEAR program can make use of these documents to familiarize themselves with relevant terminologies and obtain information from the FAQs. The North Carolina DIT, who is responsible for the technical creation, development, and management of the portal, performed updates to the forms, improved verbiage on the workflow, and is working on an improved search feature for keywords. This will make the site more user friendly to encourage participation.
Communicate Lessons, Exchange Advice, Record (CLEAR) landing page on Connect SharePoint portal.
For enhanced user participation in the CLEAR program, a few strategies were developed for various stages of implementation. The first is a systematic roll-out that includes a recorded and live training portion. This targeted training, allows the VMO to explain how the program would fit into the everyday tasks of the specific group. The more recent training gives a live demonstration of how the submission is done, how it is reviewed, and where it goes to show the end-user how quickly a submission can be made and how the identified experts can provide a review. In addition, the VMO team is working with the Communications Department to share some “wins” or outputs with the Department showing how the end-user’s submission is being implemented or integrated into the Department. The last mitigation measure that is being worked on is a department-wide “Challenge” that would solicit participation from all NCDOT personnel to share their response to a Challenge topic or problem. This idea of a Challenge is used by other DOTs and has been very successful in California ( 25 ). To gauge the effectiveness of a Challenge, the VMO hosted an Innovation Coordinator-only challenge that allowed the team to pilot the Challenge initiative on a small group. The pilot was successful and captured nearly 40 submissions from the Innovation Coordinators who participated. The communication efforts, the topic question requests, the implementation of the ideas, and the recognition were all practiced and adjusted as a part of this pilot effort. The VMO team has been able to take lessons garnered from the pilot initiative and plans to conduct a department-wide Challenge in the spring of 2021.
Lessons Learned to Lessons Remembered
In line with the organizational goal of the CLEAR program to institutionalize knowledge, the research team prepared a sequence of steps to make the lessons learned easy for users to remember. During the lessons-learned data gathering phase for the CLEAR database, the research team realized significant project concerns that relate to utilities. For example, utilities not being moved within the agreed timeframe, unknown utilities being discovered during construction, and other unexpected utilities conflicts led to claims and supplementary agreements which ultimately increased project costs and schedules. The research team performed careful analysis of NCDOT utilities claims data for 1996 through 2018 extracted through HiCAMS. The NCSU team also carried out a literature review to understand how other state DOTs mitigate potential utilities issues on their projects. A major takeaway from this data analysis is that, for the projects affected by utilities claims, project costs increased by about 2.4%, with schedule delays increasing by 70 days on average. Therefore, the research team developed possible interventions and key mitigation strategies for the utilities issue based on the existing literature, personnel responses from interviews, and HiCAMS claims data provided by the VMO. The NCSU research team provided this information to the NCDOT utilities group for further action. This sharing of knowledge may lead to revising contract language that pertains to utilities providers and specifications to detect underlying utilities by ensuring that proper subsurface investigations are performed on all projects, thus turning the lessons learned into lessons remembered.
Training Materials
The NCSU research team developed training videos using the video-making software VideoScribe. Appendix C shows sample screenshots from a training video on how users can enter lessons into the CLEAR database. These training videos were created for all three forms and describe how an end-user can enter information in the appropriate forms. The research team also prepared training materials as “kiosk” forms that are designed for maintenance personnel who do not have access to the Connect NCDOT portal to enter information. All the training materials, including the videos, have been uploaded to the CLEAR portal so that end-users can become familiar with ways to share information using the CLEAR program.
Online Training
Currently, the VMO is organizing online training sessions on the CLEAR program for personnel from various divisions/units. This training is a part of the systematic roll-out efforts to bring units on board one at a time or in small groups; by focusing the roll out on a smaller scale, the team has been able to pilot the training materials and training methods. The purpose of these training sessions is to introduce CLEAR and to explain CLEAR’s potential benefits to both the participants and the NCDOT as a whole. The first online training session was held in May 2020 using Microsoft Teams for about 30 participants within the Hydraulics and Aviation Divisions. The training was an event lasting nearly two hours with 1.5 PDHs. It reviewed the history of the development of the program, the purpose of the program, the programmatic efforts, and the technical how to. The event gave a live look of the online portal and identified where the supplemental training materials could be found. There was a break before the next training event to measure the effects of the training and allow for the VMO to conduct follow-ups with the offices. In the second training session held in July 2020, about 100 participants from the Photogrammetry and Location & Surveys units participated. This was the first training event for the entire Technical Services Unit comprising 10 different design units. Each training event was done for two groups at a time. This was to alleviate the number of training events conducted by the VMO and was to show that cross-discipline communication is a key function of the CLEAR program. Following these training events, the VMO decided to take a new approach when they did the next roll-out to the Central Construction Unit (CCU) in October 2020. This included a shortened pre-recorded training event on the history and background of the development, an explanation of the programmatic efforts, and a brief view of the portal. The CCU personnel were given two weeks to watch the pre-recorded training. Two lessons-learned workshops were conducted by the VMO to follow up. At these workshops, brainstorming activities took place to encourage the sharing of lessons learned and best practice in addition to a live demonstration of how to submit these into the portal. The VMO team also demonstrated how the submission moved through the review process and became an accepted item. The VMO team also showed how to identify if the submitter wished to be part of the greater implementation plan and what it meant to work on the implementation of the idea. This format will be used in future roll-outs because it allowed for better hands-on participation in the program. This meets the objectives of giving personnel background information, a thorough understanding of the vision, and makes it very clear what the participation looks like for everyone. The goal of the live demo is to show how quick and straightforward a process it is to make a submission. This training format will be altered slightly for personnel in the field. In this instance, a video of how to enter the submission into the kiosk will be included by soliciting support from one of the Innovation Coordinators who work in the field.
Throughout 2021, the systematic roll out will continue to on-board the entire Department. As new employees come on board during 2020 and 2021, units that have already gone through the roll-out will rely on Innovation Coordinators to share the CLEAR program with new employees. All training material is recorded and can be shared at any time. Once all units have gone through the roll-out, the VMO will work with the New Employee Orientation process to determine if there is a way to engage new hires at that time, if it is best to continue to use the Innovation Coordinators with this effort, or if annual refresher training will be provided by the VMO team.
Although these sessions were planned initially for a face-to-face gathering, because of the COVID-19 situation, the training sessions were converted in an online format. The purpose of the training session was to introduce CLEAR and to explain CLEAR’s potential benefits to both the participants and the NCDOT as a whole. The research team worked closely with the VMO in preparing the training materials and providing support to obtain feedback about the presentation materials. A feedback form provided at the end of the presentation allowed the participants to share their opinions about the CLEAR program and the efficacy of the presentation.
Data analysis of the feedback survey revealed the following information:
32 valid responses were received, of which 28 were complete in all aspects.
The total NCDOT work experience of the users was 516 years with a mean of 16.65 years.
The users ranked the order of usage preference for the CLEAR forms as Lessons Learned, Best Practices, and Solutions Needed, in that order, as can be seen in Figure 7.
A majority of the respondents strongly agreed that the training met their needs and that they would be willing to contribute to the CLEAR program.
Ranking of three submission forms based on user feedback.
Conclusion
Lessons learned can be an effective mechanism to document and retrieve wisdom gained from previous experiences and apply this knowledge for better outcomes. The CLEAR lessons-learned/best-practices database within the NCDOT will facilitate improved inter- and intra-departmental coordination between personnel. The overall aims of this database are to achieve superior design performance and thus reduce the frequency and impacts of change orders, enhance cooperation, and ultimately accomplish improved operational performance. Two important considerations here are that: 1) project teams are dynamic and seldom repeat themselves in different projects; and 2) the aging workforce will retire before their knowledge can be documented. In either case, a significant amount of wisdom would be lost if this information were not documented in a proper lessons-learned/best-practices database. The CLEAR program will provide scope for the next generation to implement these lessons learned/best practices to realize desired project goals.
This research effort resulted in an internal-only web-based database that is housed within the Connect NCDOT SharePoint portal and contains information about lessons learned and best practices from ongoing or previous projects. Authorized personnel now have the ability to input data as well as search for information through this web-based database. The CLEAR training materials, including SOPs and training videos, will assist NCDOT personnel to contribute effectively to this program. The CLEAR program is expected to encourage end-users to share knowledge gained on projects and search for relevant lessons using the search function. Project teams across divisions, units, and departments at the NCDOT will greatly benefit from this rich and robust knowledge database.
Future Scope
The ultimate success of the CLEAR program will depend on the extent to which end-users are proactive about entering and searching for relevant lessons to be applied to their projects. A data dashboard is envisioned to ensure timely intervention by the experts (e.g., upper management personnel, ERP, TAG) to institutionalize knowledge based on the lessons learned and best practices entered. The dashboard will display the most frequently recurring words/phrases based on text-mined data obtained from the submissions. A word cloud generated from the text will be an initial exploratory way to view the most frequently occurring words. Figure 8 presents a sample word cloud embedded within the map of North Carolina that has been generated from the lessons learned entered to date. It is however to be noted that the words in no way denote or are an indication of its geographical representation on the map. That is, placement of any word on the word cloud is completely randomly generated. The NCSU research team is working on other functionalities such as visualizing bi-grams and tri-grams (phrases with two words and three words, respectively). Based on preliminary discussions with the VMO at the NCDOT, Microsoft PowerBI is anticipated to be used as the data visualizing platform since NCDOT has access to Microsoft-related products and thus the program would be easy to integrate within CLEAR’s workflow processes. The research team is also considering Tableau and Smartsheets as other data visualization options, although the organization-wide implementation of these tools must be explored carefully before a final decision can be made.
Word cloud generated from text entered for lessons learned.
The NCSU research team will also use machine learning techniques such as natural language processing to automate the identification of topics from entered lessons learned/best practices. To facilitate effective communication among units, the team will identify the top lessons learned/best practices that relate to the daily workflow process of future projects and encourage their perusal. This process will help users to review past knowledge and make the changes necessary to ensure that the knowledge gleaned from past experience is applied and that mistakes are not repeated. At this point, the NCSU research team is trying to conceptualize ways to use state-of-the-art machine learning techniques effectively to yield best results for the NCDOT. In the long run, the CLEAR program will integrate efficiently with the NCDOT’s work culture through accountability and innovation on the part of NCDOT personnel.
Supplemental Material
sj-docx-1-trr-10.1177_0361198121995195 – Supplemental material for Development of North Carolina Department of Transportation’s CLEAR Program for Enhanced Project Performance
Supplemental material, sj-docx-1-trr-10.1177_0361198121995195 for Development of North Carolina Department of Transportation’s CLEAR Program for Enhanced Project Performance by Clare E. Fullerton, Alyson W. Tamer, Siddharth Banerjee, Abdullah F. Alsharef and Edward J. Jaselskis in Transportation Research Record
Footnotes
Acknowledgements
The research team acknowledges the NCDOT for supporting and funding this project. We extend our thanks to the NCDOT personnel who participated in this research project for their time and hospitality. Without the help of all these individuals, the project could not have been completed in such a successful manner.
Author Contributions
The authors confirm contribution to the paper as follows: study conception and design: Clare E. Fullerton, Edward J. Jaselskis, Alyson W. Tamer; data collection: Edward J. Jaselskis, Siddharth Banerjee, Abdullah F. Alsharef; analysis and interpretation of results: Siddharth Banerjee, Edward J. Jaselskis, Abdullah F. Alsharef; draft manuscript preparation: Siddharth Banerjee, Clare E. Fullerton. All authors reviewed the results and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the North Carolina Department of Transportation (grant number 2019-15).
The opinions expressed in this paper represent those of the authors and not necessarily those of the NCDOT.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
