Abstract
Abstract
Some small businesses and entrepreneurs find it challenging to interact with the U.S. National Aeronautics and Space Agency (NASA). In 2016, the Small Business Innovation Research and Small Business Technology Transfer (SBIR/STTR) programs adapted their goals and objectives to be more customer focused and collaborative to make SBIR/STTR a go-to program for U.S. innovators. SBIR/STTR is NASA's largest small business program that provides approximately$190M in awards annually for research, development, and demonstration of innovative technologies that fulfil NASA needs and have significant potential for successful commercialization. From 2016 to 2018, NASA SBIR/STTR undertook a series of initiatives to become more business friendly by eliciting and incorporating multiple stakeholder voices and values in determining program direction, providing regular touchpoints in annual program processes for engagement between industry and NASA, and improving the user experience for small businesses and NASA personnel interacting with the program's information technology systems. This article provides insights on the design and results of these efforts in making the NASA SBIR/STTR program more business friendly. In addition, the article presents key lessons learned and takeaways for other government agencies to consider when undertaking transformational change within their own organizations.
Introduction
NASA's Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs (âthe programâ) leverage the nation's innovative small business community to aid research and development (R&D) in support of National Aeronautics and Space Agency's (NASA's) mission in human exploration, science, space technology, and aeronautics. These programs provide small business concerns (SBCs) with an opportunity to develop technology for NASA and to commercialize that technology to spur economic growth.
This 36-year-old program has historically focused on meeting Congressional and Small Business Administration (SBA) requirements for program execution through complex program processes that have evolved and grown over that time. The problem is that by implementing complex processes for 36 years, the program became too focused on process implementation as a measure of success, unwittingly distancing itself from focusing on program outcomes, including impact on small businesses when interacting with the program.
Thus, in 2016, the program began to intentionally consider what it meant to fundamentally change its focus: from a process-focused program to one focused on overall program impact and customer experience. This would require the program to define both its desired impact and customers much more intentionally. A relentless focus was applied by program leadership to ensure those 2 driving forces become the priority for the program through both the design of new initiatives and execution of recurring program operations. The objective of this article is to present a case study of how the NASA SBIR/STTR program pursued this transformation, primarily from 2016 to 2018, and to share the lessons and findings from that pursuit for future study.
This transformation required the program to reflect seriously on whether the present program structure and methods were maximizing the impact of the program. Thus, starting in 2016, program leadership prioritized engagement and collaboration with all customers and stakeholders as core values of the program, leading to extensive engagement with customers to seek input on desired program impact, priorities, and direction from 2 identified primary customers: problem seekers and problem solvers. The problem âseekersâ are the employees of NASA programs and projects with R&D needs identified by the NASA Mission Directorates (MDs), centers, topic managers (TMs), and subtopic managers (STMs) that participate in the program. The problem âsolversâ are the small businesses and entrepreneurs with capabilities and ideas that could solve problems through an SBIR or STTR award. The program launched several new initiatives to capture insights from these primary customers, including the following:
Conducting a collaborative strategic planning exercise internal to NASA to understand the strengths, weaknesses, opportunities, and threats within the program from the vantage point of the âseekersâ when articulating the desired program impact, goals, and objectives. Partnering with 18F, the U.S. General Services Administration's (GSA) digital services agency, to use human-centered design (HCD) to develop a baseline service blueprint and customer experience roadmap from the SBCs' âsolverâ perspective to understand the roadblocks and pain points when engaging with the program.
Collectively, these initiatives informed the revision of the program goals and objectives in 2016 to be more business friendly and collaborative to make SBIR/STTR a go-to program for U.S. innovators and increase the program's overall impact.
As an indication of the lack of collaboration mechanisms between seekers and solvers, before 2016, the program did not have basic means for companies to get clarity on NASA's needs with respect to the subtopics published in the annual solicitation. Furthermore, the program traditionally did not engage with SBCs uniformly in understanding their technology solutions to make the SBIR/STTR solicitation relevant for both NASA and small businesses. To address these problems, the program initiated several new mechanisms for engaging with SBCs and pivoting the program to improve understanding of the small business customer and to incorporate those insights into designing program priorities, including the annual solicitation:
Developing an annual request for information (RFI) process to provide small businesses a routine and predictable opportunity for input into programmatic barriers, previous year's subtopics, and a mechanism to submit ideas for new subtopics.
Establishing industry day as an annual small business-NASA event to provide a forum for SBCs and NASA subject matter experts (SMEs) to convene, have input into the annual program solicitation, and increase the likelihood for commercialization of innovations and successful infusion of technologies within NASA and across the U.S. Federal Government.
Finally, in 2015, with the rising costs of technology ownership of a legacy information technology (IT) system, NASA also prioritized the modernization of its decades-old Electronic Handbook (EHB), an IT system that is used to manage the end-to-end proposal solicitation and award process. The EHB is the primary program interaction point for thousands of customers. With the focus on the customer introduced in 2016, this initiative would come to prioritize improving user experience (UX) and reducing pain points for NASA internal end users and external small business users through both technology and process improvements.
These transformational efforts would become a classic example of rebuilding an airplane while continuing to fly it, as it mirrors the complexity and immense challenge of executing existing operations that are fundamental to an organization while experimenting and improving on them at the same time.
This article discusses how the program embarked on its business transformation journey toward becoming more impactful and business friendly by describing a select subset of initiatives undertaken by the program between 2016 and 2018. The New Initiatives to Incorporate Multiple Stakeholder Voices and Values section through the New Initiatives to Improve Business Processes and User Experience Through Technology Modernization section describe the goals and implementation approach for each effort as well as the initial results and lessons learned, which are also summarized at the end of each section in table format. The Discussion and Conclusion section reflects on overall lessons learned across the 3 initiative areas and presents 3 key takeaways for government agencies undertaking large-scale transformation efforts.
New Initiatives to Incorporate Multiple Stakeholder Voices and Values
Since its inception in 1984, the SBIR/STTR program has made significant contributions to NASA's R&D efforts and spinoff technologies 1 for infusion and commercialization. At the same time, the program has sometimes suffered from being seen by internal NASA stakeholders as a âmandateâ or âtax,â given the requirement to set aside budget to spend on small businesses, instead of a value-added seed fund to empower SBCs and a critical onramp for innovative research. These differing views about the value and potential impact of the program from inside NASA sometimes caused the program to pivot from one focus to another in a reactive manner, having no guiding focus to evolve the program thus hindering its ability to have periodic internal and external stakeholder engagement and gain traction from business improvement initiatives.
Furthermore, the program did not have a present or comprehensive understanding of the perspectives and pain points of its small business customer. Instead, the program would hear from SBCs only when there was a problem to be addressed, making it difficult to understand the full value proposition for the program from the SBCs' perspective. As best described by the principal authority on leading successful organizational transformations, Kotter, âwithout an appropriate vision, a transformation effort can easily dissolve into a list of confusing, incompatible, and time-consuming projects that go in the wrong direction or nowhere at all.â 2
The program thus embarked on 2 systematic customer engagement and discovery efforts: one internally focused effort through the development of a strategic framework (engaging primarily the âseekersâ), and another externally focused effort through the development of a customer emotional journey map and service blueprint (engaging primarily the âsolversâ). Collectively, these initiatives re-energized the role and the perception of the NASA SBIR/STTR program as a more accessible seed fund for SBCs and a desirable onramp for MDs to further innovative research.
Development of a Program Strategic Framework and Performance Scorecard
Goal and purpose
In 2016, the program embarked on an effort to develop a new strategic framework to provide an actionable vision and mission, goals, and objectives for a program beholden to many, and often competing, customers and internal stakeholder interests. Historically, SBA's 4 articulated goals for all government-wide SBIR/STTR programs were used to describe the overall goals of NASA's SBIR/STTR program. However, this was limiting from a strategy alignment vantage point for a couple reasons: (1) the SBA articulated goals could actually be in competition with one another, requiring agencies to make choices about which goals were prioritized above others since all could not be completely achieved and (2) the generalness of the SBA goals made them open to interpretation, strengthening the hold of internal competing interests since there was not a common language and understanding about the NASA SBIR/STTR program's purpose and future direction.
Thus, it was important for the program to develop a consistent, relatable view of its business value as a wholeânot only within SBA's requirements but also considering NASA's strategic role for the nation both in developing and spinning out technologies and in supporting the development of commerce. What proved to be a catalyst for moving forward on this path was the introduction of a program leadership team that found it important to align team members around a common vision, the development of that vision collectively with internal stakeholders, and a means to continuously measure how successfully the program was operationalizing the vision and achieving its stated goals.
Approach
The program's leadership team embraced the need for establishing a strategic framework in 2016 and understood that the development of this strategy required complete upward and downward alignmentâgetting internal stakeholders (both within the program and with the executive leadership responsible for the program at the Space Technology Mission Directorate [STMD] at NASA headquarters) to buy into the vision.
A working group comprised thought leadersâsome new to the program to add new perspectives to the conversationâhand picked by program leadership to brainstorm on program goals and outcomes. At the same time, the program leadership team started introducing and providing early thinking on the need to orient the program to consider its stakeholder base and customers more fully. The team developed a framework structure to meet both the intent of the SBIR/STTR program as defined by the U.S. SBA policy directive and also to align directly with STMD's strategic thrusts 3 and NASA's overall annual performance indicators. 4 The new strategic framework included the vision, mission, and goal statements (Fig. 1) supported by actionable implementation objectives.
NASA SBIR/STTR program strategic framework. NASA, National Aeronautics and Space Agency; SBIR/STTR, Small Business Innovation Research and Small Business Technology Transfer.
Results
To obtain feedback on this program strategic framework, a âwikiâ site was used for the first time within STMD to crowdsource inputs on strategy components from internal stakeholders across NASA, allowing for broad and honest feedback from all levels of the program; 139 unique responses were provided, demonstrating the strong desire by internal stakeholders to have input into the strategy. The team read and considered each of these inputs and developed appropriate actions to address them before presenting the finalized strategic framework to STMD's senior leadership for final approval before implementation.
The development of a new strategic framework also compelled the program to rethink how it measured how well it was achieving the goals and objectives articulated in the framework. Thus, it also became necessary to revisit performance indicators as program priorities evolved.
For example, it had been a long-standing priority for the program to make room for more participation by first-time awardees and underrepresented communities and this was measured by analyzing Phase I SBIR and STTR awards made by the program. Then in more recent years, it became a program priority to increase the support provided to SBCs and help them commercialize technologies and/or transition their innovations across NASA MDs. Phase II and post-Phase II awards made served as a key metric. In order for STMD leadership to assess impact of the program, output metrics such as counting awards were made by the program, while a key metric in the past was no longer a sufficient or suitable impact metric on its own. This required the program to think fundamentally differently about its processes, engagement approaches with SBCs and MDs, and budget distribution to various transitions and commercialization-focused activities.
As a result, the program leadership team developed a program performance scorecard with distinct output and outcome metrics against the established program goals and objectives. This was a critical step in operationalizing the strategic framework as a âlivingâ document and providing a way to track progress on the strategic realignment effort. The scorecard was introduced to the internal stakeholders and, as with the framework, feedback was obtained, considered, and acted on. What has resulted is the beginning of a top-level view into the program's overall performance and impact/outcome-oriented measures while balancing a complicated set of objectives and the continued need to have output measures that monitor the transactional/operational health of the program.
With administrative funding made available to the SBIR/STTR program by the Congress for the first time after 2011, the program has also begun to introduce and implement initiatives in alignment with the new impact metrics. New initiatives funded include outreach focused on attracting new awardees, engagement of underrepresented communities in a more targeted manner, and strengthening support provided to SBCs through collaboration with other SBIR/STTR agencies such as the National Science Foundation (NSF).
Administrative funding has also enabled the program to make improvements in the capture and visualization of program performance data to measure progress toward the goals, using the EHB as well as integrated customer relationship management (CRM) and business intelligence (BI) tools such as Salesforce and Tableau. The use of these tools allows for more transparency and insight to drive decision-making and assess impact of inputs, outputs, and outcomes of program routine operations and strategic initiatives.
In 2018, additional efforts were undertaken by the program to continue to drive alignment between program strategy and execution (routine and new initiatives). A visual program strategy and execution alignment roadmap was developed documenting the alignment of execution priorities against the strategic framework to facilitate shared understanding within the program. This visualization also serves as a useful link with the program scorecard, helping further the understanding of program staff on how the operations and initiatives they support directly contribute toward progress and achievement of program goals.
The new strategic framework also served as a catalyst to focus inward and assess the program's internal capability and capacity to drive ambitious goals and the required change. With the introduction of the program strategic framework and the purposeful seeding of an environment where collaboration, experimentation, and ideation were encouraged, the program began to get a large influx of ideas for new initiatives and improvement projects. While new initiatives created significant pressure and stress on program routine operations, ultimately these efforts increased the program's understanding of SBCs' external customer as well as provide insight into internal stakeholder tolerance and preferences regarding types of program changes and how change initiatives should run.
The pressure on internal capacity resulted in program leadership making a concerted effort to make decisions on identifying initiativesâlow hanging fruit or more ambitiousâto help set the program for long-term success and sustainability. As such, by 2017, administrative funding began to be set aside by the program for internal improvements, such as organizational design and operational excellence culture strengthening and experimentation efforts. This was done in addition to continued funding for technology modernization and outreach, which were renewed to have a sharper focus on customer orientation and engagement. The investment in organizational design represented the first concerted effort the program had undertaken to design the program management office organization, including enabling talent development and workload management processes.
In addition to the organizational design efforts, administrative funding was also used to further commercialization objectives in alignment with the program strategic framework. Entirely separate articles could and should be written on the efforts undertaken by the program to increase overall impact, that is, through commercialization-focused initiatives or investments undertaken in organizational design to help the program more effectively and sustainably deliver on a changed and enhanced program purpose. However, organizational design and not commercialization efforts is the subject of this article.
Lessons learned
The often unspoken mission and vision of the program are SBC oriented; however, over time, this focus can be clouded by the daily realities of implementing a program with many complex processes and emergent issues. Thus, for many years the SBIR/STTR program's efforts allowed strategy to be dictated by strict process limitationsâas oppoposed to strategy determining business rules and process improvement decisions. This was a paradigm shift in and of itself that required the program to move from a NASA-centric view (SBCs exist to serve program interests) to a hybrid customer-NASA view (empower SBCs while meeting SBA/NASA R&D priorities). This mind-set change was a key foundational step that would anchor and enable every other new initiative that sought to help advance program goals and increase the program's potential for greater impact.
Furthermore, the collaborative manner that the strategic framework and performance scorecard created continued to be the primary approach for initiating and executing new initiatives within the program. Program leadership needed to raise awareness of planned initiatives and facilitate alignment with NASA counterparts whose support or buy-in was needed to help drive the required organizational alignment. The strategic framework activity was a bottom up development but involved significant upward socialization, which has led to the strategic framework's validity and relevance over several years, despite competing views of the program's purpose. This approach has allowed for differences in opinion while increasing the likelihood of buy-in on forward leaning efforts. In 2020, the strategic framework will be reviewed and refreshed to capture strategy for the next 3â5 years based on the results against the scorecard and evolving NASA needs.
18F Customer Emotional Journey Map and Service Blueprint
Goal and purpose
To complement the development of the program strategic framework, which was fundementally developed as an internal stakeholder alignment and collaboration exercise, the program embarked on a user research initiative that also engaged external customers as a tenet of the program's push toward a more business friendly program. This initiave would also inform how program business transformation efforts can best make customer experience a priority. In 2016, the program developed a customer emotional journey map and service blueprint with the intent of helping the program better understand potential barriers faced by small businesses in doing business with the NASA SBIR/STTR program.
Approach
The primary approach taken in developing the customer emotional journey map and service blueprint was centered on conducting user research and faciliated working sessions to document the program's core business processes and tools from an SBC applicant and customer experience perspective. GSA's 18F partners with federal agencies to build and buy digital services designed to improve the UX of government products and services. 5 Due to 18F's expertise in conducting user research and discovery services within the government, the program engaged 18F by entering into a reimbursable interagency agreement.
The tools developed with 18F centered on the idea of improving customer experience, which GSA's âDigitalGovâ team defines as the sum of all experiences a customer has with your organization. 6 For NASA, recruiting 18F's support in examining its customer's experience, a perspective of an entity outside the program, was a vital step in becoming a more customer-oriented, business-friendly organization.
Approximately 23 interviews were conducted over the phone by 18F to identify major themes within the SBC experience to develop the customer emotional journey map. Specifically, the journey map graphically depicts the emotional highs and lows SBCs experience throughout their interaction points with the program solicitation and award lifecycle process. The interviewees were from 2 primary SBC groups: 2016 Phase I SBIR/STTR awardees, and 2016 SBIR Industry Day (discussed in the New Initiatives to Increase Engagement Between NASA and Industry section) participants.
18F then worked with the program to document the service blueprint, which depicts the typical path that an SBC customer may encounter with the NASA SBIR/STTR solicitation process. The business capabilities used and the process steps executed by SBCs, which are depicted in the service blueprint, are also a subset of the overall program's solicitation and award lifecycle process. The service blueprint was primarily informed by program participants and comprised 4 SBC interaction types: SBC actions, touchpoints, frontstage, and backstage.
The SBC actions indicate the process that the small businesses will take when the small business/individual has an idea/technology to explore. The touchpoints indicate when the SBC has a point in the process dependent on interacting with the NASA SBIR/STTR team. The touchpoint, for example, indicates when the SBC receives a notification via e-mail announcing the annual NASA SBIR solicitation; the resulting SBC action is when an SBC hears about the NASA SBIR/STTR solicitation and then applies.
The frontstage indicates what the SBC actually sees, and the backstage indicates what the program team does but is not actually visible from the perspective of the SBC. For example, a frontstage interaction would be when the NASA team schedules a workshop and presents subtopics for discussion with SBCs. The backstage part of that interaction is the work required for the NASA team to curate and create draft subtopics to present at the workshop. Ultimately, the frontstage and backstage depict all the various processes and action steps involved leading up to what the SBC eventually experiences and/or sees. The exercise surfaced valuable insights on the SBC experience.
Results
This initiative created a more digestible and approachable form of understanding the SBC experience while concurrently providing the program hands-on experience in how to improve the application of user research and create a tool to inform program business transformation. This initiative contributed to program goals 2 and 4 by identifying roadblocks and inefficiencies in program processes and business capabilities, which when removed or mitigated for should make the program more business friendly and easier for both SBCs and NASA to collaborate.
As a means to gain an understanding of the NASA customer experience, the initiative with 18F was also successful in promoting a business-oriented culture within the program, identifying clearer connections between program process and SBC customer impacts, and supporting business-oriented decision-making by facilitating a richer understanding of what it means to be a NASA SBC customer. The customer emotional journey map and service blueprint has also helped the program gain insights into common customer âpain points,â such as the need to define clearly what present and prospective customers will require to submit a âwinning proposal.â Ultimately, the program continuously works to try to mitigate these âpain pointsâ for the SBCs through the integration of new programmatic strategies and improving present processes.
In addition, by better understanding the SBC customer identity, the program also benefits by working to improve outreach and engagement strategies. Being able to understand the customer means being more relatable in the language and communication being utilized. One key insight from the customer journey map was that there are expectations for the program to know and remember who the small business awardees are and what they may be able to offer; in the awardee's mind, a relationship has been created and it requires a certain level of personalized attention. Improving the program's interactions with SBCs will not be done in isolation but will require using other program tools, such as the program performance scorecard to promote desired customer-centric behaviors of NASA employees in interacting with the SBCs, which is not only critical for having a successful outcome from this initiative but also for helping to shift the program toward a more business-friendly organization.
Lessons learned
One specific result that demonstrates a positive change for the program based on the customer emotional journey map and service blueprint was the creation of a newly defined role within the program team, known as the âcustomer advocate.â While stakeholder engagement was identified as an essential function of the program, it was still in a nascent stage of the strategic development process at the start of the 18F project. With several âfrontstageâ tools for small businesses getting modernized and updated, such as the program's EHB, the customer ddvocate perspective initially provided focus on the much-needed connection and human-touch element for yielding the highest value and impact for internal and external stakeholders. UXs go beyond the digital tools through which the program interacts with customers, and those nontechnical interactions are often the ones that have the highest impact on the UX, ultimately attracting potential solvers to the metaphorical âwelcome matâ of the program.
The customer emotional journey map and service blueprint helped the program blend technological innovation with a business mind-set. As a government âseed fundâ program, the challenges and achievements the program experiences are rooted in the program's SBCs. The insights obtained from this initiative underscored that being business oriented starts at the SBCs' very first interaction point, such as visiting the website to learn more about the upcoming solicitation, and extends well past receiving award notification. Through this experience, the program has learned the importance of promoting a program culture that reflects on the specific ways SBCs interact with the program and express appreciation of their experiences in the design and provision of business capabilities and services.
See Table 1 for a summary of the key results and lessons learned from the program undertaking the program strategic framework and performance scorecard and 18F customer emotional journey map and service blueprint initiatives.
Executive Summary of New Initiatives to Incorporate Multiple Stakeholder Voices and Values
NASA, National Aeronautics and Space Agency; RI, research institution; SBCs, small business concerns; SBIR, Small Business Innovation Research; STMD, Space Technology Mission Directorate.
New Initiatives to Increase Engagement Between NASA and Industry
The program's tagline is âAmerica's Seed Fund.â As it suggests, the focus of the program is to ensure SBCs with innovative ideas are provided the opportunity to succeed while addressing NASA's R&D needs at the same time. The 2016 program strategic framework exercise and resulting revision of program goals and objectives sought to make the program more business friendly by adopting a customer-focused and collaborative approach in all new initiatives.
As highlighted by the SBCs' customer emotional journey roadmap and service blueprint exercise, the program realized that to achieve program goals it needed to move away from a process- and policy-centric mind-set to a more customer-centric mind-set in designing and implementing new initiatives. Foremost of these new initiatives were 2 efforts specifically focused on opening multiple channels of communication with SBCs to provide opportunities for meaningful and useful engagement with the program.
The first initiative was the creation of an annual industry day event in 2016 to help SBCs better understand NASA and its SBIR/STTR program, its structure, and future R&D needs. The event provides an opportunity for SBCs, research institutions (RIs), industry primes, and the program to engage in dialogue across the board. The SBCs and RIs benefit from networking with their counterparts for collaboration and pursuing potential commercialization opportunities with primes, while the program and NASA MDs benefit from listening to SBCs to get industry insight and be responsive to their feedback when developing the technical subtopic portfolio for the annual solicitation as well as to inform improvements in program business capabilities and processes.
At the first industry day event in 2016, the program heard 2 specific suggestions from multiple SBCs. First, that they would like a forum where they can provide direct feedback to the program on subtopics without sharing their technical ideas in front of their competition (other SBCs competing for contracts). Second, that sometimes a minor change in a subtopic requirement, which might have been otherwise very narrowly and unnecessarily focused on a very specific NASA application, could also help NASA MDs take advantage of the latest developments in industry. At the same time, this requirement adjustment may help SBCs leverage expertise and access potential paths for commercialization of their technologies. To further explore and operationalize this feedback, the program implemented another initiative, an annual RFI process.
Introduced initially in 2017, the RFI provides small businesses with a mechanism to provide direct feedback on the previous year's subtopics and guidance for the upcoming year's subtopics so that SBC innovators could now influence a subtopic in its early conception. This feedback loop provides an opportunity for NASA STMs (âseekersâ) to tweak or modify a subtopic to take advantage of where the industry expertise lies and helps SBCs to align their commercialization plans with market trends. The RFI also occasionally asks SBCs to provide feedback on the program's present processes, systems, and services, which gives the program a routine mechanism to understand the barriers for doing business with NASA and the program and how well new initiatives are working to remove those barriers in business capabilities and underlying processes.
Having started to realize benefits of conducting an RFI before fully developing the subtopic portfolio and solicitation requirements, the program has now made the RFI a routine and recurring effort to precede the annual industry day event and kick-off the program solicitation development life cycle.
Collectively, these initiatives provided SBCs more opportunities for meaningful and useful engagement with the NASA and the SBIR/STTR program to improve commercialization outcomes.
Annual Industry Day Event
Goal and purpose
Outreach events are an opportunity for 2-way engagement and dialogue between SBCs and the program. Thus, in 2016, the program conducted the first of 3 annual industry days to date. This initiative seeks to make progress toward 3 of the program's 4 strategic goals: (1) making NASA's technical needs, as described in the subtopics, more understandable while also allowing SBCs to influence the development of subtopics by sharing their capabilities and ambitions; (2) reaching out to reliable partners and also encouraging participation by new applicants, and underrepresented communities, programs, and geographies; and (3) increasing the likelihood of commercialization and infusion of development efforts by making connections to customers and other funding programs.
For years NASA has conducted routine program outreach engagements, to include participating in SBA-coordinated regional and national small business and research institute events and road tours to underrepresented states. These events allow innovators to see how NASA differs at a high level from the other federal agency SBIR/STTR programs but do not dive deeply into particular R&D needs or the nuances of working with NASA. The purpose of the program's annual industry day is to show in one event the entire scope of NASA's needs that could be addressed by small businesses and entrepreneurs and help them get to know NASA better as a customer, which involves understanding its organization and structure, and the vision and priorities of the MDs. Thus, the annual industry day seeks to complement smaller, routine outreach efforts where by design the NASA SBIR/STTR program is not the main focus of the agenda but one of many such participants.
Approach
In the past, there were only a few ways for SBCs to interact directly with NASA experts. Industry day has sought to encourage more interaction, humanize and demystify the SBIR/STTR program for SBCs, and demonstrate the importance of this R&D program to NASA's overall mission. Industry days 2016 and 2017 were held in California, where most of NASA's current applicants and awardees are located. Because most of the program management staff are also located in California, the program was best positioned to produce its first 2 events there relatively quickly and inexpensively. The program's third industry day, titled the Innovation and Opportunity Conference (IOC), was held in November 2018 in the Denver, Colorado areaâa strategic choice, as it is a growing innovation hub for the program.
Since 2016, the IOC and its predecessor industry days have brought together more than 20 NASA experts from across all 4 MDs to publicly explain NASA's R&D priorities and needs. As with any program initiative, alignment with the priorities of internal stakeholders is critical to making industry day meaningful for all involved. Accordingly, event design and content development for industry day is a collaborative process, achieved in partnership with the MDs so that their priorities as customers and âseekersâ can be best expressed at the event as well.
For NASA and program leadership, this is an opportunity to listen to the stakeholder base and communicate firsthand the necessary program information and strategic priorities. In executing high-visibility engagement events such as industry day, this type of upward strategic alignment is critical. The full and active participation of NASA and program leadership not only models the way for the rest of the program but is also a best practice in leading transformational changeâit is an opportunity to visibly demonstrate to stakeholders NASA's commitment and internal alignment in shifting to a more internal and external customer-friendly stance.
A significant change in approach and the main unique feature of 2018 IOC was that it was produced in partnership with the Colorado Business Development Foundation (CBDF), collaborating with the Colorado Small Business Development Center (SBDC) network and the American Institute of Aeronautics and Astronautics (AIAA). Utilizing a nonreimbursable NASA Space Act agreement, 7 in which the CBDF was responsible for event logistics and NASA SBIR/STTR maintained the majority of agenda slots, allowed the program to successfully scale the reach of its existing resources.
The organizations assembled a working group across multiple states and quickly came together as a virtual team to successfully execute the IOC in 6 months. The 2018 agenda highlighted the NASA SBIR/STTR program and also included participation from other SBIR/STTR agencies, NASA programs, large business prime contractors, RIs, and small business support organizations. This collaboration model was used to exploit synergies and create value for SBCs, ensuring attendees could learn about and connect with various opportunities in one conference.
Results
As illustrated in Figure 2, there has been a high degree of continued interest and participation at industry day. In 2016, more than 200 people attended in-person and online. By 2017, participation doubled, with more than 200 people attending in-person, and 200 more online. In 2018, in-person participation increased to 350 with 89% of attendees stating they are likely to attend again.
Summary event and participant statistics from industry day 2018.
Feedback overall has been overwhelmingly positive, and the insights shared have been informative and valuable to the program. SBC attendees have expressed special appreciation for relevant NASA information presented at the event, availability of NASA personnel, and constructive one-on-one sessions. One of the most popular activities at industry day, one-on-one sessions pair small businesses directly with a NASA expert for a 15-min session to discuss program basics or technical needs in a more tailored setting. The program conducted more than 100 one-on-ones at each of the 2016 and 2017 events, and doubled this number in 2018.
Participant anecdotes from qualitative feedback collected include:
ââŚThe workshop was useful in providing context around general NASA technical needs/problems and understanding the focus for the upcoming solicitation.ââ2016 Event Participant ââŚThe one-on-one sessions were the most useful part of the meetingâŚ.ââ2017 Event Participant âLots of great information to improve my application in 2019.ââ2018 Event Participant âIt gave me answers to questions I had and to ones I didn't realize I had.ââ2018 Event Participant âThe number of mid and high-level NASA folks was remarkableââ2018 Event Participant âThis was by far the most interesting, informative, and opportunity generating conference.ââ2018 Event Participant
As described previously, the SBIR/STTR program has previously hosted industry day events to engage small businesses but they were hosted by NASA, on NASA facilities, and budgetary and policy restrictions limited their scope and effectiveness. The 2018 IOC was implemented through a partnership via a nonreimbursable Space Act Agreement. The partners took on the burden and financial risk of securing a venue, staffing the event, providing audio/visual support, and registration. They additionally worked hand-in-hand with the program team to develop the agenda, recruit additional speakers, market and promote the event, and collect feedback from the attendees.
This approach was not without risk or complication and the individuals had to quickly come together as a team and work under a challenging time line. Furthermore, planning and coordination had to be performed virtually, across multiple locations and time zones. Despite these challenges, this collaboration significantly reduced the logistical and financial burden that the SBIR/STTR program would otherwise bear and allowed the program team to focus more directly on increasing and improving the content of the event as well as increasing the impact of the limited program funds spent on this activity, such as increasing the number of NASA attendees at the event that could support future transition of technologies across MDs.
The IOC provided significant value to multiple STMD programs beyond the SBIR/STTR program while effectively leveraging the resources and capabilities of partners. The participants from within NASA, from the SBC community, and from other government agencies were extremely pleased with the results of the conference. Planning is now underway to hold another IOC in 2019. This collaboration serves as a model for future stakeholder engagement efforts (both inreach and outreach) and it is anticipated that the experience and lessons learned from the 2019 IOC will help further streamline the process and increase impact of the event in the future.
Lessons learned
Building on past successes, the program seeks to evolve the annual industry day to complement program priorities and other engagement efforts to more systematically further the strategic goals and objectives of the program. There is a focused effort made to capture, analyze, and operationalize lessons learned, allowing for continuous improvement and providing added value for participating internal and external stakeholders, especially SBCs. For example, industry day 2016 sought primarily to enable SBCs to provide feedback to inform the technical aspects of the annual solicitation. While this focus was well received and contributed to the program's solicitation development efforts, it still left many participants with basic program questions unanswered. Based on this feedback, the industry day agenda in 2017 was expanded to include a full day of program information topics, such as âSBIR/STTR 101,â aimed specifically at new SBCs and RIs seeking to understand how to get involved with the NASA SBIR/STTR program and gain a foundational knowledge on the program and application process.
Feedback from industry day 2016 also showed that participants wanted more visibility into the agenda so an interactive micro-website 8 was created for the 2017 event, allowing all attendees to track speakers and their biographies, and to learn about SBIR/STTR successes and more about the SBIR/STTR program. Users were able to set up their one-on-one sessions on the micro-website, as well as in-person at the event. The final agenda was available on the website and also in the traditional printed program agenda format for distribution at the event, as it was again in 2018.
With the advent of the RFI, to be described in detail in the Annual RFI section, which now effectively serves as the primary mechanism for obtaining SBC input on existing and potential new subtopics, the program decided to decouple industry day 2018 from the solicitation development process and explore incorporating a theme based on NASA and SBIR/STTR program priorities, and STMD priorities.
In alignment with this decision, the 2-day IOC in 2018 focused on SBC technology commercialization. The goal was to provide opportunities for companies at every stage of maturity, from those just starting out with a great idea, to experienced innovators looking to expand and actively participate in tomorrow's aerospace and defense. The 2018 agenda maintained the popular âSBIR/STTR 101â content and introduced new activities and presentations that provided information, advice, and networking opportunities for innovators who are active or graduates from the SBIR and STTR programs and are seeking to transition their technologies into NASA's aeronautics and space programs; commercialize to non-NASA markets; or otherwise seek continued funding for their concepts and prototypes.
The value of collaborating with local partners for the 2018 IOC was seen in relation to this focus on commercialization as well. In order for SBCs to thrive, and to commercialize their technologies, they need support well beyond the NASA SBIR/STTR program. A strong network among the entrepreneurs, RIs, and support organizations in the local community is critical for SBC success. IOC represents an engagement strategy that allows the program to encourage connections within regional ecosystems and across SBIR agencies to help advance not only the program's focused efforts to become more business friendly but also to help advance the priorities for all involved stakeholders. The 2018 IOC paved the way for expanded partnership opportunities, internally within NASA and externally with industry. With this strong foundation set, the SBIR/STTR program is now planning the 2019 IOC as mentioned previously and intends to continue these events in the foreseeable future.
Industry day and IOC events have proven to be an invaluable tool for increasing the voices of internal and external stakeholders and for making the program more transparent, participatory, and customer centered. While this initiative's tactical aims are many (improving the quality of proposals; enhancing industry awareness of the solicitation process; providing insight into NASA and MD R&D priorities and needs; building awareness of small businesses and industry market needs; and promoting further collaboration and networking among the programs, small businesses, and regional ecosystems), they all contribute to an overall push to making the program more approachable and accessible. While the program will constantly evolve its approach to align with program strategic priorities and industry input or trends, industry day will always focus on meaningful engagements with SBCs and bringing their innovations to NASA and industry.
Annual RFI
Goal and purpose
As one of the largest SBIR/STTR programs in the U.S. Federal Government, NASA can best meet its goals and objectives when it proactively engages its many present and prospective small business innovators in a variety of ways. The RFI seeks 1-way feedback, from SBCs to the program, and is ideal for companies that may be hesitant to share closely guarded technical perspectives in an open public forum, such as the program's industry day.
Before 2016, actionable feedback was collected informally from SBCs, and no structure was in place to track and process the feedback. This is a disparate approach from most large procurements where a solicitation is published on the government-wide point of entry, a question submission period is opened, and all collected questions are cataloged, answered, and published in a systematic way. This tried and true structure works well for procurements with up to about 35 interested companies, but in the SBIR/STTR program there are thousands of proposals submitted in response to about 100 subtopics in any given year.
For example, in 2018, SBCs submitted 1,662 Phase I proposals. This volume makes a traditional question and answer cycle challenging to implement with available NASA and program resources. In addition, this would introduce another hurdle for SBCs because the onus would be on them to read not only the solicitation and all the subtopics, but the potential thousands of questions and answers that would be posted online as a result as well. The impediments for a solution were abundant, real, and practical. Thus, in addition to providing a means for confidential communication to help inform the subtopic portfolio, the program also sought a way to seek and respond to constructive SBC feedback arising from this additional mechanism for engagement, to draw related insights to improve the program.
Approach
The development of an RFI approach to seek feedback was a collaboration between program leadership, NASA procurement, and the NASA office of general council. This collaboration was required because at NASA, using an RFI to collect programmatic input is a nontraditional way to leverage an RFI; in the procurement world, RFIs are typically used to survey industry for sources for procurement requirements when a justification for other than full and open competition (JOFOC) is being considered, or if there are impediments to full and open competition that an industry survey might help remove. The structure of an RFI does not have any regulatory requirements for a response by the contracting officer or the issuing agency to the respondents. This feature of an RFI led the program to consider its potential for gathering actionable SBC feedback for the program.
The scope of the initial RFI, which was held in 2017, was wide ranging. There were multiple sections within the RFI asking for feedback on the SBIR/STTR solicitation, programmatic implementation, technical subtopic content, and suggestions for new subtopics. As another example of applying insights from one program initiative to benefit another, with SBCs and a business-friendly focus in mind, feedback from SBCs attending industry day 2016 indicated that while the networking portion of the event was valuable, having access to a list of participating SBCs would have facilitated better connections after the event. Based on this insight, a networking opportunity was built into the RFI that allowed respondents to opt-in to provide their own SBC information to an index of SBC information that would be provided to all SBCs that opted in. A total of 163 companies took advantage of this opportunity in the first year. This option was highly subscribed to and is a practice that has been repeated in the subsequent RFI.
Results
The inputs received through the 2017 RFI process were encouraging. While there were reservations from some within the program that no SBC would participate in the RFI, it turned out that program stakeholders and SBCs were eager to provide feedback and appreciative of the opportunity. The RFI resulted in 291 ideas for new subtopics being submitted, 217 pieces of feedback on subtopics from the 2017 solicitation, and 163 small businesses responding to questions about programmatic improvements. Eighty-nine percent of SBC respondents said they would participate in an annual RFI, demonstrating the desire for engagement by SBCs.
The feedback on existing subtopics proved helpful in the refinement of established subtopics as it permitted additional market research for NASA STMs. At NASA, STMs are the technology experts and program managers who craft the annual subtopics for the solicitation. STMs reviewed the SBC feedback provided in the 2017 RFI and indicated whether they thought it was actionable. Of the 217 comments, 170 (78%) were considered by the STMs in solicitation development. STMs agreed with 51% of the feedback and indicated that 18% was actionable during the solicitation development processâhighlighting the quality and relevance of the feedback provided by the SBCs. When creating subtopics for the subsequent year, STMs were asked whether they had revised their subtopic based on feedback received from the RFI, and more than half indicated that they had.
In addition, the RFI sought suggestions for new subtopics in the 2017 RFI. As a result, 9 new subtopic ideas were implemented in the subsequent year's solicitation, representing 8.7% of the total subtopics published that year.
The annual solicitation in 2017 was 214 pages long. Most of those pages described that year's subtopics, to which SBCs could prepare proposals to address. However, 54 pages of the solicitation content, known as the âfront endâ described, in detail, the rules for applying. A 54-page set of rules and processes itself could be seen as anything but business friendly. Hence, the 2017 RFI also sought feedback on the front-end material contained in the 2017 solicitation.
There were hundreds of pieces of feedback pertaining to the front end that were incorporated as changes in the subsequent 2018 solicitation; a special tracking dashboard was developed to make sure the program could address each piece of feedback and still meet the annual solicitation publication deadline. One practical example of the type of change undertaken thanks to the front-end feedback was more clearly listing all the government databases and systems that a company would have to sign up for to have a government contract in one place at the beginning of the solicitation. In the past, all these required registrations were scattered throughout the solicitation and it was difficult for a company to know if they had completed all the requirements. This change took weeks to implement and, based on SBC feedback, resulted in a solicitation that was more user friendly.
Combined, the efforts to seek and consider feedback both on the front-end and the subtopics in the solicitation have led to a significantly more collaborative solicitation development process between industry and NASA than previous years.
Based on results of the 2017 RFI and lessons learned, the second program RFI was released in May 2018 with a slightly reduced scope, limiting feedback to the following: (1) comments on the 2018 solicitation subtopics; (2) the networking opt-in that had been used in the 2017 RFI; and, (3) customer service provided by the program's help desk during the proposal submission process. The 2018 RFI had 169 respondents, with 136 comments on the 2018 solicitation subtopics, 86 user ratings for the help desk, and 76 opting in for the networking portion.
In 2019, the RFI was conducted again and, similar to 2018, focused on obtaining feedback on the 2019 solicitation subtopics along with feedback on the modernized EHB proposal submission module. The option to participate in networking was also continued. Furthermore, a new technology capability was added to better support the annual RFI via the EHB, the program's end-to-end contract management and core technology solution. It is envisioned that the RFI will continue to give the SBCs a voice in subtopic portfolio development and programmatic improvements undertaken by the program. Starting in 2019, the RFI feedback has also been seamlessly integrated into the solicitation development module of the EHB to reduce the manual burden associated with prior RFIs for both STMs (âproblem seekersâ) and SBCs (âproblem solversâ).
Lessons learned
One of the most encouraging lessons learned from the 2017 RFI was how willing SBCs are to provide useful, actionable, and critical feedback. The volume of actionable input was so large that the program faced a programmatic risk to time lines as it raced to implement it all, particularly changes to the content of the solicitation. This highlights the second lesson learned that such a broad scope of feedback should not be requested annually; the scope of an RFI should be limited to what the program can handle in its actionable response and follow through. There is a risk of exhaustion or disillusionment by the respondents if actionable and reasonable feedback is provided but nothing comes of it. Therefore, when requesting feedback through an RFI, building a response plan that includes time and staffing for follow through and potential implementation of program improvements is a critical consideration.
In both the 2018 and 2019 RFI efforts, based on the lessons learned in 2017, the program chose to ask only for feedback on program subtopics and one other targeted area. This not only allows the program to ensure that it addresses the programmatic feedback raised by SBCs but also provides the program sufficient time to implement solutions and demonstrate results before asking for feedback again.
The program's proposal subtopics are always evolving. Therefore, from an internal stakeholder perspective, there is a tentative plan going forward to continue to collect feedback on existing subtopics every year since the feedback is most beneficial to STMs, and the volume of feedback from SBCs for each subtopic is relatively low. New subtopic ideas will be requested less frequently since these inputs are reviewed by a central group within NASA and the process is time-consuming. Program feedback will likely be collected every 2 to 3 years since the process of working through the feedback is similarly very time-consuming and depending on the nature of the suggestions, they may take longer to implement. Some solutions may also need to be tested and refined before implementation in order for the program to mitigate risks.
As a final lesson learned from both industry day and the RFI, SBCs have very limited business development dollars. Thus, it is extremely important for them to be confident that the time and money they invest in writing a proposal have a very good chance of winning. The RFI and annual industry day event combined provide multiple mechanisms for small businesses and the program to engage and understand each other's perspectives while facilitating the program to make progress toward all 4 program goals. Small businesses also have the benefit of influencing subtopic requirements, understanding NASA requirements better, and exploring opportunities to partner with other firms to fill gaps in their expertise or develop plans with industry primes to commercialize their innovation and improve their chances of winning a subsequent award.
The RFI is of further benefit to NASA since the huge technical reviewer community that volunteers time for the program now spends time reviewing higher quality and responsive proposals from SBCs as a result of the inputs obtained through the RFI process in addition to the information sharing that takes place during industry day.
The RFI is now a cornerstone of the SBIR/STTR program strategy and informs the program's ongoing stakeholder engagement efforts. Key aspects of the program have improved as a result of this feedback, and great steps have been taken to improve the program as a result of this effort. One critical part of being business friendly means conducting real listening sessions; there is an implied reciprocity in a relationship where there is give and take between the customer and the program. That was the goal with the RFI, and by making industry feel heard and applying the insights to continue making the program more business friendly, it is starting to create real value for both industry and NASA.
Table 2 summarizes the key results and lessons learned from the program undertaking the industry day and RFI initiatives.
Executive Summary of New Initiatives to Increase Engagement Between NASA and Industry
DARPA, Defense Advanced Research Projects Agency; DOE, Department of Energy; R&D, research and development; RFI, request for information; SBA, Small Business Administration; STMs, subtopic managers; STTR, Small Business Technology Transfer.
New Initiatives to Improve Business Processes and UX through Technology Modernization
As mentioned earlier, the program has long leveraged a powerful system, the EHB, to manage NASA SBIR/STTR program solicitation and award business processes (referred to as independent âmodulesâ in the EHB) from cradle to grave stages: solicitation development, proposal submissions, review and selection, contract negotiations, administration and closeout, and transition to commercialization and/or infusion within NASA. The EHB system is the primary technology interface for firms and internal NASA users and typically handles more than 2,000 proposals, 4,000 evaluations, 500 awards, and 4,000 deliverables annually on an ongoing basis throughout the year.
The legacy EHB system itself began as an innovation developed by REI Systems through consecutive rounds of NASA SBIR awards. The EHB has since then been successfully transitioned and commercialized by REI Systems for the benefit of other U.S. federal agencies, such as the Department of Health and Human Services. For the NASA SBIR/STTR program, the EHB was implemented in stages starting around 2003 and then, recognizing the increasing costs of ownership of the legacy system and the need for improved UX, in 2015, program leadership made a commitment to move forward with plans to modernize the system.
The program's success is highly dependent on making the solicitation and award process as easy as possible, particularly for the vocal and influential internal stakeholders (âproblem seekersâ), and thus, the program must acknowledge and be responsive to end-user suggestions for system improvements. In the legacy system, enhancements were expensive due to the complexity and cost of maintaining a legacy monolithic code. With the new modernized system, the goal was for future enhancements to be accommodated at a lower cost to evolve the EHB and enhance usability. Once the program framework and new business-friendly orientation were introduced to the program, the challenge was to determine how and when to best engage external stakeholdersâthe past and potential SBC applicantsâgiven the increased customer focus.
The modernization of the EHB, although initiated as a technology modernization effort, became a major opportunity within the overall program toward becoming more customer focused. At the same time, it provided an opportunity to address needed process efficiencies and to position the program to provide increased business value for NASA as well as more innovative business capabilities to meet the needs of the future.
As mentioned in the previous sections of this article, the possibility of identifying and securing a budget to undertake a large-scale technology modernization started to become a reality for the program in 2014, thanks to a new, limited-time authorization in the SBIR legislation, which allowed SBIR/STTR programs to use a portion of program funding for program improvements. This coupled with an overall program budget that could concurrently support the implementation of prioritized technology modernization components (without adversely impacting ongoing mission-critical program operations and the SBIR/STTR SBC award budget for R&D contracts) made it possible to move forward with the modernization. In sum, this appeared to be a once-in-a-decade opportunity to begin modernizing the EHB, improve user friendliness, and also begin addressing needed improvements to dated program processes.
Goal and Purpose
The EHB modernization âprojectâ began in the Summer of 2016 with the initially scoped requirements completed in Fall 2018. As described previously, the initiative was also moderately rescoped in 2016 to incorporate internal and external customer feedback received on the program business processes and capabilities collected during the development of the strategic planning framework, the 18F customer journey map and service blueprint, as well as several rounds of industry days and RFIs, some of which were undertaken concurrently with the early stages of the EHB modernization effort.
In designing the EHB modernization project goals, a structured approach was taken to map the project objectives and success criteria to the newly developed program strategic framework to ensure the greatest possible impact and return on investment. As illustrated at a high level (Fig. 3), the key objectives for the modernization were to provide an enhanced system that provides seamless UX with intuitive and user-friendly interface; user-centric rather than process-centric business rules; scalability and flexibility for future program and user needs; multilayer security to protect sensitive and SBC proprietary intellectual property; integrated data analytics and visualization capabilities; accessibility of EHB content to people with disabilities (Section 508 compliance); responsive design for multiple device platforms; focus on federated data access rather than duplicative systems; prevention of data duplication; and increased interoperability with other systems (NASA or other government agencies).
Desired outcomes.
Once the decision was made to undertake EHB modernization, the program also developed a modernization roadmap that clearly articulated the business case and drivers for undertaking modernization. This roadmap was subsequently udpated to outline the envisioned results of modernization in alignment with the program strategic framework and scorecard. Once drafted, this roadmap was used as an internal sponsor/stakeholder expectation-setting and program communication tool to explain why modernization was being undertaken and what the envisioned results were. These same messages were also reinforced in user adoption and change management activities in support of the rollout of the modernized EHB to internal stakeholders.
Approach
In the early stages of planning the modernization, the program sought to develop an end-to-end analysis of business capability needs based on program and customer inputs (classified as follows: existing but process intensive or manual, existing or partially implemented in a legacy system, or desired). The vision extended far beyond just modernizing the EHB legacy technology and laid out the spectrum of technologies and processes required to meet the previously mentioned objectives based on the desired business capabilities for the program, such as external stakeholder engagement efforts (outreach to SBCs and RIs, including historically black colleges and universities and minority serving institutions); request for proposals; proposal evaluation and selection; budget management and contract negotiations and management; postaward infusions and partnerships; EHB user support; and data management and analytics.
The project was executed through a contract, scoped and negotiated in 2015 between NASA and former SBIR awardee REI Systems, which outlined an approach where the government and the contractor would define requirements and hold preliminary design reviews (PDRs) and critical design reviews (CDRs) of each major system component (âmoduleâ) to manage risk and align expectations. There was also the need to address a large risk for failed user adoption if users were not properly informed and engaged throughout the process. To address the risk, conducting user research (as illustrated in Fig. 4) at the beginning of the project became a core principle, with more than 100 user feedback sessions conducted to develop the draft requirements, and rollout plans for each major module emphasizing change management support to smoothly transition users to the modernized environment and facilitate adoption.
Agile user-centric approach (methodology from REI Systems).
After the initial feedback was collected, the project used an agile development approach to continuously address input received during joint application design (JAD) sessions to define the requirements and conducted user acceptance testing (UAT) sessions that refined the implementation based on user inputs provided on early releases. After the first release of the submission module, issues identified by users were prioritized and addressed either through frequent subsequent releases or deferred through a rigorously maintained backlog to be addressed as future enhancements of the EHB.
After the rollout of the first module and as the project progressed, both the government and the contractor saw a need to increase involvement of the users up-front during initial design sessions and sprints; thus, for subsequent module developments (including the JADs, PDRs, CDRs, and UATs), usability testing was incorporated into the project before module deployments to help steer the interface and ensure its applicability to everyday usage. This approach allowed a more user-centric focus to the effort.
The legacy EHB system was mostly a text-based experience, although it was a graphical user interface. The new interface sought a UX consistent with present technologies and also sought to enable mobile device access. The design of the system took advantage of newer technologies that simplified not only the development of the system but also long-term maintenance and enhancements for future needs, utilizing a scalable and shared microservices architecture that reduces cascading effects during any design change (Fig. 5). The new system also prioritized the development of an EHB data warehouse, as the single source of truthâirrespective of the technology used to create the data, for example, custom application, platform-as-a-service built application, and/or COTSâto provide data analytics, data visualization, and CRM to drive new business capabilities.
Microservices architecture.
The EHB was built around modules that deliver the needed business capabilities. As noted previously, the submission module was built first for firms to submit their 2018 Phase I proposals, and subsequent modernized modules were built so that the whole Phase I cycle was conducted in the new EHB, including review and selection, contract negotiation, and contract administration. Because other proposal phases and processes were still ongoing, a key constraint was the need to deliver new modules while legacy modules were still operational and achieving near-zero down time; the program addressed this by delivering updated modules in a just-in-time sequence. In addition, user support for 2 very different interfaces needed to be provided as users potentially would need to switch between the 2 systems, depending on the role and/or task needing to be performed.
Beyond the technical aspects, as the core business tool for the community, the EHB is often the first point of interaction with external users; therefore, user adoption support and change management were critical to address any impacts to users, comprising both internal and external customers. Although business processes may force internal users to interact with the EHB, the success of modernization hinged on successful user adoption. Change management and communications support were integrated to support users for a smooth transition, and this focus was also in alignment with insights obtained from the customer emotional journey map exercise described in the 18F Customer Emotional Journey Map and Service Blueprint section.
It is important to note that different EHB modules are used by different groups of users at different times depending on when the business process had to be executed (e.g., SBCs are the primary user for submissions, while many roles within MDs and centers, as well as within the program management itself, have functions in review and selection). Therefore, change management had to consider the impact of changes on different audiences based on their interaction points with the system as well as the overall timing, that is, solicitation life cycle. The program considered and applied best practices in user adoption and UX (Fig. 6) to influence the required engagement (such as leveraging change champions and a super user network) as well as the development of messaging platforms, training and user support, and detailed communication plans. The resulting change management effort included a combination of awareness, communication, outreach, and training tactics customized for each module rollout that sought to smooth the transition of users from the legacy to the modernized system and to minimize user-centric transition risks.
Tenets of user experience (methodology from REI Systems).
In order for the vendor to deliver the required functionality in a timely manner to users in alignment with the program's solicitation schedule, process and usability improvement enhancements were initially deprioritized by the program. With the initial phase of modernization completed in the Fall of 2018, the program recognized that the system's overall usability should be the next priority both in terms of UX and critical process improvements. A second phase of modernization in now underway to address enhancements prioritized by users and it is being guided by a vision for the next iteration, or evolution, of the EHBâwhere the key result of the initial phase of EHB modernization was rearchitecting the EHB and replicating the required business functionality; this new phase of modernization focuses on seamless integration of tasks across applications and improving flow between and across modules from a user's perspective.
Results
The modernization âprojectâ success criteria were defined and agreed upon during the preliminary design review (PDR) process. The success criteria were revisited by assessing results after each module was rolled out, that is, the business functions and processes associated with each module were executed by users. One of the key ways that results were determined was through analyzing user feedback on questionnaires containing both multiple choice and open-ended questions.
As an example, the initial rollout of the first EHB module (âsubmissionsâ) was generally well received by users, especially newer users who had not built up experience in the older system (Fig. 7). For example, when users were asked about the various features of the submissions module on a scale from 0 (lowest) to 5 (highest), the average scores were around 3.5 or 4, or âgoodâ based on the scale provided. Based on these scores, the program determined that the effort invested in modernizing the technology infrastructure for the program provided a better experience than the legacy system from the SBC perspective, although it still had room for improvement.
User ratings of submission features. EHB, Electronic Handbook.
The program also found from the submission module rollout that a more proactive approach to certain change management-related user engagements would be more critical than the initially thought for subsequent modules starting with more rigor in integrating change management activities into technology development and deployment schedule. Through this more integrated approach, the program instituted up-front usability testing, longer support hours, and Q&A sessions right after module deployments. After modernization of each subsequent EHB module, the program also began to more systematically incorporate feedback received during the initial module deployments to feed subsequent releases.
Interestingly, as demonstrated by the submission module's questionnaire results, the EHB modernization in 2018 was seen as net-positive by external SBCs, the primary user of the submission module, whereas most of the user resistance and issues were encountered by the internal NASA users, primarily in the âreview and selectionâ and âcontractsâ modules. A key reason for this was that the modernized EHB was not seen by many internal users as being as usable as the legacy system. This was largely driven by internal users being comfortable with how to do things in the old systemâeven if the system itself was not ideal. To help address this concern, the program conducted its first user steering group in 2018 for the review and selection module. The goal was to engage the internal users themselves in prioritizing enhancements to increase the usability of the system. Since both internal and external customers are critical, a recurring and consistent theme and continued strategic focus for the program is the EHB UX.
Both user vantage points are being considered in the second phase of modernization underway, and starting to yield results with the internal NASA community. The enhanced 2019 review and selection module, which was informed by the 2018 user steering committee group, has resulted in significantly fewer complaints than the 2018 rollout of the module. This user steering group approach was so successful that the program plans to conduct similar steering groups for each EHB module in 2019.
Moreover, the hard work the program put into listening to users and proactively communicating the change showed a commitment to both external and internal end users and stakeholders; the program is not only willing to engage them as partners to influence modernization efforts but also the program is committed to undertaking the necessary improvements to better meet stakeholder needs when interacting with the program's business processes and technologies.
Ultimately, while the EHB modernization effort presented challenges both in terms of planning and execution, it also laid the foundation for a more capable technology and business process framework that will be much more responsive to evolving user needs, in a secure and low-cost way, compared with the legacy system. The EHB project team is also now able to fully leverage the microservices architecture to more nimbly take on ambitious process improvement efforts; integrate enhancements to improve UX; and drive efficiencies in task execution. Given the program's focus on solicitation process innovation and reducing process waste, future EHB enhancements will likely continue to focus on creating additional efficiencies in task execution for âproblem seekersâ and âproblem solversâ alike.
Lessons Learned
EHB modernization was the broadest technology and process transformation the program has seen in more than a decade, impacting thousands of users within and outside NASA. The changes include both the new technology itself and the beginning of new business processes intended to address capabilities needed in the coming years. However, the program recognizes and plans to continue to invest in re-evaluating and improving the business rules and processes underpinning the technology, as well as to maintain a continued focus on improving the usability of the system for internal and external customers. These enhancements will take place in parallel with future generations of initiatives focused on improving the program and making it a higher impact.
A program with complex processes requiring extensive stakeholder buy-in to modify and technology-dependent business processes that cannot be delayed to meet a congressionally mandated schedule is an extremely challenging environment for a technology and process modernization âproject.â Despite all the planning leading up to EHB modernization, and the prioritization of module sequencing according to the program cycle phases, the program faced many challenges related to user adoption of the new technology and modified processes. Furthermore, having 2 different user interfaces (the legacy EHB and the modernized EHB), depending on the module a user was working with, caused much frustration for the user base.
As can be expected, the user issues identified during the initial phase of modernization varied. In some cases, while actions in the newer user interface were simplified, some users sought out the traditional processes; for example, deleting a record could now be done by clicking on a wastebasket icon next to the record, but users were still looking for a menu option to delete a record. Or, for example, users searched for specific reports from a drop-down menu based on their experience with the legacy system functionality instead of using a new flexible and user-configurable single report option. In these cases, users experienced frustration when the new system seemed to be less efficient based on their familiarity with dated functionality even though the new functionality mimicked technology features they might see in their personal life (e.g., on Gmail or TurboTax).
It was a key lesson learned: given that users were unaccustomed to seeing this type of functionality in legacy government systems, even simplified business functionality would require significant awareness and training investment and time for adoption.
In other cases, the updated interface in practice was not necessarily simpler and required more steps than the prior system, which is concerning from a UX standpoint. These cases illustrate the very real difficulty that users have in imagining their actual needs during a requirement definition activity and why it is so important to roll out new interfaces to a smaller subset of users and a smaller portion of the process as a pilot to work out major usability issues early.
Although the project did usability testing before rollout of each module, the program's aggressive schedule based on meeting the program's solicitation life cycle milestones did not allow for all the usability input to be addressed before the deployment of modules, which also led to some change management and usability issues with the initial releases. This again illustrates the key point that extensive technology mock-ups and demos cannot fully replace the value gained from investing in usability and following the agile philosophy guidelines of making incremental improvements through multiple releases.
The program learned from the challenges of the initial phase of modernization. With a publicly stated program priority and program leadership imperative to improve the program's business friendliness, the program made a concerted effort to prioritize and fund EHB UX and change management interventions in the second phase of modernization.
Thus, in the phase of modernization presently underway, user steering committees have been established to prioritize and validate process and user improvements. The project team is building mock-ups (prototypes) versus using wireframes in order for steering committee members to better understand and provide feedback on how their requirements are been translated and addressed. An HCD and UX SME has also been added to the project team to complement the customer advocate's efforts. The use of prototypes and access to HCD/UX industry expertise greatly facilitates the project team's ability to observe and analyze user interaction with the EHB as a supplement to user feedback in informing value-added enhancements.
In retrospect, the program identified the following root causes of the difficulties faced during modernization:
A senior member of the program management team should have been fully allocated to the modernization effort starting as early as 2015 to provide a single point of contact to the program execution and contractor team for the strategic direction of the technology project, supported by the program's customer advocate and HCD/UX SME to ensure adequate change management and user engagement were undertaken.
The technology modernization evolved to also become an opportunity to improve business processes (more of a âbusiness transformationâ effort). As business processes are coded into the technology system, these process changes in some instances delayed the finalization of technical requirements, where delays simply could not be accommodated (given inflexible program solicitation deadlines) without significant risk to quality and undue stress to the team and users.
Key internal user and stakeholder engagement availability for the JADs, PDRs, CRDs, usability testing, and UATs was limited due to competing demands on user and stakeholder time.
Adequate HCD/UX expertise to guide a customer-centric vision for the EHB modernization as well as change management resources were not budgeted during project definition and were added during project execution, but the early stages of the project were impacted as was the plan to provide integrated technology and user adoption support.
The overall modernization schedule was aligned with the program solicitation schedule, which drove the timing for the release of each modernized module. This resulted in the team having to design and develop the complex business capability and associated processes for just in time delivery to support business process at full scale (instead of starting with a smaller scale implementation of a simpler capability). As a result, the project team was challenged from the get-go to deliver much more than a minimum viable product to ensure users were able to perform the same business functions they were able to accomplish in the legacy system.
The rollout of the EHB modernization roadmap was undertaken at the same time the initial module development began. The lack of staging and concurrent awareness building efforts directed at internal stakeholders both at the high level and at the detailed module levels may not have allowed for the program to fully realize the benefit of the roadmap exercise.
Ultimately, these issues with the modernization project can be consolidated into 6 major lessons-learned categories for other programs embarking on technology modernization efforts affecting numerous types of users in a fast-paced and requirement-rich environment:
A single accountable program official must provide oversight and management for technology, customer experience, and change management aspects of a large-scale modernization effort with HCD/UX expertise readily available to guide efforts from the onset, such as the benefit of actual user observation and the ability to obtain enhanced feedback from users through interaction with mock-ups versus static screenshots and wire frames.
Process- and technology-related modernization âprojectsâ need to be clearly delineated and scoped from the start with success criteria understood, that is, as a business transformation effort versus a pure play technology modernization, as appropriate, with leadership aligned to reduce the likelihood for scope creep for the vendor.
A customer-centric vision to drive design and development efforts from the onset coupled with early, clear, and integrated UX and change management approach that is appropriately funded must be a critical priority when the change affects thousands of users, many of whom work with the program on a volunteer basis and thus prioritize ease of use when deciding whether to participate.
Given a demanding and compressed program operational schedule and âprojectâ of this scope, key personnel across all stakeholder levels must be available to dedicate time on an ongoing basis and ensure their active participation at the right time, such as for usability testing or participation in assigned steering committee meetings for requirement prioritization or validation.
Technology and process changes should be piloted with a smaller set of users for smaller scale business capabilities and processes before being rolled out program-wide before initiating a more complex change. This would allow the team to obtain insights from usability studies with a smaller community, and then apply those insights for larger scale rollout efforts.
The modernization roadmap activity, while intended to secure aligned expectations for the modernization project fell short of internal sponsor and stakeholder expectation setting due to being drafted well after modernization began.
Despite significant efforts in planning and the best of intentions, not every change associated with the modernization effort went as smoothly as hoped. There are valuable lessons learned that were captured and continue to be captured and applied for the second phase of the modernization effort presently underway. This is in alignment with the program's learning culture. A key lesson learned to be underscored is the difficulty involved in making process changes concurrently with complex and large IT system changes. Given the issues identified with dated business rules and processes, in hindsight, it makes sense to prioritize and improve those without only focusing on modernizing the technology layers.
However, full appreciation and understanding of the required business process change were not realized until the program had started undertaking initiatives that yielded this type of data and insights, and by then, the EHB modernization effort had also begun. Therefore, while not best practice, the program made the purposeful decision to undertake the essential process changes simultaneously with the technology modernization. This created significant pressure and stress on top of program routine operations. It also posed a large learning curve for both the program team and internal stakeholders who participate in these initiatives.
Furthermore, because of the âhigh collaborationâ participation model sought in designing and running the variety of innovative initiatives described in this article, it also became paramount for the program to demonstrate that it was being responsive in tackling issues as they arose to maintain trust with participants, that is, the feedback and insights that were solicited were in fact valued and being considered. Many of the process changes undertaken with respect to the EHB modernization were quite incremental in nature, but they seemed much more substantial due to the sheer amount of changes experienced by the stakeholders.
Thus, the program learned valuable lessons on how to successfully set the stage for future technology modernization âplanning and executionâ efforts as well as how to best engage and support users when introducing future technology innovations or process improvements. In addition, through planning and implementing this technology initiative, the program also has a much greater understanding of external customer and internal stakeholder tolerance and preferences regarding types of program changes and how change initiatives should run.
Table 3 summarizes the key results and lessons learned from the program undertaking business process and UX improvements through a technology modernization initiative.
Executive Summary of New Initiatives to Improve Business Processes and User Experience Through Technology Modernization
EHB, Electronic Handbook; UX, user experience.
Discussion and Conclusion
For many small businesses, the mere idea of working with the federal government can be daunting and engaging with NASA is no exception. As the NASA SBIR/STTR program seeks to encourage new participants in alignment with program goals, particularly from outside the usual pool of small business proposers, the program will need to use new strategies and innovation through an entrepreneurial approach. This is necessary to attract SBCs that are expected to have similar qualities. The program ultimately needs to create a customer-friendly front door and bring down barriers, both real and perceived.
The SBIR/STTR authorizing legislation directs agencies not only to advance R&D that meets agency needs, but also to stimulate innovation. Invention and R&D only become a true innovation if they are used. Historically, SBCs have also been the workhorse for economic growth and innovation in the United States. Thus, for NASA SBIR/STTR to have the greatest innovation impact, the program must also empower U.S. SBCs to grow into viable businesses with commercialized products to make an impact on the U.S. economy.
Furthermore, MDs and centers have very specific R&D priorities they seek to advance ranging from autonomy and robotics to small spacecraft technologies by leveraging this program. Given NASA's mission, these investments truly have the possibility to have global impact and beyond for the benefit and future of all humanity. These are the 2 fundamental focus areas where the program engages with the âproblem solversâ and âproblem seekersâ stakeholder base, balancing the program's mandates to solve particular R&D problems relevant to NASA while also stimulating the U.S. aerospace economy. Thinking of these stakeholders as program customers is starting to have a major ripple effect on the program. As described in the previous sections of the article, there were 3 main initiative areas that came out of this shift in thinking:
Incorporating multiple stakeholder voices and values (refer to the New Initiatives to Incorporate Multiple Stakeholder Voices and Values section and accompanying summary in Table 1) to inform program strategy and become a more customer-oriented and business-friendly organization.
Increasing engagement between NASA and industry (refer to the New Initiatives to Increase Engagement Between NASA and Industry section and accompanying summary in Table 2) to re-energize the role and the perception of the NASA SBIR/STTR program as a seed fund to empower SBCs and onramp innovative research.
Improving business processes and UX through technology modernization (refer to the New Initiatives to Improve Business Processes and User Experience Through Technology Modernization section and accompanying summary in Table 3) to make the solicitation and award process as easy as possible for internal and external users while providing more innovative business capabilities.
These initiatives, undertaken by the program primarily from 2016 to 2018, are the beginnings of a business transformation anchored by a relentless focus on customer values to get even closer toward the program becoming a more customer- and business-friendly organization. Looking across these initiative areas and the accompanying results as well as lessons learned by the program, there are more generalized takeaways that emerge for organizations seeking to reorient their strategy, process, and technology while still operating core programs and delivering ongoing commitments. The rest of the Discussion section elaborates on the 3 main takeaways for government organizations or programs undertaking transformational efforts, as follows:
Building vertical and horizontal organizational alignment on strategic direction is essential for successfully leading and implementing transformational change.
Engaging external customers can be challenging for the government sector but it is worthwhile and requires a mind-set change and resources.
Embracing transformational change is the starting point; success is facilitated by investing in upfront planning, ongoing risk mitigation, and time for reflection.
Building Vertical and Horizontal Organizational Alignment on Strategic Direction Is Essential for Successfully Leading and Implementing Transformational Change
As described earlier, to help execute on the business-friendly and customer-centric initiatives, it took visionary and committed program leadership as well as a capable and willing execution team, all grounded by a core set of values and a shared vision of the program's future. A pivotal role in creating a vision for the program was to have an executive champion (âchange sponsorâ) that could inspire and engage the diverse NASA SBIR/STTR program stakeholder base in developing a common understanding of the program's value and the increased business results and the impact it could have both within and outside of NASA. This executive change sponsor was essential to driving improvement across the people-technology-process spectrum.
The program's leadership embraced this role and encouraged a consensus-driven strategic framework for the programâgiven the strategic importance of this collaborative exercise, the program executive leadership team had an active hands-on role in driving this process and analysis versus delegating the work to program staff. This was intentional to demonstrate the leadership's commitment to listening and organizational change. As mentioned previously, the program is unique in that it touches every MD within the agency and the NASA centers. The program not only relies on representatives from the MDs to lead many of the program's processes, but the MDs are also the recipients of the results of many of the initiatives. It was extremely important that the program had their buy-in, so they were involved in the planning and reviews of not only the strategic framework but also in most of the potential program changes or new initiatives.
Since NASA is a largely matrixed organization, the NASA centers also play a pivotal role in both leading and supporting the program and its processes. Program participants come from all levels across the centers and representatives from each center are a part of the program team. To engage this matrix of stakeholders within NASA, the program actively engaged the MDs and the centers in working groups and encouraged them to provide vital feedback to any process updates.
It is important to note that the center-level stakeholders also have a substantial amount of interaction with SBCs through the program's ongoing communications and outreach efforts. The MD and center stakeholders are key supporters who help promote the value of participation in the programâand their recognition of the value and potential of the program was key. Without this shared understanding and alignment established upfront, it would have been challenging to garner support of the various business-friendly initiatives that were undertaken.
At the same time, the agency's senior leadership within STMD were given regular opportunities to provide direct guidance through briefings by the program's leadership, both at the concept stage and implementation stages. It is critical to add that most of this alignment work would not have been possible without several shifts in STMD and program leadership between 2014 and 2016. Collectively, these shifts paved the way for complete vertical alignment in strategic direction allowing the program to begin embracing its business transformation and the supporting initiatives described within the article.
In hindsight, one of the single most important portions of these shifts in leadership responsibility starting in 2015 was making a senior STMD leader (the deputy assistant administrator for programs) the source selection official for the program, and not the program executive. This meant that a senior level decision maker, who had the great respect of the other MD and center partners, was available to help drive and demonstrate support for changes in the culture. This senior leadership support unlocked more change across the entire program (which spans the entire agency) than possible in the previous decade.
Although the strategic framework was collaboratively developed across the program, program leadership recognized that not all internal stakeholders agreed with the direction described in the strategic framework and that there were still long-rooted and differing views on the role of the SBIR/STTR program for NASA. Thus, the program needed to work to change the perception that many had of it to increase its strategic value. While buy-in from NASA STMD and program executive leadership helped foster buy-in from many internal stakeholders, horizontal integration within NASA was the key focus for many individual projects and initiatives. It was necessary to have the support of those who were ultimately either responsible for implementing the change or were impacted by the change.
With the tracking and communicating of progress underway against the scorecard, the framework was gaining momentum and recognition on how to make strategic planning efforts stick versus being a compliance exercise once a year. Investments in organizational design have been made to ensure that the program leadership team aligns the organizational foundation and all ongoing routine activities and new initiatives with the strategic frameworkâa key focus is to provide consistency and transparency in priorities articulated and executed by the program so that program staff can clearly see how their work contributes to the achievement of strategy.
There is still much work to be done to make certain that the framework continues to be widely understood by internal stakeholders and that their activities remain aligned. While it will be an ongoing activity of the program to ensure that the framework's content remains relevant and understood, the approach toward developing it in a collaborative manner versus a closed-door activity by program leadership can be a considered a mainstay for future strategy refresh efforts.
Engaging External Customers Can Be Challenging for the Public Sector/Government but it Is Worthwhile and Requires Mind-Set Change and Resources
An equally necessary aspect of aligning program work with the program's vision is the ability to evaluate the program's effectiveness regularly from a customer-centric viewpoint. While it is a commonly accepted and important business practice for the private sector to engage customers, it is not a common nor an easy practice for government agencies to do so, especially where external stakeholders or program âbeneficiariesâ are concerned. Therefore, it requires a mind-set change. Not only did the program need to decide to more proactively and purposefully engage stakeholders, the program also needed to be aware of how and what it was asking from stakeholders. This was especially the case when the program was engaging with SBCs. The program needed to be transparent and fair in these interactions since the program must operate under government-wide procurement guidelines. Therefore, rather than not engage SBCs in feedback due to complexity and risk, the program chose to look creatively at options in its efforts to seek feedback.
Given the challenges of obtaining direct feedback, a common practice in the government sector is the use of an internal stakeholder as a âproxyâ to represent the interests of a user group. The program understood the benefit of working through proxies where there were limitations in obtaining direct SBC feedback, especially for the EHB modernization effort. However, as a lesson learned from the deprioritization of UX and user adoption support during the first phase of modernization, the program understood the need for and created an internal customer advocate role. In addition, the program prioritized and funded change management and HCD/UX resources. While these were all good first steps and good headway is being made during the second phase of modernization, these resources proved to be insufficient from a program perspective.
The key reason was 2-fold: First, the customer advocate role was performed by a civil servant suited for the task from an experience/attitude perspective. Drawing on the experience from the 18F customer emotional journey map and service blueprint exercise, this individual's role was to question if what seemed intuitive to those in the program management and execution team was really intuitive or beneficial from the viewpoint of the customer. However, this was a duty assigned on top of existing duties and not a core or a primary focus of the resource.
Second, while the program recognized there was value in drawing industry HCD/UX experts to support technology modernization efforts on the vendor side, there was still no civil servant counterpart within the EHB modernization team who was sufficiently versed in HCD/UX disciplines to effectively make the multitudes of decisions required for a modernization of this complexity while maintaining a program (and not just a technology) perspective throughout.
The EHB modernization initiative emphasized the need to listen to customers throughout the design, testing, and implementation process for a new IT systemâin even more extensive ways than required for engagements such as industry day, the RFI, and the 18F engagement where the implementation schedules and overall user impact were less constrained. The program has recognized that working to relentlessly improve the customer experience (both internally and externally) considering people, process, and technology of the program is a full-time job. Thus, in 2019, the SBIR/STTR program intends to hire its first chief customer officer (CCO) to join the program team full time. The CCO will define a customer experience vision for the program as a whole, build awareness and capability within the program team, and help the program raise the bar in how goals are executed with the customer in mind, above and beyond a UX focus for program business technology.
The new CCO role will also play a vital role in shaping useful and positive stakeholder engagements, including in the collection of feedback. Given that the program anticipates continuing with initiatives that will yield valuable data, the program is also making additional systematic investments in its suite of BI (data analytics and visualization) tools. The goal is to move toward a more evidence-based approach in measuring both the impact of the program and results across these types of strategic initiatives to drive program decision-making and future improvement efforts.
The results of each of the initiatives described in this article demonstrate that stakeholder engagement efforts were a critical component in the success across the initiatives undertaken by the program. It also showed a genuine interest by the program to shift its mind-set toward a more customer-centric orientation and to improve the overall program based on plentiful and thoughtful feedback received from stakeholders throughout the course of designing and implementing initiatives. SBIR program leadership has come to appreciate that for program staff to truly deliver impact through their work, they not only have to start with the âresultsâ in mind, but they also have to develop âcustomer intimacy,â a deep understanding and appreciation of the stakeholders or customers they serve. This is a well-established best practice in the commercial sector where the customer voice and âvotesâ directly impact the bottom line.
Embracing Transformational Change Is the Starting Point; Success Is Facilitated by Investing in Upfront Planning, Ongoing Risk Mitigation, and Time for Reflection
With administrative funding made available to the SBIR/STTR program by the Congress in 2014, the program was able to introduce significant initiatives to increase its customer focus and implement much needed updates to processes, technology, organizational underpinnings, as well as the engagement opportunities available to program customers and stakeholders. Coupled with an emerging change in mind-set due to initiatives described in the New Initiatives to Incorporate Multiple Stakeholder Voices and Values section such as the program strategic framework and performance scorecard and 18F customer emotional journey map and service blueprint, there were significant energy and commitment within the program and a desire to take advantage of this administrative funding.
Given the need to make the best possible use of this newly available funding and the guiding direction program leadership brought to prioritize and focus the efforts undertaken by the program, a decision was made to undertake concurrent investments to improve and mature program operations and technologies while exploring and experimenting with customer-focused initiatives, such as industry day and the RFI, described in the New Initiatives to Increase Engagement Between NASA and Industry section. Ultimately, these formed the foundation of the hard work needed to rewire and rebrand the program toward a more business-friendly organization and adopt applicable private sector best practices.
However, as any government sector organization that is undertaking transformational change would share, it is not an easy undertaking to build and improve a highly visible program while maintaining routine mission-critical operations with a demanding schedule in and of itself. The program needed to make thoughtful decisions both proactively and reactively at every turn to ensure it continued to move forward with the business transformation initiatives without potentially alienating stakeholders or risking mission-critical routine operations in executing the program.
While change is often necessary and can result in significant benefits, when organizations try to pivot from an existing way of operating, it can be quite difficult, especially if a fundamental change in mind-set is required. The organization must think through how it will organize work and deliver on routine operations and existing business process commitmentsâelse the entire change effort could fail. It is also important to be aware going in and be realistic in determining what can be accomplished by program resources and how quickly. Furthermore, it also becomes critical to set and manage expectations of the various stakeholder groups that need to participate and devote time to help inform and execute the changes. Done right, through early engagement, these people become change champions for the program and its initiatives. They become critical to any program leader's efforts in building vertical and horizontal momentum and buy-in.
As demonstrated by the program's decision to identify a CCO, which highlights the need to bring the ârightâ talent to bear at the ârightâ time, an additional key lesson learned was that it is necessary to dedicate time and resources early on when planning change initiatives. If program team members and internal stakeholders are fully burdened with keeping their respective routine operations and present processes running during the planning phase, they will also likely not have the focus and energy needed to commit to or participate in implementing any change initiatives effectively. Thus, from a risk mitigation perspective, the program learned it was critical to have a dedicated civil servant point of contact for each initiative, who is held accountable for results and armed with the required resources. This includes ensuring that there is a ârealisticâ plan for communications and user engagement, be it timely decisions from internal stakeholders or obtaining user-feedback in a timely manner to inform programmatic improvements.
Often overlooked in the importance in planning is allowing for periods of reflection both during and after significant change initiatives. Perhaps the most important reason is to allow time to follow up with those affected by the change and see what worked really well, what worked but was not necessarily an improvement over the old, and what did not work as planned. For critical items that did not work as planned, corrections should be made as soon as practical, but for noncritical items that did not work or for things that just did not provide the level of improvement expected, time should be spent conducting the necessary analysis to understand what could be done better before making any additional improvements.
In addition, some initiatives will simply take a longer period of time before they will demonstrate results, and this needs to be accounted for before thinking about making additional changes. For example, for the NASA SBIR/STTR program, improved outcomes related to program participation by underrepresented groups could take 3 to 5 years to assess given program outreach and proposal schedules, and commercialization success may take anywhere from 5 to over 7 years to assess impact given the time required to commercialize new products. However, some efforts can be more easily assessed in the near-term, such as the impact of industry day and the RFI based on internal and external customer feedback. In each of these cases, it is important to reflect, identify, apply lessons learned, and, not to be forgotten, celebrateâboth successes and failures.
Finally, undertaking transformational change is hard. Building organizational and program resilience and the capability to undertake change on top of running and maturing routine operations takes time and involves experimentation and learning from mistakes. Expecting zero failures in running change initiatives or expecting excellent results consistently across all program initiatives (uninformed by priority) is a recipe for disaster, that is, high risk of failing to deliver on mission-critical routine operations and/or creating a breeding ground for wide-spread employee burnout or disillusionment.
In addition to the need for reflection time described earlier, recovery time is also necessary for improvements and new processes to take hold and mature. Taking a pause to let changes settle is also important to allow customers to feel comfortable with changes; if how a program interacts with them is constantly changing, it can adversely impact the customer relationships and make the prevalent perception that âdoing business with the government is challengingâ a reality. In addition, one of the most important reasons for taking time between periods of significant change is to combat change fatigue and inertia to allow for some down time to provide those planning and implementing change some much needed time to rest, reflect, and ready themselves for the next wave of inevitable change.
Conclusion
No government sector organization should undertake transformational change for the sake of change, even with the availability of budget or the right team of guiding leadership in place. Without identifying an underlying data-driven driver (case for change) for undertaking transformational change or securing program commitment toward the results envisioned from that change, the organization will set itself up for yet another statistic of âfailedâ change in public sector.
That being said, if there is an accepted driver to undertake change, if the organization has visionary and inspired leadership to sponsor and fund the change, and if a committed program team is ready to take advantage of the change, not seizing the opportunity would be a significant loss, especially for a program such as NASA's SBIR/STTR program, which prides itself on a result and mission-first orientation. As described earlier in the article, the fact is that opportunities for this type of transformational change may only present themselves to government agencies once in 10 years, and that is with some luck.
However, having a solid business case for change, momentum presented by a visionary and committed leader, and adequate funding or resources, can also be insufficient by itself for an organization to experiment with and drive transformational change. Fully capitalizing and taking advantage of this confluence of success enablers also require an organization to have a certain type of organizational culture or DNA to be presentâone that is resilient, promotes a âcan-doâ mentality, and a result orientationâas NASA SBIR/STTR program learned in undertaking the types of change initiatives described in this program.
So, Where to Next?
The program is now building on what has been a largely outward focus of engaging stakeholders to now making the necessary investments within the internal organization to strengthen the organizational structure as well as the program's innovation and business-friendly culture, all primed by the initial set of initiatives described by this article. Finally, every initiative described in this article was built or refined to contribute directly to the program's strategic vision, framework, and performance scorecard. New initiatives will be similarly guided by this framework up until it is beneficial to re-evaluate and refine the program's strategic framework and goals.
The program team also continues to strengthen the daily embodiment of program valuesâcollaboration, transparency, and customer centricity. The team is also hiring its first CCO. This guiding focus will continue to make sure that a customer perspective is applied across program strategy and execution efforts. Most importantly, it will set the stage for the program to successfully undertake the next wave of business transformation change while firmly ingraining business friendliness into the DNA of the program.
Footnotes
Acknowledgments
There were more than 100 NASA, REI Systems, and other support contractor employees, in addition to the authors, that made the initiatives in this article a reality. The program is indebted to these individuals' leadership, creativity, diligence, and passion in creating an increased focus on customers and impact: Chandu Adabala, Lindsay Aitchison, Kenneth Albright, Suneel Allareddy, Tahani Amer, Sujatha Ammanabrolu, Teresa Anthony, Kimberly Augone, Christy Avila, Khuram Batt, Vinay Beeravolu, Michael Benning, Benjamin Benvenutti, Tigisty Berhe, Dean Bitler, Quenton Bonds, Jacqueline Bourgeois, Derek Bramble, Maxwell Briggs, Steven Brockway, Emily Brown, Alexandra Budz, Sally Cahill, Kim Cannon, Victor Carlstrom, Daniel Carpenter, Camilo Castillo, Nithesh Chauhan, Salomea Chereysky, Christopher Christie, Bruce Cogan, Dale Corey, Gray Creech, Tony Damian, Mark Davidson, Mark Davis, Matthew Deans, Glenn Delgado, Julie Delgado, Jason Derleth, Prasun N. Desai, Geetha Dholakia, Kaushik Dighe, Racheal Down, Michael Dudley, Prashanth Duttuluri, Lavanya Elluri, Jennifer Eng, Joe Famiglietti, Tom Fazio, Robert Fisher, Rosa Fletcher, Virginia Foley, Richard T. French, Srikantha Ganti, Lynn Garrison, Parminder Ghuman, Bhaskar Gogoi, Doug Goodman, Joseph Grant, Mike Green, Deborah Gresko, Therese Griebel, Kapil Grover, Jianwen (Jeff) Guo, Deepak Gupta, Christy Hernandez, James Hibbs, Morris Hicks, Rafat Hossain, Priya Jain, Gwen Jasper, Robert Jones, Jitendra Joshi, Stephen G. Jurczyk, Tabisa Kalisa, Bill Killam, Raymond Kim, David Kirkpatrick, Lk Kubendran, Sandeep Kumar, Tina Marie Landes, Judah Lawrence, Magda Lech, Carol Lewis, Bailey Light, Laxman Lopchan, Max Lustyan, Eve Lyon, Samidha Manu, Robert McWilliams, Marie Metzger, Jamani Mohanta, Michael Mooney, Kasey Morrow, Heather Morgan, Diana Munoz, Sasikaran Muthuchamy, Nirmala Devi Nallasamy, Keerti Nandihalli, Maria Nazari, Eileen Nelson, Raju Niraula, Hung Nguyen, Janelle Nguyen, Mau V. Nguyen, Erin Nomiyana, Carita O'Rourke, David Oubre, Kathy Packard, David Pendell, Denice M. Philipps, Rich Pisarski, Dila Pokhrel, Latessa Poole, Cheryl Quinn, Mohammed Rahman, Jim Reuter, Helen Roberson, Gabriela Rodriguez, Voleak Roeum, Libby Romaguera, Alison Rosenfeld, Dina Salazar, Paul Schomburg, Mike Seablom, Saurabh Sharma, Andre Sheppard, Saurabh Sharma, Brijinder Singh, Karanjeet Singh, Ramsey Smith, Brian Stanford, Theresa Stanley, Tom Stanley, Zbigniew (Zak) Staszkiewicz, Mukesh Kumar Sunder, Florence Tan, Damian Taylor, Santosh Terala, Rich Terrile, Cory Thomas, Thomas Tighe, William Toscano, Nguyen Trang, Ramona Travis, Mike Vicory, Mike Vinje, David Vivian, James Whittington, Cassandra Williams, Juanita Williams, Keith Woodman, Bruce Yost, and Roman Zadorov.
Author Disclosure Statement
No competing financial interests exist.