Abstract
The introduction of electronic referrals in Norway started in 1996, when the first standard for an electronic referral was developed. Subsequent implementations of electronic health record (EHR) systems have been tested by a national testing and approval service. Almost all the Norwegian EHR systems comply with the recommended standards, but the volume of electronic referrals remains low: only 8200 referrals were sent electronically in September 2007. In comparison, 125,000 electronic discharge summaries were sent during the same month. There are four elements that need to be in place before smooth communication can occur between the EHRs of the hospitals and the general practitioners. First, the regional health authorities must have a strategy for electronic cooperation. Second, various technical problems must be solved. Third, certain organizational problems must be addressed. Finally, electronic referrals should be used to support enhanced work processes and joint reception for referrals should be considered at larger hospitals.
Introduction
Electronic referrals are requests for medical examination and evaluation by a specialist, outpatient clinic or a hospital. The referral can be sent from a general practitioner (GP), specialist or from a hospital (Figure 1). The referral transfers the responsibility for further treatment of the patient either completely or partly.

Communication between GP and hospital
In Norway, GPs send 1.9 million referrals to specialists or hospitals each year. It has been calculated that doctors in Norway annually spend 150 man-years on administrative work related to referrals. 1 Patient flow in Norwegian health care depends on the referrals. It is therefore important that the referrals can be written, transferred and handled effectively.
In Norway there are five Regional Health Authorities (RHAs). The RHAs are funded according to the type and volume of the services they provide to patients. Each RHA is responsible for the hospitals in their region. There are almost 80 hospitals in Norway and all have electronic health records (EHRs). In order to reduce the workload related to ICT at each hospital, the RHAs have established an organization (National ICT or NICT) that provides limited funding for projects that can support all the RHAs. The Ministry of Health instructs the RHAs regarding requirements that should be fulfilled annually. One of these requirements is that the RHAs are obliged to provide services for sending and receiving discharge summaries and referrals.
Norwegian GPs have had access to EHR systems for more than a decade. The EHR systems are used as tools to support the GPs' work processes. The ELIN project (Electronic Information Exchange) was established by the Norwegian Medical Association. The main purpose is to ensure that GPs are provided with electronic solutions to ease shared care with other organizations. The GPs require that electronic messages must be sent via modules that are integrated with their EHR system. They also require that national standards must be supported and that the national cooperation architecture must be used (Figure 2). Innovation Norway has partly funded the vendors of EHR systems to implement the necessary standards and develop new functionality in the EHR systems. This has been an important factor in making communication solutions from all vendors available for the GPs. The standards have been developed by the Norwegian Centre for Informatics in Health and Social Care (KITH) based on the user's requirements from the ELIN project. All the vendors have implemented modules to support exchange of electronic laboratory results, discharge summaries and referrals.

Cooperation architecture
KITH has developed the national cooperation architecture in cooperation with the RHAs, the Directorate of Health and Welfare and the ELIN project. The basic requirements for this architecture are:
all messaging traffic should use the national broadband network, the Norwegian Health Net; only standardized messages should be used; the vendor's message implementations should be approved by the Norwegian Testing and Approval Service at KITH; the ebXML framework should be used; application receipts should be sent for all messages.
The first version of the cooperation architecture is based on messaging, but an extended version of the architecture will be used for web services.
The network is also available for most hospitals and GPs, and it is already used for exchange of laboratory messages and discharge summaries. One could expect that the ELIN project and the efforts from the Ministry of Health and the RHAs would contribute to quick deployment of electronic referrals. In fact only 8200 referrals were sent electronically in September 2007 and 6300 of them were within the RHA of North Norway. In comparison, during the same month, 125,000 electronic discharge summaries were sent.
RHA meetings
Early in 2006, NICT contacted the the ELIN project staff to decide what the main challenges for the RHAs were, and to help them to develop plans for deploying electronic referrals and other messages. This was part of NICT's annual action plan. 2 A series of 10 meetings with the RHAs was held. The findings are described below.
The organizers for the first meetings were the representatives of each RHA. The representatives are senior managers for ICT, and they are members of the RHA's management group. Both technical staff and clinicians were present at the meetings, although fewer clinicians were involved than the ELIN project had requested. A person from the Norwegian Centre for Telemedicine, with experience from regional referral projects in North Norway, was also present at some of the meetings in addition to representatives from KITH and ELIN.
Strategy for deploying electronic cooperation
All the RHAs seemed to be well informed about the requirements from the Ministry of Health, and they had taken steps to develop a plan for deploying the messages and communication modules. However, many of the RHAs did not have a strategy for electronic cooperation in general. Even though the managers knew the requirements of the Ministry of Health, many of the people who were supposed to implement the strategy at an operational level were not familiar with it. It seemed clear that top level commitment was necessary, although this was not enough to ensure that the messages were implemented and used. Most of the RHAs also have very restricted budgets. Activities that do not lead to obvious and rapid economic benefits for the hospital seemed to be given low priority.
Strategies in relation to the national plans should be developed in cooperation with the users, but also the IT workers who are going to implement plans and strategies. The RHAs must allocate sufficient resources, and the work must be given priority according to the requirements of the Ministry of Health.
Responsibility for deployment
Within RHAs, the responsibility for implementation of the communication strategy is assigned to several people at a hospital level. The reason for this is partly historical, because many of the larger hospitals have a long tradition of selecting and/or developing their own ICT-solutions. One person should be given the overall responsibility for implementing the communication strategy within each RHA. A team with members from the hospitals, representatives from GPs and community care should be established to support the process. The team members should preferably have mixed competence regarding the different information systems, e.g. EHRs, laboratory systems and PACS. Clinicians should also be included in the team.
Work processes
The introduction of electronic referrals in a large organization such as a hospital can be a major challenge. The hospitals have routines to handle the referrals that are sent by paper from GPs, specialists and other hospitals. These routines are fairly good, but it will sometimes take weeks before the person who sent the referral will obtain information about the patient's scheduled appointment in secondary care. Quite often additional information about the patient's health status and previous treatments will be requested by telephone or letter before a decision can be made about further treatment. Sometimes the referrals get delayed because they are sent to the wrong address. The requested doctor could be on leave, or the GP might have misunderstood which ward the referral should have been sent to.
Some hospitals can probably benefit from establishing a reception service where referrals are handled before they are sent to the specialist who will have the responsibility for further treatment of the patient. The reception service can request missing information from the GP, make sure that the referral is sent to a specialist with adequate competence for the task that needs to be performed, and follow-up so that information about scheduled appointment is sent back to the GP and patient.
It is likely that the limited number of electronic referrals compared to the number of electronic discharge summaries can be linked to the complexity of the cooperation process between the GP and the specialist to establish a ‘good’ referral. Further steps should be taken to address this matter and suggestions made about how this process can be improved.
Practice consultants
There were complaints from the RHAs about referrals from the GPs that included too much information. The referrals also did not satisfy the specialist's needs at the hospital.
Many hospitals already benefit from the use of practice consultants to improve cooperation. The practice consultants are GPs who are employed in part-time positions at the hospitals. They try to improve procedures to support the work processes for both GPs and specialists. A typical task would be to look at the procedures for production of electronic referrals and discharge summaries, and initiate processes to reach agreement about the structure and content of these documents. The needs of GPs for information in the discharge summary often differ from the needs of hospital doctors, for example. The GP needs the document to make sure that he or she can provide the patient with a proper treatment plan. The specialists at the hospital, on the other hand, find the discharge summary useful as a means of obtaining access to summarized information when the patient is readmitted to the hospital.
Automatic production of discharge summaries from existing EHR documentation is also possible from some EHR systems. If this is not done with care, the result can be documentation that is too voluminous for the next care provider in the treatment chain. This can be seen as a parallel to the specialist's complaints about voluminous referrals. In both cases a common understanding of each others' needs is necessary in order to agree how much information should be included.
Knowledge
During the meetings, people from the hospitals could ask questions directly of those who had been involved in other implementation projects, contact with the vendors, and testing and development of the standards. Knowledge from the resource person could be shared with those who were initiating new projects.
The Norwegian Centre for Telemedicine (NST) has managed two regional electronic referral projects. The Centre has gained valuable experience and knowledge that can be shared with other projects. There have also been several other referral projects, and it would be useful for others to know more about their experiences. Establishing meeting arenas for ongoing and planned referral projects should be considered. Web pages with easily accessible information and communication channels to a resource person should also be provided. Currently KITH has an information service available for those who implement standardized messages and NST has also been instructed by the Ministry of Health to assist in implementation projects. However, this is not sufficient.
Technical problems
There were complaints about the Norwegian Health Net. Some complaints seemed to be due to a lack of understanding about the role of Health Net. The Norwegian Health Net mainly provides a network infrastructure and some basic services. The customers have to provide content to the network and this is also a responsibility for the RHAs. The complaints also indicated that Health Net did not satisfy the customer's expectations regarding response times for handling requests and downtime. Health Net should therefore provide better and more easily accessible information about their services. This includes both their web pages and direct contact with the RHAs. In addition, technical problems need to be solved more quickly.
Testing and approval service
The national Testing and Approval Service has so far been used to test the vendor's implementation of messaging standards. The experience of the RHAs is that even if the implementation of the standard is tested separately at the endpoints, it does not necessarily mean that communication in the treatment chain with many people involved will run smoothly. For example, a table that is included in a message by a specialist at the hospital, might be unreadable when it is shown in the EHR system at the GP's office.
The Testing and Approval Service should be extended to include the whole cooperation chain. How this can be done remains to be established, but regional test environments at the RHAs might be used in cooperation with vendors' test sites, the test-server at KITH and the testing facility at the Norwegian EHR Research Centre (NSEP).
Outdated EHR systems
The RHAs claimed that outdated versions of the EHR systems at GPs' offices represented a problem. The GP EHR systems have been tested through the National Testing and Approval service, but if the GPs do not install the newest version of the software, there will still be problems with communication. The number of offices that used outdated versions was not quantified, but it was claimed that it was substantial.
One means to help solve this problem would be contracts between the hospitals and GPs. The GPs would then have to commit themselves to upgrading to new versions if they want to communicate with the hospital. Some of the hospitals are reluctant to do this, because they fear that they may lose customers to competing companies, such as private laboratories.
Conclusion
There are four elements that need to be in place before communication between the hospitals' and the GPs' EHR systems can run smoothly. First, the RHAs must have a strategy for electronic cooperation, the responsibility for implementing the strategy must be given to one person and a supporting team should be established. Second, technical problems regarding availability to the Norwegian Health Net, EHR system version and testing need to be solved. More attention should be paid to the fact that the number of electronic referrals is so much lower than the number of discharge summaries. Third, organizational issues must be addressed, and the GPs must be included in planning organizational change. Practice consultants can be used. Finally, new solutions for electronic referrals should also be used to support enhanced work processes, and joint reception for referrals should be considered at larger hospitals.
