The Association Between Hospital Characteristics and Nonresponse in an Organization Survey: An Analysis of the National Healthcare Establishment and Workforce Survey in Malaysia

Abstract

In any survey where some of the invited participants fail to respond estimates may be biased. The literature on survey nonresponse is substantial, and the intellectual focus has typically been on the nonresponse of individuals. An important yet less scrutinized area in the analysis of nonresponse is in organizational surveys, particularly surveys of health-care organizations. This study used data from the 2010 National Healthcare Establishment and Workforce Survey in Malaysia to examine the relationship between a set of measurable hospital attributes and their probability of survey response and the relationship between this probability and the differences in survey estimates. We found that readily measurable hospital characteristics such as size and geographical location are useful predictors of survey response likelihood. Larger hospitals and hospitals located in less developed geographical regions responded more favorably than their counterparts. We have also illustrated that the resulting response pattern affected some key survey estimates. These findings have the potential to extend our understanding of nonresponse to organization surveys in the health-care sector, potentially allow for the prediction of nonresponse, and help researchers to identify profiles of “reluctant responders” before a survey commences, so that additional engagement strategies may be used.

Keywords

hospital Malaysia survey nonresponse health-care organization organization survey

In any survey where some of the invited participants fail to respond estimates may be biased. The bias occurs because of systematic differences between those who did and those who did not respond (Draugalis & Plaza, 2009; Halbesleben & Whitman, 2013; Snijkers, 2013). This is an important and challenging issue in the field of survey research because biased estimates can have undesirable consequences; overcoming nonresponse has proven difficult (Pratt, Sellon, Spencer, Johnson, & Righter, 2012). While survey response rates remain a common measure of survey quality, nonresponse bias should also be the focus of survey quality assessment (Johnson & Wislar, 2012; Loft, Murphy, & Hill, 2015). The risks associated with nonresponse bias are particularly critical in the health-care sector, where policy decision can affect patient management (Abel, Saunders, & Lyratzopoulos, 2016; Rinne et al., 2015).

The literature on survey nonresponse is substantial, with an intellectual focus generally focused on the nonresponse of individuals such as survey of patients (Harkanen, Kaikkonen, Virtala, & Koskinen, 2014; Multone et al., 2015) and physicians (Riedl, Gower, & Chrvala, 2011; Steffen et al., 2015). There is a class of surveys, however, that seek a response from organizations, for example, the National Health Care Surveys (www.cdc.gov/nchs/dhcs/) and the AHA Annual Hospital Survey (http://www.ahasurvey.org). Health-care organizations, such as the hospitals, are the environment within which patients and providers interact. Important decisions on clinical care and management are often made within this context (Loft et al., 2015). Access to high quality, organizational information is important for health service planners and policy makers to support their decision-making (Klabunde et al., 2012). Surveying organization has been regarded as an effective way to obtain this critical organizational information (Klabunde et al., 2012).

Recent developments in the field of organizational surveys in health care have offered valuable insights into improving our understanding of the problem of nonresponse bias (Digaetano, 2013; Dykema, Jones, Piche, & Stevenson, 2013; Lewis, Hardy, & Snaith, 2013). However, the investigation of organizational nonresponse bias in these settings remains nascent (Lewis et al., 2013; Loft et al., 2015). Recent evidence suggests that those methods commonly employed to manage nonresponse in surveys of individual respondents are not as useful when applied to surveys of organizations (Lewis et al., 2013). This suggests that the mechanism of organizational nonresponse unlike that of individuals and, therefore, needs further research to tease out the effects of complex relationships between the surveyors and the target organizations.

Significant differences exist between surveys of organizations and surveys of individuals. Respondents in organizational surveys are not reporting personal behaviors and opinions but facts and practices of the organization they represent (Loft et al., 2015). This difference brings into play additional considerations above and beyond those in traditional surveys of individual respondents (Loft et al., 2015; Tomaskovic-Devey, Leiter, & Thompson, 1994; Willimack & Nichols, 2010). Organizations can have complex structures and internal processes, and these structures and processes can vary from organization to organization. To collect organizational data successfully, therefore, a survey team needs to know about the complexities of each organization and be able to navigate those complexities in order to locate informants who possess appropriate knowledge about the subject matter or have access to the relevant information. A common navigational challenge in surveying organizations is the existence of “gatekeeper.” Gatekeepers are persons who stand in between the data collector(s) and the potential respondent(s) within the organization (Keesling, 2008). A gatekeeper can take many forms, including security personals, secretaries, administrative assistants, and office managers. They can control access to the target informants because of their relationship with the informants. Survey researchers need to be able to negotiate with them to achieve their survey objectives. Individual and household surveys do not suffer from this kind of complexity and understanding these differences is critical to the outcomes of survey (non)response (Biemar et al., 1993).

Hospital Characteristics and Survey Response Model

Hospitals have fundamental characteristics that may affect response rates. Size and geographical location are readily measurable characteristics that may affect response rates. There are also less obvious aspects of organizational culture that may affect nonresponse. Tomaskovic-Devey, Leiter, and Thompson (1994) proposed a theoretical model for thinking about the likelihood of nonresponse in organization surveys. The model consists of three interacting components: (i) the authority to respond, (ii) the capacity to respond, and (iii) the motivation to respond.

Organizational characteristics that influence the authority, capacity, and motivation to respond will vary from industry to industry. In their cross-industry analysis, Tomaskovic-Devey et al. (1994) found that the capacity to respond declined as organizational size increased. They speculated that this may have arisen because of a greater propensity for information dispersal in larger organizations (Tomaskovic-Devey et al., 1994). There is some evidence in the health-care sector, however, that the size-response effect may not always operate in one direction. The American Hospital Association (AHA) Annual Survey, for example, showed a better response rate from larger hospitals (“AHA Annual Survey Database,” 2013; Mullner, Levy, Matthews, & Byre, 1981). This may be because larger hospitals invested more heavily in health information management systems and therefore had a greater capacity to respond (Jha et al., 2009). A good hospital information system (HIS), for instance, will support requests for detailed statistical information about hospital outputs such as the number of discharges across diagnostic groups. Whether the informant has the authority to access the HIS, or whether the organization is motivated to respond interact to affect the nonresponse rate.

A health-care organization’s capacity to respond will also be affected by the nature of the surveys (Tomaskovic-Devey et al., 1994). For instance, a single informant of a large hospital may have difficulty providing information on the technical approach of delivering a particular type of care if there are differences in clinical practice across subunits within the hospital. However, such an informant may have no difficulties determining the volume of various clinical procedures performed within the hospital. This form of information has been increasingly sought after to help policy makers and industry planners to assess the development of the sector as well as to examine the changes resulting from changes in sector policies and incentives (Loft et al., 2015).

The practical challenges of the Tomaskovic-Devey approach are (a) one of measurement, and (b) one of determining the magnitude of the bias of a Tomaskovic-Devey characteristic on survey estimates. The constructs of authority, capacity, and motivation are important for understanding why an organization does or does not respond; but how, for instance, would one measure the motivation of a nonresponding organization? They may be highly motivated but have no capacity, and simply inferring motivation from nonresponse would not work. Furthermore, knowing about motivation or capacity does not provide any insight into the extent to which the characteristic biases any estimates.

An alternative empirical approach looks at the associations between nonresponse and readily measurable, structural characteristics of an organization. This empirical approach has the potential to extend our understanding of nonresponse to organization surveys in the health-care sector, potentially allowing for the prediction of nonresponse, and helping researchers to identify profiles of “reluctant responders” before a survey commences. Armed with this additional insight specific strategies can be targeted to organizations at high risk of nonresponse to try and overcome the bias. In addition, one can investigate the relationship between the likelihood of nonresponse and survey estimates to gauge the likely direction and magnitude of the potential biases.

To investigate the potential of this approach, we leverage on the Tomaskovic-Devey model and used data from the 2010 National Healthcare Establishment and Workforce Survey (NHEWS) conducted in Malaysia to examine the relationship between a set of measurable hospital attributes and the subsequent probability of a hospital to respond to survey requests. The hypothesized relationship is tabulated in Table 1. We then examined the relationship between the probability of nonresponse and differences in estimates of key hospital outcomes.

Table 1.
Hypothesized Relationships Between Hospital Characteristics and Survey Response Outcomes.

Hospital Characteristics Hypothesized Relation to Survey Response Theoretical Basis

Hospital size Larger hospital is related to higher response tendency Larger hospitals often have better use of information technology, belong to one of the major health-care groups and are members of the private hospital organization, together this lead to higher capacity and motivation to respond; hospital type, being closely related to the establishment size, will have a positive but likely nonindependent association with response propensity as described above (Tomaskovic-Devey et al., 1994)

Hospital type Hospital with wider scope of clinical services nonindependently associated with better response

Hospital location Regional differences of survey response tendency exist Geographical location of a hospital also is hypothesized to play an independent role in determining the response tendency of the organization due to a different perception of survey request and level of cooperation among different part of the country. However, we lack the knowledge to develop a priori hypothesis for the direction of their effect

Ownership type Corporate owned and NGO hospitals are more likely to respond to survey request The distance to a decision maker from the institution gatekeeper in a corporate-owned (CB) and nongovernment organization (NGO) hospital as well as a multiestablishment (as compared to single establishment) hospitals is likely longer compared to sole proprietorship/partnership type hospitals, thus CB/NGO and multiestablishment hospitals likely will have a negative association with response propensity (Tomaskovic-Devey et al., 1994)

Multiestablishment firm Multiestablishment hospitals have lower survey response tendency

Organizational age Older hospital respond better to surveys Higher establishment age, in general, signifies that they have “survived” longer in the industry and thus likely to be less environmentally dependent and perceive to have lesser “threat” due to competition. They also may have a more stable group of staff, thus more willing to cooperate and easier to be followed up during the survey; we, therefore, hypothesize that there is a positive correlation with response propensity

Corporation group Difference corporate group has different survey response tendency Hospitals of various management groups may have different management practice and corporate policy toward responding to external surveys, thus respond to survey requests differently. However, the direction of the effect is unknown without advanced knowledge on their management and policy differences. We hence hypothesized that they may differ in their propensity to respond but do not specify the direction a priori (Tomaskovic-Devey et al., 1994)

Method

Contextual Consideration

Malaysia is an upper middle-income country situated in Southeast Asia. It has a population size of 29.9 million (2014) and a per capita gross national income of US$11,120 (World Development Indicators, 2014). Geographically, it is separated by the South China Sea into Peninsular (West) Malaysia and East Malaysia (EM; the two states on the island of Borneo). The peninsular is conventionally divided into four socioeconomic regions: the Klang Valley (KV; central region of the peninsula) where the capital (Kuala Lumpur) is located is the most developed region; the Northern and Southern West Coast (SWC) which is the second most developed region; and the East Coast (EC) which is the least developed region of the peninsular. In this study, we considered the EM and EC of peninsular Malaysia to be economically similar.

The health-care system in Malaysia is a dual, nonintegrated public and private system (Jaafar, Noh, Muttalib, Othman, & Healy, 2013). The public sector consists of hospitals and other facilities operated by the Ministry of Health, Ministry of Education, and Ministry of Defence. The private sector operates on a fee for service model with some specific services (e.g., dialysis treatment) conditionally subsidized by the Ministry of Health. This study focuses on the private sector only because the response rate from government hospitals was 100%.

Data Source and Survey Procedure

The NHEWS is a provider-based survey. It is part of the National Healthcare Statistics Initiative—an effort led by the Ministry of Health Malaysia (MOH) to make information of health-care resources and utilization readily available. Details of the study framework, methodology, and related reports are accessible online (National Healthcare Statistics Initiative [NHSI], 2008–2013). The National Medical Research Ethics Committee approved the study protocol (NMRR-09-842-4718) and the Malaysian Research Grant funded the project. The National Clinical Research Centre was the project management center for this survey.

The sampling frame included all 203 private hospitals in Malaysia. Of the eligible hospitals, 121 (60%) responded completely to the main survey. We defined complete response as a submission of data from an eligible establishment with no missing elements for all predefined key survey estimates. The key survey estimates were number of functioning inpatient beds, the number of inpatient admissions, the average length of stay, turnover interval, and availability of medical technologies such as computed tomography scanner and magnetic resonance imaging unit, the number and details of the clinical workforce including physicians, nurses, and pharmacists.

The sampling frame was established by aggregating two data sources: The Medical Practice Division (MPD) Private Hospital Registration and Licensing Database and the previous year’s NHEWS hospital list. MPD is the MOH licensing body for the private health-care facilities. All private hospitals in Malaysia are required by law to license their facility with the MPD. Information on the facilities and services are updated biennially when these facilities renew their licenses.

Unmatched records from these sources were verified by contacting each hospital to confirm their operational status. Duplicates of establishments were removed prior to the survey. Eligibility criteria were given in the NHEWS 2010 report (available online; NHSI, 2008–2013). We used a set of disposition codes to track the outcomes of each hospital similar to that suggested by the American Association for Public Opinion Research ( 2016).

The NHEWS 2010 was divided into eight sections: general hospital information, maternity, paediatric, surgical, emergency and trauma, anesthesiology, oncology, and psychiatry services. Organizational participation in the survey was voluntary. The project team invited all hospitals to a briefing session in each state. The survey forms were distributed to those who attended the session. Absentees were posted the surveys with complete instructions and followed up by phone.

Each hospital was followed up by trained research officers every 2–3 weeks until the project team confirmed their response status (agreed or refused to participate). Hospitals submitted their data by post, fax or email of the scanned questionnaire, or via a web-based application. Definition of data elements was standardized and provided with the survey forms. Details of data quality are available (and downloadable) for each published technical report (NHSI, 2008–2013).

Statistical Analysis

The analysis was conducted in two stages. In the first stage, a nonresponse risk model was developed to estimate the probability of nonresponse conditional on a set of predictors (i.e., hospital characteristics). In the second stage, responding hospitals were stratified by the nonresponse risk model into four strata of nonresponse risk, and differences in key hospital outcomes estimated. If nonresponse was a random occurrence, one might expect to see no difference between hospitals with high and low probabilities of nonresponse.

Variables

A dichotomous outcome variable was used to represent the response (1) or nonresponse (0) of a hospital to the survey. The response was operationalized as a complete survey (as described above). Anything else such as a partial response or refusal was coded as a nonresponse.

The predictors of response/nonresponse were based on an initial pool of 10 hospital attributes, later reduced to 7 attributes (Table 2) to remove issues of multicollinearity, and some continuous variables were recoded as ordered categories. The predictors are described below.

Table 2.
Characteristics of the Hospital Cohort and the Differences Between the Respondents and the Nonrespondents.

Hospital Characteristics All Hospitals Respondents Nonrespondents Bivariate Analysis

N = 203 N = 120 N = 83 Coefficient (SE)^a

Geographical region, n (%)

East Malaysia (EC) 17 (8.4) 15 (12.5) 2 (2.4)* —

East Coast (EM)^b 11 (5.4) 10 (8.3) 1 (1.2)

Klang Valley (KV) 85 (41.9) 41 (34.2) 44 (53.0) −2.19 (0.65)^*

Northern West Coast (NWC) 48 (23.6) 30 (25.0) 18 (21.7) −1.61 (0.68)^*

Southern West Coast (SWC) 42 (20.7) 24 (20.0) 18 (21.7) −1.83 (0.69)^*

Ownership, n (%)

Corporate body/ NGO 174 (85.7) 107 (89.2) 67 (80.7) –

Sole proprietor/ partnership 29 (14.3) 13 (10.8) 16 (19.3) -0.68 (0.40)

Hospital type, n (%)

Maternity center^c 73 (36.0) 35 (29.2) 39 (47.0)* –

Secondary hospital^d 80 (39.4) 41 (34.2) 38 (45.8) 0.24 (0.32)

Tertiary hospital^e 50 (24.6) 44 (36.7) 6 (7.2) 2.13 (0.49)^*

Hospital age (year), n (%)

0–5 Years 44 (21.7) 21 (17.5) 23 (27.7) –

6–15 Years 74 (36.5) 44 (36.7) 30 (36.1) 0.47 (0.38)

16–25 Years 47 (23.2) 30 (25.0) 17 (20.5) 0.66 (0.42)

25–150 Years 38 (18.7) 25 (20.8) 13 (15.7) 0.74 (0.46)

Hospital corporate group, n (%)

Independent 165 (81.3) 86 (71.7) 79 (95.2)* −0.83 (0.85)

Group A 7 (3.4) 5 (4.2) 2 (2.4) –

Group B 20 (9.9) 19 (15.8) 1 (1.2) 2.02 (1.32)

Group C 11 (5.4) 10 (8.3) 1 (1.2) 1.39 (1.34)

Independent hospital, n (%)

Yes 165 (81.3) 86 (71.7) 79 (95.2)* –

No 38 (18.7) 34 (28.3) 4 (4.8) −2.06 (0.55)^*

2.89 (0.76)*

Hospital size (by number of bed), n (%)

1–25 Beds 107 (52.7) 45 (37.5) 62 (74.7)* –

25–150 Beds 68 (33.5) 49 (40.8) 19 (22.9) 1.27 (0.33)^*

>150 Beds 28 (13.8) 26 (21.7) 2 (2.4) 2.89 (0.76)^*

Note. SE = standard error; NGO = nongovernmental organization.

^aLogistic regression. ^bIt represents East Coast/East Malaysia (ECM) in bivariate analysis. ^cMaternity centers are private hospitals that only provide obstetric or delivery services. ^dSecondary hospitals refer to those that have any of the six basic specialty services (general medicine and surgery, emergency care, paediatric, general orthopedics, and obstetrics and gynecology); single specialty hospitals such as heart centers are grouped under this category. ^eTertiary hospitals refer to those that have more than the six basic specialty services. For examples, hospitals that provide oncology services in addition to the six basic specialties.

p < .05 by Fisher’s exact test.

Geographic region

KV, Northern West Coast (NWC), SWC, EC, and EM. These divisions represent the variation of social–economic difference within the country as described earlier.

Hospital ownership

Corporate body/nongovernmental organization (NGO), sole proprietor/partnership

Both corporate bodies and NGOs are registered as limited liability companies in Malaysia. NGO hospitals may have different organizational attributes from the for-profit hospitals, but their numbers were too small for any independent analysis.

Hospital type (per type of care provided)

Maternity centers, secondary hospitals, and tertiary hospitals

Maternity centers refer to a private hospital that only provides obstetric or delivery services. Secondary hospitals refer to those that have any of the six basic specialty services (general medicine and surgery, emergency care, paediatric, general orthopedics, and obstetrics and gynecology); single specialty hospitals such as heart centers are grouped under this category. Tertiary hospital refers to those that have more than the six basic specialty services. For examples, hospitals that provide oncology services in addition to the six basic specialties.
Hospital age as categories: 0–5 years, 6–15 years, 16–25 years, and 25–150 years.

Hospital corporate group owner (anonymized): Group A, Group B, Group C, and independent.

Hospital size as categories: 1–24 inpatient beds, 25–150 inpatient beds, and >150 inpatient beds. The number of inpatient beds and the number of physicians in a hospital was found to be highly correlated (r > .8), and it was decided to use the bed count to categorize hospital size.

Nonresponse Risk Model

Bivariate logistic regressions were first performed for each predictor. A multivariate logistic regression analysis was used to develop a final nonresponse risk model. We computed the response likelihood (RL) for each hospital by calculating the predicted probability of response using only the significant variables with a p ≤ .05 found in the multivariate analyses. Hospitals were stratified into five strata: Stratum-1 is hospitals with the highest likelihood to respond (between 0.8 and 1.0); stratum-5 with the lowest (0.0–0.2).

Relationship of Nonresponse Risk and Key Estimates

Eight NHEWS survey estimates (median) were compared by the RL strata. These were the number of inpatient admission, the average length of stay, bed occupancy rate, turnover interval, the number of CT and MRI scanning performed, and the number of staff nurses and pharmacists. Differences between estimates in each stratum were tested using nonparametric Kruskal–Wallis test. We adjusted our significant level using Hochberg Procedure to minimize Type I error (α value was prespecified at .05 prior to analyses). All statistical procedures were performed using R version 3.1.0 (2014-04-10).

Results

The NHEWS 2010 for private hospitals achieved a response rate of 59.6%. There were no missing data in the predictor variables. Table 2 shows the differences in respondent and nonrespondent hospital characteristics as well as the whole hospital population. Hospital location, type, corporate group, and hospital size appear to differ between the two groups but not ownership status and the establishment age.

In the bivariate analyses, hospitals located in more developed geographical regions (KV, NWC, and SWC) and hospitals categorized as “independent center” were negatively associated with RL. Tertiary hospitals and hospitals of larger sizes were associated with higher RL (Table 2). After adjustment (Table 3, Model 1), only the geographical location of the hospitals and their sizes remained associated with response status. We found no significant differences in response tendency between the three hospital corporate groups (Table 3, Model 2). A test for interaction between location and hospital size found no significant effect.

Table 3.
Multivariate Analysis of the Relationships Between Hospital Characteristics and Survey Response Tendency.

Predictors Coefficients (SE)

Model (1) Model (2)

Geographical region

East Coast and East Malaysia (ECM) — —

Klang Valley (KV) −2.56(0.7) −2.60(0.70)*

Northern West Coast (NWC) −1.94 (0.74)* −1.97(0.74)*

Southern West Coast (SWC) −2.11(0.74)* −2.14(0.75)*

Ownership

Corporate Body/NGO — —

Sole proprietor/partnership −0.25(0.53) −0.25(0.53)

Hospital age

0–5 Years — —

6–15 Years 0.34(0.46) 0.29(0.46)

16–25 Years 0.66(0.51) 0.60(0.51)

25–150 Years 0.19(0.56) 0.12(0.57)

Hospital type

Maternity center^a — —

Secondary hospital^b −0.67(0.50) −0.66(0.50)

Tertiary hospital^c 0.10(0.79) 0.08(0.79)

Hospital group

Independent — −0.33(0.96)

Group A — —

Group B — 1.18(1.40)

Group C — 0.95(1.43)

Independent hospital

No — —

Yes −1.03(0.65) —

Hospital size (number of bed)

1–25 Beds — —

25–150 Beds 1.25(0.53)* 1.25(0.53)*

>150 Beds 2.60(1.01)* 2.58(1.01)*

Note. NGO = nongovernmental organization; SE = standard error.

^aMaternity centers are private hospitals that only provide obstetric or delivery services. ^bSecondary hospitals refer to those that have any of the six basic specialty services (general medicine and surgery, emergency care, paediatric, general orthopedics, and obstetrics and gynecology); single specialty hospitals such as heart centers are grouped under this category. ^cTertiary hospitals refer to those that have more than the six basic specialty services. For examples, hospitals that provide oncology services in addition to the six basic specialties.

p < .05, logistic regression.

RL was computed using a model containing only the geographical region and hospital size (range: 0.0–1.0). No hospital scored between 0.0 and 0.2, thus only 4 strata were compared. Table 4 shows that all except average length of stay differ between strata. A higher RL appeared to have a higher median for all estimates except for turnover interval, where the pattern reversed.

Table 4.
Comparing Survey Estimates (Median) Across Strata of Response Likelihood.

Survey Variables Strata of Response Likelihood^a

Stratum-1 Stratum-2 Stratum-3 Stratum-4

Number of hospital (n) 51 19 38 12

Total number of inpatient admission 7,629 6,872 1,071 187

Average length of stay (days) 2.7 2.5 2.2 2.5

Bed occupancy rate (%) 62.0 58.1 40.8 15.5*

Turnover interval (days) 1.9 1.8 3.3 10.0*

Number of CT scanning performed 2,347 1,262 0 0*

Number of MRI performed 1401 468 0 0*

Number of staff nurse 117 52 10 5*

Number of pharmacist 2 1 0 0*

Note. CT: computed tomography; MRI = magnetic resonance imaging.

^aStratum-1 has the highest propensity to respond; Stratum-4 has the lowest.

*p < .05, Kruskal–Wallis test.

Discussion

The purpose of investigating nonresponse and its related biases is to improve the overall quality of survey research and ultimately achieve an unbiased conclusion based on the data (Klabunde et al., 2012). Because there are important differences between surveys of organizations and surveys of individuals, an investigation of the effect of nonresponse in organization surveys requires considerations beyond those of the individuals representing their organizations (Loft et al., 2015; Tomaskovic-Devey et al., 1994; Willimack & Nichols, 2010). In this study, we examined the association between hospital characteristics, RL, and the survey estimates to extend our understanding of nonresponse and its related biases to organization surveys in the health-care sector. Our analyses show that readily measurable hospital characteristics, such as size and geographical location are useful predictors of RL. Specifically, larger hospitals and hospitals located in less developed regions responded more favorably to the survey than their counterparts. We have also identified that the response pattern is related to key survey estimates.

The findings that larger hospitals are more likely to respond to the NHEWS may be a result of several factors that relate to the nature of the NHEWS and the private health-care sector in Malaysia. First, the NHEWS sought detailed quantitative information about hospital resources and activities not qualitative knowledge or opinions from hospital informants. Hospitals with greater quantitative information processing power are therefore more likely to be able to respond. Commonly, hospitals with greater quantitative information processing capability are often the larger ones and are more likely to be part of a major health-care group. This finding is consistent with previous observations (Mullner et al., 1981). Interestingly, studies conducted in nonhealth-care settings (Gupta, Shaw, & Delery, 2000; Phipps & Jones, 2007; Seiler, 2014; Tomaskovic-Devey et al., 1994) demonstrated an inverse relationship between organization size and RL, which contrast with our finding. The inverse relationship was interpreted by the earlier investigators to reflect the greater propensity for information dispersal and higher organizational complexities of larger organizations and by extension a lower capability and motivation to respond. Those findings, however, were based on weighted average effect across a wide range of industries. Our singular focus suggests that organization survey in the health-care industry may operate somewhat differently, and insights gained from other industries may not generalize. Additional research is required, however, to identify why there was a divergence of findings.

Second, we suspect that larger hospitals are perhaps easier for the survey teams to reach because they are more visible. In addition, they are more likely to be members of the private hospital association (Association Of Private Hospitals Malaysia, described earlier), and the survey team had previously engaged with APHM and obtained their prior endorsement. Thus, much of the resistance commonly experienced in surveying hospitals was likely mitigated. In addition, the 2010 NHEWS was conducted after a pilot implementation in 2009. Issues in navigating through the institutional gatekeepers and developing cooperative relationships were identified and managed in the full survey. Hence, it is likely that the negative impact of organizational complexity (argued in some earlier works; Gupta et al., 2000; Tomaskovic-Devey et al., 1994) was further mitigated.

Although it might be less relevant to our survey, structural factors affecting organizational survey response should not be overlooked. These include, whether the hospital has established protocols to channel survey request to the authorities, whether there exist dedicated human resources to respond to requests, the presence of favorable institutional policies for survey response, and the availability of incentives to encourage staff support. We believe that these kinds of structural factors will drive the RL by influencing the authority, capacity, and motivation of target organizations and their informants.

Private hospitals located in EC/East Malaysia (the less developed regions) are found to be more responsive to the NHEWS. The reason behind this is not immediately obvious. The possibilities include (1) the concentration of hospitals is lower in this region and relatively more visible to the survey team and (2) they may have a greater willingness to cooperate because they operate in a less competitive commercial environment. These reasons are largely speculative, however, our findings do indicate the importance of taking potential geographical differences and into account and their effects on the targeted survey estimates. Although we did not identify any significant differences in term of the key survey estimates across the four geographical regions, it is not uncommon for regional differences in various health service parameters to exist. Quality and resource levels, for instance, are two variables often found to vary widely by geography (Del Vecchio, Fenech, & Prenestini, 2015; Krim et al., 2011; Laskey et al., 2010). Hence, a good level of understanding of the survey content and setting is of critical importance.

Several limitations should be considered when interpreting these findings. First, although we have found that the survey RL was related to geographical differences, we were unable to examine this in greater details. Second, we lacked the necessary data to depict the effect active (those survey recipients who made conscious decision not to respond) and passive (nonrespondents who may have wanted to respond but prevented from doing so by circumstances such as loss of the survey form) nonresponse have on the survey estimates. This may have some impact on our conclusion. Third, our population frame was mainly developed using the national licensing agencies database for private hospitals, and it is possible that some data elements may not have been recorded consistently. However, the NHEWS team has taken the necessary steps to verify some of the inconsistencies noted during the pretest phase in 2009 and the short cycle of compulsory renewal of hospital license in our setting suggest that the problem is likely to be minimal.

Finally, our findings are likely a result of the factors investigated and several additional interacting elements not modeled in the analyses. The un-modeled elements include the nature of the survey (highly quantitative) and the mode of the survey (mix of mail and internet survey), the setting of the Malaysian private hospital sector, and the interaction between the survey team and the target respondents. The relationship established during the presurvey engagement with the hospital association (APHM) and pretest period, and the identity of the research center (a government body) may also influence the response outcomes. The conclusions should be considered along with these other factors if the approach is applied to a different setting. We believe that this work will stimulate further discussion that will enrich the current state of knowledge in this area.

Conclusion

Readily measurable hospital characteristics such as hospital size and geographical location are likely useful predictors of RL in hospital survey. Where data permit, survey researchers should leverage this information to anticipate the likely response pattern and strategize accordingly. Consideration should also be given to how the resulting response pattern may affect the survey estimates to minimize biases.

Hospital Characteristics	Hypothesized Relation to Survey Response	Theoretical Basis
Hospital size	Larger hospital is related to higher response tendency	Larger hospitals often have better use of information technology, belong to one of the major health-care groups and are members of the private hospital organization, together this lead to higher capacity and motivation to respond; hospital type, being closely related to the establishment size, will have a positive but likely nonindependent association with response propensity as described above (Tomaskovic-Devey et al., 1994)
Hospital type	Hospital with wider scope of clinical services nonindependently associated with better response
Hospital location	Regional differences of survey response tendency exist	Geographical location of a hospital also is hypothesized to play an independent role in determining the response tendency of the organization due to a different perception of survey request and level of cooperation among different part of the country. However, we lack the knowledge to develop a priori hypothesis for the direction of their effect
Ownership type	Corporate owned and NGO hospitals are more likely to respond to survey request	The distance to a decision maker from the institution gatekeeper in a corporate-owned (CB) and nongovernment organization (NGO) hospital as well as a multiestablishment (as compared to single establishment) hospitals is likely longer compared to sole proprietorship/partnership type hospitals, thus CB/NGO and multiestablishment hospitals likely will have a negative association with response propensity (Tomaskovic-Devey et al., 1994)
Multiestablishment firm	Multiestablishment hospitals have lower survey response tendency
Organizational age	Older hospital respond better to surveys	Higher establishment age, in general, signifies that they have “survived” longer in the industry and thus likely to be less environmentally dependent and perceive to have lesser “threat” due to competition. They also may have a more stable group of staff, thus more willing to cooperate and easier to be followed up during the survey; we, therefore, hypothesize that there is a positive correlation with response propensity
Corporation group	Difference corporate group has different survey response tendency	Hospitals of various management groups may have different management practice and corporate policy toward responding to external surveys, thus respond to survey requests differently. However, the direction of the effect is unknown without advanced knowledge on their management and policy differences. We hence hypothesized that they may differ in their propensity to respond but do not specify the direction a priori (Tomaskovic-Devey et al., 1994)

Hospital Characteristics	All Hospitals	Respondents	Nonrespondents	Bivariate Analysis
Geographical region, n (%)
East Malaysia (EC)	17 (8.4)	15 (12.5)	2 (2.4)*	—
East Coast (EM)^b	11 (5.4)	10 (8.3)	1 (1.2)
Klang Valley (KV)	85 (41.9)	41 (34.2)	44 (53.0)	−2.19 (0.65)^*
Northern West Coast (NWC)	48 (23.6)	30 (25.0)	18 (21.7)	−1.61 (0.68)^*
Southern West Coast (SWC)	42 (20.7)	24 (20.0)	18 (21.7)	−1.83 (0.69)^*
Ownership, n (%)
Corporate body/ NGO	174 (85.7)	107 (89.2)	67 (80.7)	–
Sole proprietor/ partnership	29 (14.3)	13 (10.8)	16 (19.3)	-0.68 (0.40)
Hospital type, n (%)
Maternity center^c	73 (36.0)	35 (29.2)	39 (47.0)*	–
Secondary hospital^d	80 (39.4)	41 (34.2)	38 (45.8)	0.24 (0.32)
Tertiary hospital^e	50 (24.6)	44 (36.7)	6 (7.2)	2.13 (0.49)^*
Hospital age (year), n (%)
0–5 Years	44 (21.7)	21 (17.5)	23 (27.7)	–
6–15 Years	74 (36.5)	44 (36.7)	30 (36.1)	0.47 (0.38)
16–25 Years	47 (23.2)	30 (25.0)	17 (20.5)	0.66 (0.42)
25–150 Years	38 (18.7)	25 (20.8)	13 (15.7)	0.74 (0.46)
Hospital corporate group, n (%)
Independent	165 (81.3)	86 (71.7)	79 (95.2)*	−0.83 (0.85)
Group A	7 (3.4)	5 (4.2)	2 (2.4)	–
Group B	20 (9.9)	19 (15.8)	1 (1.2)	2.02 (1.32)
Group C	11 (5.4)	10 (8.3)	1 (1.2)	1.39 (1.34)
Independent hospital, n (%)
Yes	165 (81.3)	86 (71.7)	79 (95.2)*	–
No	38 (18.7)	34 (28.3)	4 (4.8)	−2.06 (0.55)^*
				2.89 (0.76)*
Hospital size (by number of bed), n (%)
1–25 Beds	107 (52.7)	45 (37.5)	62 (74.7)*	–
25–150 Beds	68 (33.5)	49 (40.8)	19 (22.9)	1.27 (0.33)^*
>150 Beds	28 (13.8)	26 (21.7)	2 (2.4)	2.89 (0.76)^*

Predictors	Coefficients (SE)
Geographical region
East Coast and East Malaysia (ECM)	—	—
Klang Valley (KV)	−2.56(0.7)*	−2.60(0.70)*
Northern West Coast (NWC)	−1.94 (0.74)*	−1.97(0.74)*
Southern West Coast (SWC)	−2.11(0.74)*	−2.14(0.75)*
Ownership
Corporate Body/NGO	—	—
Sole proprietor/partnership	−0.25(0.53)	−0.25(0.53)
Hospital age
0–5 Years	—	—
6–15 Years	0.34(0.46)	0.29(0.46)
16–25 Years	0.66(0.51)	0.60(0.51)
25–150 Years	0.19(0.56)	0.12(0.57)
Hospital type
Maternity center^a	—	—
Secondary hospital^b	−0.67(0.50)	−0.66(0.50)
Tertiary hospital^c	0.10(0.79)	0.08(0.79)
Hospital group
Independent	—	−0.33(0.96)
Group A	—	—
Group B	—	1.18(1.40)
Group C	—	0.95(1.43)
Independent hospital
No	—	—
Yes	−1.03(0.65)	—
Hospital size (number of bed)
1–25 Beds	—	—
25–150 Beds	1.25(0.53)*	1.25(0.53)*
>150 Beds	2.60(1.01)*	2.58(1.01)*

Survey Variables	Strata of Response Likelihood^a
Number of hospital (n)	51	19	38	12
Total number of inpatient admission	7,629	6,872	1,071	187*
Average length of stay (days)	2.7	2.5	2.2	2.5
Bed occupancy rate (%)	62.0	58.1	40.8	15.5*
Turnover interval (days)	1.9	1.8	3.3	10.0*
Number of CT scanning performed	2,347	1,262	0	0*
Number of MRI performed	1401	468	0	0*
Number of staff nurse	117	52	10	5*
Number of pharmacist	2	1	0	0*

Footnotes

Authors’ Note

All authors contributed to the study conception, interpreted the findings, and approved the manuscript. Chee Yoong Foo performed the data analysis and drafted the manuscript. Daniel D. Reidpath contributed significantly to the improvement of the manuscript. Sheamini Sivasampu contributed to the project management and critical review of the manuscript.

Acknowledgments

We would like to thank the Director General of the Ministry of Health of Malaysia for his permission to publish the findings. We also like to thank the private hospitals in participating in this study and the NHEWS team of researchers for their effort in the conduct of the survey.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Association for Public Opinion Research. (2016). Standard definitions: Final dispositions of case codes and outcome rates for surveys (9th ed.). AAPOR.

Abel

G. A.

Saunders

C. L.

Lyratzopoulos

(2016). Post-sampling mortality and non-response patterns in the English Cancer Patient Experience Survey: Implications for epidemiological studies based on surveys of cancer patients. Cancer Epidemiology, 41, 34–41. doi:10.1016/j.canep.2015.12.010

AHA Annual Survey Database. (2013). from https://www.ahadataviewer.com/additional-data-products/AHA-Survey/.

Dutka

Frankel

L. R.

(2004). Measurement Errors in Business Surveys, in Measurement Errors in Surveys (eds Biemer

P. P.

Groves

R. M.

Lyberg

L. E.

Mathiowetz

N. A.

Sudman

), John Wiley & Sons, Inc., Hoboken, NJ, USA. doi: 10.1002/9781118150382.ch7.

Del Vecchio

Fenech

Prenestini

(2015). Private health care expenditure and quality in Beveridge systems: Cross-regional differences in the Italian NHS. Health Policy, 119, 356–366. doi:10.1016/j.healthpol.2014.10.014

Digaetano

(2013). Sample frame and related sample design issues for surveys of physicians and physician practices. Evaluation & the Health Professions, 36, 296–329. doi:10.1177/0163278713496566

Draugalis

J. R.

Plaza

C. M.

(2009). Best practices for survey research reports revisited: Implications of target population, probability sampling, and response rate. American Journal of Pharmaceutical Education, 73, 142.

Dykema

Jones

N. R.

Piche

Stevenson

(2013). Surveying clinicians by web: Current issues in design and administration. Evaluation & the Health Professions, 36, 352–381. doi:10.1177/0163278713496630

Gupta

Shaw

J. D.

Delery

J. E.

(2000). Correlates of response outcomes among organizational key informants. Organizational Research Methods, 3, 323–347.

10.

Halbesleben

J. R.

Whitman

M. V.

(2013). Evaluating survey quality in health services research: A decision framework for assessing nonresponse bias. Health Services Research, 48, 913–930. doi:10.1111/1475-6773.12002

11.

Harkanen

Kaikkonen

Virtala

Koskinen

(2014). Inverse probability weighting and doubly robust methods in correcting the effects of non-response in the reimbursed medication and self-reported turnout estimates in the ATH survey. BMC Public Health, 14, 1150. doi:10.1186/1471-2458-14-1150

12.

Jaafar

Noh

K. M.

Muttalib

K. A.

Othman

N. H.

Healy

(2013). Malaysia health system review. In Healy

(Ed.), Health systems in transition, 15–30, Geneva, Switzerland: World Health Organization.

13.

Jha

A. K.

DesRoches

C. M.

Campbell

E. G.

Donelan

Rao

S. R.

Ferris

T. G.

… Blumenthal

(2009). Use of electronic health records in U.S. hospitals [research-article]. The New England Journal of Medicine, 360, 1628–1638. doi:10.1056/NEJMsa0900592

14.

Johnson

T. P.

Wislar

J. S.

(2012). Response rates and nonresponse errors in surveys. Journal of the American Medical Association, 307, 1805–1806. doi:10.1001/jama.2012.3532

15.

Keesling

(2008). Gatekeeper. In Lavrakas

P. J.

(Ed.), Encyclopedia of survey research methods, 300–301, Thousand Oaks, CA: Sage.

16.

Klabunde

C. N.

Willis

G. B.

McLeod

C. C.

Dillman

D. A.

Johnson

T. P.

Greene

S. M.

Brown

M. L.

(2012). Improving the quality of surveys of physicians and medical groups: A research agenda. Evaluation & the Health Professions, 35, 477–506. doi:10.1177/0163278712458283

17.

Krim

S. R.

Vivo

R. P.

Krim

N. R.

Cox

Hernandez

A. F.

Peterson

E. D.

… Bhatt

D. L.

(2011). Regional differences in clinical profile, quality of care, and outcomes among Hispanic patients hospitalized with acute myocardial infarction in the Get with Guidelines-Coronary Artery Disease (GWTG-CAD) registry. American Heart Journal, 162, 988–995.e4. doi:10.1016/j.ahj.2011.09.006

18.

Laskey

Spence

Zhao

Mayo

Taylor

Cannon

C. P.

… Fonarow

G. C.

(2010). Regional differences in quality of care and outcomes for the treatment of acute coronary syndromes: An analysis from the get with the guidelines coronary artery disease program. Critical Pathways in Cardiology, 9, 1–7. doi:10.1097/HPC.0b013e3181cdb5a5

19.

Lewis

E. F.

Hardy

Snaith

(2013). Estimating the effect of nonresponse bias in a survey of hospital organizations. Evaluation & the Health Professions, 36, 330–351. doi:10.1177/0163278713496565

20.

Loft

J. D.

Murphy

Hill

C. A.

(2015). Surveys of health care organization. In Johnson

T. P.

(Ed.), Handbook of health survey methods (1st ed.)., 545–560 Hoboken, NJ: John Wiley.

21.

Mullner

Levy

P. S.

Matthews

Byre

C. S.

(1981). An investigation of institutional characteristics associated with response rates in mail surveys of community hospitals. Public Health Report, 96, 128–133.

22.

Multone

Vader

J. P.

Mottet

Schoepfer

Fournier

Burnand

… Pittet

(2015). Characteristics of non-responders to self-reported questionnaires in a large inflammatory bowel disease cohort study. Scandinavian Journal of Gastroenterology, 50, 1348–1356. doi:10.3109/00365521.2015.1041150

23.

NHSI. (2008–2013). National Healthcare Establishment and Workforce Survey. National Healthcare Establishment and Workforce Statistics. Retrieved from http://www.crc.gov.my/nhsi/category/publications/hospital-establishment-workforce/

24.

Phipps

P. A.

Jones

C. K.

(2007). Factor affecting response to the occupational employment statistics survey. Paper presented at the 2007 Federal Committee on Statistical Methodology Research Conference. Retrieved from http://www.bls.gov/osmr/abstract/st/st070170.htm

25.

Pratt

C. L.

Sellon

R. K.

Spencer

E. S.

Johnson

T. W.

Righter

D. J.

(2012). Systemic mycosis in three dogs from nonendemic regions. Journal of the American Animal Hospital Association, 48, 411–416. doi:10.5326/JAAHA-MS-5799

26.

Riedl

Gower

R. G.

Chrvala

C. A.

(2011). Current medical management of hereditary angioedema: Results from a large survey of US physicians. Annals of Allergy, Asthma and Immunology, 106, 316–322.e4. doi:10.1016/j.anai.2010.12.012

27.

Rinne

S. T.

Wong

E. S.

Lemon

J. M.

Perkins

Bryson

C. L.

Liu

C. F.

(2015). Survey nonresponders incurred higher medical utilization and lower medication adherence. American Journal of Managed Care, 21, e1–e8.

28.

Seiler

(2014). The determinants of unit non-response in the Ifo Business Survey. AStA Wirtschafts- und Sozialstatistisches Archiv, 8, 161–177. doi:10.1007/s11943-014-0142-9

29.

Snijkers

(2013). Designing and conducting business surveys. Hoboken, NJ: Wiley.

30.

Steffen

M. W.

Hagen

P. T.

Benkhadra

Molella

R. G.

Newcomb

R. D.

Murad

M. H.

(2015). A survey of physicians’ perceptions of their health care needs. Occupational Medicine (London), 65, 49–53. doi:10.1093/occmed/kqu145

31.

Tomaskovic-Devey

Leiter

Thompson

(1994). Organizational survey nonresponse. Administrative Science Quarterly, 39, 439.

32.

Willimack

Nichols

(2010). A hybrid response process model for business surveys. Journal of Official Statistics, 26, 3.

33.

World Development Indicators. (2014). Retrieved November 04, 2016, from http://data.worldbank.org/country/malaysia

Hospital Characteristics	All Hospitals	Respondents	Nonrespondents	Bivariate Analysis
Hospital Characteristics	N = 203	N = 120	N = 83	Coefficient (SE)^a
Geographical region, n (%)
East Malaysia (EC)	17 (8.4)	15 (12.5)	2 (2.4)*	—
East Coast (EM)^b	11 (5.4)	10 (8.3)	1 (1.2)	—
Klang Valley (KV)	85 (41.9)	41 (34.2)	44 (53.0)	−2.19 (0.65)^*
Northern West Coast (NWC)	48 (23.6)	30 (25.0)	18 (21.7)	−1.61 (0.68)^*
Southern West Coast (SWC)	42 (20.7)	24 (20.0)	18 (21.7)	−1.83 (0.69)^*
Ownership, n (%)
Corporate body/ NGO	174 (85.7)	107 (89.2)	67 (80.7)	–
Sole proprietor/ partnership	29 (14.3)	13 (10.8)	16 (19.3)	-0.68 (0.40)
Hospital type, n (%)
Maternity center^c	73 (36.0)	35 (29.2)	39 (47.0)*	–
Secondary hospital^d	80 (39.4)	41 (34.2)	38 (45.8)	0.24 (0.32)
Tertiary hospital^e	50 (24.6)	44 (36.7)	6 (7.2)	2.13 (0.49)^*
Hospital age (year), n (%)
0–5 Years	44 (21.7)	21 (17.5)	23 (27.7)	–
6–15 Years	74 (36.5)	44 (36.7)	30 (36.1)	0.47 (0.38)
16–25 Years	47 (23.2)	30 (25.0)	17 (20.5)	0.66 (0.42)
25–150 Years	38 (18.7)	25 (20.8)	13 (15.7)	0.74 (0.46)
Hospital corporate group, n (%)
Independent	165 (81.3)	86 (71.7)	79 (95.2)*	−0.83 (0.85)
Group A	7 (3.4)	5 (4.2)	2 (2.4)	–
Group B	20 (9.9)	19 (15.8)	1 (1.2)	2.02 (1.32)
Group C	11 (5.4)	10 (8.3)	1 (1.2)	1.39 (1.34)
Independent hospital, n (%)
Yes	165 (81.3)	86 (71.7)	79 (95.2)*	–
No	38 (18.7)	34 (28.3)	4 (4.8)	−2.06 (0.55)^*
				2.89 (0.76)*
Hospital size (by number of bed), n (%)
1–25 Beds	107 (52.7)	45 (37.5)	62 (74.7)*	–
25–150 Beds	68 (33.5)	49 (40.8)	19 (22.9)	1.27 (0.33)^*
>150 Beds	28 (13.8)	26 (21.7)	2 (2.4)	2.89 (0.76)^*

Predictors	Coefficients (SE)
Predictors	Model (1)	Model (2)
Geographical region
East Coast and East Malaysia (ECM)	—	—
Klang Valley (KV)	−2.56(0.7)*	−2.60(0.70)*
Northern West Coast (NWC)	−1.94 (0.74)*	−1.97(0.74)*
Southern West Coast (SWC)	−2.11(0.74)*	−2.14(0.75)*
Ownership
Corporate Body/NGO	—	—
Sole proprietor/partnership	−0.25(0.53)	−0.25(0.53)
Hospital age
0–5 Years	—	—
6–15 Years	0.34(0.46)	0.29(0.46)
16–25 Years	0.66(0.51)	0.60(0.51)
25–150 Years	0.19(0.56)	0.12(0.57)
Hospital type
Maternity center^a	—	—
Secondary hospital^b	−0.67(0.50)	−0.66(0.50)
Tertiary hospital^c	0.10(0.79)	0.08(0.79)
Hospital group
Independent	—	−0.33(0.96)
Group A	—	—
Group B	—	1.18(1.40)
Group C	—	0.95(1.43)
Independent hospital
No	—	—
Yes	−1.03(0.65)	—
Hospital size (number of bed)
1–25 Beds	—	—
25–150 Beds	1.25(0.53)*	1.25(0.53)*
>150 Beds	2.60(1.01)*	2.58(1.01)*

Survey Variables	Strata of Response Likelihood^a
Survey Variables	Stratum-1	Stratum-2	Stratum-3	Stratum-4
Number of hospital (n)	51	19	38	12
Total number of inpatient admission	7,629	6,872	1,071	187*
Average length of stay (days)	2.7	2.5	2.2	2.5
Bed occupancy rate (%)	62.0	58.1	40.8	15.5*
Turnover interval (days)	1.9	1.8	3.3	10.0*
Number of CT scanning performed	2,347	1,262	0	0*
Number of MRI performed	1401	468	0	0*
Number of staff nurse	117	52	10	5*
Number of pharmacist	2	1	0	0*