Abstract
Three forms of inductive generalization - statistical generalization, variation-based generalization and theory-carried generalization - are insufficient concerning case-to-case generalization, which is a form of analogical generalization. The quality of case-to-case generalization needs to be reinforced by setting up explicit analogical argumentation. To evaluate analogical argumentation six criteria are discussed. Good analogical reasoning is an indispensable support to forms of communicative generalization - receptive and responsive (participative) generalization — as well as exemplary generalization.
Keywords
Introduction
Empirical research evokes the question of generalizability. 1 When persons, groups, organizations, situations, social processes, aid programs, et cetera, have been researched, do the results and conclusions of this research also hold for other persons, groups, organizations et cetera, as with those that were the object of the research? Of course, every research project need not lead to results and conclusions that are generalizable. For example, the results and conclusions of an evaluation research of one particular aid program in a specific, unique context, need not be generalizable to other contexts. Another example: a case study of a so-called ‘negative case’ to falsify a theory or a hypothesis need not be generalizable to other cases. This article discusses research in which generalization is indeed important, but in which the researcher does not simply assume that the results and conclusions may be generalizable. It particularly intends to supply criteria to enhance the argumentative quality of generalization based on analogy.
My aim is to point out that statistical generalization does not have to be the only form of generalization, suited for all situations. Besides statistical generalization, there are at least two good alternative forms of ‘inductive generalization’ — theory-carried generalization and generalization based on covering the variation (or variation-based generalization). However, even these two forms of generalization are not always applicable to all situations. Generalization based on good analogical reasoning is sometimes more suitable, especially when research results obtained from one case are to be generalized to another case. I shall call this form of generalization ‘analogical generalization’.
Good analogical argumentation can support certain forms of inductive generalizations. Moreover, analogical argumentations can play an important role in generalizations by the readers of the research report. In that case, the researcher tries to supply the reader with the information in the research report, whereby the reader can set up an analogical argumentation by himself. 2 In fact, it is even possible that the researchers, other participants, the readers and any other potential users of the research really interact. I shall call these social forms of generalization ‘communicative generalization’. 3
Inductive generalization
Quite a number of methods are used to obtain a certain generalizability. Among the most familiar methods are those in which different forms of random sampling are carried out, intended to draw a statistically representative sample of a population that is too large to research as a whole. The basis for generalization, supposedly, is the statistical representativity of the sample. Statistically representative sampling may be combined with qualitative research. For instance, a quantitative survey may help to discover which cases are extreme, unique, or typical (for ‘using numbers’, see Bryman, 1988; Silverman, 1993; and Seale, 1999). However, it is not always possible to achieve a good statistical representation. A reason could be, for example, that the population is relatively unknown: what does and what does not belong to the population? Which characteristics are relevant to the research question and which are not? Without an answer to the first question, it is impossible to ensure that each person, case, et cetera, has an equal chance of becoming part of the sample. Without an answer to the second question, it is not possible really to split the population into so-called strata, within which different random sampling can be done. If the population is unknown to a large degree, it is rather difficult, if not impossible, to do random sampling in a well-stratified manner.
The statistical approach, also known as the distributive approach, is not suited for all types of research. For example, one cannot necessarily generalize from a single case to another case if both cases are part of a statistical representative sample of a population that includes both cases. Whether or not research results are generalizable is a question that, in this case, definitely can not be answered affirmitively when the population is heterogeneous, nor when elements in that population are not all known. Indeed, in this case, allowances have to be made for the fact that the population could be heterogeneous. This would mean that case-to-case generalization is almost completely based on chance-capitalization, more a guessing game or a stroke of luck.
Yin (1984) rejects statistical generalization that is totally connected with the aforementioned statistical representativity in case studies. He recommends a replication logic instead of a sampling logic that is aimed at statistical representativity. This replication logic is quite often applied in experimental research designs. In experimental research, there are seldom, if ever, measures taken to ensure that the group of subjects (those in the experimental condition together with those in the control condition) consists of a representative sample of a population. For example, sometimes it is simply assumed that students who are paid ten dollars to participate in the experiment are representative of others. But sometimes the experiment is replicated a few times with other subjects. The replication logic of Yin fits in at this point and entails that the researcher selects not only cases where one might expect that results obtained in a previously conducted study are repeated and thus affirmed, but also cases where one might expect the research results to contradict the previously affirmed substantive hypothesis. In this way, subsequent case studies enable the researcher to test — to affirm and to falsify — repeatedly conjectures and hypotheses, and to adjust them and thereby develop them into a theory. The theory that is ultimately formulated must then become the vehicle for generalization to other cases that have not been studied. For Yin (1984), this is analytical generalization. The cases that have not been studied, of course, must belong to the scope of the theory. There is indeed no generalization from a statistical representative sample to a population, but there is generalization from one case to other cases that belong to the scope of the theory involved. I would prefer to call this ‘theoretical generalization’ as Seale (1999) does, or better still ‘theory-carried generalization’. The main reason is that Yin (1984, 1994), and following him Firestone (1993), as well restricts analytical generalization to ‘generalization to a theory’. By this they mean that research results are generalized by means of a suitable theory to its scope or domain. 4 The expression ‘theory-carried generalization’ is a clearer and more accurate description of this concept. It indicates that the theory functions as a carrier or as a vehicle.
Theory-carried generalization can be recognized in the original idea of analytical induction of Znaniecki (1934), which aims at forming a theory by systematically testing conjectures and hypotheses and reformulating them by means of intentionally chosen cases. This can be called theoretical sampling, a term that is applicable to the replication logic of Yin as well (see also Maso & Smaling, 1998). Ideally, the researcher stops researching new cases when the theory formed thus far is contradicted no longer, not even when the researcher thoroughly tries to find ‘negative cases’, or cases that would contradict the theory. When the researcher does not have to modify the theory, after examining about five ‘negative cases’, a point of saturation called ‘theoretical saturation’ is reached (see Glaser & Strauss, 1967; Strauss & Corbin, 1990; Wester, 1995). At that point, the researcher can stop looking for new cases and the theory is now a suitable carrier or vehicle for generalization to cases that have not been researched. This form of theory-carried generalization may be seen as a special sort of argumentation that may be called ‘abductive reasoning‘. It is a sort of reasoning that aims at hypothesis-forming and/or theory-forming. This type of reasoning could possibly be distinguished from inductive reasoning, but this is not my concern in this article. Nevertheless, the term ‘abductive generalization’ can be used. Moreover, theory-carried generalization, as the type of generalization whereby an existing theory already sufficiently affirmed acts as a carrier or vehicle, could be called ‘subsumptive generalization’, that is generalization of research results or conclusions by placing them within the context of a more general expression, rule or law or, as in this case, within the context of an existing theory. In this article however, these two forms of theory-carried generalization will not be explored. Rather, the focus is on the importance of good analogical argumentation.
Several objections can be made against theory-carried generalizations. Some research is more descriptive than it is theory forming or theory testing. Even research aimed at theory forming or theory testing does not in every case lead to a good, solid theory. The result is sometimes merely a typology. Moreover, a point of theoretical saturation often fails to be reached. Even if the result is a theory, that theory hardly ever indicates precisely in which cases and under which circumstances the theory holds. Furthermore, existing theories are all too often not strong enough, not precise enough, nor sufficiently supported, to serve as a vehicle for generalization.
Theoretical sampling is a form of purposive sampling (see Patton, 1990). Not every purposive sample needs to aim at formulating a theory. When conducting descriptive, explorative, or purely practical research, the researcher may strive for some form of generalization of the research results by focussing on variations in which a phenomenon occurs. For this purpose, the researcher intentionally looks for cases that deviate from those already researched and even looks for extreme cases. He keeps searching for such cases within the population involved until no more new information turns up after examining several of these cases. At this point, saturation has been reached with regard to the description of the variation, thus resulting in descriptive saturation rather than theoretical saturation. The assumption then becomes that the variation occurring in the population has been sufficiently represented by the variation in the researched sample. The researcher does not have the intention of counting how often something occurs. The purpose is not to cover statistical distribution but merely to cover the existing variation. The sample is not representative in a statistical manner, but rather it represents the variation.
The researcher can aim at generalization based on covering the variation by means of the aforementioned form of purposive, non theory-directed, descriptive sampling. In other words, the researcher can aim at variation-based generalization. However, even with this descriptive form of sampling, he does not always succeed in achieving a point of saturation in the short term, when he can say that no new variations are likely to occur. For this reason, he will usually have to categorize collected data in time, otherwise there is the risk that all new data is considered new information. The point of saturation is thus reached when it is no longer necessary to adjust the system of categorizing even after repeatedly collecting new data. Note that the case discussed here mainly concerns descriptive saturation, not so much theoretical saturation. The categories are not derived from an explicit theory, but are the result of interpretative descriptions of the collected material. In practice, it is rather difficult to achieve such a system of categorizing in the short term; categories do not emerge automatically. Nevertheless, generalization based on covering the variation is still a good option in addition to statistical and theory-carried generalization. However, these three forms of generalization are insufficient when determining whether or not the results of the research can be generalized to other concrete situations. This is especially true when it concerns generalization from a single case that has been researched to another that has not been researched.
These three forms of generalization — statistical generalization, theory-carried generalization (both abductive and subsumptive generalization) and generalization based on covering the variation (variation-based generalization) — can be regarded as generalizations based on inductive reasoning in a broad sense. Each is concerned with drawing conclusions pertaining to (a class of) subjects, cases, situations, et cetera, that have not been researched, based on a limited number of persons, cases, situations, et cetera, that have been researched. In all those cases, a generalization is made from research results to a population or to a scope belonging to a theory. In this sense, these forms of generalization can be seen as forms of inductive generalization: methods to arrive at general or universal propositions. However, these forms of inductive generalization are not explicitly guided by a detailed knowledge about similarities and differences between situations, cases, and so on. The criteria for good analogies, as discussed in the next section, could help to reinforce inductive generalizations by means of explicit analogical reasoning, in as much as these forms of inductive generalization actually presuppose that the researcher has prior knowledge of similarities and differences between situations, cases, et cetera. For the more one knows about similarities and differences between a case that has been researched and one that has not, the more firmly based any possible generalization (see figure 1).
Analogical generalization
Thus far, explicit analogical reasoning has not been discussed. What we have seen is that analogical reasoning implicitly plays a part in the replication logic of Yin (1984, 1994), for example, when the researcher looks for cases that are expected to give the same research results. After all, in this instance the researcher is looking for analogous cases. Yin does not explicitly mention analogies nor does he discuss measures or criteria for good analogical reasoning. The same holds for Mitchell (1983), Bryman (1988), Firestone (1993), Silverman (1993) and Seale (1999).
Generalization based on analogy does not need an explicit theory. However, analogical argumentation can play a role when statements are made concerning a new case that has not yet been researched, based on a case that has, by means of a preliminarily developed theory.
Theory-carried generalization implies that the researcher knows in which sorts of cases the theory will probably hold. These cases that have not been studied must possess a certain analogy with those that have been researched. Theory-carried generalization (and thus also the analytical generalization of Yin) would benefit if the analogical argumentation connected with it was made explicit and was tested. Empirical testing of analogical reasoning can contribute to more stringent formulation of the conditions under which the theory holds and thus of the scope of the theory. The other forms of inductive generalization could be argumentatively supported, in my opinion, by analogical reasoning. In other words, good analogical reasoning can help to avoid the fallacy of hasty generalization, especially regarding case-to-case transfer.
Analogical reasoning can be seen as a special sort of inductive argumentation, as Copi (1982) does, or as a separate form of argumentation which must be distinguished from inductive argumentation, as does, for example, Walton (1989). Walton restricts inductive argumentation to situations in which the researcher, based on a particular sample, generalizes to a population (or scope) of which that sample is a part. Analogical reasoning, however, is more concerned with the apparent similarities between a case that has been researched and another particular case that has not. In this article, I shall follow Walton's view, as the main purpose of this article is to show that good analogical argumentation is of special importance for diverse forms of generalization. Whether or not analogical reasoning is seen as a sort of inductive argumentation, analogical reasoning nevertheless has its own character. For instance, it is not aimed at arriving at general or universal propositions, let alone natural laws. In the present discussion, it is important to know how analogical reasoning can become an acceptable argumentation or even a powerful argumentation. Analogical reasoning is often mistrusted. People say: ‘Yes, but this comparison doesn't hold’, or ‘That's comparing apples and oranges', or ‘That's a horse of a different color’. In other words, analogical reasoning is often regarded as fallacious. But that is not always correct, hence the question is what is it that makes analogical reasoning argumentatively acceptable. How can the fallacy of false analogy be avoided? This section is concerned with answering this question.
Analogical generalization is based on analogical reasoning. Such reasoning is only plausible when there are solid arguments that, when a particular researched case has characteristics which are relevant for the research conclusions, another case that has not been researched also has these relevant characteristics. The knowledge about the relevant similarities can be based on present experience, on existing literature, or on the judgment of a group of experiential experts based on argumentative dialogues. There may possibly be an accepted or well-founded theory as a support, or a separate empirical study of the relevant similarities between two or more cases.
As a component of external validity of the results and conclusions in an empirical study, analogical reasoning is concerned with the plausibility and acceptability with which these results and conclusions could hold for phenomena, cases or situations that have not been studied and that display similarities with phenomena, cases or situations that have been studied. The plausibility of analogical reasoning cannot be derived from deductive logic or inductive statistics. Analogical reasoning is not deductively valid reasoning nor does it lead to quantitative statements of probabilities. As discussed in the previous section, analogical reasoning differs from non-statistical inductive reasoning that plays a role in generalization supported by theory and generalization based on covering the variation. Hence, the question is what makes analogical reasoning plausible? When do two situations compare with each other sufficiently to make it plausible that research results in one situation will also hold in another? On this score, it does not matter whether the analogy is demonstrated by the researcher, by a group of stakeholders, or by the reader of the research report. I shall discuss six criteria (or canons or rules) that will make analogical reasoning as I have discussed more acceptable. These six criteria are a compilation of what can be found in the literature by authors such as Copi (1982), Freely (1976), Govier (1985), Kennedy (1979), Rescher (1964), and Walton (1989) 5 (see also Smaling, 2002).
Six quality criteria for analogical reasoning
The analogy between case (or phenomenon, or situation) P and Q will be considered. Analogical argumentation could then be as follows:
the cases P and Q (e.g., two companies) have the characteristics a, b, c, d, and e in common; P also has characteristic k (e.g., the organizational form O has proven to be a success with P); the (plausible but not certain) conclusion is that case Q also has characteristic k (e.g., we expect that the organizational form O also will prove to be successful with Q).
This analogical reasoning is plausible when the following rules are applied. Presupposed, in connection with the following formulations, is that all other things being equal. In other words, the ceteris paribus condition applies. 6
The relative degree of similarity
Analogical reasoning is argumentatively stronger insofar as there are more similarities and not more differences. For instance, the analogy between P and Q is stronger when P and Q have four similarities — s1, s2, s3, s4 — and three differences — d1, d2, d3 — in comparison to the analogy between P and Q when these have only three similarities — s1, s2, s3 — and the same three differences — d1, d2, d3. Hence, the first analogy makes the conclusion that O will also be successful with Q more plausible than the second analogy (all other things being equal). In addition, analogical reasoning is also stronger insofar as there are fewer differences between the cases and the number of similarities is not smaller. This means, for example, that the analogy between P and Q is stronger when P and Q have four similarities — s1, s2, s3, s4 — and three differences — d1, d2, d3 — than when the analogy between P and Q has these same four similarities but five differences — d1, d2, d3, d4, and d5. Again, the first analogy makes the conclusion more plausible. It should not be thought that there is any simple numerical connection between the ratio of the number of similarities and differences and the probability of the conclusion. The team of researchers (possibly in cooperation with co-researchers and other stakeholders) have to decide how to cope with other comparisons. The following rules may be helpful.
The relevance for the conclusion
Insofar as the similarities between P and Q are more relevant for the conclusion, the analogical reasoning is more plausible. Similarly, insofar as the differences between P and Q are more relevant for the conclusion, the analogical reasoning is less plausible. For example, when P and Q are similar regarding the willingness to work together by those persons who have to do so, the analogical reasoning that organizational form O will also be succesful with Q is far more plausible than when, instead, P and Q match on the color of the wallpaper (all other things being equal). To say it more loosely, the more relevant the similarities and the less relevant the differences, the stronger the conclusion of the analogical reasoning that O, which has proven to be succesful with P, will also appear to be succesful with Q. Of course, the relevance of similarities and differences has to be assessed on the basis of theoretical and empirical evidence or pre-knowledge, as well experiential expertise.
Support by other, similar cases
The analogical reasoning for P and Q is stronger insofar as there are more cases, A, B, C, et cetera, that are similar to case P regarding the same points a, b, c, d, and e as P and Q are similar, and in which the organizational form O is also successful. The conclusion that O will probably also work in Q is in that situation more plausible compared to the situation where P is the only example. In other words, examples of other cases like P in which O is successful support the analogical reasoning.
Support by means of variation
When the other cases, A, B, C, et cetera, compare with case P, and when A, B, C, et cetera, additionally have characteristic k (organizational form O is succesful), just as P, the analogical reasoning for P and Q is more plausible insofar as P, A, B, et cetera, differ more on their points of difference. For example, suppose that case A has characteristics a, b, c, d, and e, just like P (as well as Q), and that in both cases A and P organizational form O is successful. Suppose further that A and P display large (relevant) differences on the points of difference. In that case, the possibility that the organizational form O will work well also in case Q, is more likely than in the case that A and P displayed smaller differences. In sum, larger differences between A and P, which compare on a, b, c, d, and e, and in which organizational form O is successful, make it more plausible that the differences between P and Q do not matter either and that O will also work in Q.
The relative plausibility of the conclusion on its own
The more plausible or probable the conclusion is on its own, apart from the analogy, the more acceptable the analogical reasoning. For example, when the organizational form O is vulnerable, the analogical reasoning is weaker than when O is a more solid form of organization. There is obviously a greater risk that a vulnerable form of organization does not work in a new and different situation.
Empirical and theoretical support
The analogical reasoning is more plausible insofar as the knowledge about the similarities and differences between the cases and their relevance has been supported more firmly, empirically, and theoretically.
When all these demands of quality have been fulfilled, analogical reasoning can be stronger than inductive reasoning based on a statistical representative sample, theory-carried generalization, or variation-based generalization as described earlier. After all, coincidence, unfamiliarity with relevant variables, and too little specificity for one separate, other case can thoroughly weaken inductive reasoning, if it must serve to generalize from a case that has been studied to one that has not been studied. However, it will not always be possible to fulfill the demands of each of these six criteria. A criterion may also function as a reminder or an evaluative dimension. The six criteria may help to evaluate and criticize case-to-case generalizations based on other forms of (inductive) reasoning. The fewer criteria fulfilled, the weaker the claim that one can generalize from one case to another.
When a researcher tries to choose a case in advance that is typical for a set of cases, for example a nursing home, then such a typical or exemplary case must compare on certain characteristic relevant points with the other cases from that class and at the same time not differ too much. In order to select such a typical case, the researcher will need a certain amount of prior knowledge. For this purpose a theory or experiential knowledge can serve him well. He can even conduct empirical research as to discover similarities and differences. This empirical research may also be a quantitative survey study. The selection of a typical case can be seen as a sort of purposive sampling. Purposive sampling, therefore, does not only play a role in trying to cover the variation or in testing and further developing a theory so that it may serve as the support for generalization. Purposive sampling also plays a role in the selection of a case that is typical or exemplary because of its analogies with other cases. Researching a typical or exemplary case requires good analogical reasoning (see the previous section).
Communicative generalization
In the first two sections, forms of inductive generalization and analogical generalization were discussed. The question of who decides whether research results and conclusions are generalizable, the researcher or the reader of the research report, was, however, left unanswered. In this third section, I shall discuss this social dimension which I shall call ‘communicative generalization’. 7 In this form of generalization, the researcher and the reader (or potential user) may or may not interact. If they do not interact, the researcher may still be of help to the reader. The report may enable the reader to answer a question about what the research results and conclusions could mean for a situation that he does not know at all, but which is of interest to the reader. Communicative generalization is partly supported by analogical argumentation, in the case as seen from the reader's point of view. Concerning communicative generalization, it is not so much the researcher but the reader (or potential user) of the research report who figures out to what degree the research results and conclusions are generalizable to people, situations, cases, et cetera, that are relevant to him. In this form of generalization, two main variations can be distinguished — receptive generalization and responsive generalization. The last form can be regarded as interactive.
Receptive generalization takes place mainly after reading the final research report. The researcher and the reader of the research report do not interact. With regard to responsive and exemplary generalization, interactions between the researcher and the potential users are of crucial importance before the final research report is written.
Receptive generalization
In receptive generalization, there is hardly any interaction between the researcher (or writer of the research report) and the reader. The only thing the writer can do is provide sufficient information so that the reader can decide for himself whether the study is relevant for him and to what degree. A good example of such a case is what in the literature is called ‘transferability’. The ‘naturalistic generalization’ of Stake (1978, 1995) is a special case; the reader generalizes based on his or her everyday or specific practical experience with a situation in mind and not based on scientific knowledge. Utilization value can also be seen as an example of receptive generalization, an example that is especially aimed at practice-oriented research.
Transferability
Transferability (see Lincoln & Guba, 1985; Guba & Lincoln, 1989; and ‘Übertragbarkeit’ in Klafki, 1976) has been mentioned by some authors as the only qualitative characteristic of quality as a replacement for the traditional external validity or generalizability. As seen in this article, this view is too restrictive. Several other forms of generalization have already been discussed, inductive as well as analogical. Nevertheless, transferability is an important form of communicative generalization which suits research aimed at knowledge as well as practice-oriented research. Transferability is implicitly based on analogical reasoning, which the above mentioned authors do not discuss. The fact of the matter is that but the reader of the research report, not the researcher, determines whether analogies exist between the situation that has been researched and another situation which is of interest to the reader. The reader must have an adequate knowledge of the researched situation so that he can determine by himself whether there are sufficient relevant similarities that make it plausible that the research conclusions should also hold in the other situation. The researcher has to take the trouble to help him, so as to enable the reader to identify the similarities and the differences between the researched situation and the unresearched situation. He can help the reader if the research report describes the following matters:
the status, position and roles of the researchers in the research situation (after all, these can have an influence on the research results); the selected fellow-workers, researched subjects and informants; the situations, conditions and social contexts of the research project; the selected methods, techniques and concepts; the (meta-)theoretical orientation; the reasons used to make the diverse choices mentioned above; and ‘thick description’.
‘Thick description’ is not just an extensive description of all relevant phenomena, but especially a description of several meanings that are possible within their context. In particular, this means a description of ‘a stratified hierarchy of meaning structures’ (Geertz 1973: 7). The researcher cannot describe everything extensively and in detail. He has to choose matters that could be relevant in other situations.
The reader of the research report would do well not only to observe that there are some similarities between the research situation and another situation, but especially to check the six criteria for a good analogical reasoning, before making a decision about transferability. The argumentative quality of transferability would profit, as well as the argumentative quality of naturalistic generalizability. For example, if the research concerned an analysis of the failure of a recently started electrical engineering firm, then for someone who reads this report and who is also involved in plans to set up an electrical engineering firm elsewhere, it is important that he can read all sorts of information that allow him to determine relevant similarities and differences between the two projects. Style of leadership would be an example of a relevant characteristic.
Utilization value
The utilization value of a research project can be increased as follows (see also Rink & Rijkeboer 1983; van de Vall 1980, 1987; Patton 1986, 1990; Smaling, 1994):
The language used in the research report should coincide with the language of the potential users; this requirement may mean that several versions of the report must be written for the different potential users. Detailed information and detailed research conclusions should be given, which specify the uniqueness of the context of what is researched. The research report should contain a few comparisons with situations in which similar research has been satisfactorily used. This should be motivating for potential users. The research should join in with the way of life of potential users and go on from there, instead of starting with a theory from the professional literature. Naturalistic research or research that is set up from the bottom is often used more than research that is rather far removed from everyday reality. Potential users should participate in the setup and conducting of the research. (The value of participation will be discussed explicitly and more extensively further on in this article.) Methodological measures should be taken and explicitly mentioned in order to increase the external validity, especially measures that enhance transferability (see above). In general, a high methodological quality can motivate potential users.
Information that is important to discover analogies and to assess them is referred to especially in the third point above, but also in points two and four. The utilization value of a research report would be increased, in my opinion, if the writer and reader would explicitly take into account the criteria for good analogical argumentation.
Responsive generalization
In responsive generalization, interaction between researcher and reader or potential user is present, opposite to receptive generalization where it is not present. Those who have participated in the research can decide, based on their experience in the research and the conversations with other participants, what the value of the research is for his or her problem or situation outside the research. With the term responsiveness, we mean here the way it occurs in connection with what has come to be known as ‘responsive evaluation research’ (see Stake, 1975; Guba & Lincoln, 1989; Abma, 1996). In responsive evaluation research, all involved ‘stakeholders’ are included as much as possible in the research, and the final research report is also a joint construction (see Guba & Lincoln, 1989). The idea of responsive generalization can also be recognized in participative research, in which the researched subjects participate in the research as a kind of co-researcher and in which the researchers take the position as co-subjects (see, for example, Heron, 1981; Reason, 1994). Instead of ‘responsive generalization’ the term ‘participative generalization’ could therefore be used. In my view, a connection to participative generalization is found in the literature, for example in Guba and Lincoln (1989) when they discuss their so-called ‘authenticity criteria‘. When these criteria are met, the research has what I call participative value.
Participative value
I consider ‘participative value’ to be a better term to cover what Guba and Lincoln (1989) describe as authenticity. Indeed, their concern does not seem to be about whether a researcher is personally authentic, but about the measure in which justice is done to all participants in a study. The concern is not just to increase the methodological quality of the study but, apart from that, especially to serve the interests and equality of those involved. Guba and Lincoln obviously mean that a researcher who wants to serve the best interest and equality of all stakeholders, as they do themselves, can only be authentic on this score if he follows the measures they propose.
The participative content of a study can become apparent by the manner in which the diverse stakeholders are involved in the study — stakeholders such as researchers, subjects, informants, commissioners, interested parties — but also possible recipients of the effects of the implication of the results of the study. This involvement has a number of aspects; the term used by Guba and Lincoln (1989) is mentioned in brackets with each of the following aspects (see also Erlandson, Harris, Skipper, & Allen. 1993):
Everyone's point of view is discussed with equal serious attention (fairness). This can be encouraged by open and equal negotiations. Everyone's conscious experience of ‘the world’ is improved (ontological authenticity). Everyone's understanding of the point of view and of the interpretations of the other stakeholders increases (educational authenticity). The study makes taking action or making changes easier, even stimulates it (catalytic authenticity). This overlaps with utilization value. The study and carrying it out leads to empowerment of all participants (see also Smaling, 1996) by the fact that they are recognized as a sort of co-researcher (tactical authenticity).
When a study has a large participative value, it will have great after-effect, certainly within the context in which the study was carried out, even after the study itself has been concluded. It may be expected that those involved will use their experiences positively in new situations. Even those who were not involved but who read the report may be given new ideas by means of the extensive description of the ways in which participative value was sought.
The plausibility of analogical reasoning by those involved can be supported by checking the six criteria, or demands for quality, mentioned above. When they have to judge possible generalization of responsive or participative research to new cases, situations, persons, et cetera, these six criteria can be meaningful to help them check the relevant similarities and differences between these new cases and those that were studied.
Exemplary generalization
The third variation of analogical generalization can be called ‘exemplary generalization‘. This form of generalization is very well suited for what is known as ‘exemplary action research’ (Coenen, 1987; Doets, 1982; Boog, Coenen, Keune, & Lammerts, 1998). I borrow the adjective ‘exemplary’ from this term. Exemplary generalization can partially be seen as a special type of responsive or participative generalization. What makes it different is that the reseacher has three extra responsibilities. First, he must choose a research situation that can be exemplary for a set of situations. This first concern is not necessarily of a communicative nature. The purposive sampling of an appropriate case with exemplary quality may be seen as a task of the researcher. Second, he must realize organization and cooperation in the research activities themselves in such a way that these can become examplary for the activities that are targeted for improvement by the research. Put simply, the researcher must set a good example as a co-worker as well as an organizer. Third, he must write a research report in which the former two responsibilities are illustrated as clearly as possible, so that the report achieves ‘exemplary value‘. All those involved in the study can, on their own, generalize their experiences to other situations whereby an explicit analysis of the nature of the analogies can be very helpful. Additionally, people who are able to just read the report, but did not participate in the study, can also profit by this setup.
Although exemplary generalization has communicative aspects, it is mainly a case of analogical generalization. Therefore, exemplary action researchers might benefit from the six criteria or checkpoints for promoting the quality of their analogical reasonings.
Summary
Statistical generalization is not always the best form of generalization. There are alternatives that can be better, certainly in situations in which little is known about the population. Generalization based on covering the variation (in short, variation-based generalization) is a good possibility, especially when it is more important to display the variation than to know how often a certain variant occurs. Theory-carried generalization is a good possibility when a good theory is available in which case it is subsumptive generalization. Theory-carried generalization is also a good option when a hypothesis or theory is to be tested and further developed, in which case it is abductive generalization.
These forms of inductive generalization — statistical generalization, variation-based generalization, and theory-carried generalization — are insufficient concerning case-to-case generalization. However, they can be reinforced by setting up explicit analogical argumentation. This form of reasoning plays the key role in analogical generalization. Its main concern is with making plausible that research results in a certain situation also hold in a certain other situation. To evaluate good analogical argumentation, six criteria or checkpoints are necessary.
In addition, good analogical reasoning can support forms of communicative generalization, whereby especially those involved in the study and the readers of the research report have to decide on their own, or on the basis of a dialogue, as to the measure in which the research results are generalizable to another situation. On this score, the content of the information of the research report plays an important role. There are two forms of communicative generalization: receptive and responsive generalization. The last one is interactive by nature, because the researcher and those involved communicate interactively, mainly before the appearance of the final research report. Transferability and utilization value are forms of receptive generalization. In responsive (or participative) generalization, the so-called criteria of authenticity can play a part. These criteria are indications of the participative value of the research.
Exemplary generalization has receptive and responsive (or participative) aspects, but its analogical character is not necessarily of a communicative nature. Choosing a case which has exemplary quality may be the exclusive task of the researcher.
Footnotes
1.
In this article I use the term ‘generalizability’ as a synonym for ‘external validity’.
2.
When the author refers to the researcher, the researchee, or the reader using ‘he’, ‘him’, ‘his’ or ‘himself’, these may be read as also meaning ‘she’, ‘her’, ‘hers’, or ‘herself’.
3.
Firestone (1993) distinguishes the following three forms of generalization: sample-to-population extrapolation, analytical generalization, and case-to-case translation. Sample-to-population extrapolation coincides completely with statistical generalization. Analytic generalization comes from Yin (1984, 
). In this article, I discuss these two forms of inductive generalization, but I prefer to use the term ‘theory-carried generalization’ rather than analytical generalization. I distinguish two forms of theory-carried generalization: abductive and subsumptive generalization. Moreover, I discuss a third form of inductive generalization that is not dealt with by Firestone: generalization based on covering the variation (variation-based generalization). To me, it seems reasonable to conceive all these types of generalization as forms of generalization from a sample (or a case) to a population (or a class of cases). In case-to-case transfer, Firestone does not explicitly deal with the quality of analogical argumentation, although analogical argumentation plays an important part at this point of the discussion. In this article, I shall deal extensively with criteria of quality for analogical argumentation. In connection with his case-to-case generalization, Firestone furthermore discusses the situation in which especially the reader of the research rapport has to find out whether generalization is possible. In this article, I shall mention specific measures and shall discuss two forms of generalization not mentioned by Firestone, those being responsive (or participative) and exemplary generalization.
4.
From the discussion and meaning of generalization in this article, ‘generalization from findings to theory’ can only mean that one generalizes from research results to phenomena, cases, et cetera, that belong to the scope or the domain of the theory involved, and not to the theory itself. It seems that Yin (1984) also considers the theory or the theoretical framework merely as a vehicle. discusses the idea of theoretical generalization extensively, but he does not use this term, nor does he use the term ‘analytical generalization‘. Instead he speaks of ‘logical inference’ (theoretical reasoning) in contrast to ‘statistical inference’. However, he does not mention any criteria for the strength or plausibility of this reasoning.
5.
The six presented criteria (or canons or rules) may show some resemblance with the canons of John Stuart Mill (see Ryan, 1970; Copi, 1982) of inductive logic. However, the canons of Mill are aimed at inductive generalization: the method of arriving at general or universal propositions on causal relationships, particularly natural laws. The nature of the connections between the six criteria for analogical reasoning, which are not aimed at general or universal propositions, and Mill's five canons of inductive logic are not discussed in this article. Nevertheless, at least one thing is clear: Mill's five canons of inductive logic are particularly relevant to theory-carried generalization as a form of inductive generalization.
6.
I give a clarifying example of the function of the ceteris paribus condition concerning the first criterion: Suppose that we compare situation S1, in which the companies P and Q are similar regarding a, b, and c and differ regarding r and s, with situation S2, in which P and Q are similar regarding a and b and differ regarding r, s, and t. If k (organizational form O is successfull) is true in P, then the conclusion of analogical reasoning is that it is more plausible that k will also be true for Q in situation S1 than in situation S2, unless S1 and S2 appear to differ on other points than the mentioned ones, for then the conclusion that it is more plausible that k will also true in Q in S1 in comparison with S2 is less acceptable! To preclude this unless-situation, the ceteris paribus condition must be added. The ceteris paribus condition requires that the situations S1 and S2 do not differ regarding aspects other than the mentioned aspects a, b, c, r, s, t, and k. In other words: in situation S1, the analogical reasoning is more plausible than in situation S2 provided that the ceteris paribus condition holds. Of course, concerning our problem we have only to check the ceteris paribus condition, in accordance with rule 2, insofar the other aspects may be considered as relevant. Concerning each of the six criteria for plausible analogical reasoning, at least two situations of analogical reasoning are compared in order to be able to say that one of the two reasonings is more plausible than the other. As for this comparison, it must be presupposed that all other things being equal, it must be assumed that the compared situations of analogical reasoning remain the same with regard to characteristics, circumstances, unforeseen occurrences, and external influences that are not mentioned in the formulation of the very criterium. In other words, it is assumed that the ceteris paribus condition applies. In empirical research, the assumption of the ceteris paribus condition should be supported by convincing empirical evidence. However, sometimes it will suffice, for instance, that there is no difference of opinion between experts, researchers, and other stakeholders about the irrelevance of certain differences between situations.
7.
The role of the reader is already indicated by Kennedy (1979) as well as .