Seeking Environmental Injustice with Help of Q Methodology on APA Algodoal-Maiandeua

Abstract

This article presents findings of a field research conducted in Brazil's Amazon in 2011. The aim of the article is to present a new method tool for environmental justice research called Q methodology by outlining the outcry for justness of indigenous people on Algodoal-Maiandeua, on an island sibling and área de proteção ambiental (APA) [area of environmental protection]. Two Q studies have been realized at the same time and compared by factor analytical means. The findings gave evidence for statistical significantly less degree of discourse differences for the sustainability development concept than for the environmental justice concept. The narratives tell that the secretary of the environment seeks to establish ecological corridors whereas the community demands infrastructure measures and job opportunities.

Introduction

The area of this research is an environmental protection area (APA) with the name Algodoal-Maiandeua, also referred to as the APA Algodoal, an island near the Amazon delta. Algodoal is located between the 00° 35′ 03″ S and the 00° 38′ 29″ southerly latitude and the 47° 27′ 42″ W and the 47° 34′ 57″ westerly longitude (Bastos 1996, 11) in the Brazilian Amazon, federal state of Pará, municipality of Maracanã. Well known for its broad ecological diversity (SEMA 2010: 1), APA Algodoal was declared a conservation unit by law No 5,621 on November 27, 1990. The aim of this policy was to initiate development based on ecotourism (Secretaria do Meio Ambiente 2010, 1). The APA contains 19 km² or 2,378 hectares and is inhabited by approximately 2,000 people of either indigenous origin or migrants. They inhabit four villages (Algodoal, Mocooca, Fortalezinha, Camboinha). Subsequent federal law 9,985/2000 requested a management plan for all APAs in Brazil. A plan of this type was then created by the secretary of the environment (SEMA) on June 6, 2006. The plan established a management council with seven governmental institutions,¹ two scientific research institutions,² nine local associations,³ and three nongovernmental organizations (NGOs)⁴ as officially registered members.

For Algodoal as a case study, it is now in this research assumed that only two concepts are in question, sustainable development (SD) the one and environmental justice (EJ) the other. Both concepts “have more in common than a cursory look at either reveals” (Kameri-Mbote and Cullet 1996, 1). However, both differ concerning the definition of environment. This article assumes that the traditional definition of “environment” causes the problems on Algodoal. These were already described (Kaufmann 2012). Since over twenty years the traditional definition assumes a purely “biological” understanding of environment. This definition is the one utilized in existing policies and institutional action. In contrast, the other definition is perceiving environment as a place “where we live, work, and play” (Gosine and Teelucksingh 2008, viii). The article now aims at revealing problem sets that were ignored before as a consequence of this limited biological definition.

The case Algodoal is thus with this purpose studied using Q methodology. Q methodology was developed by William Stephenson (1935; 1953) in 1935 and later became established in the field of social sciences by Steve Brown (1980). Basically, the method makes use of factor analysis. In this way, it discloses discourses (in the meaning of Foucault 1981) on a specific topic. Each extracted factor represents as result of the analysis one significantly different discourse in the field. Using the inverted “bell curve” (Gaußkurve) qualitative data can numerically be analyzed “by establishing patterns within and across individuals rather than patterns across individual traits” (Barry and Proops 1999, 339).⁵ Currently the EJ concept appears ambiguous as its application oscillates between either incorporation into the SD debate (as one direction, cf. Hopwood, Mellor, and O'Brien 2005, 41) or, alternatively, remaining separated in structure and orientation (Gosine and Teelucksingh 2008, 14). Q methodology is therefore used to analyze discourse differences on SD and EJ comparatively. Comparing the two qualitative discourses (SD and EJ) by quantitative measures, a systematic and traceable frame for analysis was achieved while usually the method is only used to understand one discourse.

In this article, for the first time, two discourses will be analyzed comparatively. This is done by looking whether the discourse on either EJ or SD include “more” differences. By this procedure, a new way to apply Q methodology is presented which may bear far reaching applications beyond the specific question raised here on Algodoal.

Discussion

Execution of Q study

To analyze in this way the two discourses on sustainable development (SD) and environmental justice (EJ) 98 statements are taken, both from literature and from qualitative data. The interviews have then been conducted with members of the management council and locals, in the period from September to October 2011 (28 statements). Twenty-nine statements have been taken from international literature (Barry and Proops 1999; Clapp and Dauvergne 2005; Gosine and Teelucksingh 2008 etc.) and 41 from Brazilian literature (Acselrad, Mello, and Bezerra 2009; Romeiro 1999; Nobre 2002; Figueiredo and Santana 2010). These statements have then been classified and thus been reduced⁶ through a concourse matrix (Table 1). Here John Dryzek (1997) [horizontal] and Jennifer Clapp et al. (2005) [vertical] have been used.

Table 1.

Concourse Matrix: Sustainable Development (SD)/Environmental Justice (EJ)

Total (SD/EJ)	Definitive	Designative	Evaluative	Advocative	SUM
Market Liberals	3 (2/1)	4 (3/1)	2 (1/1)	2 (1/1)	11 (7/4)
Institutionalists	2 (1/1)	3 (1/2)	3 (3/0)	2 (0/2)	10 (5/5)
Social Greens	3 (1/2)	4 (2/2)	3 (1/2)	3 (1/2)	13 (5(8)
Bioenvironmentalists	1 (1/0)	1 (0/1)	1 (1/0)	1 (0/1)	4 (2/2)
SUM	9 (5/4)	12 (6/6)	9 (6/3)	8 (2/6)	38

Source: G. Kaufmann.

For such reduction of the Q samples' statements (Table 2 and 3), at least the representation of one statement in each category was defined as requirement, here one for either the EJ or the SD concept.

Table 2.

Concourse Matrix by Defined Statements (Sustainable Development)

	Definitive	Designative	Evaluative	Advocative
Market Liberals	3, 19	12, 15, 18	5	9
Institutionalists	10	7	6, 11, 17
Social Greens	16	1, 13	14	2
Bioenvironmentalists	4		8

Source: G. Kaufmann.

Table 3.

Concourse Matrix by Defined Statements (Environmental Justice)

	Definitive	Designative	Evaluative	Advocative
Market Liberals	11	3	14	17
Institutionalists	4	13, 15		2, 9
Social Greens	1, 10	16, 19	5, 7	6, 18
Bioenvironmentalists		8		12

Source: G. Kaufmann.

By using the “concourse matrix” 98 statements have thus been reduced to a number of n=19 each, thus 38 for two Q sorts in total.

The Q sample and Q sort

As representation of the populations on Algodoal, the members of the local management council at local have been chosen as P-set for the Q sorting plus three members of the indigenous population and of the national government through SPU/PA (Superintendência de Patrimônio da União no Estado do Pará) and ORLA (O Projeto de Gestão Integrada da Orla Marítima).

Just 19 of 22 participants have participated in both sortings, which has been done so in consideration to Table 4 from October to December 2011.

Table 4.

Q Pyramid

Scale Score	−4	−3	−2	−1	0	+1	+2	+3	+4
No of statements	1	2	2	3	3	3	2	2	1

Source: G. Kaufmann.

Q analyses

The Q analysis was made with PQMETHOD (Schmolck 2013a), using centroid procedure for factor analysis and VARIMAX rotation to extract factors which are significant “ideal type” discourses in accordance to the protocols of Q methodology. In consideration of the Principle Component Analysis results, four factors have finally been extracted from both sorting using automatic flagging.⁷ Further testing hasn't improved the results.

Procedure of Q analysis

As for the following narratives all Q samples have been used (22 for the SD sorting and 19 for the EJ sorting). In Appendices A and B, the normalized factor scores for each discourse are listed.⁸ Strength of agreement and disagreement to single statements of the discourses are presented by both the ideal type discourse sort and the (better) comparable Z-score.

In the following, each factor will be narratively presented as a discourse A, B, C and D. Distinguishing statements⁹ are presented in Tables 5 and 6, classified as agreement, no agreement and disagreement.

Table 5.

Distinguishing Statements for Each Factor in the Sustainable Development (SD) Set

Discourses	Agreement ^a	No Agreement^b	Disagreement ^c
A	7 (+2; 1.09), 8 (+2; 0.68)	18 (−1; −0.57)	3 (−3; −1.36), 5 (−2; −0.98)
B	10 (+4; 1.58), 4 (+2; 1.35)	17 (0; 0.09), 3 (0; −0.08), 9 (−1; −0.53)	13 (−3; −1.27)
C	18 (+4; 1.50), 14 (+2; 0.66)	12 (+1; 0.59), 19 (−1; −0.28)	17 (−2; −1.09), 10 (−3; −1.10), 3 (−4; −2.44)
D	3 (+2; 0.86)	−	−

Rank +2 to +4.

Rank +1 to −1.

Rank −2 to −4.

Source: G. Kaufmann.

Table 6.

Distinguishing Statements for Each Factor in the Environmental Justice (EJ) Set

Discourses	Agreement ^a	No Agreement ^b	Disagreement ^c
A	19 (+4; 2.29)	10 (+1; 0.44), 1 (−1; −0.25), 15 (−1; −0.27)	3 (−2; −1.06), 4 (−3; −1.07), 17 (−3; −1.45)
B	18 (+4; 1.60), 12 (+3; 1.18), 3 (+2; 0.93), 7 (+2; 0.76)	9 (+1; 0.31)	10 (−2; −0.61), 1 (−2; −1.17), 5 (−3; −1.72)
C	17 (+3; 0.98)	1 (+1; 0.77)	4 (−2; −0.90), 3 (−3; −1.85), 19 (−4; −1.95)
D	4 (+3; 1.24)	3 (−1; −0.25)	9 (−2; −0.87), 12 (−3; −1.10), 1 (−4; −2.11)

Rank +2 to +4.

Rank +1 to −1.

Rank −2 to −4.

Source: G. Kaufmann.

Four Discourses on Sustainable Development

Discourse A: institutional eco-preservation driven, market-skeptical, pro “zero growth”

The most distinctive argument (7; +2/1.09) of this discourse and in opposition to all others (B −1/−0.75, C −2/−0.72, D −3/−1.37) emphasizes absolute limits to economic growth: this is why growth must stop at a certain point, since technical progress cannot qualify the environmental limits (5; −2/−0.98). This opinion criticizes the increasing anthropocentrism in the community of inhabitants as one of the most defining negative impacts of sustainable development on Algodoal-Maiandeua (8; +2/0.68) and does not follow the free-market liberal arguments that see positive impacts of the economic development on an individual (18; −1/−0.57) and societal level (3; −3/−1.36). The latter is of special importance since it is, like the first argument, the most distinguishing statement in the field and the only one which is declared significant for all four discourses. In other words, all four discourses have labelled this statement as important to defining their opinion, but with different loading (B: 3, 0/−0.08; C: 3, −4/−2.44; D: 3, +2/0.86). Discourse A's negative loading locates it close to discourse C, who refuses the “limits to growth” (C: −2/−0.72) and anthropocentrism (C: −3/−1.29) argumentation uncompromisingly.

Discourse B: environmental admonisher, institutionalism focussed

This viewpoint favors at the first place, the understanding that SD on APA Algodoal is here defined as originated by an equilibrium of human and environmental development without giving priority to one or the other (10, +4/1.58). It also warns of an approaching catastrophe for humanity in case of such non-consideration of environment's absolute limits to economic growth (4, +2/1.35). Very strongly, too, this perspective refuses the claim of drugs abuse on the islands as caused by human development (13, −3/−1.27). Rather no agreement is reached on the statement that economic growth were accompanied by felicity (9, −1/−0.53).¹⁰ It is the only discourse of the four that doesn't agree with the statement that the creation of the management plan should rather be proceeded via the inhabitants than via governmental institutions (17; +0/0.09). Neither was the argument of economic growth as a positive force able to eliminate the social disparities not at least for any acceptable environmental costs (3; +0/−0.08). In regards to the latter argument, all other perspectives in question here show a clear opinion to either agree or to disagree (3; A: 3/−1.36, C: −4/−2.44, +2/0.86).

On the other hand, strong agreement is found in regard to statements that deliver a more “technical” definition of the concept, whilst distinguishing arguments in regard to “political” questions. These questions are not refused, but avoided apparently due to commitment. This more “technical” approach considers less the very local, specific, and not generalizable opinions. It favors the more abstract and scientific understanding of the problem. Nevertheless, this abstract view brings about inside knowledge to the field problem too: Drugs for example are not at all a phenomenon arising from contemporary human development, but from tradition, at least in Fortalezinha¹¹ (Kaufmann 2003: 64–65). It might be questioned however whether the abuse of drugs is a consequence of human development.

Discourse C: anti-capitalist, social-liberal, practitioner

Basically, this discourse is framed by two statements, found on each end of the scale: Refused are notably positive impacts of economic growth on the elimination of existing social disparities by acceptable ecological costs are refused (C: 3, −4 compared to D: 3, +2; B: 3, 0; A: 3, −3). The negative Z-score loading here is also the strongest position within all discourses (C: 3, −2.44 vs. D: 3, +0.86; B: 3, −0.08, A: 3, −1.36). This is so not just with regards to the named statement, but also compared to all other statements, at least those that are recognized as either significant (ranging from B: +1.58 to A: −1.36) or non-significant (D: +1.894 to D: −1.881). The disbelief in free market forces in this discourse is accompanied by its strongest support (18, +4/1.50) for any of the other options, such as liberty and a better well-being [in the socio-economical life]. None of the other discourses would disagree to this fact, but none would agree too. Thus, discourse C alone would consider this statement as significant with a slightly negative loading (18, −1/−0.57). Furthermore the discourse here considered fairly agrees to the opinion, that prostitution is an impact of present sustainable development on Algodoal-Maiandeua (14, +2/0.66). It also agrees—though less univocally—to the viewpoint that nowadays (mass-) tourism on the islands is the superior solution to develop the island siblings. In particular, since fishing as hitherto is no source of revenue for all periods of the year (12, +1/0.59). As remarkable exception, this discourse does not disagree to this opinion as do all the other discourses. The only thing the discourse misses on the conservation unit, would be cars (19, −1/−0.28). On the other hand, the discourse either doesn't agree to the idea that inhabitants of the islands themselves should be those who create the management plan for development (17, −2/−1.09). By no means, sustainable development is seen as an equilibrium without priorities (10, −3/−1.10) and therefore this discourse opposes discourse B, which most strongly favors this argument as a definition of sustainable development in the regional context of islands Algodoal-Maiandeua (B: 10, +4/1.58).

Discourse D: market-liberal

This discourse is just based upon one statement. This however is the most distinguished statement in the field. Discourse D's agreement to statement 3 (+2/0.86) stands clearly for economic development without limits. It favors growth in order to resolve both the environmental and the social inequality problem set. The predominance of this argument bears to favor the resolution of social inequality claims to obtain the welcome benefits by accepted, since inevitable, dis-recognition of an environmental challenge to obtain the benefits.

Four Discourses on Environmental Justice

Discourse A: institutionalism focussed, communitarian, social

This discourse favors the waste problem as the major environmental justice problem on APA Algodoal (19, +4/2.29). It doesn't see the paupers as responsible nor as victims of the environmental damage (3, −2/−1.06). Even though injustice is recognized as results of stakeholder influence that injustice is strongly linked to their economic power only (17, −3/−1.45), the discourse disagrees with the opinion that institutions were unable to resolve the environmental question (5, −3/−1.07). For defining the concept of environmental justice, consequentially, communitarian rights to resource use have to be considered but aren't centered (10, +1/0.44). The latter conclusion then is also true for the representation of natives in the management council and for their cultural marginalization, a fact to which this is rather in disagreement (15, −1/−0.27). Even beyond, this one discourse in contrast assumes that it is rather in the nature of humans to simply pursue the goal of more and more money (1, −1/−0.25).

Discourse B: institutionalism focussed, environmental racism awareness, eco-analytical

This discourse argues that the majority of the population on the islands is excluded from environmental goods whilst suffering more than others from environmental costs (18, +4/1.60). Evidence is seen by cases of malaria and yellow fever (12, +3/1.18). The poor were both responsible and victim of environmental damages are the poor (3, +2/0.93), where the costs were carried by non-white people and future generations (7, +2/0.76). The obligations seen as key figure were to create a new concept of modernity through invention of a new civilization based on the foundation of respect, of knowledge about and of love to nature (9, +1/0.31). This viewpoint rejects any reconsideration of antique understandings in modern society (10, −2/−0.90). It assumes that institutions are generally able to answer the environmental question (5; −3/−1.72).

Discourse C: economical individualism, subjection of environmental concern

This discourse refuses that waste is the major problem on the islands Algodoal-Maiandeua (19, −4/−1.95). It supports the belief that stakeholders with more economical power have and should have more influence than those without that power (17, +3/0.98). Additionally, this opinion assumes (Z-score difference 0.21) that is not naturally given to mankind to want more and more money (1, +1/0.77). Environmental justice cannot be defined nor understood in terms of the current satisfaction of needs without sacrificing the needs of future generations (4; −2/−0.90). Last but not least, the understanding of paupers as the causers as well as the victims of environmental damage is neglected (3; −3/−1.85).

Discourse D: SD based environmentalist, EJ incorporating

This last discourse is defined through requesting not to sacrifice the needs of future generations in order to fulfil present needs (4, +3/1.24). It refuses the argument that it is mankind's nature to want more and more money (1, −4/−2.11). The discourse disagrees to the statement that asks for consideration of regional specifics and love to nature (9, −2/−0.87). Cases of malaria and yellow fever are no more the failure for environmental justice on Algodoal-Maiandeua (12; −3/−1.10).

Conclusion

As can be seen from the above analysis, the environmental problem set on Algodoal is more driven by thoughts on just participation in resource division than on broad ecological diversity (“biological” environment). The distinction between institutional perception on mitigating the existing problem set and the demands from the native community gives reasons to suggest the combination of the two claims by “buying” the support of locals. Consequently, the article asks for the attempt to realize institutional SD objectives through satisfying the demands of the indigenous community. As for the initial hypothesis on the extent of environmental justice discourses compared to SD discourses, the analysis could show that the difference between the two discourse sets is evident and proves the hypothesis correct.

Controversies within the given frame of chosen statements in the given field and with the chosen participants about the concept of EJ are more disputed than in the SD set: First of all, there has been not one consensus statement in the EJ set whilst the SD set had two. Additionally, in the EJ set more than three times as many statements have been significant for a minimum of two discourses than in the SD set (seven to two), and only one statement has been significant for all discourses in the SD set, but two have been in the EJ set. For interpreting these results, one might consider that a consensus statement would suggest that there is no dispute about this defining statement. So, if no consensus statement is found, a dispute about all statements consists. Second, the more factors (discourses) label one and the same statement as significant, the more controversial is the statements set. This is due to the fact that significant statements are defined by being loaded most differently compared to loading differences on all other statements. Consequentially, in both other cases the EJ set results trump the distinction of statements in the SD set.

Finally, Q methodology could demonstrate its high value for environmental justice research due to its qualitative and quantitative conceptions. The method helps to not miss the jungle of “environments” for the trees, so to speak, and thus should be taken into the list of powerful tools to reveal environmental injustice in the world.

Footnotes

Acknowledgments

My primary thanks go to the people from island Algodoal for taking part in the research, to the environmental secretary of Pará (SEMA) for allowing and supporting this research, to the local police, and all NGOs for helping me conducting this research. Additionally, I want to thank the institute for higher Amazon studies (NAEA) of the Federal University of Pará (UFPA) for their kind cooperation and support before, during, and after the research. Finally, I want to thank my father for encouragement and assistance in final proof reading of this article.

Author Disclosure Statement

The author has no conflicts of interest or financial ties to disclose.

Appendix B:

Normalized Factor Scores for Each Factor in the Environmental Justice Set

No	Statements/Discourses	A	B	C	D
1	It's not the nature of humans to want more and more money.	−1/−0.25	−2/−1.17	+1/+0.77	−4/−2.11
2	The government has to take more responsibility to create a better environmental legislation than actual are.	+3/+1.30	0/+0.03	−1/−0.08	+2/+0.83
3	The poor are both victims and causers of environmental damage.	−2/−1.06	+2/+0.93	−3/−1.85	−1/−0.25
4	Sustainable development is defined by satisfying the needs of today without sacrificing the needs of future generations.	+1/+0.16	+1/+0.41	−2/−0.90	+3/+1.25
5	Institutions cannot solve the environmental question.	−3/−1.07	−3/−1.72	+1/+0.28	+2/+0.90
6	The solution for the environmental question would be protests of the civil society.	0/+0.07	+1/+0.39	0/−0.03	−2/−0.74
7	The costs of development are paid by the poor or non-white people or by future generations.	−2/−0.68	+2/+0.79	−1/−0.26	−1/−0.60
8	The biggest problem isn't the depletability of resources but the consequences of their usage.	0/−0.20	0/+0.21	+2/+0.86	+1/+0.44
9	To understand Sustainable Development it is necessary to consider regional specifics, ideas of human respect, deep knowledge and love to nature.	+3/+1.15	+1/+0.31	+4/+1.57	−2/−0.87
10	In the past, work relations and relationship to the natural resources based on communitarian rights. Environmental Justice means to reconsider the antique understanding.	+1/+0.44	−2/−0.61	+3/+1.40	+4/+1.98
11	Development on Algodoal-Maiandeua is efficient and produces environmental justice.	−4/−1.74	−4/−1.82	−1/−0.55	−3/−1.08
12	The failure of environmental justice on Algodoal is significant due to cases of malaria and yellow fever.	−1/−0.35	+3/+1.18	0/+0.15	−3/−1.10
13	There are many cases of illegal land buying, selling and ownership of land. Environmental justice means to finish this illegal possession.	+1/+0.46	−1/−0.07	0/+0.16	−1/−0.62
14	Environmental Justice means the monitoring of enterprises to provide better environmental services.	+2/+0.76	−1/−0.16	−2/−0.80	+1/+0.78
15	The problem's cause is the under-representation of Indians in the participation organs. Indigenous culture there is marginalized.	−1/−0.27	+3/+1.35	+1/+0.53	+3/+1.12
16	The institutions of the government don't know the needs of the traditional population at local.	0/−0.25	−3/−1.48	−3/−1.08	0/−0.03
17	As fair salary for their success, the influence of the different stakeholders on Algodoal differs in accordance to their economic power.	−3/−1.45	−1/−0.20	+3/+0.98	0/+0.11
18	The people's majority has no opportunity to partake the environmental goods. Additionally, they suffer disproportionately.	+2/+0.68	+4/+1.60	+2/+0.82	+1/+0.16
19	Waste is the major problem in the APA Algodoal.	+4/+2.29	0/+0.04	−4/−1.95	0/−0.15

Source: G. Kaufmann.

References

Acselrad

, Campello do Amaral Mello

, and das Neves Bezerra

. 2009. O Que É Justica Ambiental?. Rio de Janeiro.

Barry

John

, and Proops

John

. 1999. “Seeking Sustainability Discourses with Q Methodology.” Ecological Economics 28 (March): 337–345. doi:10.1016/S0921-8009(98)00053-6. <http://linkinghub.elsevier.com/retrieve/pii/S0921800998000536>.

Bastos

Maria Nazaré do Carmo

. 1996. Caracterização Das Formações Vegetais Da Restinga Da Princesa, Ilha de Algodoal—Pará. Belém.

Brown

Steven R.

1980. Political Subjectivity. Applications of Q Methodology in Political Science. New Haven: Yale University Press.

Clapp

Jennifer

, and Dauvergne

Peter

. 2005. Paths to a Green World. The Political Economy of the Global Environment. Cambridge: MIT Press.

Dryzek

John S.

1997. The Politics of the Earth. Oxford.

Figueiredo

Elida Moura

, and Santana

Graca

. 2010. “A Transformação Da Princesa. Relatos de Pesquisas Na APA Algodoal/Maiandeua, Maracana, Pará, Brasil.” Amazônia: Ciencia and Desenvolvimento, 6: 211–229.

Foucault

Michel

. 1981. “The Order of Discourse.” In Untying the Text. A Post-Structuralist Reader, edited by Young

Robert

, 48–78. Boston, London, Henley: Routledge & Kegan Paul Ltd. <http://webspace.qmul.ac.uk/sbaumgarten/Foucault_The Order of Discourse.pdf>.

Gosine

, and Teelucksingh

. 2008. Environmental Justice and Racism in Canada: An Introduction. Toronto: Edmond Montgomery Publications Limited.

10.

Hopwood

Bill

, Mellor

Mary

, and O'Brien

Geoff

. 2005. “Sustainable Development: Mapping Different Approaches.” Sustainable Development, 13 (February): 38–52. doi:10.1002/sd.244. <http://doi.wiley.com/10.1002/sd.244>.

11.

Kameri-Mbote

Patricia

, and Cullet

Philippe

. 1996. “Environmental Justice and Sustainable Development. Integrating Local Communities in Environmental Management”. 1996-1. <www.ielrc.org/content/w9601.pdf>.

12.

Kaufmann

Götz Ferdinand

. 2012. “Environmental Inequality Patterns on Island Algodoal-Maiandeua. A Q Methodological Case Study.” Kieler Geographische Schriften, 123: 263–296.

13.

Kaufmann

Götz Ferdinand

, and Hurtienne

Thomas

. 2011. “Inglehart's World Value Survey and Q Methodology.” Journal of Human Subjectivity, 9 (2): 41–70.

14.

Kent State University. 2014. “Q-Method@listserv.kent.edu.” Official Website. <https://listserv.kent.edu/cgi-bin/wa.exe?A0=q-method>.

15.

McKeown

, and Thomas

D. B.

. 2013. Q Methodology. 2nd ed. Los Angeles: Sage Publications.

16.

McKeown

Bruce

, and Thomas

Dan

. 1988. Q Methodology. Newbury Park, London, New Delhi: Sage Publications.

17.

Nobre

2002. Desenvolvimento Sustentável. A Institucionalização de Um Conceito. Brasília.

18.

Previte

Josephine

, Pini

Barbara

, and Haslam-McKenzie

Fiona

. 2007. “Q Methodology and Rural Research.” Sociologia Ruralis, 47 (April): 135–147. doi:10.1111/j.1467-9523.2007.00433.x. <http://doi.wiley.com/10.1111/j.1467-9523.2007.00433.x>.

19.

Romeiro

Ademar R.

1999. “Desenvolvimento Sustentável E Mudança Institucional.” 68. Texto Para Discussão. Campinas. <http://www.eco.unicamp.br/docprod/downarq.php?id=1698&tp=a>.

20.

Schmolck

Peter

. 2013a. “The QMethod Page.”<http://schmolck.userweb.mwn.de/qmethod/>.

21.

Schmolck

Peter

. 2013b. “The QMethod Page.” PQMethod Software. <http://schmolck.userweb.mwn.de/qmethod/>.

22.

Secretaria do Meio Ambiente. 2010. “Voce Sabia? Unidade de Conservação Em Destaque.” Journal Informativo, 2: 1–2.

23.

Stephenson

William

. 1935. “Technique of Factor Analysis.” Nature, 136: 297.

24.

Stephenson

William

. 1953. The Study of Behavior. Chicago: University of Chicago Press.

25.

Webler

Thomas

, Danielson

Stentor

, and Tuler

Seth

. 2009. Using Q Method to Reveal Social Perspectives in Environmental Research. Greenfield, MA: Social and Environmental Research Institute.