Evaluation provides evidence about whether and in what ways an initiative (e.g., programme, service, policy) is effective or successful. Evaluative evidence can inform decision-making about how an initiative can grow and develop, as well as how it might be implemented in other settings. This page looks at:
Kaupapa Māori is literally ‘a Māori way’. Taki (1996, p. 17) expands on this further in her consideration of the word ‘kaupapa’; namely that ‘kaupapa encapsulates…ground rules, customs, the right way of doing things’. The importance of the Māori language within Kaupapa Māori is further reiterated by Graham Smith (1997) who writes that the Kaupapa Māori paradigm in education is founded on three themes: taking for granted our right to be Māori, ensuring the survival of te reo Māori me ōna tīkanga (Māori language and customs), and the central place occupied by our struggle to control our own cultural well-being. According to Smith (1995), Kaupapa Māori:
In other words, the core of Kaupapa Māori is the catch-cry: ‘to be Māori is normal’. Tied to this is the recognition that Māori worldviews, ways of knowing and mātauranga Māori (Māori knowledge) are valid and legitimate. Kaupapa Māori is about our right to operate within this context, within Aotearoa (New Zealand). ‘Knowing’ is therefore sourced within our own values and beliefs. In addition, this knowing is positioned within a reflexive cycle of being willing to evolve, grow and update our knowing (Henry & Pene, 2001). Thus, our ancestors existed within a research culture whereby knowledge was updated ‘as part of ongoing information management practices’ (Reid 1999, p. 61). We are therefore not limited to merely responding to mainstream constructions of us, we can facilitate the revitalisation of traditional constructions as well as the formation of new constructions of what it means to be Māori within Aotearoa. There is a growing theorisation and practice of Kaupapa Māori across, for example, education, health, justice and social services. Kaupapa Māori informs practice, research and policy within these disciplines and within mainstream (where Māori groups operate), Māori, and Iwi (tribe) contexts.
Even though Māori and Iwi development, incorporating the provision of services and programmes to reduce Māori/non-Māori disparities, has always existed (Smith, 1995), the number of ‘flax roots’ initiatives has grown dramatically over the past 2-3 decades. This growth in the provision of ‘by Māori and Iwi, for Māori and Iwi’ services can be seen as one response to hapū (sub-tribe), Iwi (tribe) and Māori community dissatisfaction with mainstream services, including the perception that these mainstream services are responsible for maintaining Māori–non-Māori disparities rather than working actively to address them.
One of the things that may not have been working in the mainstream delivery of services is the “dominant, professional, and ‘expert’ driven service delivery model and system” (Ricks, Charlesworth, Bellefeuille & Field, 1999, p. xiii). Within this model services respond to people as individuals in isolation from their families, communities and social context; with an emphasis on the assessment of individual pathology and deficits (ibid.). This mainstream framework does not work for Māori.
In research on Māori and Iwi provider success, many Māori and Iwi organisations (of services across six different sectors including social services) described their motivation for beginning their service was the services that were not working for their community (Pipi et al., 2003).
Māori and Iwi initiatives represent aspirations by Māori and Iwi to regain control over their own lives and deliver services that are more in tune with whānau (family), hapū, Iwi and Māori community needs and aspirations. Māori and Iwi organisations told us that their dreams and goals revolve around tino rangatiratanga and mana motuhake (self-determination). They are also motivated by their desire to improve conditions for Māori, with their effectiveness reflected in the positive changes they see in the attitudes and behaviour of individuals and their whānau (family) (Pipi et al., 2003).
William Trochim writes that the most frequently used definition of evaluation is "the systematic assessment of the worth and merit of some object." He critiques this definition as being 'hardly perfect' and provides a second definition that "Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object." This second definition bypasses worth (that is, cost-effectiveness or value) and merit (that is, quality) and acknowledges that evaluation is about the collection, and shifting and sorting of information to assess its validity (that is, soundness), followed by a process of deriving inferences or conclusions from this data in order to provide useful feedback to those who are interested (e.g., organisations, funders, communities). Trochim has also provided a definition of programme evaluation as distinct from the other sorts of evaluation we may do in our everyday lives.
Kaupapa Māori evaluation with Māori and Iwi organisations encompasses a multitude of ideas that are sourced within what it means to be Māori:
Evaluation is therefore about ‘capturing’ what organisations are trying to do, including their motivations, their goals, and their understandings of how their programme or service provides something that their community needs and wants to be involved with. This is important, as the role of evaluation is often to document whether or not an organisation is achieving the outcomes contracted for by a funder. An in-depth understanding of the context in which organisations are working can, in turn, both mediate expectations and provide a platform for partnering relationships between organisations and funding agencies.
Evaluation can provide important feedback to an organisation about how well they are servicing their client group and their community. This feedback can enable an organisation to refine and tailor its services and programmes so that they are more responsive to, and successful at meeting the needs and facilitating the aspirations of, the organisation's client group and community. Evaluation is therefore an important decision-making tool that can contribute to an organisation's legitimacy and relevance within its community.
It's important that an organisation has the opportunity to partner in any external evaluation (i.e., an evaluation done by evaluators who are not part of the organisation). In a partnering relationship, such as that promoted within Kaupapa Māori evaluation, the organisation should be able to ensure that the evaluation meets its needs and requirements, as well as funder accountability requirements. Organisations are also encouraged to build their own evaluation capacity so that they understand more about evaluation, and can conduct their own, internal evaluations if desired. This building of evaluation capacity is embedded within Kaupapa Māori evaluation; that is, by being involved in a Kaupapa Māori evaluation organisations should expect to learn more about how evaluation is undertaken.
While being assessed or evaluated may carry with it a sense that an organisation is being judged, Kaupapa Māori evaluation is less about judgement and more about finding out how and why organisations are working for the benefit of their clients, whānau and communities, and what outcomes they are achieving through this work.
Cram, F. (2016). Lessons on decolonizing evaluation from Kaupapa Māori evaluation. Canadian Journal of Program Evaluation, 30.3 (Special Issue), 296-312.
Cram, F. & Mertens, D. (2016). Negotiating solidarity between Indigenous and transformative paradigms in evaluation. Evaluation Matters - He Take Tō Te Aromatawai, 2, 161-189.
Cram, F., Kennedy, V., Paipa, K., Pipi, K. & Wehipeihana, N. (2015). Being culturally responsive through Kaupapa Maori evaluation. In S. Hood, R. Hopson & H. Frierson (Eds.), Continuing the journey to reposition culture and cultural context in evaluation theory and practice. For the Evaluation and Society Book Series. Charlotte, NC: Information Age Publishing. pp.289-312.
Cram, F. (2004). Kaupapa Māori evaluation: Theories, practices, models, analyses. Paper prepared for the 2004 Evaluation Hui Summit, January 11-16, Waikiki. [KMEvaluation04.pdf] This paper was prepared for an indigenous evaluation hui between evaluators from Hawai'i and Aotearoa New Zealand.
Cram, F. (2004). Celebrating the ordinary: The practice of 'by Māori, for Māori evaluation. Paper presented at the American Evaluation Association Conference 'Evaluation 2004 Fundamental Issues', November 3-6, Atlanta, Georgia. [KMEvalAEA04.pdf]These last two papers can be downloaded below.
An overview of Maori programme evaluation, The evaluation hikoi, has been developed by Helen Moewaka Barnes and Te Rōpū Whariki, Massey University.
American Evaluation Association (2011) Public Statement on Cultural Competence in Evaluation.
Joan LaFrance (2004). Culturally competent evaluation in Indian Country. New Directions in Evaluation, 102, 39-50.
LaFrance, J., & Nichols, R. (2009). Indigenous evaluation framework: Telling our story in our place and time. Written for the American Higher Education Consortium. Alexandria, VA: American Indian Higher Education Consortium.
LaFrance, J., & Nicols, R. (2010). Reframing evaluation: Defining an Indigenous evaluation framework. The Canadian Journal of Program Evaluation , 23 (2), 13–31.
Kien Lee (2007). The importance of culture in evaluation. A practical guide for evaluators. The Colorado Trust.
Dawn Mackety (Ed.) (2012). Community-based participatory research and evaluation approaches in Native American communities. Washington, DC: National Indian Education Association.
National Collaborating Centre for Aboriginal Health (2013). Indigenous approaches to programme evaluation.
W.K. Kellogg Foundation (2000, 2003). Handbook of program evaluation. Battle Creek, MI: WKKF.
Cram, F., Kennedy, V., Paipa, K., Pipi, K. & Wehipeihana, N. (2015). Being culturally responsive through Kaupapa Maori evaluation. In S. Hood, R. Hopson, K. Obeidat & H. Frierson (Eds.), Continuing the journey to reposition culture and cultural context in evaluation theory and practice. For the Evaluation and Society Book Series. Charlotte, NC: Information Age Publishing.
Henry, E. & Pene, H. (2001). Kaupapa Māori: Locating indigenous ontology, epistemology and methodology within the academy. Organisation, 8, 234-242.
Pipi, K., Cram, F., Hawke, R., Hawke, S., Huriwai, TeM., Keefe, V., Mataki, T., Milne, M., Morgan, K., Small, K., Tuhaka, H. & Tuuta, C. (2003). Maori and iwi provider success: A Research report of interviews with successful iwi and Māori providers and government agencies. Wellington: Te Puni Kōkiri.
Reid, P. (1999). Te Pupuri i te Ao o te Tangata Whenua. In Davis, P., Dew, K. (Eds.), Health and Society: Perspectives from Aotearoa/New Zealand. Auckland: Oxford University Press.
Ricks, F., Charlesworth, J., Bellefeuille, G. & Field, A. (1999). All together now: Creating a social capital mosaic. Victoria: Frances Ricks & Ottawa: Vanier Institute of the Family.
Smith, G. (1995). Whakaoho whānau: New formations of whānau. He Pukenga Kōrero, 1, 18-36.
Smith, G. (1997). The development of Kaupapa Māori: Theory and praxis. Unpublished Ph.D. Thesis, Education Department, University of Auckland.
Taki, M. (1996). Kaupapa Māori and contemporary iwi resistance. Unpublished MA Thesis, Education Department, University of Auckland.
Trochim, W. (1998). An evaluation of Michael Scriven's "Minimalist theory: The least theory that practice requires." American Journal of Evaluation, 19(2), 243-249.
Wehipeihana, N., Oakden, J., Cram, F., Spee, K., Pipi, K. & Porima, L. (2011). Evaluation of the Māori oral health project. Wellington: Ministry of Health.
Cram, F. (2009). JR McKenzie Trust - Te Kāwai Toro Evaluation Report. Prepared for the JR McKenzie Trust. Wellington: Katoa Ltd. See the evaluation summary here.
Kawakami, A., Aton, K., Cram, F., Lai, M.K. & Porima, L. (2007). Improving the practice of evaluation through Indigenous values and methods: Decolonizing evaluation practice – returning the gaze from Hawai’i and Aotearoa. In P. Brandon & N. Smith (Eds.), Fundamental issues in evaluation. New York: Guilford Press. pp.219-242. Downloadable below