Slovensko društvo evalvatorjev

"Complex Society in Radical Middle" (December 2012), Bojan Radej*, Mojca Golobič**, Mirna Macur***. University of Ljubljana, Biotechnical Faculty, Department for Landscape Planning, 240 pp.

* Slovenian Evaluation Society, Ljubljana, Corresponding author (Ta e-poštni naslov je zaščiten proti smetenju. Potrebujete Javascript za pogled.)

** Biotechnical Faculty – Department for Landscape Planning, University of Ljubljana, Slovenia

*** Faculty for Applied Social Sciences, Nova Gorica, Slovenia

Society is complex because it is composed of irreconcilable constituents. As a consequence, every important social issue is evaluated conflictingly as being both horizontally incommensurable (among interest groups) and vertically incommensurable (micro – macro). Social incommensurability (Munda) as a category concerns incompatible but legitimate social claims. When different principles of social primacy are applied, no objective basis exists for rational choice between alternatives (Wacquant). This raises concern for the possibility of holistic evaluation of complex social matters. Because of an aggregation problem (Scriven; Virtanen Uusikylä) our ability to reach wide social consensus on the content and process of social transformation is critically undermined. Hence it appears that increased complexity leads to the disintegration of contemporary societies.

We are of the opposite opinion! By definition, holistic evaluation of incommensurable social issue is not achievable in a direct way. An indirect approach is needed – such as in a mesoscopic perspective. Its core theoretical background derives from mathematics (Pythagorean Hipassus) and philosophy of science (Kuhn) because of their elaboration of incommensurability concept, complexity theory with the concept of meso (Prigogine, Wallerstein, Leontief, Dopfer, Easterling, Kok, and Rotmans), and theory of social integration (Giddens, Bourdieu and Wallerstein) in sociology.

The major characteristic of meso evaluation is its capacity for addressing incommensurable oppositions in intersectional way. “Intersectional” means through overlaps between oppositions which take place only in non-essential instances that are only marginally important to the corresponding agents who pursue incommensurable claims. For example, social-economy is an overlap between conventional economic and social policies as two incommensurable components of sustainable welfare.

Intersectional character of the meso logic is schematised with Leontief’s input-output matrix and Venn’s diagram. They are suitable because they both combine non-overlapping meanings as primary or constitutive but also divisive, with overlapping meanings as only secondary in importance but correlative and thus capable of indirect synthesis of evaluation conclusions.

Meso approach to holistic evaluation is operationalized with a set of hybrid categories which are then applied in evaluation of five study cases. These categories are: weak in/commensurability, weak balance, weak cohesion, weak and double exclusion, bimodality, two step aggregation procedure, dual embeddedness, dual hermeneutics, concept of partial whole, and classification system of meso levels (meso 1, 2, 3a, 3b, 3c).

Study cases are selected from the realms of policy evaluation, public choice, organizational quality management and citizen’s self-organization. In each case we first identify conventional oversimplifications in the standard approach which does not take into account underlying social complexity. In a second step – with the introduction of hybrid categories – study cases are explained in meso perspective and resolved as complex ones. At the end of each experiment broader implications are drawn for the mesoscopic research of socially complex matters.

Meso approach has demonstrated its capacity to broaden and deepen the understanding of complex social matters. Practical implications of obtained conclusions are relevant for the policy programming, evaluation, coordination of policies, for integration of society, in particular for possibility of deantagonised handling of deep and substantive social confrontations.

First Case. Synthesis without a common denominator: The case of policy evaluation

Impact evaluators of large-scale and multi-domain policy interventions have had difficulties in aggregating detailed assessments results into a summative evaluative conclusion. Assumptions about how to aggregate assessed impacts from micro to macro level across multiple evaluation domains (economic, social, and environmental) differ and different approaches produce different end results. An aggregation problem (Scriven; Virtanen, Uusikylä) arises because different policy impacts are not commensurable across evaluation scales and across evaluation domains. Leopold et al. recognised this and decided to present evaluation results in a disaggregated manner. However, they failed to observe that evaluation of economic impacts on the environment are not strongly but only weakly incommensurable (strongly incommensurable are impacts of economic measures on economy with impacts of environmental measures on environment); they are only weakly incommensurable which allows for partial aggregation of detailed results. Ekins and Medhurst have appropriately defined the problem of incommensurable of impacts but failed to implement the observation consistently. They happened to overlook that incommensurability exists not only between different evaluated impacts (effects) but also between different policy measures (causes).

A new approach to the synthesis of evaluation results is proposed. Detailed impact assessments results are first partially aggregated into an input-output matrix of assessment domains (meso level) and then non-diagonally situated partial aggregates, defined as secondary impacts, are correlated. The methodological solution is illustrated by the comparative assessment of the sustainability of the development programme for the Pomurje region in Slovenia, using three methods: micro (no aggregation of impact results), macro (full aggregation) and meso (partial aggregation and correlation) approach. Only the latter is found consistent with the complexity of the challenge.

With the horizontal extension from three to four set model, meso logic is further elaborated. We comply with Dopfer’s et al. classification of meso 1, meso 2 and meso 3 levels, but we are able to upgrade it so as to add meso 3a, meso 3b and meso 3c levels of synthesis (Picture). This is possible because our approach is not only vertical but also includes a horizontal dimension and horizontal domains can be multiplied (for instance, human impact-H is added to the economic-G, social-S and environmental-O evaluation of the programme).

Picture: Four set Venn diagram of sustainable development

Three conclusions are drawn. Precondition for neutral evaluation is not only an objective analysis of data but also a consistent synthesis of analytical findings. Synthesis is not meant as a cumulative aggregate in a quantitative way, it has to be obtained correlatively between diverse qualities. Meso logic is extensive into itself.

Second case. Integration beyond polarities: The case of territorial cohesion

Social integration is classically presented as a dichotomy between its mechanic and organic counterparts (Durkheim). The former concerns preservation of structural order with the means of balancing principal system oppositions (negative integration). The latter is concerned with correlated interactions among members (positive integration) which produce cohesion. Mechanic and organic integration are opposite but also elementary for social integration.

Contemporary sociologists (Giddens) explain integration as a process of double hermeneutics. People produce structural order through their habituated interactions; and when established, structural order refines a framework inside which people socially interact. This explanation is circular. Circular explanations of emerging properties of phenomena are ordinary in social sciences, like for instance the invisible hand of the market, or evolution. It is not wrong to apply circular explanation to the phenomena which are circular in their nature. However, it fails to explain how one can interfere with integrative processes.

One needs to go beyond binary theories of social integration, to account for social complexity. For this purpose a new meso category of weak balance is proposed which translates the classical model from dual (mechanic/organic) to triadic, consisting of strong balance as a measure of mechanic integration, and weak cohesion with weak balance as the two organic measures of integration. Weak balance as a concept derives from correlation of secondary meanings. In evaluation of socially incommensurable matters, correlation cannot produce scalar result as it does conventionally, but decays into its two components – covariance which describes weak cohesion of organic integration and product of standard deviations which describes weak balance of organic integration.

As an aspect of balance, the concept of weak balance is closely linked to mechanic integration. Simultaneously, weak balance is also related to organic integration since it arises through correlation. Concept of weak balance is therefore doubly embedded in mechanic and organic aspect of social integration, which thus cease to be treated as binary divided. This demands reformulation of social integration equation with a concept of weak balance in the centre – measuring not the strength of secondary relationships (weak cohesion) but its mutuality. In a contemporary society where relative comparisons (resulting from differentiation) are important, interactions are valued higher when they exhibit multifunctional and plural character and when they share social content with the means of mutual understanding, recognition of legitimate oppositions, sharing the benefits of economic transactions fairly, global environmental and local social responsibility. The issue is illustrated by the example of evaluation of national energy programme’s impact on territorial cohesion of Slovenia.

Third case. Excellence Squared: Organizational self-deception

Internal evaluation is an intrinsic approach to the assessment of complex social concerns. These are in a large extent contextual and thus accessible for evaluation only by those involved in a specific context. Internal evaluation thus exposes evaluator to the risk of bias and self-deceptive assessment of complex social matters.

The issue is demonstrated on the case of self assessment of organization O from Slovene public sector with the Common Assessment Framework (CAF). It has been introduced as “a generic approach”, but there is a logical inconsistency in CAF between initial holistic aspirations of the assessment model and subsequently piecemeal assessment methodology. The problem is nested in the fragile theory of organizational quality (EFQM) borrowed from private firms. These are conceptually simple as they posses merely one and strictly consistent operational scope – profit, and require assessment on a single level – of the firm. On the other side, public sector’s organizations are complex, with differentiated scopes of operation and competing levels of judgment.

The problem is particularly evident in the synthesis of self-assessment results: CAF invests disproportionately large input in collection of detailed assessment data, sometimes surpassing two thirds of time required for the whole assessment. But in interpretation, aggregated quantitative results are left aside as less important assessment achievements in comparison to internal discussion about organizational quality which accompanied quantitative assessment.

To resolve the inconsistency between logical structure of the model and its practical implementation, a synthesis module is added to the standard CAF. Its meso logic first reorganizes the assessment from scoring of criteria to process evaluation which evaluates relationships between main organizational domains (leadership, employees, partners/users, wider society). Processes are assessed in relation to their contribution to organizational “added value” in transformation of its various allocated inputs into its prescribed outputs. Organizational inputs and outputs are usually administratively imposed from above, so the process is the most autonomous component of a given organization in public sector. Organization “adds value” in process terms only when its various inputs are assessed (scored) lower than its differentiated outputs. So in evaluation it is not only important how high inputs and outputs score but also how scores are distributed between them.

Organization is schematised into a matrix of sub-processes which operate partly independently from each other and partly overlap. Internal tensions and associations provide the possibility for nonbiased internal assessment on the level of organization as a whole. The experiment also points out that internal evaluation should always remain open externally. Lack of external communication of assessment results is one of the main self-reported weaknesses of standard CAF (Staes et al.) and the source of doubt about internal validity of its results.

Fourth Case. Equalsamong the firsts: Setting collective priorities

The neo-liberal model of economic development has proved to be strategically ineffective – because of its social injustice and environmental pollution with devastating welfare consequences. The model’s ineffectiveness is particularly observable in Pomurje region, Slovenia, with majority of population already categorised as vulnerable. A substitute development model of social economy is proposed. Its prime function is to employ inactive local resources for meeting unsatisfied local needs.

Social economy is conventionally defined as an overlap between economic and social motives of the entrepreneur. Instead of this binary view, an alternative three-part (or complex) definition is adopted. It sees social economy as an overlap between its economic, social and autonomous aspects. New understanding is exemplified on the attempt for ranking of ten project alternatives.

In principle, project or policy alternatives, which aspire for being selected as regional priorities, relate to the concerns which are of primary importance for specific interest group, but only of secondary importance for the region as a whole. Primary concerns on the regional level, such as major health and safety standards are always presented as urgent so there is no dilemma about their relative importance. Secondary context suggests that one needs to apply mesoscopic prioritisation of social economy’s alternatives. It puts forward the alternatives with the most intensive secondary impacts and thus projects which are the most synergetic for the region. As a result, regional priorities in social economy are achieved indirectly through side effects of regional stakeholders’ primary aspirations. Conclusion is that spill-over effects, positive externalities and multifunctionality are critical for regional development based on social economy.

Fifth Case. Through Exclusion to the Community: Organization and structure

Emergence of the antisystem movements, claiming to represent the 99% majority of the excluded population, has brought forward again demands to study society from the aspect of social exclusion. Exclusion in this case is not concerned with the dark side of contemporary societies (poor, marginalised – classical approach), but establishes itself as a starting point for reasoning about alternative social strategies which give rise to autonomous antisystem sociality (Landauer).

There are many modalities of social exclusion and these mirrors in diverse programmes of antisystem groups. As a result, these groups are not comparable in programmatic terms and when they need to interact, they favour unstructured interactions. For this reason they are not capable of organizing and structuring their antisystem potentials on a more operational and representative basis. This seriously obstructs their otherwise outstanding political potentials.

However, antisystem groups are not only programmatic units but also activist units. And there is evident discrepancy between the two – movements are more radical in programme (such as classical trade unions and NGOs) or in action (new movements, like Occupy). The idea is that it is exactly the »inconsistency« in their programme-activist footprint which is able to connect and structure the movements in a way that does not endanger their programme diversity. Movements with inversely similar footprint will tend to correlate into antisystemic hybrids to achieve balance between programme and action as a precondition for their internal consistency. The novelty approach requires a hybrid solution so it can be framed in the meso perspective. Three correlates are obtained: quasi, semi and orto-antisystemic correlate which complement each other in their capacity for mobilisation, production of autonomous alternatives and in provision of means to impose and defend the boundaries between the occupied autonomous zones and the system.

The emergence of antisystem structures (such as World forums, Anonymous, autonomous zones…) may have a profound impact on conventional conceptualisation of social conflicts. A classical “antisystemic conflict” between a system and society is in meso view reframed into a “system conflict” between system’s structure and antisystem's structure of correlated movements. Society can be now seen as located in the middle between two structures and being able to migrate between them depending on the need for more safety (which can be obtained in a system structure) or more autonomy (migrating back to antisystem structures).

Antagonistic struggles, previously taking place in the form of antisystemic conflict, are now enclosed in the system type of conflict. This leaves society deantagonised which is also why meso interference is seen as radical. This of course does not mean that society will eventually become less conflictual. Deantagonised milieu in a meso society gives rise to a post-binary and multi-polar class conflict. Now it is understood in meso logic as an irresolvable disagreement between multi-polar interest groups concerning the definition and appropriate implementation of their irreconcilable visions of social good. When society sets itself free from any particular type of structure (systemic or antisystemic), it releases a large part of internal tensions. More importantly, it also establishes capacities for self-ordering in decreasingly structured ways. In this way society decreasingly relies on the system as a negative precondition for social integration.

Complex society in the meso

The five methodological studies have been accomplished with the aim to illustrate difficulties in holistic evaluation of complex social matters. The main findings: it appears that trend of social disintegration is not a result of increasing social complexity but only of oversimplified approaches (micro or macro) to evaluation of socially complex concerns. For eliminating oversimplification, a meso frame is proposed. In meso frame, social totality does not emerge from commensurable elementary parts piled up together but from partial and incommensurable wholes, which remain separated and as such cannot directly coalesce toward ever higher unity. Instead, these partial wholes sometimes compete and sometimes cooperate for more holistic characterization of social good and for more integral strategies for achieving it. Hence, social totality can be emphasised only indirectly, preliminarily, in an open way and educationally.

Routine practices of oversimplification and generalisations are diagnosed in simplistic social research. Standard scientific approach stretched its logic far beyond its cognitive capacities. But this does not mean that standard approach failed in social sciences. Only its validity ought to be constrained to the narrow framework from which it is obtained. Normal science (Ravetz) is thus not useless in researching a complex world, but it needs a socially humble application to utterly acknowledge a plurality of incommensurable social truths.

Mesoscopic evaluation is not like science. It is internal – otherwise it cannot become authentic. It is also process-oriented – otherwise it cannot understand society as sets of interactions which undergo permanent transformation. And finally, it is holistic; without synthesis it cannot integrate the social meaning beyond its incommensurable origins.

Despite its enhanced capacity to deal with complex concerns, meso evaluation can not resolve the initially given dichotomy between primary and secondary meanings. But it nevertheless develops apparatus for their consistent treatment. Primary and secondary aspects need to be logically separated in the analytical phase of the evaluation but cognitively integrated in synthesis of meanings. The finding is that social matter has to be evaluated in a complex way. This simply means that it has to be evaluated in the golden ratio of its duality, stretched between the explanation of its primary meanings – which are constitutive for it, but in an incommensurable and deeply dividing way – and the explanation of its secondary meanings that are the only ones that lead to a holistic view, but merely in contents that are not of primary importance to any participating agent.

Incommensurability after all is not an obstacle to holistic thinking but a constituent of social complexity which requires treatment of social matters on a plural platform. When incommensurability is fully acknowledged and implemented methodologically in evaluation, further possibilities of integration become abundant in meso frame. Its double hermeneutics imposes a permanent request for advancing hybrid social forms to enable improved translation of social contents between primary Us and secondary Others.

In our view, new possibilities for social integration will be enhanced in at least three independent directions: in direct relations between each other privately, either locally or globally; in relation to all Others with whom we are only indirectly related; and finally in relation to the world as a whole and to the truth about it.

In the first case, social body is integrated through mutual recognition of aspirations of parties in interaction. This is achieved with the provision of weak balance between them. Ordinary market mechanism, fashioned in a neo-liberal style, for example, performs very poorly in this regard as it is driven by the philosophy of winners and losers.

The unknown Other is integrated by means of global responsibility of local interactions when participants evaluate local achievements in global context, beyond their primary concerns. Justification for global responsibility is not ethical but rational – in relation to everybody’s aspirations for the acquirement of a particular form of social good. As this is incommensurable, it cannot be achieved directly but only cohesively through secondary means with the provision of something valuable to the unknown Other – who is in this way invited to recognise our aspirations and so broaden our strategic chances.

As far as holistic aspiration for social world as such and its truths are concerned, meso logic is extensive into itself. It can proceed only with permanent disqualification of one’s particular origins from which these aspirations are uttered. In the end, every quest for truth and holistic understanding of social matters ought to enlighten the evaluator about the particular way in which his or her holistic aspirations have actually not yet been consistently socially complex.

Selected bibliography:

Bourdieu P. 1977. Outline of a Theory of Practice. Cambridge University Press, prevod R. Nice.

Derrida J. 1978. ‘Genesis and Structure’ and Phenomenology, v Derrida J., Writing and Difference. London & New York: Rutledge, transl. Alan Bass.

Dopfer K. 2006. The Origins of Meso Economics Schumpeter’s Legacy. Jena: Max Planck Institute of Economics, Evolutionary Economics Group, The Papers on Economics and Evolution št. 610.

Dopfer K., J. Foster, J. Potts. Micro–meso–macro. Springer, Journal of Evolutionary Economy 14/3(2004):263–79.

Easterling W.E., K. Kok. Emergent Properties of Scale in Global Environmental Modelling – Are There Any? Integrated Assessment 3/2–3(2002):233–246.

Ekins P., J. Medhurst. The European Structural Funds and Sustainable Development: A Methodology and Indicator Framework for Evaluation. Sage: Evaluation, 12/4(October 2006):474-95.

Feyerabend P. 1975. Against Method. London: New Left Books.

Foucault M. 2001. Arheologija vednosti. Ljubljana: Studia Humanitatis, Transl. U.Grilc, foreword M. Dolar.

Gödel K.F. Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme, I., Monatshefte fuer Mathematik und Physik, 38/173(1930).

Habermas J. 1984. The Philosophical Discourse of Modernity, Cambridge: Polity.

Hayek F.A. The Fatal Conceit: The Errors of Socialism, The University of Chicago Press. 1991.

Kuhn T.S. 1970. The Structure of Scientific Revolutions. Chicago: The University of Chicago Press.

Landauer G. Kroz odvajanje do zajednice; Durch Absonderung zur Gemeinschaft, predavanje 18. junij 1900 v Friedrichshagen-Berlin, v Izbor tekstova i odlomaka, transl. Aleksa Golijanin, Bojana Golijanin-Radonjić, 2011, Žurnali/ Lista blok-45.

Leontief W. Environmental Repercussion and the Economic Structure: An Input-Output Approach. Cambridge Mass.: The MIT Press, Review of Economics and Statistics, 52/3(1970):262-71.

Leopold L.B., F.E. Clarke, B.B. Hanshaw, J.R. Balsley. 1971. A procedure for evaluating environmental impact. Washington: Geological Survey Circular 645.

Munda G. Social multi-criteria evaluation: Methodological foundations and operational consequences. European Journal of Operational Research, 158/3(2004):662-77.

Radej B. Synthesis in policy impact assessment. Sage: Evaluation 17/2(April 2011):133–50,

Ravetz J. What is Post-Normal Science? Futures 31/7(1999):647–54.

Rotmans J. Scaling in Integrated Assessment: Problem or Challenge? Swets & Zeitlinger, Integrated Assessment, 3/2–3(2002):266–79.

Scriven M. The Final Synthesis. Sage, American Journal of Evaluation, 15/3(1994):367-82.

CAF 2006. 2007. Common Assessment Framework. Maastricht: EIPA – European Institute for Public Administration,

Staes P., N. Thijs, A. Stoffels, S. Geldof. 2011. Five Years of CAF 2006: From Adolescence to Maturity. Maastricht: EIPA, CAF Resource Centre.

Virtanen P., P. Uusikylä. Exploring the Missing Links between Cause and Effect A Conceptual Framework for Understanding Micro–Macro Conversions in Programme. Sage, Evaluation, 10/1(2004):77–91.

Wacquant L. 1997. Reading Bourdieu’s »Capital«, foreword to English translation “The State Nobility”, Cambridge, Polity Press.

Wallerstein I. 2004. The Uncertainties of Knowledge. Philadelphia: Temple University Press.


This is a long abstract from Radej B., M.Golobič, M.Macur. 2012. Complex society in the radical middle. Ljubljana: Department for Landscape Architecture BF/UL, foreword S. Dragoš, 248 pp. (in Slovene language),

Final version III/2013 (II/2013, XII/2012).

Proposal for citation: Radej B., 2013. Complex society in the radical middle - Long abstract. Ljubljana: Slovenian Evaluation Society,