Consider this yet another commitment to trying to write here a little more regularly. Lots of thinking has been going on but not much writing! At least not writing as I’m going…
Three things colliding over the past few weeks have led me to want to try and get some ideas down. The first was a conversation – one of a set really – with Titus Brown and Dan Katz relating to the application of political economy and collective action theory to communities building research.
The second was the posting of video of a talk I gave a month or so back to our research centre, the Centre for Culture and Technology at Curtin University. In this, I tried to work through the full story of some recent thinking for the first time. I’m trying to articulate what has evolved in my thinking from a view driven by network theory that open is good, more open is better, to an understanding of where and how things have gone wrong.
The final piece was the publication of a new paper (preprint version is also available, actually its what I read so that’s what I’ll link to) by Sabina Leonelli which tackles the value of reproducibility as a concept head on. Leonelli’s work has been a big influence on my developing thinking. Two pieces are particularly relevant. The idea that data is not a thing, but is a category defined by use. That is, we can’t say whether a particular object is data, only that something is data when it is used in particular ways. This is true of many things in research, the objectivist in us wants the qualities of objects like data, methods, and knowledge itself to be eternal platonic concepts inherent in the objects themselves. In practice however these qualities only reveal themselves in certain activities.
These activities depend in turn on ‘repertoires’ another concept I’ve taken from Leonelli’s work, this time from a paper published with Rachel Ankeny. The concept here is that research communities are defined by practices and that these practices are not just learned, but displayed and internalised, in a similar manner to a musician learning repertoire as a set of hurdles to be jumped in their development. I first read this paper at the same time as I was reading Lave and Wenger’s Situated Learning and the parallel between craft learning (see also Ravetz), identity making (see also Hartley and Potts) and community had a significant resonance for me.
In the new piece Leonelli tackles the range of different, and largely incommensurate meanings that are ascribed to the concept of ‘reproducibility’. The paper tackles a range of disciplinarily bound practices ranging from computational sciences, where there is ‘total control’ (yes I could quibble about machine states and compilers but this is philosophy, let’s accept the abstraction) through to ethnography, where there is the expectation that the life experience of the researcher affects the outcome, and that a different researcher would reach different conclusions.
The article focuses on how the necessity for documentation changes as different aspects of accountability rather than reproducibility are taken into account. These concepts of responsibility and accountability for sources of variability that have implications for the claims being made seems more productive and more readily contextualised and generalised. The article finishes with a call to value ‘epistemic diversity’. That is, it is through differing kinds of cross referencing, with different sources of variability, accountability, and reliability that we can most usefully build knowledge.
All of this resonates for me because I’ve been working towards a similar end from a different place. My concern has been thinking about how to operationalise social models of knowledge production. There are a broad class of constructivist models of knowledge where validation occurs when groups come together and contest claims. I have a paper currently in review that casts this as ‘productive conflict’.
In such models the greater the diversity of groups in contact the more ‘general’ the knowledge that is created. However the more diverse those groups are, the less likely it is that such contact will be productive. Indeed it can be harmful – and in most cases involving disadvantaged groups (which is practically everyone in comparison to a professional, permanently employed, western academic) it is at the very least a significant burden and frequently significantly damaging.
In the talk I’m trying to work through the question of how such contacts can be made safe, while equally privileging the value of closing ranks, returning to community to regain strength and re-affirm identity. If diversity is a first order principle in the utilitarian goal of making knowledge, what are the principles of care that allow us to protect the communities that contribute that diversity?
Part of the argument I make is that we need to return to the thinking about network structures. Five years ago I was making the case that ‘bigger is better’ underpinned by an assumption that even if benefits flowed to those who were already well connected the new opportunities of scale would still spread benefits more widely. That seems less and less to be the case today, and the key problems appear to be ones where the benefits of locality are reduced or broken, boundaries are too readily traversed, and groups come into contact in non-productive conflict too frequently.
In the talk I discuss the need to privilege locality. This is the opposite of what most of our systems do today, both in the wider world (‘you need to go viral!’) and in academia (‘international rankings! international journals!’). Many social models draw simple lines with groups inside or outside. Even with a mental view of groups within groups and overlapping groups this tends to miss the messiness that results from non-homogenous interactions across group boundaries. Network structural analysis is harder to do and harder to visualise but it seems to me crucial to get a view of what the optimal dynamics are.
One way to frame the question of network and community dynamics is economically. That is, to ask how the differing groups and communities in contact are sustained and succeed or fail. This in turn is a question of exchange (not necessarily purely financial exchange, but there are limitations to this framing!). Models of club economics and collective action from political economy are useful here and this leads us back to the first conversation I mentioned above.
Titus Brown has been working through a series of posts, relating to the NIH Data Commons project looking at the goods produced in open source projects through the lens of commons and clubs goods. Dan Katz was raising good questions about issues of time and scale that are not well captured by the classical analysis of goods.
I have quibbles with Brown’s analyses and often with others who use this kind of goods-class analysis. However I only just realised that my issue was similar to Leonelli’s point about data. The question of what is ‘the good’ in any particular case is relational, not an absolute. Our discussion of these models tends to turn on an implicit assumption that something (prestige, code, knowledge, expert attention) is the specific good in play, something that in turn surfaces a common assumption in most classical economics that it is appropriate to assume some kind of ‘numeraire good’ that allows exchange of all the others (this is what money is for, and if there’s anything wrong with economic analysis, its money..).
This doesn’t leave me with any firm conclusions beyond some possible routes to analysis. It seems likely that simple analysis based on static descriptions of community and the environment aren’t going to give us answers. We need to understand dynamics of groups opening up and enclosing over time, what sustains them, and what they might find valuable in exchange. We need agent based modelling of the network structures these occur over, and ways of translating from local structure in networks to more ‘bright-lined’ social models of groups. We need to move beyond the simple ‘economic mathematics’ that is often developed from the the club economics of Buchannan and Olson and to develop much more sophisticated and flexible models that allow the nature of exchange and goods to change when viewed from different perspectives.
All in all it emphasises the conclusion I’d reached, that diversity is a first order principle. All forms of diversity and the more diverse the better, provided the coordination between groups is principled. There’s a lot to unpack in that word ‘principled’ but I feel like its the link to the ethical dimension I’ve been missing. I need to dig deeper into notions of ‘care’ (Moore, Priego) and ‘flourishing’ (Holbrook) to tease this out.
