My forthcoming book: “Learn to speak Evalian”
I have been working for a while now on a book provisionally entitled “Learn to speak Evalian”. This little post is just a place-holder with information about the book; I will update it when there is more to say.
I intend to publish it open-access, i.e. it should be primarily available online free of charge, perhaps with on-demand printing too. There will be a beta version and comments will be very welcome. It may even stay in a kind of permanent beta phase as I update and add to it.
If you are interested, drop me a line (steve AT pogol.net) and I will let you know when the first version is ready.
The book is not meant as an introduction to monitoring and evaluation (M&E)1. It will make most sense to people who have some practical experience and who are familiar with the theoretical outline of more than one well-known approach to these topics such as the logical framework approach (LFA) or Outcome Mapping.
The book might be of interest for social scientists in general but I am writing in particular for monitoring and evaluation (“M&E”) professionals - people who are responsible for monitoring and evaluating the success of and processes within many different kinds of projects and programmes, for example in international development, education and so on.
The book is disguised, in a light-hearted way, as an introduction to “Evalian”, the fictional language of a fictional country, Evalia. Evalian is a language very like English in which M&E concepts can be more easily expressed than in English. In particular Evalians are more precise in the way they express “Variables” and their possible (factual and counterfactual) “Values”, and the way in which these can be linked up into more or less deterministic “Mechanisms” which they describe with “Theories” about those Mechanisms. So concepts like “Effectiveness” and “Impact” are quite easy to define and use in Evalian. What’s more, many of the key ideas of a wide range of evaluation approaches from Outcome Mapping to RCTs can be quite neatly expressed in Evalian.
As a bonus, well-formed Evalian can be pasted into the Theory Maker web app to produce corresponding diagrams.
Many of the key ideas are based on the theory of causal networks set out in Pearl (2000)2
So, this book will teach you to speak Evalian, the fictional language of a fictional country, Evalia.
Why a fictional language?
Because the Evalians manage to communicate with one another about things without the misunderstandings which plague English and (I assume) the other languages which M&E staff use today. They can recite a Theory of Change or a project proposal as if it was an amusing story or an inspiring poem, and no-one argues about what it means or the words it contains. They often conduct Randomised Controlled Trials, almost for fun, but they also love Outcome Mapping and many other approaches to evaluation; and they see no contradiction in this.
So, “learning Evalian” helps us to introduce some words, concepts and conventions which we can use to think and write about projects and programmes from an evaluation perspective. Using these conventions, we can have a fairly standard way to communicate how we think projects work and what effects can be attributed to them.
So we will define some slightly new way of making causal statements, as well as some key words like Variable, Value of a Variable, and so on, for talking about these statements. We will do it in a way which is (I hope) compatible with most approaches in social science textbooks, but which is a bit different from them too:
- Evalians, although they are skilled at mathematics, are very comfortable with fuzzy and vague formulations and are very good at drawing precise conclusions from vague premises; they do not feel they are being sloppy when they don’t express a Variable in terms of numbers
- Evalians are particularly good at describing Differences between factual and counterfactual states of affairs; the concept of “Difference” is very important in evaluation.
Preview of first section
One of the most convenient things about Evalian is that it is very like English, with just a few tweaks. When Evalians are speaking about evaluation matters - variables, efficiency, outcomes, etc - they use a special kind of intonation which they capture in writing by using a special grey background like this:
This sentence is in Evalian.
For everything else, Evalians just use English. They do use quite a lot of Evalian though, because they like to pass time in the long evenings talking about things like attribution and contribution, bless them.
Written Evalian should be pretty easy for non-native speakers to read and write. However, it can be difficult for non-native speakers to speak.
For example, this piece of Evalian:
Teacher skills Teacher presence on training course
says something like “Teacher presence on training course contributes to Teacher skills”. The second line is indented by one space, and Evalians have a special kind of intonation for this which is difficult for non-native speakers to hear or say themselves.
In Chapter xx, we discuss a bit more how to speak Evalian. Right now, let’s concentrate on learning the written language. As unfortunately many Evalians are hard of hearing, they also use diagrams which exactly mimic statements in Evalian. We are grateful to them, because non-Evalians too often find diagrams easier to understand (especially with longer texts).
Another convenient thing about Evalian is that we English speakers can just use bits of it without learning the whole language. Hopefully this will lead, at least piecemeal, to better understanding between English speakers when speaking about things that matter in evaluation. So just as you can sparkle at a party (sometimes) by throwing in a bit of French, you can make a mark with your M&E colleagues with even just a few phrases of Evalian.
Theorymaker.info, a website which understands Evalian
The book is a companion to Theory Maker, at theorymaker.info, a (free and open-source) web app for quickly sketching out many different kinds of Theory of Change and other kinds of diagram like project theories and evaluation plans relevant to M&E.
Theory Maker speaks Evalian! At the Theory Maker website, you will see a text window on the left which you can type into. If you type in Evalian, a corresponding diagram is produced on the right.
Below most of the Evalian phrases in the book, you will see a diagram, the same one you would get at theorymaker.info if you typed it into the text window. In the book, you will see an “edit” button below each diagram. If you are reading the electronic version of this book, when you click the button you will be taken to the website where the text will be conveniently pasted in for you. There, can play with the text to see how it works, and perhaps adapt it to your own needs.
Sometimes “M&E” is used to mean a relatively low-level function, namely the mere monitoring of projects and programmes, contrasted with the more illustrious discipline of evaluation. I make no such distinction and refer to both as “M&E”.↩
Causality: models, reasoning and inference. Cambridge Univ Press.↩