John Daley: Give your decision-makers the report they need

Independent policy consultant and former Grattan Institute CEO John Daley explains how to write reports so that decision-makers will be able to use them to actually make policy. (That's what we want, right?)

Season 2 episode 7
Presenter: David Walker

Play the podcast

Download the podcast

    Click for MP3 file, 54.0 mb

Subscribe to the podcast in your preferred software

Apple Spotify Pocket Casts RSS

Share on social media

Transcript

David Walker: In this episode of Shorewalker on Reports, we talk with an independent policy consultant – John Daley.

Now, John Daley looms large in the world of reports. That’s in part because he was the founding head of an independent, centrist Australian policy think-tank called the Grattan Institute. He spent a decade running Grattan, and building it. He thinks more analytically about the report-production business than just about anyone else I know.

One of the most important points that John Daley makes in this podcast is that reports have to be written for decision-makers, with an eye to letting them make a decision as soon as possible. So here’s a preview, where Daley explains the situation of the people who probably commissioned that report from you, or asked someone else to.

John Daley: You’re talking to senior decision-makers. The thing that is invariably true about senior decision-makers is they are really, really time-poor. They just don’t have time to do anything. And they certainly don’t have time to get across the detail very much. And consequently, they are paying you to get across the detail and to think through that detail and synthesise it and realise that the consequence of all that is, you know, a conclusion. But what they want is the conclusion. And of course, that business of writing an elevator pitch is actually much harder than it looks.

David Walker: We’ll return to those time-poor decision-makers. But first, let’s go back to the start of John Daley’s career and find out how he gathered all that expertise at creating persuasive reports.

John Daley: I grew up in Canberra. Aged 17, I caught the first bus out of Canberra to go to Melbourne University. I did a degree in science and law, which, in retrospect, meant that I was one of those unusual people who both gets taught how to put together an argument, and who gets taught how to manipulate data and numbers. That turned it out to be very useful later on, when I wound up in the policy game. Also did a lot of music and theatre at university ...

David Walker: After earning university degrees in both science and law he chose a startling variety of different posts.

• He worked as a researcher for two big legal names – leading academic Cheryl Saunders, and the High Court’s Darryl Dawson – and for one luminary of Australian medicine, David Pennington.
• He also worked in the Victorian Premier’s Department.
• And he got himself an Oxford PhD in legal philosophy. As Daley puts it: “Of course, at the end of it, I had a doctorate from the University of Oxford and was otherwise unemployable, so I joined McKinsey.” He spent two years at McKinsey & Company, the world’s most famous management consultancy.
• And then he joined ANZ Bank as group head of strategy, and went on to run the bank’s Etrade Australia online stockbroking business.
• In 2010 John Daley was chosen as founding chief executive of the Grattan Institute, a government-established but mostly privately-funded think tank. It was this that really made his name in the reports business. At Grattan, he seems to have applied everything he had learnt across all those previous jobs.
• He grew Grattan from nothing into Australia’s leading policy think-tank in just eleven years.
• In the process, he re-established the idea that the centrist policy think-tank had a future in Australia, which some people – me included – had begun to doubt.
• Also as Grattan CEO, he oversaw production of more than 130 reports, and authored or co-authored more than 20 of them.

John Daley: And after finishing up at Grattan, after 12 and a half years, I went to EY Port Jackson Partners, as a consultant there for a bit – and then early in 2024 have set out on my own as a consultant, able to focus on things that I really want to do. So I’m doing a mixture of tax policy work, higher education policy work, some stuff on a brand new industry called online conveyancing, which has, if nothing else, the interest that it’s a brand new industry ... And so when we’re looking at how to regulate it, we really are working it out as we go along.

So one thing you can take away from that is that I have a very short attention span, and have moved from one thing to another as I found them really, really interesting.

David Walker: That’s John Daley. Few people bring as much experience to bear on the report-authoring business. So it makes sense to ask for his insights on how to make reports work.

 If you’re in charge of writing a report – for a senior executive, or a department head, or a minister who will have to publish it – John Daley lists three vital tasks at which you must succeed. They can make or break the project.

John Daley: You've got three things you've got to worry about when you're writing a report.

The first is making sure you are asking, and therefore answering, the right question. Secondly, you've got to do the analysis that actually answers the question. And then thirdly, you've got to communicate whatever you found in a way that's effective.

Vital task 1: Find out how to answer the right question(s)

John Daley: So to take the first of those: in terms of asking the right question, it's always worth, I think, spending quite a lot of time with the person who is commissioning the report, talking to them about: what do they really care about, what's their understanding of the problem, and being prepared to go back to them and say: "You think that your question is x, but actually I think your question is something slightly different, because this slightly different thing is, in fact, going to solve your real life problem, even though that's not necessarily how you might have characterised it to start with."

And that process – of really sitting with someone and understanding what problem are they trying to solve and getting clarity on it – takes longer than most people think.

But it's really worth the investment. Because if you get that wrong, or you don't get it quite right, then you wind up either doing a lot of work that you didn't need to do, or not doing a lot of work that you should have done. So that's part one, get the right question.

There's a lovely sort of one-page format that McKinsey used to use around the problem statement that was aimed at getting clarity around the question. I found it a really useful framework.

One of the most important parts of that framework was getting clarity about what was out of scope. So write down the things that, as you've heard it, you don't need to worry about. And make sure that the person who's commissioning it agrees that those indeed are the things that are out of scope. And that to-and-fro about what's in and out of scope, I think, can be really helpful.

Set up your management framework

David Walker: As you start your report project, you also need to establish a basic project management structure: you need to establish who has what kind of say in the project, what resources you control, the time you have before delivering the report, and how you will check in with the decision-makers while you are carrying out your investigation.

John Daley: Some of your listeners may be familiar with the so-called RASCI framework around just getting really clear about who’s got decision rights, who has the power to recommend, who has the power to advise, who has the right to be consulted (and) whose views are influential because they’re their views (even if they’re wrong), who has to be informed afterwards, but otherwise doesn’t need to know. Just getting really clear about who is in each of those categories is very helpful up front. [RASCI stands for Responsible, Accountable, Supportive, Consulted, and Informed.]

And then, of course, the other kind of classic project management things are around getting clarity around what resources you’ve got, what timeframe have you got, when are the check-ins? What is the routine that you want to set up so that you can keep bringing the emerging thinking back to the decision maker? Because the last thing you want is to go away for four months and then kind of come back with a finished report that’s actually probably going to be useless.

What you want to do is routinely engage with them to say: “This is how things are going”. Because, by definition, the person who’s commissioned the report is working in this area. They know a lot. They’ve got very good priors. They probably know more than you do. They’ve certainly probably got better instincts than you do, and being able to engage with them regularly on how things are emerging means that you get the benefit of those instincts and those steers around: “You’ve missed the following argument; you haven’t spoken to the following person; your evidence says this, but the real thing that’s going on is something completely different”. You know, you need a process that allows you to engage with your senior commissioner, that enables you to extract all of that out of their head. And they may not be particularly good at giving you that up-front. They may need you to kind of routinely be engaged with them to make sure that that gets extracted.

Work with a summary in mind

David Walker: And while you’re doing all this, Daley suggests you should remember an important rider: the people who commissioned you are paying you to provide an answer, not a book.

John Daley: You’re talking to senior decision-makers. The thing that is invariably true about senior decision-makers is they are really, really time-poor. They just don’t have time to do anything. And they certainly don’t have time to get across the detail very much. And consequently, they are paying you to get across the detail and to think through that detail and synthesise it and realise that the consequence of all that is, you know, a conclusion. But what they want is the conclusion. And of course, that business of writing an elevator pitch is actually much harder than it looks.

I wrote a thesis on why you should obey the Australian Constitution, to which my elevator pitch is now, “because that means, if you do obey a constitution, fewer people get shot and more hospitals get built”. Now as an elevator pitch, that’s very short and sharp – but it did take me about four years to get there.

So that business of sitting back and working out, “how do I compress what I want to say for a senior decision-maker” actually takes real time. So make the time – and that goes also for the summary of a report.

If you look at Grattan reports, you will discover that the first page, the summary page, is literally one page. It is only ever one page. It has never been more than one page. No matter how big, how complicated the report, we set ourselves a limit, which was that you only get one page. And if you can’t fit it onto one page, rewrite it. And it was an immensely powerful discipline in terms of thinking through, “if I have really limited time with a decision-maker, what am I going to tell them?” And I think that that’s a really good discipline for reports – that your executive summary, your true executive summary, should be literally one page. And because of the way that Grattan’s technology worked, there was no way to change the margins or the font spacing or anything like that. You couldn’t do any of those kinds of cheats. You just had to get it down to one page.

That spending the time to do that is really important. It will take way longer than you think it will. That’s the thing over which most of the arguments will happen within your team. That’s fine. That’s normal. Make sure you have those arguments and work it through. And that’s hopefully what will be really important and valuable to a decision-maker. And if they want to dive into detail, you will be fine, because you’ve written 80 pages, or whatever it might be, about all of the detail, and invariably, you will know way more about the detail than they do. And so if they’ve got questions, you can go there. But actually getting that top-level argument right is important.[DJW1] 

I think the other thing is making sure you’ve really thought about where is the audience coming from. And again, I think one of the vices of academia and policy and all the rest of it is that we do tend to think about logos [as the ancient Greeks called it] the whole time. You know, what’s the logical region, reasoned argument? And don’t get me wrong; I love a logical reasoned argument. It’s definitely better than the opposite.

But you also must think about the ethos and the pathos. What are the values that the decision-maker brings to this? What are the emotions that they bring to this? And how are you going to deal with those? You know, it’s an interesting reflection that up until the 18th century, you know, rhetoric was considered a major area of it was, in fact, considered a major discipline in its own right. It’s effectively now more or less disappeared. But all of the things that are in Aristotle’s rhetoric and were taught for literally thousands of years still work. The fundamental things that cause people to change their minds are not just the logical arguments; it’s also the values and the emotions. And if you haven’t figured out where your audience is coming from and how your argument and how your presentation is addressing those emotions and values, chances are they’re not going to change their mind.

Vital task 2: Analyse your problem

David Walker: So now you’ve set up. You’ve found out what questions you need to answer. You’ve established your management framework. You’re working with a summary in mind.

Now it’s time to actually figure out what it is that someone should do about your problem.

John Daley: So then the second thing you've got to do is, actually having worked out your question, do the analysis. And that's looking at the literature, and doing surveys, or running analysis on the numbers, or whatever it might be.

The trap here, I think, is that it's very easy to go and do the things where you know you've already got the data, or you know you've already got access to the right people, and you can write a whole lot of things down. The trap, however, is that those may or may not answer the question – and therefore they may or may not be particularly helpful. And it is very easy to think: "I'm being, you know, paid a bunch of money to do this; therefore I need to show that there's lots of activity; and if I look at this data source, which is readily available, I can produce lots of pages very quickly. And that is, you know, demonstrating my value." And the catch there, of course, is that that's really just an anxious parade of knowledge. It's not necessarily helping the person who has commissioned you.

And so starting with that question that you've established as part of the first step and saying "What analysis will really help to answer this question?" ... And one of the ways that I try and think about that is: "Let me just imagine what the possible answers of this analysis might be, the extreme answers – it might be 10, it might be 1. If it were 10, would that change my answer to the overall question, compared to if the answer was 1?"

And if the analysis is not going to change your answer to the overall question, well, don't do the analysis – because it's, by definition, not necessary.

So that business of just, before you start doing the work on any particular piece of it, asking "iIs this going to change the answer", I think is really also very valuable. It's one of the things that keeps scope manageable and means that you don't burn your team on the way through.

But it's psychologically difficult, because psychologically, we'd rather do things that look familiar, and where we know where we're going. And often the things that are going to answer a difficult question are precisely the things where we don't know where we're going. So having the self-discipline to ask "is this analysis going to change the answer" I think really makes a difference. And that means that maybe you spend more time doing analysis that's a bit tedious and where the data sources are a bit ropey, and where it's hard to kind of get at it. But if that's the work that's really going to answer the question, well, that's where you should spend the time.

Avoid the problem description trap

David Walker: Daley particularly advises you to beware filling your report up with data that merely describes the problem.

John Daley: I think the problem description trap is a real problem. Problem description is usually, as the name suggests, a description about the world. And we've got lots of data about what the world looks like – and usually describing the world is relatively uncontroversial. We are simply saying: "Here are the facts that we can observe about the world – lots of data, lots of numbers." We can draw pretty graphs and so on. But of course, for most people who are commissioning a report, they don't want a description of the world. They want to know: what should they do? And that's a much harder question in some ways, because that is inherently a normative question, not a descriptive question. It's one that asks not "what does the world look like", but "what should the world be".

That's inherently a much more controversial question. It invariably involves, to some extent at least, some value judgments. As soon as you start making those kind of value judgments, people start shooting at you. As soon as you say, not just "this is how the world is", but "this is how we ought to make the world different", people tend to start shooting at you. And so we tend to spend a lot of time describing the problem and then keep the "therefore this is what you should do" pretty short. And unfortunately, I don't think that that's usually the right balance.

And out in the real world, you can see this happening a lot. The New Democracy Foundation did a lovely piece of work where they commissioned two different think-tanks – one on the right, one on the left – to look at a whole series of legislative processes and ask, how had the problem worked? How had the policymakers worked through it? And despite the fact that the think-tanks were coming from very different ideological positions, they came to very similar conclusions – that by and large, on each of these individual measures, there had tended to be a lot of discussion about what the problem was, often quite a lot of discussion about   "therefore we should do X", but very little discussion about: "What are the different things that we could do to solve this problem, what is the evidence that one of them might be better than the others, and therefore, which answer should we choose?" And I think that's also true in policy work, it's very easy to spend lots of time on description and then jump to a policy answer that sounds as though it's got something to do with the problem we've just described, and then declare victory.

And often you don't find much time working through given the problem you've described: "Here are the various plausible policy levers that might help, here are the pros and cons of each of those levers, and given all of that, this is the best lever we should choose."

Now, all of the sort of regulatory impact statements and all of those kinds of processes are designed to try and make people do that process properly, of thinking through the alternatives. But that is the step in the process that people most often jump over, because it's hard, and often the evidence is pretty ropey. But the fact that it's ropey doesn't mean that you shouldn't do it. It just means that you've got to give it the best shot you can and be as disciplined as you can – so that at the end of the day, instead of just coming up with something where you say, "Well, this policy action seems to have something to do with the problem I've described", instead, you're saying: "This policy action is one that is most likely to actually improve the world, given the problem I have described in the first half of the report." So yes, I think there's a real danger of falling into the trap of spending way too much time describing the problem and not nearly enough time working through what are the different policy solutions that might help – and of those, which would be the best one in terms of really solving the problem that's been identified.

A simple analytical technique

David Walker: Now, say that you’ve accepted that you can’t just answer the question with problem description. You’ve accepted that you really need to come up with answers. Now your task may suddenly feel much more intimidating. You will have to perform some real analysis that has real explanatory power. How do you go about that?

John Daley uses one technique which is remarkably straightforward: start ranking your potential answers according to the various impacts they will have. He argues the powerful explanation may jump straight out at you when you finish this exercise – and if it doesn’t, this technique will at least point you to places where you need to do more research.

John Daley: The thing I found most useful is a bizarrely simple tool which says: "What are the various criteria that we care about in terms of the policy solutions?" How much, for example, is it in fact going to improve housing affordability, if that's the problem we're trying to solve? So, if you want to put numbers around that: How many people will be able to buy houses that couldn't buy houses before? How many people will spend less on their housing than they used to, and by how much? But then we also care about – because it's always the collateral impacts that kill us – how much is this going to cost the budget? What is its economic effect going to be? How will it change distribution? Will it make distribution more unequal or less unequal, and all of the other things that you might care about. And then for each of your policy solutions, just rank them on each of those criteria.

[That] is not particularly complicated. One of the things that I think that people get very hung on about is that they need to be able to weight all of these criteria, and they need to be able to add them up to come up with a single number. And of course, the problem is that often these criteria are very difficult to add. I actually think that by and large, you do better not to worry about that weighting. Just describe each of these policy solutions against each of these criteria.

And often what happens is that the answer is then blindingly obvious. For example,  I've got lots that are going to have no impact, or at least very little impact, on housing, on the number of people that buy houses, or on the percentage of people's income that's they're spending on housing. So we can knock all of those out to start with. I've got a number that have got horrible economic impact. So we can probably knock those out. Let's see what's left. And so that simple table – here's all of down the left hand side, here's all of my policy solutions, and this is how they rank on each of the major criteria that I care about – [is] often not actually particularly hard to do, and it can make the answer to your problem much, much clearer, very quickly. And occasionally, there's some analysis that you could do that will say: "to what extent will these things really help?" And then you've got to go and do that analysis.

[INSERT GRATTAN HOUSING REPORT TABLE]

Vital task 3: Communicate the answer(s)

David Walker: So you go and do the analysis. Are you finished now? Not even close. Because your analysis is not the main product that the people who asked for the report really want to see. Most of all, they want the answer to their problems.

John Daley: And then thirdly, of course, you've got to communicate about the answer. And of course, here, the problem is that we tend to communicate the answer back essentially replicating the steps we went through in order to answer the question. And that, if you were trying to guide a future researcher who was replicating your project, would be really helpful. But by and large, that's not our audience. By and large, our audience is someone who's asked the question, or someone like someone who's asked the question. And what they want to know is the answer.

And so usually the way we should communicate back the answer is completely different from the steps that we went through in order to come up with the answer. And of course, that's completely different to your typical scientific or economic report, where ... you try and make it easy for someone to replicate it. You say: "Here's my question, and here's my literature review, and then here is my methodology, and then here is my data sources, and then here is the numbers that pop out of the data, and here's my analysis – and then finally, here are my implications, the things that I think might be the implications for the real world, and what you ought to do." That may or may not be a good way to write an economics paper, but it's a terrible way to communicate anything to busy decision makers who usually don't have a lot of time. And you will help them most, and they will be probably most grateful, if you walk in and say your question was X, and the answer is Y, and the key reasons are 1,2,3. And then if they want to dig into the methodology, off you go. But make sure that you communicate in that way, which is very much answer-first.

Constructing answers and arguments

David Walker: John Daley wants to warn you that communicating your answers may be a new sort of challenge. For a report, you construct your argument backwards from the answer you’ve reached. That’s the opposite of how you probably do it most of the time.

John Daley: This is something that academics really struggle with, because if you look at the typical learned journal article report, it basically says, you know: “We’ve thought up this interesting question. Our methodology for attacking this question is A, B and C. So we’ve gone and collected a lot of data according to the methodology that we’ve just laid out in enormous detail. And here are our kind of complicated tables, and here’s all of the statistical analysis. And the conclusion of this is that A is better than B.”

And so that is effectively the process that you go through in terms of analysing a question. So in terms of working out “well, how should I go about doing the analysis”, that’s actually not a bad description. But in terms of communication, it’s terrible – because, of course, in terms of communicating it, what you should be doing is saying “you’re probably interested in this question and A is better than B. The reasons that A is better than B is 1-2-3, and the methodology that’s led me to 1-2-3 is here, if you’re really interested, but you’re probably not.”

So the communication strategy is totally different from the report strategy.

But if we go back to that report strategy, the way that I always think about it is also slightly reasoning from the answer, which is to ask: “What is the choice that the policy maker faces?” As Paul Keating would put it, what are the policy levers they’ve got ... Their choice is between pulling policy lever A and policy lever B. Then your report is essentially about asking: “Is A better than B?” And then you go and ask: “What is all the evidence that ... would lead me to conclude that A is better than B, or not?” And then go and collect that evidence and then find out what it says.

Now that’s partly an iterative process. As you get into it, you discover that there are things that influence whether or not A is better than B that you didn’t start off with. But ... you at least start with: “Here are all of the dimensions on which you know we can compare A and B. Let’s go and find out about each of these dimensions, how A compares with B, and then off we go from there.” Now that’s a very simplistic thing, where you’ve kind of got something that’s very binary. If you’ve got something a lot more complicated, like housing affordability, then you start off by saying: “Lots of people care about housing affordability. Well, question one, what do they mean by housing affordability? Well, we can break that up into several different ways of thinking about that. Then let’s go and get the evidence about which of those things is, in fact, getting worse or not. Then let’s go and get the evidence about which kinds of policy levers might affect that. Let’s then get the evidence about which of those policy levers would work best. And then we can bring that all together at the end and say, if you care about housing affordability, then this is the policy lever that will actually help.” So that’s, I think, how you attack the report.

One of the other things that you need to do is, in that process – and this is very much something I learned at McKinsey – before you do the analysis, actually sit down and think: “If I do this little piece of analysis, or big piece of analysis, let’s imagine the possible answers – and if the answers are kind of at one end, how will that change my ultimate conclusion, and if the answers are at the other end, how will that change my conclusion?” And if you think about that thought experiment and say, “you know, it doesn’t actually matter whether the data comes out at the far right-hand end or the far left-hand end; I’m going to do the same thing in terms of the policy answer”, well then – and this is the crucial piece – don’t do the analysis. It’s not going to change the answer.

… And what that leads you to do is you do a smaller number of pieces of analysis. Often they’re the really hard ones, but by definition, there are the things that might actually change your view about what is the right thing to do as a result. And it’s one of the things that’s very different from academic analysis. In academic analysis, if you can find out something that’s interesting about the world and get the right kind of statistics behind it, then it’s publishable and it’s a good idea. Whereas, if you’re in the business of policy report writing, inherently, you’re in the business of practical action; you’re in the business of trying to explain to people why they should do A not B. And so the mere fact that a piece of analysis is fascinating and interesting is not good enough if it’s not going to help people decide whether or not they should do A rather than B.

Case study: housing affordability

David Walker: Let’s look at how some of this works in real life. Daley led Grattan’s 2018 work on an emerging policy problem: Australian housing costs too much. This work was, we can see now, among Grattan’s most successful. It influenced other work. All that work eventually led the New South Wales, Victorian and federal governments to change their minds about housing. By 2024, they embraced the idea that Australia’s housing problems would best be fixed by … yes, building more housing. That sounds simple. But it represented a huge and pivotal change in policymakers’ mindsets

This report has a lot to teach about how you address a complex subject and make governments change their actions. This one also has a lot to tell us about surprises: reports don’t always stick to the neat process we’ve just talked about.

John Daley: We wrote 177 pages for our housing affordability report, which I think at the time was a Grattan record by way too much. And ... given that I was the principal author on that report, there was an edict that went out from the CEO shortly afterwards to say that no Grattan report was to be more than 100 pages ever again. But it was partly because housing markets are genuinely extremely complicated – and interestingly, in Australia, and somewhat bizarrely, given how important they are, somewhat under-researched. For whatever reason, there had not been a lot of academics doing work in that space. The Reserve Bank had done a lot of very fine work, but there are limits to how much the Reserve Bank can publish, and government departments had published very little, and academia-wise not that much [research had been done]. And so there was a lot of things to work through. We did work through as many of the elements as we could. We spoke to a lot of people. So every time someone came up with a new objection, we chased it down. But we did come to the end of that report, and then when we came to write the summary of it, my summary in one line was: “If you want housing to be more affordable, you should build more of it”. Now there’s actually a very profound truth to that claim. Bizarrely, there are actually quite a lot of people, most of whom live in planning departments, who actually think that that claim is untrue. And so the report is partly 177 pages trying to show them why they’re wrong, and that if you do, in fact, build more housing, it will be more affordable. But given all of the noise in housing affordability and the way that many of the policy responses that governments come up with sound terrific, but unfortunately do not involve building more of it, almost inevitably they don’t in fact make housing more affordable. And so that’s why sometimes you have to do the very thorough analysis at enormous length. And it’s only after you have done that that you can do something very simple at the end of it. But every time you get challenged, you can go another level down and another level down, another level down, to show that this overall answer is, in fact, right.

David Walker: Once you’ve done all this research and analysis, you have a draft to circulate – to colleagues, and to trusted third parties who you’ve been consulting with. John Daley advises you not to treat this stage as a formality. Sometimes it can start your report in a new direction. That was the case for Grattan’s 2018 housing affordability report.

John Daley: So we’d written a draft. We circulated the draft to lots of people, because one of the ways that you find out all the ways that you’re wrong or things you missed, is to circulate people to draft report and ask them what they think – and then take their answers seriously. So one of the things that that means is: Don’t circulate a draft report and then plan on publishing it four weeks after you’ve circulated it. Circulate your draft report and plan on publishing it three months later, because it will genuinely take you that long to really think about, incorporate, where necessary change, where necessary go and do more research, all of the things that come back, and that’s the difference between not a bad piece of work and a really robust piece of work.

And so we put out a draft, and one of the things that came back was: “Look, you’re saying that, you know, it’s all hopeless, and no one’s actually doing any substantial development in middle rings of Sydney and Melbourne. But actually, that’s not true. In the last two or three years, we’ve actually done a really good job in Sydney.” And we went: “Okay, okay, well, we better go and find out whether or not that’s true.” So we went and found a dataset and came to the conclusion that actually we were wrong, and that particularly in the previous two or three years, there’d been a lot of development in Sydney, particularly in the middle suburbs. We thought, “well that’s interesting”. And so we certainly brought that up and showed that this had really changed things. And then we ... I can’t remember how I found it, but I stumbled across a map of where all of the cranes are in Sydney and Melbourne. Someone puts out this regular report, literally mapping them ... and the patterns were really different. So in Melbourne, basically there were a stack of cranes in inner Melbourne, central Melbourne, so obviously in the CBD, but particularly in Southbank, as all of those towers, accommodation towers were going up in in Southbank, but, but frankly, not very much anywhere else. Sydney, the map was opposite. There was almost nothing going on in central Sydney. But there were these lines of cranes basically on all of the major transport routes out of Sydney, out of central Sydney, so all of the big roads into the suburbs. And by definition, a crane is building a building, at least for probably more like about 10 stories. And so we went and talked to some people about in Sydney, and said: “This is the map. Please explain to us what is, in fact, going on here. And by the way, we also see the evidence that there’s lots of new apartments being built in middle Sydney.” And they said: “Oh yeah, what’s happened is that lots of people have been building six- to eight-storey apartments in Sydney along the major roads. Because the planning scheme has always said that that was possible, but that never happened, because local councils always said no, but the New South Wales government has brought in this new independent planning panel, and all of the developers are showing up in front of the planning panel, saying ‘we want to build our eight story development, and by the way, it’s consistent with the planning scheme and what the rules say.’ And the planning panel, which is not particularly responsive to the views of local residents, just keeps saying, ‘well, that’s what the planning scheme says,’ and the planning scheme’s there for good reason [so] knock yourselves out, go and build your building.’ And that’s why so much has been built.”

And without the cranes map, we would probably never have asked the right question. We, of course, did ask the subsequent question, which was: “So was this independent planning panel a brilliant idea of the New South Wales government to solve their housing affordability problems?” And people said: “Ah, no. No, no; the independent planning panel is there because the New South Wales government thought that a lot of the local councils were corrupt and were being excessively influenced by developers.” Can’t imagine why; it would never happen in Melbourne. And consequently they set the planning panel up because it would be less corrupt than the local councils. So ironically, yes, it is true that the planning panel has made a material difference to building in Sydney, and consequently, over the long run, building housing affordability in Sydney is marginally less bad than it would be otherwise. But that was, in fact, not the problem that government was trying to solve at the time. And, you know, would not be the first time that we had done the right policy thing for the wrong reason.

David Walker: One of the lessons of that is that to really understand the problem, you just have to get inside it and walk around a bit looking for interesting things.

John Daley: Yeah, and in particular, you have to look for things that that that are different from your priors. I mean, we were talking earlier about, you know, the difficulty sometimes of really incorporating evidence that’s different to your priors. But one of the ways I think about that is if you come across something that you think is surprising, it doesn’t fit your priors. by definition, it’s interesting. You probably someone who’s been playing this policy game for a while. You your priors. are probably pretty good. So if your guess is A and in fact, the evidence turns out to be something completely different. Well, not only is that news to you, but it’s probably going to be news to a lot of other people, and it almost certainly is revealing something which is not obvious about the world, and is therefore probably quite important and quite interesting, because you know, public policy, the thing we all are terrified about is unintended consequences, and the way that you forestall unintended consequences is to find out stuff that is hidden and so looking at those things that are surprises and genuinely understanding them is really important, because by definition, almost they are going to tell not only you but your readers something that they will find genuinely new and different and interesting and probably important. I mean, if it’s contrary to your priors, it’s almost certainly important as well as being interesting.

David Walker: With a full set of facts, analysis and arguments on the Australian housing market, the Grattan team released their housing affordability report to the public. It did not immediately provoke governments to change their housing policies. But Daley did not expect it to.

Daley and his Grattan co-author, Brendan Coates, then set out to both stress-test and publicise their views. And that involved a trip to the bank – the Reserve Bank, Australia’s central bank and the government institution which pays most attention to the dynamics of housing markets. One of the Reserve Bank’s economists was Peter Tulip, who was already thinking along the same lines as the Grattan team.

John Daley: The Reserve Bank, as I said, has always been very interested in housing markets. Lucy Ellis, who, of course, now is at Westpac, but at the time, was very senior at the Reserve Bank, I think is perhaps the most expert of all analysts of the Australian housing market. So we went and presented the draft work to the Reserve Bank, which I think is probably the toughest crowd that one can present in front of in Australia. The senior group the Treasury is pretty close, but the Reserve Bank certainly seems to have a culture, when external people come, of asking a lot of very difficult questions. So we did present it to them. We got a lot of very good questions and a lot of things we had to follow up. But we did also inspire Peter Tulip a little bit, because we said: “There’s this fantastic piece of analysis that’s been done by a guy called Ed Glaeser in America that essentially works out how much of housing prices are essentially because of the cost of building, and how much is essentially the cost of land – you know, per square meter of land – and how much is effectively the scarcity value of being allowed to subsidise. And of course, you can’t change how much land there is, and it’s not easy to change building costs, but you can change how easy it is to subdivide.” And Peter followed up with that, and then promptly did that analysis – which is a long and complicated piece of analysis that requires a lot of detail, a lot of access to data which the Reserve Bank had. And he published it. And it was a fantastic piece of work that proved what we very, very strongly suspected, which is that a lot of the price in Australian major cities, particularly in Sydney and Melbourne – I think from memory about 35 per cent of the price is essentially what’s known in the trade as the zoning value, the fact that you’re not allowed to subdivide, as opposed to the value of the land – and that therefore, changing planning policies, implicitly, will have quite a big impact over the long term on housing affordability.

David Walker: After writing that analysis, economist Peter Tulip wrote or co-wrote more Reserve Bank papers on housing affordability. Then in 2020 Tulip moved to the centre-right think-tank The Centre for Independent Studies, and began advocating for governments to allow more home-building with fewer conditions. That advocacy caught the attention of what was then the New South Wales Productivity Commission, which one of Tulip’s Reserve Bank co-authors had joined. By 2023, a newly-elected New South Wales government had adopted those views as one way to ease the New South Wales housing shortage. The Victorian government soon followed.

Six years after Grattan’s original report, its impacts were still being felt. As Daley points out, public policy moves slowly – but it does move.

John Daley: Governments’ levers are, in a funny way, very large. The cost of missteps is extremely high. And consequently, we have designed ourselves government systems that that are not quick to move. Democracy is not designed to be fast. It’s designed to avoid really bad outcomes. And therefore, if you’re trying to get good outcomes, as the famous Weber phrase goes, it’s slow boring through hard boards. So yes, it’s slow. And ... the cumulative effect of multiple people looking at something and doing their report and saying, “Yup, it really looks like A is going to be better than B”, progressively changes the conversation about something. Certainly at the time that we wrote Housing Affordability, by that stage, the debater got to look you’re probably right that changing planning stuff will make a big difference, but it’s all too hard. And today, interestingly, we’ve got a Victorian premier going full bore on “We’re going to really, really, really change planning policy,” in ways that lots of people in Brighton are very upset about, and which, if it came off, would genuinely make a difference over the long run. So what was politically impossible now has a major political party in power, in Victoria, advocating it full bore. So things do change.

I think it very much fits with some of the work that I did in ... The other report that I wrote in my final year at Grattan was one called Gridlock, which was essentially looking at [the question]: “Why do some policies get changed, and why do lots of good ideas not happen?” And one of the things that we concluded out of that report was that the state of the evidence base makes a really big difference. If the evidence base is weak – if the only thing going for you is a Grattan report – then chances are it won’t happen. And in particular, major lobby groups will almost certainly succeed in overturning it, or at least stopping it happening. Whereas if the evidence base is really strong, then chances are it will happen.

And there’s a lovely case example of this. Grattan, as part of that housing affordability work had done some work on rental conditions in Australia. A third of the population rents. Australia, by world standards, has very, very pro-landlord rental laws, so tenants have relatively little rights. So until relatively recently, for example, in most Australian states, you had no right to hang things on the walls. You had no right to have a pet. The landlord had ability to basically keep the tenants out very easily. And that’s in big contrast to how, for example, a lot of European tenancy works.

In Victoria, partly as a result of Grattan, maybe, but partly as a result of lots of other people, no doubt, the government got interested in this commissioned the consumer affairs area commissioner to do a big report. And they did a very, very large piece of work. They consulted with literally thousands of different people. They wrote literally hundreds and hundreds of pages on this, essentially working through all of the arguments that landlords put up to say that “if you do this, then you know it’ll be the end of the rental market as we know it, and every landlord will sell their property” and so on. And they came to the conclusion that a lot of these rental laws could be changed – that, you know, tenants could have a right to have a pet, and chances are that the sun would still come up tomorrow. And the Victorian Government, in fact, then changed all of the rental laws accordingly.

By contrast, in New South Wales, none of that work got done. This stuff did make it up in front of Parliament. The minister got interested. The landlord groups made a huge song and dance. They scared the government into thinking that maybe this was not such a good idea, and it never happened. And so it’s a really good little case example of the way that the strength of the evidence base can make a difference. And there were lots of other examples we had, but not one that was so starkly different between two different states with two different evidence bases. And I think that’s what goes on in practice and the reason why these reports make such a difference. And I’ve spoken to a number of ministers and public servants about this subsequently, and they’ve all said: “Yes, this is exactly how it works”. So a minister comes up with a bright idea, and inevitably some lobby group hates it. So the lobby group walks into the minister’s office and says, “If you do this, then you know, the sky will fall in and the sun will not come up tomorrow.” And the minister turns around to his public servants, and says: “This is the people who are involved in the industry. You know, they know a lot about the sky falling in and the sun coming up tomorrow in their industry. They must be right  – what have you got? And the public servants go: “Well, it kind of looks right, minister.” And the minister says: “Well, what would you know? You’re a bunch of public servants. This is not your industry.” And so they get scared out of doing anything.

Whereas if, on the other hand, what happens is that the industry comes in and says, “Well, if you do this, minister, the sky will fall in and the sun won’t come up”, and the minister turns to its public servants, and they say, “Well, yes, minister, of course they would say that, but on page 127 of this very learned report written by this person who clearly knows what they’re talking about, and who spent several months looking at this, it says that this is the argument that gets put up, and the evidence why this is, in fact, a lot of rubbish is A, B, C and D, and so minister, we’re pretty confident that you can do this, and the sun will, in fact, keep coming up, and this is just a lobby group talking their own book,” then invariably, the minister turns around to the lobby group and says, “Well, you know, the evidence is in and I don’t believe you, because, yes, you work in this industry, but of course, you’ve got an interest to protect, and you know your interest is being contradicted by the evidence, so I’m not very interested in what you have to say, and so we’re going to do it.” And that, apparently, is the dynamic that plays out just time and time again – that when you’ve got good evidence that’s accessible in a report, when the lobby groups turn up and start saying, “this is going to be a disaster”, if you’ve got that evidence, it’s way, way easier for the minister to face them down. And by and large, they do.

It’s one of the things that I was completely surprised about in that report, that in that Gridlock report, I had expected to find that the opposition of powerful vested interest groups was a really major reason why policy didn’t change. And by the end of the research that we’ve done, I’d come to the conclusion that that was actually not true – that actually, in a funny way, big lobby groups are not that powerful in the face of really good evidence. Now, one of the things that big lobby groups do is convince governments never to collect the evidence in the first place. And of course, that’s much harder to see, and one of the reasons why they can often win. But if you do do the work, and if you do commission the evidence, it’s surprising how powerful it can be. Now, that’s a very comforting thing to find out, as someone who used to run a think-tank. But it’s also, I would add, not something I expected to find. I expected to find that lobby groups would be a lot more powerful. And you know, that’s not what the evidence told us.

David Walker: So you need to anticipate poor arguments and show why your recommended course of action is better. This sounds simple. But in practice, this requires you to do something which runs against much of human nature. You need to show you understand the strongest form of each argument you are rejecting, and explain why you are rejecting it. Only then can you show why the recommended course of action is better on balance. You need effective defence as well as offence.

John Daley: In my time at Oxford – as I said, I was doing work in legal philosophy, which meant I spent quite a lot of time essentially having to argue with the philosophers, and as I said, Oxford philosophy is a blood sport, hunting to hounds has got nothing on it – the way that they used to talk about this was what they call the principle of charity. You have to take your opponent’s best argument; you have to construe it not necessarily the way they’ve put it, but in the way that would be the most powerful that you could possibly make it; and then you need to show why it’s wrong. So that makes the bar really high, but it’s a really good way of writing this kind of report. Work out: what are the arguments against this? How do I how do I pick, not the easy arguments that I can kind of dismiss easily, but how do I pick the arguments that really have weight, that really might sway someone’s opinion? How do I put them the strongest way I possibly can, and then how do I assemble the evidence that shows that you shouldn’t worry about them?

And so absolutely yes, a report has got both offence and defence. It explains this is why we think this is the right answer. And also, here are the major counter arguments, and this is why we think they are outweighed – either because, you know, they’re not borne out by the evidence, or because they’re not as strong as or not as important as the other things that we care about. So yes, that is a really important part of the way that I think good policy work is done. If you haven’t thought about the major counter arguments and explained why they are less powerful than the argument that you’re putting, you’re not really helping.

Because one of the other things that became very obvious to me running Grattan Institute ... One of the things people always ask is, you know, Grattan is 20-odd researchers; how can it possibly have this much impact relative to an army of public servants and veritable armies of academics? And one of the answers to that question is because we thought about it the way that policy makers think about it. And the way that policy makers, whether those are senior public servants or ministers, [think about it] is, every policy decision is an all-in all-things-considered policy decision. You have to consider all of the things that influence you and say: “On balance, I think that A is better than B.” It’s to simply say: “We could minimize the number of homes that get burnt if we build them all out of, you know, reinforced concrete three meters thick”. You know, I’m sure that that’s true. And I’m sure you could do some very elegant academic work to show that building houses three meters thick out of concrete means that they’re less likely to burn down. But that doesn’t help a policymaker. A policymaker has to think about, well, how much does this change how you’re going to build things reduce the fire risk – but also, how much does it cost? And, you know, what’s the carbon impact of that? And what’s going to be the impact on building, the building industry, etc. They’ve got to worry about all of those things.

And so if you’re going to go to them with a recommendation about, look, this is how you should change the ... building laws, in the light of Black Saturday and bushfires. You have to answer all of those things at once. Otherwise they can’t make a decision. Similarly, if you’re going to talk about tax policy, it’s not good enough to say: “Oh, look, this will mean a more equitable distribution of resources.” You’ve got to show that it’s not going to have a terrible impact on the economy. You’ve got to show that it’s not going to have a terrible impact on vested interests, etc, etc. And so thinking about all things considered, including the counter-arguments, is really important, because otherwise you are not really helping the decision-maker, whose job it is to make a decision, all things considered, thinking about all of those different considerations.

David Walker: Why do people find it so hard to come up with those all-things-considered approaches? Why do they so often instead latch onto one particular argument and just pursue that?

John Daley: I think it’s easier. I think we’re hardwired like that. And there’s some elegant psychological theories floating around about why we prefer certainty. We have strong confirmation biases, so we dismiss evidence that doesn’t fit our priors. We put more weight on evidence that does fit our priors, irrespective of the actual strength of that evidence. And we’re hardwired to do that a bit. And it’s more comfortable, right? No one likes changing their mind all that much, so I think it’s very much we’re hardwired, and it’s easier to simply say: “Look, I think that A is better than B, and I’ve just lined up all the evidence I can find showing that A is better than B.” But as I said, it doesn’t actually help, and of course it’s not intellectually honest.

And I think that the the people who are really, really good at this and who really make a difference are the people who have intellectual self-doubt. And, you know, they can sound very convincing and sound as though they’re very sure of themselves, but at the same time, hopefully they’ve got a little voice at the back of their head going: “Yeah, but are you really sure about that? And when so and so has run this argument against you, have you really thought about where they’re coming from and maybe that there might be a germ of truth in that.” That takes quite a lot of humility, and not many people have got that humility. I think partly, in order to counter those biases, it’s about having the self-discipline to find the time where you worry about that stuff. And I think for everyone, that’s different. For me, I do it in the shower, and I do it when I’m swimming, and I let those kind of thoughts bubble up to the surface about the things that I’m worried about.

And when you’re worried about it, the trick is not to kind of stop worrying about it and push it aside. The trick is to actually look at it in the face and say: “Why am I worried about that? What might I have missed here? Let me really try and get inside the shoes of the person who’s making this argument and understand where they’re coming from.” And I think that’s one of the other reasons why this is hard. That business of ... I mean, we all talk about walking a mile in someone else’s shoes, but actually not that many people do it. And that’s what’s involved, is getting inside somebody else’s worldview and asking understanding where they’re coming from and really understanding their argument.

Now you may well do that and come to the conclusion that now you really understand where they’re coming from, you really understand the fundamental mistake they’ve made, and now you can explain it to them. But maybe you’ll explain understand also or differently, that there’s a germ of truth in it. So I think that that’s why it requires humility, it requires making the time, it requires fighting against our own instincts. All of those things are hard, but they are what leads, I think, to really high-quality work in the long run.

David Walker: As Daley points out, policy evolves in such a way that you can rarely pinpoint one moment or one person or one document that made it happen. But more than most, John Daley knows how to create reports which persuade decision-makers.

I have to confess: I’m biased towards John Daleys’s perspective. That’s in part because he has had such wide experience. And it’s in part because he brings so much analytical rigour to the task of creating reports. (It’s also because, for better or worse, so many of his views match my own.)

John Daley’s six key points

So let’s finish by recapping six of Daley’s key points for report authors:

1. You must focus on the key decision-makers. Authors need to understand decision-makers’ priorities, constraints, and decision-making process – and then give decision-makers the conclusions they need.
2. Think carefully about your audience. Determine which conclusions that you can actually justify will be most helpful and persuasive for them. Discard material that is just academically interesting.
3. Engage regularly with decision-makers throughout the process to get their feedback and incorporate their insights, rather than just presenting a final report.
4. Break down complex issues into clear and concise points. You can put plenty of detail and necessary technical language in the body of the report, but decision-makers don’t need it up-front.
5. Anticipate counterarguments, and address them. Don’t just present your case; set out the strongest opposing arguments you can find, and then explain why your recommendation is still superior.
6. Be willing to rewrite and refine your communication multiple times to improve clarity and persuasiveness, even if the underlying analysis doesn’t change much.