diff --git a/whether_substantive.ipynb b/whether_substantive.ipynb new file mode 100644 index 0000000..4820560 --- /dev/null +++ b/whether_substantive.ipynb @@ -0,0 +1,268 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "view-in-github", + "colab_type": "text" + }, + "source": [ + "\"Open" + ] + }, + { + "cell_type": "markdown", + "source": [ + " **License:**\n", + " Copyright (c) 2024 Ayush Patnaik. This work is licensed under a [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/)." + ], + "metadata": { + "id": "F_zvNeagP37W" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Introduction\n", + "\n", + "This tutorial explains using local language models (LLMs) for natural language processing tasks in Google Colab. It covers installing packages, setting up the LLM, and applying it to tasks like summarizing YouTube video transcripts. This approach enhances data privacy and reduces reliance on external servers, applicable to various text processing needs.\n" + ], + "metadata": { + "id": "wj_35TW5ClVb" + } + }, + { + "cell_type": "markdown", + "source": [ + "## Setting up" + ], + "metadata": { + "id": "tN-5DaU9DlZJ" + } + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8boYmiDegMlf" + }, + "outputs": [], + "source": [ + "# @title Install packages\n", + "\n", + "%%capture\n", + "\n", + "# Package for running LLMs locally\n", + "!pip install langchain\n", + "!pip install langchain-core\n", + "!pip install langchain-community" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "p3Rle_LJk2cT" + }, + "outputs": [], + "source": [ + "# @title Download and install Ollama\n", + "%%capture\n", + "\n", + "MODEL_NAME = \"llama3.1\" # @param {\"type\":\"string\",\"placeholder\":\"Model Name\"}\n", + "# Find Ollama models: https://ollama.com/library/\n", + "\n", + "!sudo apt install pciutils # install library to connect with GPUs\n", + "!curl -fsSL https://ollama.com/install.sh | sh # Download and install Ollama\n", + "!nohup ollama serve & # Start Ollama server\n", + "!ollama pull {MODEL_NAME} # Download model\n", + "!nohup ollama run {MODEL_NAME}]& # Run the model in the background" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ADjelyLYkZ1L" + }, + "outputs": [], + "source": [ + "from langchain_community.llms import Ollama\n", + "\n", + "llm = Ollama(model=MODEL_NAME)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "nAvAjleZlZXz", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 55 + }, + "outputId": "125dd69c-7ff9-4cb2-bd92-bd57b93dcbfd" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'Hello! How are you today? Is there something I can help you with or would you like to chat?'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 4 + } + ], + "source": [ + "llm.invoke(\"hello\")" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## LLMs in action: Summarising YouTube videos" + ], + "metadata": { + "id": "JUyGvnF5Dov7" + } + }, + { + "cell_type": "code", + "source": [ + "%%capture\n", + "!pip install youtube-transcript-api # Package for extract transcript from YouTube videos" + ], + "metadata": { + "id": "cPRFR0WXD9di" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "from youtube_transcript_api import YouTubeTranscriptApi" + ], + "metadata": { + "id": "zMLm06XDv9xJ" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "og05kjoTlaTy" + }, + "outputs": [], + "source": [ + "# @title Function to extract the transcript of a YouTube video\n", + "\n", + "def extract_transcript(video_id):\n", + " episode_elements = YouTubeTranscriptApi.get_transcript(\"86PxrQ_lkp4\")\n", + " episode_text = [time_stamp['text'] for time_stamp in episode_elements]\n", + " episode_full_transcript = \" \".join([str(item) for item in episode_text])\n", + " return episode_full_transcript" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1-ZNL8lClkLB" + }, + "outputs": [], + "source": [ + "# @title Extract transcript of EiE episode 57\n", + "\n", + "transcript_E57 = extract_transcript(\"86PxrQ_lkp4\") # this code is in the URL of the video" + ] + }, + { + "cell_type": "code", + "source": [ + "transcript_E57" + ], + "metadata": { + "id": "P8QO2M5T29WP", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 164 + }, + "outputId": "dba0f05e-1982-4317-f4e7-12cfa244edaa" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "\"[Music] welcome gentle reader for this uh fascinating and important episode of Everything is Everything um Amit do you think the green screen will work correctly with the video insertion it is it is the best green screen known to it is a natural green screen and and I know you are particularly happy Aisha because you're a tech geek and you're sitting here today with a piece of technology that no Indian YouTuber uh you know can use as well as you what is that technology you're so excited about I have a hairlight 150 million kilm behind [Music] me so as a said it's an important episode and that's really because it's a subject that is close to both our hearts and it goes to a fundamental question about why do we care about the things that we care about at one level when we live our lives our Quest is personal I want to improve my own life I want I want every day to be as pleasurable for me I want to make money I want to become successful I want to read good books etc etc those are personal quests when we talk of things beyond the personal what do we really want when we talk of Economics what is a larger question when we talk of politics what is a larger question when we talk of society what is a larger question and I worry sometimes and I I I try to watch out for it within myself that it should not be the case that there is a package of beliefs that I have adopted that appear to explain the world to me and make me feel comfortable in my knowledge and that everything that I then say about economics or politics or Society comes from that package of belief that either an ideological tribe may have or that I find it convenient to adopt and I stopped looking at the real state of the world and I feel it's very important for all of us at different points in time to keep taking that step back and say that no I'm not supporting this idea because I have adopted this idea as part of my identity I know all the logic I know all the reasoning and to get dogmatic about it but to instead always go back to the larger question which moves Us in the first place and the larger question that I think matters to should matter to every Indian matters I know to both you and me is that how can we make this world a better place and in India context how can we make India a better place both of us are extremely fortunate to have grown up in well of households in more ways than one you know we had uh sort of a world of books at our disposal when we were growing up we've had good lives but we've looked around us and we've seen that the world is full of problems and it can sometimes get frustrating when you travel somewhere abroad and you see wow the city is so great the infrastructure is so great everybody is so well off any kid any kid in this country can aspire to the best things you know and then to come back back and see what is happening here and start thinking about it uh you know in that manner and I love this so and start thinking about it in that manner and and I I just feel that too many people get stuck in a Groove too many people forget the larger question that I think animated all of us I will say with you know I would like to imagine that everyone cares about this because they are animated by the central question and one of the central questions in India is how do we make this a great country for everyone who lives here how do we get rid of poverty how do we prog add etc etc and I just feel that this episode is incredibly important because in the field of Economics which is where the pathway uh to progress and development surely lies in that field I see too many people who have adopted uh you know ideological packages who have adopted dogmas and don't really care so deeply about what's happening in the real world what works what doesn't no you're on a career path you're in your journey that's all that matters to you and in this episode what I would like you to do is sort of of give me and all our gentle readers a sort of Journey that when we talk about development economics what do we mean what do we want this is not about numbers and fiscal deficits and policies and inflation it's about real lives real people this matters so I want you to take us on a journey through this entire field and talk about why this journey matters what are the foundational uh pieces of thinking around the subject where have we gone wrong what what should we remember so we so exactly where you started that we should not have a prefabricated belief system with glib and Ready Answers for everything we should always be able to argue it from first principles so what we should try to do today is let's do a discussion and argument and unfold from a foundational question into a sense of the agenda into a sense of the questions and some of the big ideas of the field like we will not remotely answer many of the questions but at least we'll get our frame straight of who are we what are we trying what are the important questions and you know some things that we know are dead ends and are not working too well okay so it will not be a batch of answers but it will Orient Us in the correct place in the mountain that this is where we are looking and this is what we are trying and in fact that's a great metaphor because we on a mountain right now people are paragliding off us and that's what we going to do I want you to paraglide us with with all of us all your gentle readers on separate birds and give us that Skyhigh View and talk about first principles and give us a lay of the land and then people can zoom where they want to zoom but I think that Sky High View is important yeah so we are at 2,500 M let's [Music] go uh the question that we should start at the question we should focus upon is why is the Indian per capita GDP 1110 of the United States okay that is a single important defining question and that's PPP that's PPP adjust it so you take raw numbers it's uh much worse but PPP is the correct way to measure it um Amit has promised a YouTube sh in English and in Bengali explaining what is PPP measurement right now just take my word for it I haven't PPP is purchasing power parity so it basically looks at what you can buy with one rupee here versus what you can buy with $1 in the US and then you figure out what is the real value of the money uh so there is a 10x gap between the average Indian uh output PPP versus that in the United States it's not 10% it's not 20% it's 10 times okay and that's just a huge gap so even though India is, 1400 million people and the United States is 333 million people the United States economy is vast while India is a small economy because per person that there is a 10x Gap in output and it is important to fix our minds on the sheer size of that Gap 10x is a lot 10x is a very big gap so I think that that's our Central and first motivating question that what is the source of this Gap why is there such a large gap what are the things that can be done so that a poor country like India can make progress and can become an advanced economy again I'm we're using IND India as The Guiding metaphor but this applies all across the world there is a small number of countries that are advanced economies most countries in the world are poor there are different different points in terms of their poverty but development economics is about this question why is there a 10x gap between the Indian per capita GDP versus the United States and what are all the ways in which there could potentially be a hope maybe not a certainty through which they may be a pathway for India to make progress and potentially match up with the United States uh in some years the great Economist Robert Lucas has a beautiful paper on development in 1988 and in that he says early on that once you start thinking about this question it's hard to think about anything else because 10x is such a big gap that there is such a large opportunity to make the world better by doing development properly by doing development correctly or or equally if you go back into history and ask how did the United States become a developed economy the United States had a certain development Journey or the UK had a certain development Journey how do these things happen why do they happen and what if anything can all of us bring to the table in terms of knowledge ideas values culture participation in the decisions as individuals as firms as governments through which these things can be done better this is is the question of development economics and the moment you pose this question there will be people who will say are you being a little reductionist saying that income you know fill the luer money is all that matters and we can come at this in many different ways but my rough idea is that at the Indian levels of per capita GDP it's an absolute crisis okay we have childhood stunting because kids don't get enough protein in their growing up we have vast lands of North India where it is miserably cold in the winter and people can't afford to get a hot water bath and I know how miserable I get if I am told not to get a hot water bath so the levels of deprivation in India are abominable so I feel that you may have higher Notions in your own life about the place of money in your life and you know in many many episodes we are the kind of people that say you know money isn't the most important thing in the world but at the level of the country there's no question that this 10x Gap is an Abomination and the way to think about it is that a prosperous country is one where people have many many choices then most people may squander their lives they may not create things of Truth and Beauty they may become very unhappy at a personal level but that's a different question that's you know your a question between you and your God it is your spiritualism about how you want to approach life but the point is everybody should have the level of prosperity through which they can lead interesting lives uh I remember when Amit did the cycling episode after the episode ended I only walked away with Envy at the advanced commercial and organizational culture of Europe that supports the beauty of this whole cycling world and that's a joy that's an experience it's a life experience it is satisfying for so many people people because they are an advanced economy and you know our Dream for India is that more and more people should have these choices after that what they choose to do with them is their Birthright okay so a person may choose to sit at home and not particularly do anything interesting or cool with their lives that's their Birthright but being a rich country is about Basics it's about protein intake when you are a child it's about developing that strength as an adult it is about the ability to pay for a hot bath it is about having uh mosquito screens on your windows so that you are not bitten by mosquitoes while you sleep at night these things are just basic and nobody should question them in fact I think it's a you know we had an episode on luxury beliefs and luxury beliefs essentially are beliefs that people Elites can have in their conditioned rooms but they end up hurting the poor but the elites feel virtuous when they have them and I think the notion that uh you know that money isn't everything is a luxury belief when you're poor when you don't have food to put on the table when your children are married malnourished when they are suffering because of that in the crucial years of their life and you can't get those back uh you know when you don't have a stable roof above your head where you don't have an income where you don't know where your rent is going going to come from when one medical emergency can destroy you money is a big freaking deal the GDP number is a big freaking deal I've had episodes of the scene and the Unseen on how GDP is an imperfect measure in so many ways but the bottom line is that ultimately you know uh it is about the money no matter how much you choose to measure it that matters what is your National productivity what is your income so on and so forth and as we will go on to discuss uh you know it is a great indicator of welfare in all its aspects yeah so just to summarize and recap in this chapter development economics is about the question why is India at a per capita GDP which is 1/10th of the United States and what are all the various actors and the actions that can help make progress [Music] so a let's start at the very beginning are you familiar with that song a very good place to start a very good place to start so let's start at the very beginning give me a historical sense you know let's go step by step I'm treating this episode as a master class of sords give me a historical sense of how thinking on development economics has evolved yes you have the central question then what yeah so we start at the beginning and uh an early foundational idea came from growth Theory which was begun by Robert solo MIT Economist Nobel orat and the solo model solo Swan model of growth roughly tells a story like this it says that aggregate output of a country is about the amount of Labor that goes in the amount of capital that goes in and how these are combined with a certain level of productivity for Simplicity let's focus on per capita so then it's just about the Capital stock per capita in the country and the productivity at the time productivity was thought to be something about the technology something scientific by per capita AJ means per person so when you talk of GDP as the GDP of the whole country but populations of countries differ widely so a better measure is per person what is the GDP and that's what per capita means just clarifying that and so then this model gets reexpressed saying the per capita output is something about productivity and at the time productivity was thought of as machines that what kind of machines do you have what kind of London metro do you have what kind of textile automation do you have that are you doing some handloom or do you have amazing modern spinning machines and weaving machines and so on so there is some number which is a productivity and then there is Capital stock per capita let me explain Capital stock per capita it's about again physical assets it's about the factories it's about the London metro value of all these things divided per person so all the uh the financial value of all the equipment that goes into the production process is the Capital stock and so jumping off from that immediately after the solo growth model there came to be development thinkers who started thinking hm how would I apply this to thinking about development and then the arguments came that the reason why India is poor is because the Capital stock is poor because there are people sitting with spinning wheels and they don't have these modern powerful machines because there is no Bombay Metro to match the London metro because the highways suck okay so people on this uh episode will be amazed to know that before I first went abroad to do a PhD I had never seen a highway in India there was not a single Highway in India okay that's how bad things were in India as a poor country there were no highways and there was no Metro systems so that was the idea that you need to fix the Capital stock and so it's a poor country you'll keep saving and gradually you'll build up your capital and you will get the Capital stock up and the other thing the productivity was thought to be something in the hands of the scientists something about engineers and Engineering okay this is sort of a nehruvian mahalanabis notion of Economics that we just have to buy the right machines and read the right manuals and bingo we're done and now we are in advanced economy okay so these are early primitive Notions of development that hiding in the solo growth model was a theory of development so the solo growth model was done for the United States it was done for the UK and it has very many interesting implications and explanations for the United States and the UK as advanced economies the rich countries but then it was repurposed to start thinking about development and so the intuition was you got to get the Capital stock per capita up and you got to get all those machines and all those manuals and those books and for example there was a great scare for many years about the word technology transfer where the worldview was that the rich countries are holding technical knowledge close to their chest that they will refuse to give us that scientific and Technical knowledge so there were entire United Nations bodies I think unido or anad I can't even remember which who used to obsess on creating formal State tostate Frameworks where those secret runes will be given to the poor country love the faces you make that there was some secrets and and I grew up around these things in my childhood and I remember my father used to tell me that no that's completely wrong all the knowledge is there in the books is there in the papers you buy the machine you get the manual patents release knowledge into the public domain the burden is on us it is our brain power to go read all those things and learn all those things you don't you don't request the rich countries to give us the knowledge we have to go take that knowledge and you know there's really no barrier to obtaining huge strides in e Economic Development so this is the old story of capital and labor and over the years I feel this has worked less and less well as a description of the world and by the time I'm saying all these things I hope you've started under thinking that really like how is it so hard that you're standing in 2024 and we haven't yet got all the right machines and we haven't got all the right manuals the idea that productivity is a manual a document that can be transmitted Beggars the imagination and is there really any limitation in India in terms of reading the manuals and understanding how to do things um on a ride up the mountain uh gentle reader I asked Amit that uh you know we we're very cool we're very smart we think we're making a great show but you know can we be cold that if you think of namita and vnav and our Sony equip M uh is this a good YouTube show by World standards or are there obvious ways in which we are inferior to the global State ofthe art and amit's answer was no I don't really see any important aspect of our production values where we are not Global state-ofthe-art this road microphone is is literally the global State ofth art today and there's nothing much to improve on it and yeah we are bright enough to read these manuals we use these capabilities to the hilt then you're at the global Frontier so then what went [Music] wrong so aay what went wrong is a complicated question a lot of things went wrong and we are not just talking about India we care most about India obviously but we not just talking about India things went wrong across the world things things went wrong in India and essentially this way of things the model itself the framework itself is wrong and we know that today in hindsight obviously so uh take me through the Global Experience yeah so in the in the dawn of development there was a lot of optimism because of the way I've put it to you it's just a matter of piling on the capital and getting enough you know clever people who can connect up to the world economy who can bring in those manuals so in India there was plenty of English speakers okay we are all myle children we brought in those manuals we brought in these books and like hey presto you should be done in the early years of development there was a very powerful intuition that the very word development is analogous to the development of a child the development of a puppy that there is an inevitability that the puppy grows into a full dog that a child grows into a full human and you just have to make sure that the calories and the protein come in and the growth of a child into a full adult is inevitable it's baked in and it's going to happen and you can be reasonably sure it's going to happen and maybe we can argue with each other about how quickly it will happen but that it's going to happen was never in doubt the philosopher in me wants to argue with the economist to you and say that that growth from a child to an adult and from a puppy to a dog is actually deeply tragic inevitable but deeply tragic and whereas our nation we want our nation to actually grow I'm sorry I couldn't yeah so in the early years of development again drawing on the intuition of the solo growth model and the way I deliberately used words that convey their way of thinking and their philosophy and their philosophy was yeah it's only a question of time the child will develop now can you do something to add to the nutrients and make that go faster Yeah by all means let's do it so you know we will have government to government engagement to transfer technology foreign aid will supplement the Capital stock of the country and the country will grow faster but growth is baked in it's just a matter of time it's going to happen and we stand here with the benefit of hindsight and we know this was absolutely wrong this did not happen this was a wrong theory of the world if you stand in 1945 we've said this many times on eie and this is a very critical fact that I feel should be burned into our skulls if you stand in 1945 and think of the places that were poor then only four have made it to rich country status today and they are uh South Korea uh Taiwan Israel and Chile okay there are no other countries that have really risen to the top so if you had to play statistically statistically development doesn't happen it's the exception rather than the norm there are more than 100 countries which did not make it and it's been a while since 1945 so it's not just that you know in 1965 you could say yeah just hang in there in 1985 you could say yeah just hang in there in 2024 you can't say just hang in there it is broken and it is not working so today we would make the opposite proposition that the most important question of the world once you start thinking about it you can't think about anything else is a question of development it is about why a country like India is 1/10th the per capita GDP of the United States and most countries just don't get out of that it is rare it is the statistical aberration where a country gets to development it is a statistical aberration that Taiwan and South Korea and Israel have risen to the ranks of the advanced economies most countries haven't so in fact failure is the norm success is the exception and this is something that was not understood At All by the early generation after the uh second world war for about 50 years there was a great deal of optimism and I also think that the development Community does not speak clearly and openly about this proposition there's just too much RAR there's too much optimism I think people would like to drum up money from donors funders humanitarian assistance we'll come back to incentives and an understanding of the development Community separately but right now I just want to say that we will all do better by respecting the ladder that we seek to climb okay so I mean if you ask Amit B to climb to the top of Mount Everest you have to walk in saying statistically look this is hard there's a high chance that you will not make it and then suddenly everything will work better because then Amit will train for it Amit will refashion his mind and his focus and his attention saying I'm going to do something very difficult there's a high chance I'll fail and then I'll actually do it better whereas if you tell me yeah yeah it's Elementary it's just a matter of time you eat enough protein bars you're going to reach the top of Mount Everest you'll approach it differently and I feel that that was an intellectual error and it continues to be a certain RAR that suffuses this landscape which is unhealthy for all of us don't get me wrong I want nothing more than for this stuff to work I want nothing more than for us to now learn the runes through which we can actually make development happen knowing that we failed from 1945 but the journey to that will begin when we Cast Away the mistakes of the way it has been approached earlier and a key mistake has been this sense of optimism in fact we should view it with great concern that wow this is hard and just to clarify what you're saying is not that development is hard and it's so hard that we shouldn't do it it's an aberration what you're saying is that we had an approach towards development we thought the formula is a BC do these things development will come might take 20 years might take 40 years might take 60 years but that formula was wrong it hasn't happened it wasn't as easy as we thought it and and that's a great metaphor you know you have enough protein bars you'll climb Mount Everest no you got to do a lot more and it's just such a brilliant metaphor in fact and and the point is climbing Mount Everest is possible if you really want to do it becoming a developed country is possible it is doable except that we need to be clear about what is the approach approach that we want to take and the fact that since 1945 four countries have made that leap is like such a sobering and cautionary tail but also inspiring because if they can do it so can we correct and I just want to dwell on like Taiwan Israel Korea like what a terrible beginning for each of them Taiwan is an irrelevant island off the coast of China Japanese Occupation very cruel Japanese Occupation and at the end of the uh Chinese Communist Revolution a whole bunch of people ran away from mainland China and took to Taiwan and changai shik the leader of the Chinese regime was was no great person I mean is no neeru okay is a pretty rotten person in his own ways and so they had a very bad beginning Korea the line on the map between North Korea and South Korea is essentially at Soul the capital city of Soul the most important city of uh South Korea during the Korean War six times the city of Soul was taken and retaken by the North and the South and you can imagine the devastation with World War II Quality Equipment that would have happened in Soul so you should think of Soul as being destroyed on the scale of Berlin or on the scale of staling orren orron okay and that's the beginning and there are stories about how early South Korean government functionaries used to look up to India and try to learn from Civil Service practices is in India that's the beginning or Israel okay they fought a war in 1948 every step was a struggle every step was misery they were surrounded by all hostile States if you think fighting a two front war is difficult think of what Israel has been through and they came out of that so these things are very hard and they can be done and some of the countries that were placed under very adverse circumstances did manage to claw their way out of it [Music] so a we've actually spoken a lot about the Indian experience across episodes of everything is everything on the scene and the Unseen many times we in fact had an episode which you'll see on the screen right now which is exactly about uh you know India's journey to prosperity and but I want you to in the context of development economics and what the failures are talk a little bit about how it failed in India as well yeah so you stare at the solo model you stare at the growth equation and then you think of about India okay you're supposed to put labor labor is plentiful okay and then you pause to think what is going on in the country the employment rate in the country is about 1/3 what does that mean that for every 100 people of age 16 to 64 about 33 or 35 are working and you'd fall off your chair I should not fall off my chair you'd fall off your chair saying what then our cinemat will say AJ is rolling yeah it's say what it's like two-thirds of the Indian working age population is not working something is wrong yes so to understand development is to understand why is 2/3 of the Indian labor force sitting outside the labor force and not able to come into the main line capitalism into the mainline market economy and start doing labor Supply and that is what needs to be understood just because the labor is there it's not inevitable that it will get used the puzzles are in how that comes together in productive ways similarly Part B let's take capital okay there is infinite capital in the global Capital Market the entire stock of capital required to convert India into an advanced economy within 20 years is trivially accessible in the global Capital Market okay and we've done an episode on finance there are few things as amazing as this concept that trillions of dollars are available on tap in the global Capital Market so why do they not come here the problems are all here that there are mistakes and difficulties in the Indian institutional environment which interfere with that okay and then you turn to the productivity to the technology side that is there any problem in reading books papers manuals buying the best machines and building firms and building organizations in India that are able to embed this best technology it's hard to say that the transfer of scientific knowledge is the the bottleneck okay for example in a previous episode we talked about my uh happiest moment of recent months I visited a blue jeans Factory on the outskirts of Bangalore belonging to arind limited and it's just amazing watching those people they are in touch with global State ofthe art they know every single machine that is available out there they carefully work out the ROI and they think about which machine can I adopt when and they have the full scientific and Engineering capability to go to China to go to Taiwan to go to Europe buy those machines read the manuals deploy them into India there is no technical barrier like you cannot say that there is a technological reason there is some package of knowledge that needs to be transferred that is holding back the productivity and the capability in the country of being able to do all this and then again let me turn to the firm as a site of production so firms are supposed to put all this together firms are supposed to hire the labor obtain the cap Capital at its best foreign Capital you remember we've talked a lot about firms in internationalization and firms are supposed to understand Global state-ofthe-art machines techniques license those patterns go out go abroad get the best knowledge and bring it here and yes there is a capable tiny Elite in India that is able to do all that so I can't understand why all that hasn't happened so if the solo Swan model was true if that was your the of growth then by now India should have been a rich country so to flip it around that hasn't happened and our journey in development economics is about understanding why that hasn't happened so I could turn all this conversation so far around and say development economics is a study about why India is 1/10th the per capita GDP of the United States even though the labor is plentiful the capital is infinite in the World Market all the technical manuals available so what went wrong that is a question of development economic [Music] so a you know at this point I could go on a rant and I could talk about an hour or 2 hours or 3 hours about all the things that have gone wrong with India but I don't right now want to talk about the journey India has made we'll come to that I am more interested in the destination for once in terms of thinking about what went right in the countries where things went right you know Leo tolstar has that famous line at the start of a Carina all happy families are alike all unhappy families are unhappy in different ways right so the problems of India Bangladesh Africa all the poor countries of the world they are all different they all uh you know differ from each other but actually all the developed countries that have made it including the four countries that you mentioned have made it since 1945 they have so much in common tell me a little bit about that yeah it is indeed striking how variability within advanced economies is relatively low and variability in poor countries is very high that if you deeply understand South Africa and you deeply understand India they seem like very unlike each other you deeply understand uh Burma it's completely different you deeply understand Cambodia it's completely different so poor countries are very heterogeneous and this is again a problem that bevils the field of development economics because when you think about development economics you are somehow implying a homogeneity in all poor countries actually poor countries are widely unlike each other so I have done a little bit of work in a couple of countries over and Beyond India and for me one of the biggest things I had to do in each country was to zero base that no you're not in India now think this from scratch and it it takes considerable work to understand each country and they are very different but the advanced economies actually are all a lot like each other and let me just try to Rattle off some characteristics and I'm sure you'll also have a lot of that intuition uh all advanced economies are prosperous liberal democracies they have freedom of speech there are no no authoritarian advanced economies pause to think about it not one Advanced economy is an authoritarian country they all have freedom freedom of press rule of law they have this High state of freedom and the state is a agent of the people there are many many steps to go it's always difficult but ultimately the people tend to kick the state into shape to serve the needs of the people the people are the principle and the state is the agent it is imperfect in many many ways but fundamentally those feedback loops have been built up in reasonable ways the state is not out of control and out of touch and doing whatever it wants all advanced economies have amazing firms firms are after all the site where GDP is made so you know a country like Finland has firms like Nokia and elsewhere in Scandinavia you have these amazing companies which you would otherwise not have expected in these very small economies so these countries are the host to amazing Global world-class firms who have very high levels of productivity and so their output per worker is very high and it actually pulls up the output per person of the entire economy so there are amazing high quality firms what makes the United States a great country is not the United States government it is the Google and the Microsoft and the JP Morgan and so on these are just Legends these are amazing firms they know how to do things at a mind-blowing level they are far ahead of any firms that we will see around us in India that is a secret Source how do you hold those firms who are at Supreme levels of capability so you know to quote a current Indian example uh I recommend every gentle reader to not just go to an Ikea and Shop there but walk around Ikea and admire the damn company like what a business what audacity of conception that you could dream such a business think of how the furniture is designed how it is manufactured how the stores are built how the marketing is done all over the world what an amazing business and this is just like one Advanced economy Corporation and like this imagine how in the advanced World there are just thousands of great companies the that is Toyota and Samsung and Taiwan semiconductor manufacturing and so on and these are companies that have knowledge inside the firm they're not weak companies propped up by the state they are capabilities inside the companies then all the advanced economies have separation of church and state there are no advanced economies in which religion is there in the government at all it is also the case that all advanced economies have steep declines in religiosity this began first in Europe after the second world war for a long time the United States was a bit of an outlier where it was thought that the US is unusually religious but with a lag the collapse of religiosity has started happening in America as well and it's a part of this package that secularization happens as you grow up a country become secular and the people start relating to each other as human beings and not through tribalistic concepts of ethnicity and religion so you start seeing marriages taking place around very different concepts of assortative mating rather than more tribal concepts of uh number of kilometers within each other of the birthplace or uh Indian conceptions like a jati or intra religious or inter within Faith marriages and so on so there are many many of these social cultural political economic Concepts in which all advanced economies are a bit like each other and are recognizably like each other I don't want to understate the differences between Japan and uh South Korea and Taiwan but it does not surprise me that South Korea is a home to K-pop which is a remarkable cultural phenomenon across the whole world that's the way advanced economies work that they are able to create incredible flourishing of Truth and beauty and Commercial Success and there are there are explosive situations where the energy of the people gets harnessed and the key Point here that I want to underscore is that an advanced economy is not just about people earning more money about per capita GDP it's also about that freedom that economic freedom percolating down to society that within Society you have that freedom like it's a tremendously important point that all of these advanced economies have declining religiosity and it isn't the case that uh you know you can't be religious you have personal freedom you can be as religious as you want you have conservatives in the US will go to church every Sunday or go to the synagogue whenever and you have all of that you're free to do so but others don't have to do it you you're not forcing a particular vision of the world upon you you are not necessarily most of the time demonizing others the mainstream is not like that and I feel that a lot of the ugly tribalism that happens in Society the Fishers that crop up really happen more in situations of scarcity where then you know you're always fighting fighting fighting to get ahead of the next guy increasingly Everything feels like a zero some game and then these tribal instincts come to the four and you're looking for the other and all of that but in an advanced economy some of those pressures are really not there so it's a full package of Freedom it is not only about economic freedom or a flourishing Marketplace but it is freedom of the mind it is freedom in the bedroom it is freedom in the public Square all of those things matter yeah as an example gay marriages all over the advanced world or uh I have started thinking a good indicator to see uh social progress is the rate of single moming it's it's a great and even for India once had written a controversial column which got me a lot of heat where I said that one of the great indicators of uh women's empowerment in India is a rising divorce rate I think a rising divorce rate is fantastic because it means women are being empowered to to walk out of bad marriages mostly that's what it is and I think that that's kind of mind-blowing another distinction and I want to uh you know ask you if you agree with me is that some gentle readers may say that hey Saudi Arabia is doing well and so on and so forth but the point is a lot of the countries in the Middle East became Rich because they happened to have that natural windfall of oil the key thing to remember about many of these Middle East Nations or nations of that sort is that you had one windfall you had growth but you didn't actually have the full package of Development coming and that is a historical accident and as long as it lasts that's great but you don't know how long it's going to last so you know what does one make of those so if a windfall comes under the ground and I would earnestly pray for my country that there will never be a windfall found under the ground in India but when that windfall comes you get very high levels of wealth but you don't get development and actually what we want is the development we want the people we want the people to flourish we want the people to live at a different level in their lives and in Saudi Arabia the people have you know what most of us would consider to be a pretty terrible [Music] life so we've spoken about the destination we've spoken about you know the package of attributes that all these developed Nations have let's get back to the journey when we think about how to get there you know what are the kind of uh sort of paths that people propose and what does the data now tell us because people will propose many paths in and say redistribution K we should have a welfare state etc etc that's also important and the interesting thing is that this is this does not need to be an ideological battle this can be an empirical argument because the numbers exist so tell me about what do you think the question should be and what do we know so far from the data so I began with posing the question of development economics as why is the Indian per capita GDP 1110th of the United States and now that brings us to the most important question which is what are the actions through which that Gap can be closed and so now we turn to doing it that how do you run this Marathon what are all the things that need to be done um our good friend Lan pritet uh always tells us a very important story he says that let us study poverty because we are going to be very interested in poverty in and of itself and a good measure of the poverty rate that is comparable all across the world is the fraction of of the people who have a consumption every day of less than1 us PPP per day okay so that's a definition of poverty and there are reasonably well measured rates that is what is the percentage of the population there are reasonably well measured rates for all countries across the world across the years and lant says that if you try to explain this by using one number and that number is median income then you've ended up EXP explaining 99% of the variation across the countries so one attribute median income explains all of the poverty rate that you set out to explain the poverty rate but the answer turned out to be median income that it was not a deep subtle complex story of poverty it was not by studying poverty that we got the Breakthrough inside the Insight is fix the middle just fix you could do this by mean income you could do this by median income basically get the entire economy moving forward and that solves poverty so all the other obsessions of many people issues like inequality the welfare programs the redistributive efforts by a government all these things that you might seek to do that appear to be more directly connected to the poverty rate add up to 1% of the story around poverty rates okay so this is intellectual power that the solution to something that all of us care about lies not in obsessing on uh things that are proximate to that I'm reminded of uh primitive people thinking that uh elephant dung has something to do with curing elephant aasis because elephant elephant something sounds similar it's like that but actually to solve elephantes you have to go do something completely different you have to get some antibiotics it has nothing to do with elephants even though it superficially looks alike so that's the way I think about poverty that there is a humanitarian impulse that oh we want to help people that has nothing to do with development economics so helping people directly that you're poor I will come and give you food every day that has absolutely nothing to do with development economics development economics is the business of making a country a rich country comparable with today's oecd countries and to do that we've got to keep eyes on the ball and the ball is fix the growth fix the middle get the median income up get the mean income up lift the entire country and poor people will also do better and that's the bulk of the story 99% of the variance of the poverty rate is explained by growth not by the obsessions of people who are taking interest in poverty directly so we did an episode on five economic fallacies to avoid and one of them was of course a zero some fallacy and the zero some fallacy uh that you know if the rich are getting rich the poor must be getting poorer you want to do something about poverty redistribute it seems so obvious uh to the common person but when trained economists who should know better kind of fall for it and believe that redistribution is the way out you begin to wonder now there's been a debate about growth and redistribution and I believe that there are trade-offs and the question therefore is which one do you Target and what I kind of want to underscore is that the data now shows that growth is y when he speaks about you know when L's work shows that 99% of the impact in the poverty rate comes from Pure economic growth that you do redistribution you do welfare programs and you have enough countries to look at where redistribution has been done welfare is here welfare is there etc etc but you look at what makes an impact ultimately that impact comes from economic growth and that's important to remember and I did this great episode with L pret again I recommend that you go and listen to that episode but more than that more than the episode just engage with his work look at the show note of dad have linked to his critical papers in the show notes over there we link them here as well just read them because they are eye opening and they are based on data right uh it is Hardcore numbers from The Real World where you look at outcomes you don't look at uh intentions and uh you know another paper of his shows that economic growth that growth itself is both necessary and sufficient to create a developed country that if you have growth you will get to development and without growth you cannot uh you know get to development so it's necessary and it is sufficient that's all you need to do right and and you have to read the paper because it's a masterpiece and what um you you know his marshaling of data his Clarity of language is absolutely [Music] brilliant so AJ you know we know what doesn't work but when we speak about growth it's it kind of seems Wily what are the elements of this how do we break it down what do we need to do yeah so let's take that grand question and now we will dive into what are the chapters of the book so if you had to write a book of development then what would be the subcomponents inside it uh L pret has a nice classification scheme I have deviated a bit from it because my way of thinking about it is a little different so he emphasizes uh no I would emphasize three big Ideas the development journey is three big Ideas the first piece is the emergence of a High productivity economy okay so we need to get productivity up we need high quality firms which are producing at the global Frontier okay the United States is a global Frontier so your distance from the United States has to be reduced inside the firms we need to get to firms which match the capabilities of the corresponding firms of the United States that's one pillar that the emergence of a productive economy the second pillar is the emergence of State capability that every successful country needs a state okay and while uh we are considered while Amit Varma and aay sha are considered to be uh extremist Libertarians we are not anarchists we do not Advocate a withering away of the state as some leftist thinkers have done we Advocate capability that you need a state and then the state should do those things correctly it should succeed in the projects that it should set out to uh achieve you do a few things well instead of many things badly correct and for few things you absolutely need the state to do those things well there is no escaping State capability in the journey to Economic Development and a critical component of that is rule of law rule of law is essential for State capability because otherwise the Cs of power of the state will inevitably collapse into corruption rule of law is what forces the state to do its work correctly when rule of law is lacking the coercive power that is present inside State organizations will be used in corrupt ways okay and the third pillar is the emergence of the political system whereby the state becomes the agent of the people and not the other way around that in bad countries the state is the ruler the state is a myop we all look up to the state we are afraid of the state we wait for the signal to come from the state we are subjects of the state whereas the emergence of an advanced economy is where the state becomes the agent of the people so that's political modernization and it is again related to rule of law so I feel development economics is about the pursu of these three ideas idea number one how does high productivity emerge in the firms idea two how does State capability emerge and critically linked to it is rule of law and idea three how does the state become the agent of the people and not a tyrannical master and again critical to that is rule of law so these are the three pillars of the book on development economics if we were to study development economics these are the three things that need to be studied and where you you know differ from l in in the kar and sha book is that he separates State capability and rule of law and your point is and which I agree with is that rule of law is part of State capability so it's a part of number two and number three that if you want to achieve State capability you need rule of law if you want to achieve a government that is the agent and not the principal you need rule of law so I don't think of it as a separate pillar it is the raw material that makes those two pillars happen gentle reader at this point I want to say that before this we made a joke that between us we are a Bengali Economist we don't like Bengali economists but between us like I'm Bengali he's an economist so this is what the team is and the proof that I'm Bengali is that I put on the jacket first right hopefully I won't have to put on a monkey cap also [Music] so you know I I I find everything you just said deeply resonant because I keep complaining about how in India the state doesn't serve us at rules us and we think of ourselves by default as subjects rather than citizens and the mayab state is almost you know everpresent and uh deeply corrosive and and that affects the way you know the balance of the state and the society just drives me nuts because there is so much on the side of the state so what do you think about the role of society in all of this you've just discussed the state yeah so when you go go deep into each of the three pillars the emergence of firms the emergence of State capability driven by rule of law the emergence of the political system where the state is the agent and not the principal shaped by rule of law in all these standing behind the scenes is the society okay and I think many of us make a mistake by being excessively State Centric in our thinking we just constantly looking at State coercion State coercion as the tool to do this as the tool to do that or the malfunctioning tool that does not do these things right I want to say that I believe that equally the greatness of an advanced economy is in the people it's in the society it is in culture it is in values it is in Norms it is in beliefs it is in the ability to have conversations it is in trust Capital it is in the ability to collaborate it is in communities it is in uh people coming together to solve problems okay I could say this in many different ways but uh I want to touch upon trust capital in uh the in the world values survey there is a question that is asked to households do you generally do you believe that people can generally be trusted and what you see is that in advanced economies there are numbers like 50 60 70 80 90% And in India the number is like 15% and that is such a big impediment upon cooperation like we need to talk to each other okay if you're a neighbor blasting loudspeakers I need to be able to knock on the door and talk with you and hey guy can we do this better can we collaborate and can we solve our problems with each other so if you think of market failure a lot of market failure is all about people and collaboration if people will be able to talk to each other you could solve externalities you could solve public goods you know you could do many many things then the state is not the first recourse you could solve many things at a local level but because we are so weak in that collaboration DNA in India in the Indian Society then we turn to the state for everything but the state is very week and we put too much load on the state so the citizenry is constantly screaming at the state inducing the premature expansion of scope and then you get a organizational collapse of the state because it's being asked to do too much so there is a connection between the collaboration DNA in the people but equally turning to the high end turn to the firms okay there are conversations that take place in coffee shops there are uh conversations that take place in Trust there is debate there is discussion there are oral agreements you don't need to turn everything into a contract a man's word is his honor and so on a lot of things become easier in a stable society when norms and values are established I think we in India are only at the beginning particularly in the elite capability of very diverse people from all over India who are really brought together with the common glue of English and urbanization who are now starting to impinge on each other and gradually learn to trust each each other and develop trust Capital with each other so I think that we should think of these as two distinct Journeys that are taking place in the story of development one is the development of the state which is more conventional and widely understood but equally I feel we have to think a lot about the development of the society the emergence of communities of norms because you can't have the state policing everything you need a lot to be done by people by good behavior towards each other by uh good Behavior by default where bad behavior is the exception and I think all too often in India today we're at the early stages of learning that journey and if I may say one example of that failure of social capital in India is the inability of the genders to talk to each other okay boys and girls in India don't know how to talk to each other just imagine what a Schism in the heart of the country half the people are on one side half the people are on the other side and they can't talk to each other this is so basic this is in the society this is in the culture people need to be able to talk to each other to collaborate to do things each other with each other to trust each other to work orally on a lot of things you don't need a legal contract for everything these are developments that have to happen in the society and at this point I'll uh you know plug two episodes of the scene in the Unseen my most two popular episodes the loneliness of the Indian woman and the loneliness of the Indian man which speaks exactly to that that they can't talk to each other [Music] so you know there is a framing that I have objected to to in the past and it's a framing that's in fact come from both Ragu Rajan and rohini Nik in recent times and I chatted with it on a seen unseen episode with rohini as well which is this tripartite framing of State society and market and in fact even our good friend P kutas along with Ragu jetley in their great book missing an action use the same frame and my objection to the frame is that I think there are two elements to that frame there is a state there is society and that markets are merely one of the mechanisms through which the voluntary actions of society expresses itself it is a mechanism by which people in society do things for other people in positive some ways that benefit them also it is a web of double thank you moments and yet markets get a tremendously bad name now as we have sort of spoken about across many episodes of Everything is Everything firms are critical to a nation's development like if you think about you know people will say unemployment but who will employ you firms will employ you you know who who will generate wealth firms will generate wealth in a positive some way uh you know by uh like how do firms make money by solving the problems of people by giving them something that they uh value more than the money that they pay for it so tell me a little bit about the role of firms in it and do you agree with my sort of objection to that framing I totally agree with uh what you saying the first Grand pillar of development economics is this cold statement that output is made in firms to achieve the per person output of the United States you need the firms to achieve that high productivity and firms are the magic that convert people and capital and machines and manuals and patents into output and there is real magic happening inside firms it's not trivial it's not a simple matter so if it was simple it would have happened by now so why is India poor because there's plentiful labor and there's infinite Capital out in the world economy and the Indian it has the full ability to learn any machine to read any manual we past that maybe you could have argued that 100 years ago this was a bad country where basic knowledge was lacking today the english- speaking Elite in India is able to connect into global capitalism and obtain global technology and that's not the barrier so now you've got to ask the question what is happening in the firms and that is why the first Primal objective of studying development economics and understanding India's journey is the firm and that is why in so many episodes of everything is everything we've gone after one at a time and they're all literally chapters of a book they are a book of firms of understanding the Indian firm and we've tried to construct those episodes in two ways one is they are useful for people inside firms trying to build firms because that journey of how to build high quality firms in India is a very important story about knowledge about culture about how state-of-the-art firms can can emerge from Indian soil and Part B for the thinkers for the researchers these are the ways in which you could study this Indian Journey now a lot of the bad things that happen in the firms are shaped by the government so yes we need to solve policy problems when we need to fix them but once again that will happen once you study the firms that if you put the firm under the microscope and then you see that unpleasant character at the perimeter of the firm who is menacing the firm and spoiling the ability of the firm to think and do the right things then you will get the guidance on how to fix those policy failures and get to a better functioning government but if you will not study firms you will never study that government then you'll never understand this whole journey and so I think that there is a real failure in development economics in not paying enough attention to the firms so you know somehow firms are thought of as rich people and so somehow development economics wants to study poor people and then the firms are like not my concern or my funders don't like it or my journals don't like it and effectively the main engine of the progress of India is in the firms the firms the financial system the regulation of the firms that's the story that's how India got to be a$3 trillion GDP and if we'll not study that then we will not understand it and it will hold us back in our journey to $30 trillion of GDP and for me like this is a classic illustration of that phrase is seen in the Unseen what is seen is that what all the what the firms have achieved and where they've taken us what what is unseen as what they could have achieved if they were not circumscribed by our bad laws by our bad regulations and all of it comes from exactly that the state circumscribing what firms can do what private Enterprise can do what private individuals can do U you know for example our labor laws which have the best of intentions will protect the workers but actually they the growth of firms and therefore they hurt workers and firms will often instead of uh you know building a work base will often have contract labor and etc etc and everyone is crippled we don't have enough companies growing from midsize to large size from achieving scale from generating employment from generating wealth for everyone all of that is kind of crippled by this so the to me the role of the state is really critical that you have to understand you know as we've discussed in a previous episode everything the state does has a cost and the state should only intervene in specific cases where there are market failures you know you can check out that episode and read um uh you know AJ and vij's excellent uh book as it were and and so you know all of this is so connected it's like India is really a tapestry of straight interference in our [Music] lives so AJ we've we began with what are the questions of development economics and what is development economics we've gone through the landscape of failure we spoken about a lot of the flaw thinking so I ask you that question again what should development economics in the the light of what we now know concern itself with yeah so we should be studying State failure we should be studying all the ways in which the Indian state has gone wrong and state failure is all over the landscape that there are uh interventions by the government that should not exist where just freedom suffices and then there are the legitimate important activities of the government that make public goods that do other kinds of uh addressing market failure where the state fumbles and fails and does the wrong thing so one big pillar is understanding State failure and we should also understand the society we should think deeply about the emergence of norms about the way in which the people are able to develop trust capital in each other and do better on talking to each other collaborating with each other developing communities that transcend traditional tribal Notions so for example we've often talked about the Indian Elite English speaker as an amazing Network it is people from all over the country who are able to look past traditional conception of identity and talk to each other engage with each other form communities and make progress with collaboration meta's law the power of a network is proportional to the number of people [Music] squared gentle readers you would have noted our location has shifted how did that happen why did that happen I'll tell you essentially what happened was this a leopard attacked us or almost attacked us it came close and we said leopard leopard Mr leopard as you like to talk to animals Mr leopard why are you attacking us and the leopard said I smell a Bengali Economist at which point I raised my finger and said no you smell a Bengali and you smell an economist you don't smell a Bengali Economist what's really happening is Bengali Economist for us is a bit of a pejorative term and uh maybe one day we'll explain why but the light kind of was going and we thought let's enjoy the sunset together and we'll finish the episode the next morning so here we are the next morning development is a long journey so our episodes and and uh you know you've spoken very well about what development can mean for the society what development can uh you know mean for the firms and how important it is for them to grow my question for you is what about the state is it then uh something that we can consider that once a country is on the way to development and is solving its problems of poverty and so on that the state will also correspondingly be efficient Etc so this is a big thing that confuses many people we know that improvements in state capability are an important part of the development journey and development is about GDP growth can you flip that around and say that when GDP growth has taken place the state has become better okay so a lot of people in India assumed that yeah GDP growth happened so the Indian state must have gotten better but that's not true for precisely the kind of reasons that we've described that there's a journey of firms and adding Capital stock and improve Ming the technology inside the firms and that's where the GDP comes from so there can always be periods where apparently there was GDP growth but side by side with that the state actually got worse so we should not make the mistake of jumping to the conclusion that because GDP growth happened State capability got better in fact there can be periods where the two run in contrary directions one of the great empirical regularities that we know about development is that it tends to happen in spurts so for example in India we got a great growth episode 1991 to 2011 and then many things went wrong after that and by and large economic dynamism has not come back after 2011 and there may almost be a causal connection going on there and I would conjecture something like this that there was a huge growth of the economy from 91 to 2011 along with that there was a lot of complacence and optimism that oh yeah India great 7% growth 8% growth everything is fine and under those conditions the energy to criticize and fix the government tends to go down people become complient government's okay because GDP growth is doing well you come to the false conclusion that the government's okay and in fact all along the development Journey the government needs to keep changing and reshaping itself to fit the needs of the economy that what is required at $300 per capita is very different from what's required at $600 per capita and $1,200 per capita so across each doubling of GDP the Machinery of the state needs to evolve quite a bit another way to see these problems is that when the shear dollar values in the economy become bigger there are bigger problems of corruption because there are bigger dollar values being waved for more mundane cases so a Judicial System a regulatory system that worked for a 100 rupee economy works less well for a 400 rupe economy because whatever was the check and balance that was controlling the misbehavior by state agents of wielding coercive power would break down when the dollar values being waved in front of their noses go up dramatically so in fact the causality runs upside down that left to itself if you don't have enough thinkers and enough passion to constantly criticize and reform the state it is in fact the case that high GDP growth the periods of high GDP growth like 1991 to 2011 are at the end of that Journey a mismatch between the check and balance of government and the needs of the economy and that contains the seeds of its own fall so that is why growth episodes end so what is miraculous about the UK and the US and many advanced economies in their development trajectory is they managed to do 200 years at 3% they just grew and grew at 3% for 200 years they never really faltered and that's amazing whereas the development journey in a lot of developing economies has been a far more of a hair and not a tortoise that there tends to be a spurt in that spurt everybody's RAR everybody's optimistic you'll get a cover of The Economist saying India is great and a spurt is your such at such a low base that you are going to get that spurt if you get some things right but the the Curious difference and I think this is a deep question which has not been adequately answered that why are countries like India subject to growth in episodes we get growth episodes we don't get sustained growth we got an amazing growth episode 1991 to 2011 but then it did not sustain and in some ways what is going on in India right now is the development of the people and the knowledge and the ideas to create conditions for a future next episode we'll see when that will come but we have not found the amazing Economic Development story of the UK and the US where they did not get 6% 7 % ever but they got 200 years of 3% so they just hammered away year after year and they never faltered they never had these bad collapses this is something interesting to ponder but anyway my main headline in response to what you said would be that while it is true that improvements in state capability help create conditions for GDP growth it is not true that you can read GDP growth and thereby conclude that oh yeah the Indian state must have gotten better no that's not true in fact in fact it is possible for things to go the other way around u in the book building State capability uh Matt Andrews land pred a bunch of co-authors do some very careful measurement and quantification around State capability and in fact they do find that in the 1991 to 2011 period Indian State capability got [Music] worse so you know in this particular case what you just said actually doesn't surprise me because uh you know having kind of looked at the way the state functions for a long time I would have imagine that the more money that comes into the state the more it becomes like a beast that feeds itself and feeds itself and feeds itself so you can begin with a lean mean hungry state that does what it is supposed to and doesn't do what it's not supposed to and has capability in getting the few things right that it must but once money starts coming in the incentives as we discussed in the in our public Choice episode just simply take over and it's a beast that gets bloated feeds itself and I think we've seen this in many of the developing economies and in India we kind of have the tragedy that we began with a bloated state that does a lot of things badly and only a few things very well and I would uh say that you know we had that growth episode because we fixed a few things but this fundamental thing we didn't even begin to fix right and uh and got complacent about that so that's a problem but I want to ask you about other wrong conceptions that people have about with regard to caus to development because people will often say that oh educate the people development will come or build infrastructure development will come we've had a couple of episodes on education and infrastructure where we've discussed in detail uh you know how that is actually not happening it happens nowhere in the world quite in that order so I'd like you to sort of expand on those and tell me about some of the other wrong conceptions like redistribution is a classic conception you know people will argue about growth versus redistribution to me if you look at the data it should not even be a debate if you want to eliminate poverty it is very clear that economic growth is necessary and enough of it is sufficient and redistribution doesn't move the needle at all yeah the there are many many theories of Economic Development and you know that is legitimate and healthy there should be a vibrant life of the Mind where there are debates and uh discussion around these things and uh um you know you and I would disagree with many of the following candidates the most important candidate is a big developmental state which is basically javal nehu and mahalanabis that India is a wretched poor country and we disrespect the Creative Energy of the people we disrespect the hunger and the business Acumen of the people we disrespect the private sector and uh the state is this Noble creature who's going to grab the People by the Scruff of the neck and drag the people into to development and you know I I I'm only slightly exaggerating the views of many of the Indian National Congress intellectual Community standing in 1947 see it it is actually for this kind of deeply condescending attitude that a lot of people hate The Elliots yes yeah and in this I feel that they were completely wrong that this was just entirely a wrong approach so this whole idea of big developmental state that development is the work of the state and today I feel we understand this a lot better and today we'll say the job of the state is do some basic right like make the police work make the courts work you know get the air quality solved fix the mosquitoes like as KES has this beautiful quote he says that the job of the state is not to do things a little bit better than what private people will do the job of the state is to do the things that the private sector will not do at all and you know I should have remembered to bring up that quote around UPI that private sector will build Payment Systems I mean what is your value added by building a Government monopoly and throwing in state coercion so the I think the most important rival in the block is Big developmental State the theory that state will lead the development journey and the people will sing the national anthem and follow and I just think that's wrong similarly the development Community has a great deal of delusions around some Notions of Health meaning I think public health is very important but almost everything the development people figured out about health is just wrong they think Health Care they think the private Goods of health care and that's just confused like you got to fix the water you got to fix the sanitation you got to get the air quality you got to do the mosquitoes you got to fight the AMR the anti antimicrobial resistance you got to make a road system where there are fewer Road crashes everything is preventive the site of action is the health of the people not one person at a time so population scale interventions the distinction a want made for me in a seen unseen episode we did where we spoke about this is that if you think about mosquitoes and malaria right if you get malaria and you go to get treatment for your Mal malaria that is Healthcare that is private if you think about eradicating mosquitoes so malaria doesn't happen at all that is a public good and that is what the state should do Public Health yeah and that is the job for the state and there's a vast body of development thinking which has tried to build hospitals in every village or pump Primary Health Centers which is just deluded Sly uh I'm so proud of us Amit as everything is everything we've gone after so many of these building blocks of traditional development thinking we've done an episode on uh education mostly Elementary education we've done an episode on building the knowledge society which is about higher education we've done an episode on infrastructure we've done an episode on digital public goods so these are all the standard theories of change out there and uh I think that many of them are not particularly useful and they don't take us closer to the answer of why only four countries did it because you know lots of countries built Primary Health Centers lots of countries did plenty of years of schooling and and so on so you've got to think deeper what makes an advanced economy why what did Denmark do right what did the UK do right in its development Journey what did Taiwan do right what did South Korea do right what did Japan do right what did Israel do right okay and these are the success stories whereas some of these treatments have been applied to many many countries and there are hundreds of failed journeys of development and I just want to clarify that when we are speaking development we are speaking development from a point of view of getting rid of poverty so it is not as if the elites will do well in their big glass buildings and all of that the you know we're using development Loosely but what we essentially mean by that term is getting people out of poverty which is so critical for India so when we say XYZ don't work and only a works and a is necessary and sufficient and a is economic growth it is with an ion poverty as Gandhi said for every everything that you want to do the state wants to do the fundamental question you should ask is what will it do for the poorest person in the land and and to me it is just not ethical to turn anywhere else and that that's our lens throughout so it is implicit but I'm making it explicit [Music] yeah so a you know you've just made me proud by saying you're proud of the episodes we are doing but actually I'm way more proud of a few of the book that you wrote with vij kka in service of the Republic because you know for the first time there is a book that lays out a fundamental framework for all of this almost like a textbook but much more readable than the typical textbook and you've written about this subject in that textbook at all as well in terms of what is the right way to think about this so tell us a little bit about that yeah so for me personally in my life Journey uh it has been a huge experience to distance myself from the solo model okay so as a economists as children we are brought up with the solo model so lay lay on the capital there's plenty of Labor in the country and you bring the best manuals and the machines and whoosh development is supposed to happen it took me a very long time to step out of that that that's not working that we just stuck and there are so many other features of the society why the development doesn't happen that you know about two-thirds of the Indian working age population is sitting around doing nothing and the firms are not organizing themselves to tap into the infinite capital of the Global Financial system and the infinite cheap labor of India and the best manuals and machines and turn India into advanced economy within one generation why is that not happening so it was a long journey of thinking that what the economists think about like in macroeconomics it is always said there is growth and there are fluctuations and growth is supposed to be the solo model and fluctuations is a business cycle Theory and that's just not a description of the world that we live and so over the years I started understanding more deeply the stories around state state capability rule of law checks and balances and the understanding of public Choice Theory as the central engine of thinking about the misbehavior of the state and why States go wrong and why the Indian State went wrong and also a constructive strategy of not just diagnosing and complaining about what is wrong but the journey on how to fix it that if things went wrong then these are the ways in which you can do things differently how to create limited power in state organizations how to create checks and balances surrounding the state organization how formal structures of organization design and proper drafting of laws creates State organizations with less power with less coercive power and where the poison of the coercive power is placed inside the cladding like a nuclear reactor you need a heavy cladding around the poison of coercive power so that that coercive power is used in benign ways and you know our ultimate dream is the state is supposed to be a machine that converts coercive power into welfare and that doesn't come lightly it is very hard to find that check and balance and again reminiscent of the kind of things we were talking when we discussed digital public goods that just adding power to the state is a bad idea every time you give more information to the state every time you give more power to the state you you have to correspondingly rethink the cladding of the nuclear reactor and it is wrong and irresponsible to just add power to the state and add knowledge to the state add legibility to the state then you're just getting premature State legibility and we're setting the stage for State failure and then uh difficulties in Economic Development so this is the kind of work that Vijay Kar and I did all our lives we were in the Ministry of Finance we have licked our wounds on many many projects that went wrong and you know had the luxury of ruminating over success and failure over many years and so we built the book and our informal internal conversation was that if there was one book we would like to send in a time machine to javal nehu what would that book be this is our attempt that where did things go wrong why did the Indian State fail to go at fundamental levels on that optimism about a big developmental state that was not sufficiently balanced by public Choice Theory by the whole philosophy of checks and balances of mistrusting the power of the state and the key Point here is like when you talk about sending a book to Neu the key assumption there which I agree with is that there is a man of high intellect with great intentions right but he simply doesn't have the body of knowledge around him public Choice Theory doesn't exist hak is like an you know not taken that seriously even though the road to serve D is a bestseller and when you when he looks around him the fashion of the times would indicate that we need a big state and also if we go deeper into the mind of those people they were debating like a UK type strategy which you know in hindsight would have worked better or a USSR type strategy and they were struck by the fact that from 1917 the Russian Revolution had terrible time in their history they' just been whipped in the Battle of tannenburg in 1916 and they'd lost a war to Japan in 1905 and the zaris regime was rotten and uh helped by German military intelligence the Communists kicked in the door okay and in 1917 there had a horrible starting point there was a civil war till 1921 from there to 1945 being the greatest land power in the world okay it's hard to not be impressed by that and another tragedy of that period and uh when lant was here last time we were talking that at the end of the second world war all the protagonists had turned into big state the economy had been Commander to varying extents by all the protagonists to varying extents so in the USSR it was always a communist country in Germany private firms worked at the beend call of the Nazis in the UK it was a war economy managed to a great extent by John mayard canes in the US they passed a new law I'm forgetting the exact name of the law that would give them whip hand in dealing with private firms and then they finally went and contracted with private firms but they passed one law which I think was even used in covid times uh to give the government special power to give orders to firms that are otherwise extra legal so you looked at the entire world standing in 1945 all the big countries the major countries had mobilized War economies so it was very easy to slip into a state on toop illusion so for all these reasons I felt that there is an intellectual fog which shaped conditions to 1947 but it's not just about Neu it's about today look around us in India today most people continue to have a big state delusion you go to the average private business person how will you fix the country they want government to do more not less so building government systems a Government monopoly the government will tell us what to do the government will write an engineering standard the government will build a monopoly these thoughts Run Deep so nehruvian high modernism is Alive and Well in the country even today and you know that's the dream of the book that the journey to development is first and foremost a journey of ideas first the knowledge has to come and then the knowledge can translate into the firms and the state and the people and then you can become an advanced economy so I think that very intellectual journey is the core of where we stand today that we finished a growth episode 1991 to 2011 we love those people they added Great Value to the country there are huge huge mistakes in what was done what was done was incomplete now where are we how do we rebuild the knowledge the community to lay the foundations for a next wave of Indian economic growth and hopefully 100 years at 5% rather than a spurt at [Music] 78% so you know it used to be said about nehu that at every cabinet meeting there was an empty chair for the ghost of Hari right the wrong kind of leftwing Fabian socialist sort of influence and I would have liked if at every cabinet uh meeting There Was You know the disembodied Spirit of L Pritchard from the future coming and sitting with him and the reason I'm mentioning L is that his work has been so educational for me and would be for everyone I'll you know I did a great episode with him that'll be linked from the show notes he's done very good episodes with shuti Raj goal and for the ideas of India podcast that'll be linked we'll link some of his key papers but one of the frames that I learned from him which I've been thinking about a lot and uh which made me look at the world a bit more clearly but also made me angry was a frame of national development was kinky development and his contention is that what matters to all of us is what you are talking about National development how can we raise median per capita GDP because that is what gets people out of poverty that is a central problem it is not an economics problem in some limited domain it is a problem of humanity and therefore we need to sort it out and what is kinky development kinky development is when you lose sight of the big picture and you zoom down on some narrow little thing and you say let me solve that particular problem the metaphor I came up with for our episode was that imagine there is a drunk man looking for something under a lamp post famous old metaphor for many things there's a drunk man looking for something under a lamp poost and a cop comes and says what are you looking for and he says I Dro my keys I'm looking for my keys so the cop a polite good cop and India also has many uh you know fine cops I I'm imagining a Mumbai uh cop says I'll help you and they look together and after a while there's no key there so the cop asks the drunk that you know where did you drop the keys exactly so the drunk turns around and says oh 100 M away in the bushes over there but it's completely dark so the cop says and why are you looking here he says because the light is here that's too hard you know and I feel that the entire field of mainstream development economics is doing that they are searching for easy answers that don't matter you know and part of that also is when you think about the incentives that you are working say either in uh some kind of think tank or you're working for the World Bank or whatever constantly your funders are saying your bosses are saying show me the result you know what have you done and in this long journey of 30 40 50 years of national development it's really hard to point at results you know but it is much easier to take some tiny problem and say that hey we solved that you know and RCT is being just one example of that that so many RCS tackle such ridiculously trivial questions like do treated mosquito Nets help with malaria and you know and and and like that and and lant also has papers on how they don't actually make any difference to the overall development story and for poverty itself but there are easy wins for the people involved and the best and the brightest Among Us you know for rational reasons have moved to this field and that really makes me really angry it's a difficult questions that make a difference that we need to tackle and not look at career advancement and etc etc so give me your sense of what's going on uh it's good to always think in terms of incentives right we economists people respond to incentives so I feel that the Distortion of the field came from three directions of the reshaping of incentives so what happened was that there are philanthropic organizations where the noble human being who's gifting money away is also a vain human being who wants credit okay so these things go hand in hand uh seldom do we get Noble human beings who are gifting away money who are also detached about saying look I don't want the credit I just want to contribute to the world in a good way and there are such people okay I really respect and admire the contributions of many great philanthropists who've done that but a lot of philanthropy tends to obsess around attributability and then again you have to see the organizational design problem that the philanthropist at the top maybe a good human being who's saying look I want to add value to the big problems of the world I want us to do risky things that nobody else will fund I'll add value by doing things that the rest of the world is not doing but the staff the teams the bureaucracy the officials inside a philanthropic organization are under career incentives to claim wins that I did this I contributed to this policy that got 73.4 people out of poverty so there is a thirst for attributability and that takes you to minor problems so you will think of dinky problems and uh then you'll be able to claim success and then the career of the employees inside a philanthropic organization tends to be more assured and also the philanthropist may like to claim credit that I did this so that's the first barrier that pushes in favor of attributability and so you want trivial narrow sites of intervention and people are looking for easy wins so one of the most pernicious things in India is that there is one good bureaucrat and they are severely understaffed and what they need is some English speakers some competent people who will work with them so Consultants are very acable so you get some consultant some erst and young some price waterhous some mckeny and you bring Consulting type people who will take guidance and instructions from The Joint secretary and they will do the day-to-day practical office work of the joint secretary and yeah it helps to have English speakers who will do day-to-day office work of the joint secretary but this is not going to result in change it is not going to result in policy programs and progress and not least because there is a very different uh DNA between the world of economic thinking and reasoning of the kind that we have done here versus the Practical knowledge inside consulting firms so you know that mckeny flavor the uh earnst and young flavor that we will do the automation of an HR System will pay salaries on time I mean that's really trivial it's not deep those are minor gains and those are not important so that is plank one of attributability and being able to say that I supported this important reform either for a philanthropist or the employee of a philanthropic organization the second strand of uh incentives that reshaped this field was populism Okay so we've done an episode on the problems of the populist wave that has surrounded us for many many years and Democratic politics creates pressure for governments to be able to explicitly go after inequality and poverty so it is the great populist idea that we will take from the rich and give to the poor and then you want to obsess on redistribution you want to go to poor people and say I gave you 2,000 Rupees as opposed to I created conditions for GDP growth which is the only permanent solution to Poverty it's difficult to make that claim and tell that story whereas it's visceral and many many people including in the intelligen yeah have these luxury beliefs that a Robin Hood state is a good thing we will take from the rich we are opposed to inequality Okay so we've gone through a long Arc from the French Revolution to Thomas petti where we obsess on inequality that's a second strand where populist politics has asked for economists who will do the redistribution and the welfarism and the third strand of distortion that has shaped this field is the peculiar frat boy world of the economics journals economics journals are very strange places where the referees try to shoot down a submission it is a strange thing I can barely explain to somebody who has not experienced it themselves normally what you think of as research journals and refering is I become passionate about some work I will build a paper on it it goes to the referee who will basically just check that there are no gross errors in it and for the rest it's my paper it's not the referee's paper but in economics the referees jump on board and become a co-author and ask for 20 changes and I can't say no and the referees are only looking for an opportunity to reject a paper and there is a peculiar frat boy world of the United States Journal editors and referees where they have taken Delight in rigor and in causal identification and asking big questions is not important asking important questions is not important you got to be rigorous you've got to get a definitive answer to that subject otherwise you'll be shot down and then the entire career incentive of people trying to get a job trying to get tenure becomes how do I get an 80 page paper through to econometrica where I have dotted every eye and crossed every te and that fighting through the uh frat boy world of the journals kills the ability to think then you're basically down to trivial questions it becomes like this fundraising game that you raise millions of dollars and you will do an RCT and in an RCT you will claim causal identification and you know I would be very skeptical about the usefulness about a lot of those papers but it became the fashion that the journals were ready to accept that and so careers were made around that so these three factors resulted in the Distortion of development economics and we lost touch with the questions and so through this entire episode what you and I have been trying to do is to just go back to First principles and wonder who are we what is this field what are the questions we are asking and so you know several times we refined the question we refined the question in this episode we're not necessarily offering a lot of the answers but we're just trying to align ourselves and get the community straight that Guys these are the questions come let's talk about them let's write about them let's argue about them let's debate them and for the Journey of each of the over 100 poor countries that would desperately love to become an advanced economy one day the journey lies in figuring out these things and rebuilding the firms and the financial system and the society and the state in ways that are conducive to the Machinery of an advanced [Music] economy so you know actually this episode is like a synthesis of a lot of stuff you've already spoken about you mentioned luxury beliefs inequality check out that episode on why inequality and poverty are almost opposite things and need opposite treatments our fixing the knowledge Society was about uh many of these incentives that you just spoke about and in our recommendations really we have joint recommendations on this uh and before even those joint recommendations one recommendation from me that I have learned a lot from in service of the Republic by uh Vijay kelker and aay sha just a masterful book so you must read it if you're interested in these subjects you watch this entire episode please pick up that book please gift it to people who are interested uh dip into it every once in a while it's a great book I also want we now we come to Joint recommendations want to recommend all of land pritchett's Works uh you know the key works and blog posts that he's written uh all of which I think are really seminal are are linked in the show notes below uh plus his episode with me and his episodes with shuti and William easterly's books are again great books in understanding this world of development economics where philanthropists can go wrong where economists can go wrong so that again for me is essential reading to really learn about the world and understand the world and my personal final sort of recommendation would be that too many of us fall into two kinds of easy traps easy trap number one is that there's a bunch of people you're in with you have your little ideological tribe they have these beliefs we seem virtuous we don't question them we fall into that package of beliefs that they uh hold all of which really sound nice and we should build community but we should always keep thinking independently and question everything especially when there is so much data about some of these big questions and the second thing I want to say is a common public policy mistake but a common mistake in thinking about the world that do not mistake intention for outcome you will find many policies with great intentions but the outcomes are completely in the other direction like many of the luxury beliefs he spoke about in that episode you know a lot of the policies which ostensibly are supposed to tackle poverty and help the poor actually end up hurting the poor and this is not uh you know a statement an argument that I'm making alone the data shows us it's very easy for us to actually look at all the data on everything think for ourselves intentions Aren't Enough outcomes really matter if you want to make a difference so you know just think independently we could also be full of right you think independently you read what we've given you you read criticism of that you read people who disagree with us you think about the world you look around you and make up your own mind so my plea to Independent thinking is I think the most important recommendation I can give\"" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 9 + } + ] + }, + { + "cell_type": "code", + "source": [ + "# @title Use the LLM to summaries the transcript.\n", + "\n", + "llm.invoke(\"I want you to clean up a transcript of a conversation with Ajay Shah. I will provide you with a text file in UTF-8 encoding. The character length is about 63000. Will you be able to handle that document and clean it up. It is a somewhat complex task because it is a conversation between Ajay Shah, the Economist from India who also happens to have deep knowledge of digital technologies and open source software, and Amit Varma, a Writer, Journalist and Podcaster. Broadly, Amit asks the questions or adds comments, and Ajay gives his views and understanding of the issues. The transcript is not explicitly marked with who is speaking. Since it is a conversation, the format is not always a question and answer. It can also be in the form of a comment and response. I want do some collaborative thinking with you to delve deeper in the nature of the arguments Ajay makes, what are the strengths and shortcomings of the arguments, the underlying dominant school or theories or worldviews of Economics (e.g. liberalism, neo-liberalism and several others) which might be influncing the argument, can there be an alternate ways of looking at the subject snd so on. I am only familiarity of vatious theories or schools of economics as a curious person, who can comprehend the ideas and arguments. Naturally, having the cleaned transcript is crucial for this to happen fruitfully.: \"+transcript_E57)" + ], + "metadata": { + "id": "q7kBiHLEueJf", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 164 + }, + "outputId": "78911294-19fb-482c-829d-b60b81e9ef7c" + }, + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "'This text appears to be a transcript or summary of a conversation between two individuals, likely economists or development experts. The discussion revolves around various issues in the field of development economics, including:\\n\\n1. **The role of referees in journal publications**: One person expresses frustration with the typical process of having referees review papers, which can lead to an excessive focus on minor details rather than addressing the overall importance and relevance of the research.\\n2. **The prioritization of rigor over impact**: The conversation highlights how the emphasis on rigorous methodology and causal identification has become a dominant paradigm in development economics, potentially stifling innovation and critical thinking.\\n3. **The \"frat boy world\" of journal editors and referees**: This phrase is used to describe a perceived culture of exclusivity and elitism among some economists, where the focus is on publishing papers with complex methodologies rather than exploring pressing questions or making meaningful contributions to the field.\\n4. **The emphasis on randomized controlled trials (RCTs)**: The discussion touches on how RCTs have become the gold standard for evaluating development interventions, which can lead to a narrow focus on specific projects and outcomes rather than considering broader systemic changes.\\n5. **The importance of independent thinking**: The conversation emphasizes the need for economists and policymakers to think critically and independently, considering multiple perspectives and evidence when making decisions.\\n\\nThe text also mentions several key works and authors that have influenced the conversation, including:\\n\\n1. **Vijay Kelker\\'s book \"In Service of the Republic\"**: This book is highly recommended as a masterful analysis of development economics.\\n2. **Land Pritchett\\'s work**: His writings are considered seminal in understanding the challenges facing development economics.\\n3. **William Easterly\\'s books**: These works are mentioned as essential reading for gaining insight into the world of development economics and its pitfalls.\\n\\nThe conversation concludes with a personal recommendation from one of the participants to maintain independent thinking, question established narratives, and consider multiple perspectives when addressing complex problems in development economics.'" + ], + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + } + }, + "metadata": {}, + "execution_count": 10 + } + ] + } + ], + "metadata": { + "colab": { + "provenance": [], + "gpuType": "T4", + "authorship_tag": "ABX9TyOWA5NyJuBdYs1vjsPHczNh", + "include_colab_link": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + }, + "accelerator": "GPU" + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file