(note: the parts in standard type are theory, the parts in italics are narratives that flesh out the theory and, one hopes, help combat the tedium of reading theory)
Organisms survive by finding an unoccupied niche. Look at algae. They pick off photons on the surface of water bodies. In occupying the niche, the organism does something no other organism does. Specifically, the organism finds an unexploited energy source. Algae use photosynthesis to use the Sun’s energy to power themselves. Humans have our own niche, that of intelligence. We mould ourselves to any situation, like putty. Our species asserted dominance over Earth. And we now we only have each other to compete against for control of energy sources. Now what is the very first thing humans do to survive? We form a group. The desire to belong together is drilled deep into us. And bands of humans competed for food sources.
Thus you had different societies competing with each other. Think of each society as an organism – it is the optimization of breaking up the social organism into us. Each person performs his/her function and work together with other people to ensure survival of the group. At this point, I must clarify that I recognize the primacy of the ‘Selfish Gene’ theory. That is, I understand the fundamental driving force is for genes, not groups, to propagate. This essay simply posits an abstraction of genetic forces on a large scale – a model. In large groups each individual, each you, each me, is so insignificant so as to be rendered negligible. Our survival completely bound up in group survival. It truly did become group survival then. Every society is an organism. Every social group is an integrated creature, with its own thought patterns, it’s own culture. Culture represents a social group’s sentience. Culture is the gestalt summation of a group’s laws, traditions, customs, ideas, ideals, the sum total representing an evolution that governs the organism’s behaviour.
Is a society be alive? Do not a society’s ideas evolve? Does not a society defend itself? Does not it grow? Does it not attempt to survive? Does it not show all the characteristics of life? What is life? Any entity that displays directed self-preservation.
What do individuals in society do? Society functions by understanding its environment. Society uses that understanding to extract energy to run on and propagate itself. Thus, society must process all the information it can about its environment in order to generate knowledge. But society is the gestalt sum of all its people. Hence people must do the work of information processing. Now clearly one person is inadequate to process all the input data. Each person can only process a small amount of data. But you have many people. You allocate a small set of data to each person, such that you have as much data covered as possible. There is an infinite amount of data, for all practical purposes, to be processed. Society must decide on the optimal level of data processing. To what depth should a certain area be probed? This is crucial because allocating higher resources to one area is to the detriment of the study of another area. Which areas provides the best chance of providing knowledge that leads to survival?
I just said each person processes a small amount of data. A person must know –
- What data to process.
- What level to process it to.
- How the knowledge it generates fits in with other data.
- What knowledge has been generated in the same data before it, such that they can advance that knowledge.
Looking at those points, society processes information about the environment, but it must process information about itself! And things suddenly get interesting! Society must be conscious of itself, and direct itself such that this information processing is optimized. Data must be broken up optimally, allocated optimally, progress monitored optimally, and the cycles continually tweaked as knowledge grows. Now before we try and get all technical about this, I’d rather try and give you an idea of exactly how complicated it is for a system to study itself. The prime example ofcourse is the brain. Our brain has absolutely no idea how it works. It’s hilarious and absurd. It’s even worse for society; does society understand how society works? I have no way of answering that. But here’s an entertaining news story on, literally on, studying things. The Government Accountability Office performed a study of a study the Pentagon carried out on studies the Defense Department carried out. It is really hard for society to understand itself. Think of how complex the interaction between two humans is. Then think of how complex the interactions in a network of 7,000,000,000 humans is. Okay allow me to put it in perspective. Each one of us has 23,000,000,000 connected neurons. Thus 7 billion people interacting means you have a network of 7 billion nodes, each node itself a network of 23 billion neurons. That’s 161,000,000,000,000,000,000 neurons total.
Societal information processing can be divided into two parts
- Information processing of the environment.
- Information processing of the organism, here the organism being society.
Task #2 is the higher level task. You are overseeing the generation of knowledge. That is, you are processing information about information processing. You must study how information is processed, such that this task is optimized. Because this is the fundamental ongoing process in society. And it is a complex process, requiring broad understanding at many different levels.
Let us break down the factors involved in these two points.
Task #1: Information Processing of the Environment
- Those processing data in an area must possess knowledge of previous knowledge generated, and must have ready access to this information. This is to ensure work is not duplicated, but improved upon.
- Each person should know precisely what it is they must do. Their task must be clearly defined.
- Their goals must be defined and planned, such that other people dependent on this processing get their data in time. Thus coordination of information processing is required.
- Most importantly, the people themselves. People who are capable of carrying out the required information processing are needed. Without them, the system falls apart.
People must be able to share information in real-time and debate their results to continually ensure no work is duplicated and improvements are instantly shared such that other people benefit from their work and vice-versa.
Task #2: Information Processing of Society (that is, understanding society)
- The speed of communication between people must be as high as possible.
- People must be arranged as that information flows optimally, to other people who need it.
- Methods to oversee the generation of information are required.
- Optimal ways of spreading and allocating processing tasks are required.
- Understanding and agreements on data sharing are needed between different hierarchies.
A system cannot have perfect information about itself. To describe oneself, would take memory. And memory is physical. But it would then have to describe that memory, that memory becoming part of itself. Ad infinitum. A system will always have abstracted, imperfect, condensed information on itself. That is why our society is not perfect, and never will be. It is impossible to use intellect to perfectly manage society simply because we cannot describe it perfectly. We will always run an information deficit.
The ultimate goal is to build a 100% efficient system, such that we do not need outside energy to stem the increase in entropy. If society can keep its entropy constant without using energy, we have achieved peak performance. Of course, entropy always increases. We will always need energy to maintain society.
Why is this required?
Why cannot this reduction in entropy be left to evolution? It is because we are reaching the stage where evolution does not help. In projects such as the LHC, the ISS, the UN, there is no competition. They have no competitors. These projects are not forged out of competition, they are formed out of rational, planet-wide cooperation. We do not have multiple attempts with slight variations where the environment choose the best variation.
Let us survey the main points here –
- For all practical purposes, the environment is infinitely complex.
- We assemble into societies.
- For societies to survive, the environment must be understood.
- Each of us has a finite amount of information processing power (IPP).
- If we assume intelligence is randomly distributed among humans, each human’s IPP is roughly equal.
- This IPP is applied to gain knowledge of the environment.
- The only way to increase IPP is to have more people.
- Thus humans participate in society and collaborate to increase the total societal IPP (SIPP).
- However, society imposes its own overhead costs. Hence, some SIPP is used to understand and manage society itself.
- Hence SIPP is divided into Task 1: IP of the Environment and Task 2: IP of Society.
- Clearly, it is optimal to devote as little of SIPP to task 2 as possible, because this does not serve the useful function of understanding the environment and satisfying desire. It only serves to optimize society and reduce inefficiency within society.
- The implication is there is a tradeoff between devoting SIPP to tasks 1 & 2, the condition being that any SIPP devoted to #2 must increase the efficiency of SIPP devoted to #1. But increase it enough such that the knowledge thus generated is greater than the knowledge that would’ve been generated had this SIPP been applied directly to task #1. (For purposes of convenience we shall call SIPP devoted to #1, SIPP1 and SIPP devoted to #2, SIPP2).
- A society that requires too much SIPP2 to the detriment of SIPP1 will lose its edge and its progress will slow. Thus the greater the ratio of SIPP1 to SIPP2, the better a society will function.
- On the other hand a society may generate knowledge but may be unable to distribute it efficiently throughout itself. It may underestimate the need for managing knowledge generation, not allocating enough to SIPP2, hence hamstringing itself.
- Thus, a careful investigation of the optimal allocation of SIPP2 versus SIPP1 is required.
- The demand for SIPP is proportional to what we know we do not know about the environment.
- That is to say, in the 1800s we had not invented quantum mechanics, hence there was no demand for people to process information with regard to that subject. But when we did invent it, we realised the entire field had to be fleshed out, thus creating demand for SIPP.
- As more knowledge is generated, the demand for IP grows. Hence IP generates demand for more IP.
- Thus IP does two things – (a) It helps society survive and (b) It generates demand for more IP.
- Why does (b) happen? Because when you generate new, previously unknown knowledge, that knowledge is used to fulfill wants. But to do so that knowledge must be fleshed out.
- To expand upon this with the quantum mechanics example. An understanding of quantum mechanics fuelled the semiconductor industry. However, when quantum mechanics was discovered, no one had a practical use for it. It was “just a bunch of curious physicists exploring the environment”. 
- This raises the interesting questions of how new theories are generated – the development of quantum mechanics being one.
- There was no known use for quantum mechanics when it was formulated. This means there the SIPP that went into formulating it was not being useful. Thus it must have been superfluous SIPP, not needed in any gainful tasks. There was SIPP that society did not need. Moreover society had the resources to support this SIPP. That is, the institutes, and laboratories and money to fund and support this SIPP, even though it was not providing any return.
- This means a new revolution can take place only when society possess extra SIPP. This means the demand generated for SIPP by the previously discovered theories must have faded.
Thus, to sum, society has a need for IP. Every person born goes towards satisfying this need. When this need is fulfilled, those persons can explore and a new field is found. Society then finds that social needs/human wants can be satisfied by this new field. This new field absorbs SIPP. Now when this field is fleshed out, newer fields can be discovered.
The inter-disciplinary nature of these ‘waves’ of progress is harder to discern. As progress accumulates, and that as waves feed off each, the potential for cross links between waves increases. Does progress become faster? Or, do the problems get harder keeping the rate of progress the same? For example, airplane travel has stayed the same between 1960 and 2015. Progress in experimental particle physics has become slow compared to the rate of advance of theoretical particle physics. But progress in one field could spark a wave in another field even though the original field stops progressing. This is a question for complexity theorists.
In the case of truly new paradigm shifts, for example quantum computing, the building blocks are not in place, and so perhaps there will always be a finite amount of time between waves. Perhaps progress is not necessarily exponential, an assertion which leads to the ‘Singularity’. Each ‘wave’ takes its own time to complete, because of the inherent nature of its difficulty. One must not conflate incremental progress with waves. For example, Moore’s Law is the result of incremental progress and is indeed exponential. On the other hand, the shift from traditional binary computing to quantum computing is a paradigm shift where several different approaches and techniques, both in theoretical and experimental realms, need to be tried before the field can stabilize. This is a the birth of a new wave. Hence a wave can be thought of as an advance on the order of a magnitude, which no matter what will challenge us and take its own sweet time complete.
Task #2: Information Processing of Society
- We form groups/societies to compete.
- Society must understand the environment to survive (information processing – IP).
- As the size of society, defined by the number of humans forming it, increases, each person is unable to keep up communication links with other humans.
- This serves as a disincentive for societal growth, because if I cannot gain from IP done by others, I have no incentive to share IP with them. Hence, there must be some way to ensure IP results are transmitted from every person to every other person.
- The solution is a small world network, where IP can be transmitted from any one person to any other, with only a small number of links. We are all familiar with this as “Six Degrees of Separation”.
- Thus, growing society comes down to efficient communications links. With better communication links, the incentive for people to cooperate increases because they can exchange information with each other. This is the basis of society.
- However, modern day society is enormous. Even with small world networks and communication links, we are unaware of much of society.
- If we cannot keep a check on whether we are getting the most out of society, what is the solution? The only solution is trust. You can only do your part, and repose hope in others that they do theirs.
- Hence, it is crucial to society that total strangers form an unwritten, unsaid pact to trust the work of each other, because communication links cannot keep up with society’s increasing complexity.
- However, as cooperation is enforced by society, the incentive to cheat increases. At some point people will cheat, because their actions will be screened by society’s size.
- We have a limit to societal growth. Size limited by the boundary where cheating will go unnoticed simply because society’s size ensures the part of society in which cheating takes place is walled off by communication bottlenecks.
- When such cheating is revealed, people, i.e. we, lose trust in society. Society’s fabric is strained, and we stop sharing our IP with others, bringing progress to a halt.
- Society ofcourse recognizes this, and implements structures such as moral codes, culture, societal norms et al to ensure we stay in line, trust each other and can in turn be trusted. This goes in SIPP2.
- However, these structures demand obedience and reduce a society’s ability to adapt. A turning point will come with they do not optimize SIPP1 but detract from it. Hence, every society builds structure over time to develop SIPP2, but structure, by the very nature of it, resists change and will at some point be detrimental to SIPP1. Thus, all societies will stop progressing and fail, given time. This is inevitable. Unless, a society consciously recognizes this and consistently reconfigures itself to constantly and instantly adapt to changing circumstances.
- Thus we have two inherent constraints here, (a) People will cheat and (b) Society is its own enemy. However, if we recognize and understand these processes, we can take steps to change them.
Every society has structure. And this structure is an adaption to the environment at that point. The environment will change and that structure will not longer be an adaptation. Indeed, any adaptation a society makes will become detrimental at some point. This is because societies tend to find it impossible to tear down existing structures. Structures entrench power, and entrenched power will not give up its hold. This is why every society will fail ultimately. Because their adaptations that suited them at one point in time, become liabilities, and societies resist change to those liabilities. This is why, after a point, societies either collapse, or undergo revolution. A revolution is the only way to dismantle structure and wipe the slate clean. Revolutions are an integral part of society. They enable societies to survive by purging a society of its mal-adaptations. A society can either choose a revolution, or it can die. Think of revolutions as wildfires. Wildfires clear old growth and make way for new vegetation to take root. They allow the new in, and clear the old. Similarly, revolutions throw out the old and useless, and usher in the new.
- There is another constraint, a much more fundamental one for me. That is the limit on each person’s processing power. Even if we could somehow link with every other person, we cannot process the information they give us. It is beyond our capabilities. Which means we have to trust. Either we trust, or we have no incentive to participate in a society that becomes too complex to handle.
- At some point, we will be so overwhelmed by information that we will not even be able to know the degree to which to trust what little information we do receive. At this point, we have to trust blindly, and this means we can be cheated. As is evident, this is a much deeper, more fundamental, limit on societal size. Why? Because while society’s IP has progressed exponentially over time, each individual’s IP has more or less remained static. And this static IP must deal with exponentially increasingly knowledge. I think we can see the outlines of the scale of this problem here.
- A solution you may cite to this is a hierarchical structure of information processing in society. Thus each layer abstracts and simplifies knowledge generated and disseminates it to other people. However, this control of information means control of power, and so perhaps multiplies the degree of trust you need to repose.
- This problem hides itself. If you do not even receive information and are unable to process the amount you do, you cannot see the scale of the problem.
A potential, “science-fiction” solution is to increase our brain’s processing power using technology. Anything that can be done by a computer must be hived off to a computer. Our brains and computers complement each other, as they are already doing (Wikipedia/search engines/etc).
Another pertinent issue is that we are shaped to compete with one another. That is how we progress. However, we are reaching the point where competition no longer applies. For example, does the United Nations have a competitor? Does the Large Hadron Collider has a competitor? No. These can only be built by the active cooperation of every country on this planet. Because these projects are simply too big for each country to solve. Hence there can be no competitors. Take global warming. It affects us all, and require every country to cooperate and reach a consensus. If even one country backs out, it incentivizes other countries to back out saying,”Why me if not them?”. On treaties that control nuclear weapons, global warming, world trade and so on, every country must sign on. It’ll be interesting to see how such projects progress in the future.