The formatter threw an exception while trying to deserialize the message: Error in deserializing body of request message for operation 'Translate'. The maximum string content length quota (30720) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 2, position 31674.
The formatter threw an exception while trying to deserialize the message: Error in deserializing body of request message for operation 'Translate'. The maximum string content length quota (30720) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 1, position 31873.
In late 2008, tech luminary Kevin Kelly, the founding executive editor of Wired magazine, published a critique of what he calls “thinkism” — the idea of smarter-than-human Artificial Intelligences with accelerated thinking and acting speeds developing science, technology, civilization, and physical constructs at faster-than-human rates. The argument over “thinkism” is important to answering the question of whether Artificial Intelligence could quickly transform the world once it passes a certain threshold of intelligence, called the “intelligence explosion” scenario.
Kelly begins his blog post by stating that “thinkism doesn’t work”, specifically meaning that he doesn’t believe that a smarter-than-human Artificial Intelligence could rapidly develop infrastructure to transform the world. After using the Wikipedia definition of the Singularity, Kelly writes that Vernor Vinge, Ray Kurzweil and others view the Singularity as deriving from smarter-than-human Artificial Intelligences (superintelligences) developing the skills to make themselves smarter, doing so at a rapid rate. Then, “technical problems are quickly solved, so that society’s overall progress makes it impossible for us to imagine what lies beyond the Singularity’s birth”, Kelly says. Specifically, he alludes to superintelligence developing the science to cure the effects of human aging faster than they accumulate, thereby giving us indefinite lifespans. The notion of the Singularity is roughly that the creation of superintelligence could lead to indefinite lifespans and post-scarcity abundance within a matter of years or even months, due to the vastly accelerated science and robotics that superintelligence could develop. Obviously, if this scenario is plausible, then it might be worth devoting more resources to developing human-friendly Artificial Intelligence than we are currently. A number of eminent scientists are beginning to take the scenario seriously, while Kelly stands out as an interesting critic.
Kelly does not dismiss the Singularity concept out of hand, saying “I agree with parts of that. There appears to be nothing in the composition of the universe, or our minds, that would prevent us from making a machine as smart as us, and probably (but not as surely) smarter than us.” However, he then rejects the hypothesis, saying, “the major trouble with this scenario is a confusion between intelligence and work. The notion of an instant Singularity rests upon the misguided idea that intelligence alone can solve problems.” Kelly quotes the Singularity Institute article, “Why Work Towards the Singularity”, arguing it implies an “approach [where] one only has to think about problems smartly enough to solve them.” Kelly calls this “thinkism”.
Kelly brings up concrete examples, such as curing cancer and prolonging life, stating that these problems cannot be solved by “thinkism.” “No amount of thinkism will discover how the cell ages, or how telomeres fall off”, Kelly writes. “No intelligence, no matter how super duper, can figure out how human body works simply by reading all the known scientific literature in the world and then contemplating it.” He then highlights the necessity of experimentation in deriving new knowledge and working hypotheses, concluding that, “thinking about the potential data will not yield the correct data. Thinking is only part of science; maybe even a small part.”
Part of Kelly’s argument rests on the idea that there are fixed-rate external processes, such as the metabolism of a cell, which cannot be sped up to provide more experimental data than they would otherwise. He explains, that “there is no doubt that a super AI can accelerate the process of science, as even non-AI computation has already sped it up. But the slow metabolism of a cell (which is what we are trying to augment) cannot be sped up.” He also uses physics as an example, saying “If we want to know what happens to subatomic particles, we can’t just think about them. We have to build very large, very complex, very tricky physical structures to find out. Even if the smartest physicists were 1,000 smarter than they are now, without a Collider, they will know nothing new.” Kelly acknowledges the potential of computer simulations but argues they are still constrained by fixed-rate external processes, noting, “Sure, we can make a computer simulation of an atom or cell (and will someday). We can speed up this simulations many factors, but the testing, vetting and proving of those models also has to take place in calendar time to match the rate of their targets.”
Continuing his argument, Kelly writes: “To be useful artificial intelligences have to be embodied in the world, and that world will often set their pace of innovations. Thinkism is not enough. Without conducting experiements, building prototypes, having failures, and engaging in reality, an intelligence can have thoughts but not results. It cannot think its way to solving the world’s problems. There won’t be instant discoveries the minute, hour, day or year a smarter-than-human AI appears. The rate of discovery will hopefully be significantly accelerated. Even better, a super AI will ask questions no human would ask. But, to take one example, it will require many generations of experiments on living organisms, not even to mention humans, before such a difficult achievement as immortality is gained.”
Concluding, Kelly writes: “The Singularity is an illusion that will be constantly retreating — always “near” but never arriving. We’ll wonder why it never came after we got AI. Then one day in the future, we’ll realize it already happened. The super AI came, and all the things we thought it would bring instantly — personal nanotechnology, brain upgrades, immortality — did not come. Instead other benefits accrued, which we did not anticipate, and took long to appreciate. Since we did not see them coming, we look back and say, yes, that was the Singularity.”
This fascinating post of Kelly’s raises many issues, the two most prominent being:
1) Given sensory data X, how difficult is it for agent Y to come to conclusion Z?
2) Can experimentation be accelerated past the human-familiar rate or not?
These will be addressed below.
There are many interesting examples in human history of situations where people “should” have realized something but didn’t. For instance, the ancient Egyptians, Greeks, and Romans had all the necessary technology to manufacture hot air balloons, but apparently never thought of it. It wasn’t until 1783 that the first historic hot-air balloon flew. It is possible that ancient civilizations did build hot-air balloons and left no archeological evidence of their remains. One hot air balloonist thinks the Nazca lines were viewed by prehistoric balloonists. My guess would be that the ancients might have been clever enough to manufacture hot air balloons, but probably not. The point is that they could have built them, but didn’t.
Inoculation and vaccination are another relevant example. A text from 8th century India included a chapter on smallpox and mentioned methods of inoculating against the disease. Given that the value of inoculation was known in India c. 750 BC, it would seem that the modern age of vaccination should have arrived prior to 1796. Besides safe water, vaccines reduce mortality and increase population growth more than any other means. Aren’t 2,550 years enough time to go from the basic principle of inoculation to the notion of systematic vaccination? It could be argued that the discovery of cell theory (1665) was a limiting factor; if cell theory had been introduced to 8th century Indians, perhaps they would have been able to develop vaccines and save the world from hundreds of millions of unnecessary deaths.
Lenses, which are no more than precisely curved pieces of glass, are fundamental to scientific instruments: the microscope and the telescope and are at least 2,700 years old; the Nimrud lens, discovered at the Assyrian palace of Nimrud in modern-day Iraq, demonstrates their antiquity. The discoverer of the lens noted that he had seen very small inscriptions on Assyrian artifacts that made him suspect that a lens was used to create them. There are numerous references to and evidence of lenses in antiquity. The Visby lenses found in a 11th to 12th century Viking town are sophisticated aspheric lenses with angular resolution of 25–30 µm. Even after lenses became widespread in 1280, it took microscopes almost 500 years to develop to the point of being able to discover cells. Given that lenses are as old as they are, why did it take so incredibly long for our ancestors to develop them to the point of being able to build microscopes and telescopes?
A final example that I will discuss regards complex gear mechanisms and analog computers in general. The Antikythera mechanism, dated to 100 BC, consists of about 30 precisely interlocked bronze gears designed to display the locations in the sky of the Sun, Moon, and the five planets known at the time. Why wasn’t it until more than 1,400 years later that mechanisms of similar complexity were constructed? At the time, Greece was a developed civilization of about 4-5 million people. It could be that a civilization of sufficient size and stability to produce complex gear mechanisms did not come into existence until 1,400 years later. Perhaps a simple lack of ingenuity is to blame. The exact answer is unknown, but we do know that the mechanical basis for constructing bronze gears of similar quality existed for a long time, it just wasn’t put into use.
It apparently takes a long time for humans to figure some things out. There are numerous historic examples where all the pieces of a puzzle were on the table, there was just no one who put them together. The perspective of “thinkism” suggests that if the right genius were alive at the right time, he or she would have put the pieces together and given civilization a major push forward. I believe that this is borne out by contrasting the historical record with what we know today.
It takes a certain amount of information to come to certain conclusions. There is a minimum amount of information required to identify an object, plan a winning strategy in a game, model someone’s psychology, or design an artifact. The more intelligent or specialized the agent is, the less information it needs to reach the conclusion. Conclusions may be “good enough” rather than perfect, in other words, “ecologically rational”.
An example is how good humans are at recognizing faces. The experimental data shows that we are fantastic at this; in one study, half of respondents correctly identified this image as being a portrait of Napoleon Bonaparte, even though it is only a mere 6×7 pixels.
MIT computational neuroscientist Pawan Sinha found that given 12 by 14 pixels worth of visual information, his experimental subjects could recognize 75-percent of the face images in a set accurately, where the set had a mix of faces and other objects. Sinha also programmed a computer to identify face images, with a high success rate. A New York Times article quotes Dr. Sinha: “These turn out to be very simple relationships, things like the eyes are always darker than the forehead, and the mouth is darker than the cheeks,” Dr. Sinha said. “If you put together about 12 of these relationships, you get a template that you can use to locate a face.” There are already algorithms that can identify faces from databases which only include a single picture of an individual.
These results are relevant because they are examples where humans or software programs are able to make correct judgments with extremely small amounts of information, less than we would intuitively think is necessary. The picture of Napoleon above can be specified by about 168 bits. Who would imagine that hundreds of people in an experimental study could uniquely identify a historic individual based on a photo containing only 168 bits of information? It shows that humans have cognitive algorithms that are highly effective and specialized at identifying such information. Perhaps we could make huge scientific breakthroughs if we had different cognitive algorithms specialized at engaging unfamiliar, but highly relevant data sets.
The same could apply to observations and conclusions of all sorts. The amount of information needed to make breakthroughs in science could be less than we think. We do know that new ways of looking at the world can make a tremendous difference in uncovering true beliefs. A civilization without science might exist for a long time without accumulating significant amounts of objective knowledge about biology or physics. For instance, the Platonic theory of classical elements persisted for thousands of years.
Then, science came along. In the century following the development of the Scientific Method by Francis Bacon in 1620, there was rapid progress in science and technology, fueled by this new worldview. By 1780, the Industrial Revolution was in full swing. If the Scientific Method had been invented and applied in ancient Greece, progress that would have seemed mind-boggling and impossible at the time, like the Industrial Revolution, could have potentially been achieved within a century or two. The Scientific Method increased the marginal usefulness of each new piece of knowledge humanity acquired, giving it a more logical and epistemologically productive framework than was accessible in the pre-scientific haze.
Could there be other organizing principles of effective thinking analogous to the Scientific Method that we’re just missing today? It seems hard to rule it out, and quite plausible. The use of Bayesian principles in inference, which has led to breakthroughs in Artificial Intelligence, would be one candidate. Perhaps better thinkers could discover such principles more rapidly than we can, and make fundamental breakthroughs with less information than we would currently anticipate being necessary.
A key factor defining feats of intelligence or cleverness is surprise. Higher intelligence sees the solution no one else saw, looks past the surface of a problem to find the general principles and features that allow them to understand and resolve it. A classic, if cliché example is Albert Einstein deriving the principles of special relativity working as a patent clerk in Bern, Switzerland. His ideas were considered radically counterintuitive, but proved correct. The concept of the speed of light being constant for all observers regardless of their velocity had no precedent in Newtonian physics or common sense. It took a great mind to think about the universe in a completely new way.
Kelly rejects the notion of superintelligence leading to immortality when he says, “this super-super intelligence would be able to use advanced nanotechnology (which it had invented a few days before) to cure cancer, heart disease, and death itself in the few years before Ray had to die. If you can live long enough to see the Singularity, you’ll live forever [...] The major trouble with this scenario is a confusion between intelligence and work.” Kelly highlights “immortality” as being very difficult to achieve through intelligence and its fruits alone, but this understanding is relative. Medieval peasants would see rifles, freight trains, and atomic bombs as very difficult to achieve. Stone Age man would see bronze instruments as difficult to achieve, if he could imagine them at all. The impression of difficulty is relative to intelligence and the tools a civilization has. To very intelligent agents, a great deal of tasks might seem easy, including vast categories of tasks that less intelligent agents cannot even comprehend.
Would providing indefinite lifespans (biological immortality) to humans be extremely difficult, even for superintelligences? Instead of saying “yes” based on the evidence of our own imaginations, we must confess that we don’t know. This doesn’t mean that the probability is 50% — it means we really don’t know. We can come up with a tentative probability, say 10%, and iterate based on evidence that comes in. But to say that it will not happen with high confidence is impossible, because a lesser intelligence cannot place definite limits (outside of, perhaps, the laws of physics) on what a higher intelligence or more advanced civilization can achieve. To say that it will happen with high confidence is also impossible, because we lack the information.
The general point is that one of the hallmarks of great intelligence is surprise. The discovery of gunpowder must have been a surprise. The realization that the earth orbits the Sun and not vice versa was a surprise. The derivation of the laws of motion and their universal applicability was a surprise. The creation of the steam engine led to surprising results. The notion that we evolved from apes surprised and shocked many. The idea that life was not animated by a vital force but in fact operated according to the same rules of chemistry as everything else was certainly surprising. Mere human intelligence has surprised us time and time again with its results — we should not be surprised to be surprised again by higher forms of intelligence, if and when they are built.
One of Kelly’s core arguments is that experimentation to derive new knowledge and the “testing, vetting and proving” of computer models will require “calendar time”. However, it is possible to imagine ways in which the process of experimentation and empirical verification could be accelerated to faster-than-human-calendar speeds.
To start, consider variance in the performance of human scientists. There are historic examples of times where scientific and technological progress was very rapid. The most recent and perhaps striking example was during World War II. Within six years, the following technologies were invented: radar, jet aircraft, ballistic missiles, nuclear power and weapons, and general-purpose computers. So, despite fixed-rate external processes limiting the rate of experimentation, innovation was temporarily accelerated anyway. Intuitively, the rate of innovation was arguably three to four times greater than in a similar period before the war. Though the exact factor is subjective, few historians would disagree that rapid scientific innovation occurred during WWII.
Why was this? Several factors may be identified: 1) increased military spending on research, 2) more scientists due to better training connected to the war effort, 3) researchers working harder and with more motivation than they otherwise would, 4) second-order effects resulting from larger groups of brilliant people interacting with one another in a supportive environment, as in the Manhattan Project.
An advanced Artificial Intelligence could employ all these strategies to accelerate its own speed of research and development. It could 1) amass a large amount of resources in the form of physical and social capital, and spend them on research, 2) copy itself thousands or millions of times using available computers to ensure there are many researchers, 3) possess perfect patience, perpetual alertness, and accelerated thinking speed to work harder than human researchers can, and 4) benefit from second-order effects by utilizing electronic communication between its constituent researcher-agents. To the extent that accelerated innovation is possible with these strategies, an Artificial Intelligence could exploit them to the fullest degree possible.
Of course, experimentation is certainly necessary to make scientific progress — many revolutions in science begin with peculiar phenomena that are artificially magnified with the aid of carefully designed experiments. For instance, the double-slit experiment in quantum mechanics emphasizes the wave-particle duality of light, a phenomenon not typically observed during everyday circumstances. Determining the details of how different chemicals intermingle to produce reaction products has required millions of experiments. Understanding biology has required many millions of experiments as well. Only strictly observational facts such as the cellular structure of life or the surface features of the Moon can be assessed through direct observation. To determine how metabolic processes actually work or what is underneath the surface of the moon requires experimentation, trial and error.
There are four concrete ways in which experimentation might be accelerated to speeds beyond the typical human level. These are conducting experiments faster, more efficiently, conducting them in parallel, and choosing the most useful experiments to begin with. Kelly argues that “the slow metabolism of a cell (which is what we are trying to augment) cannot be sped up”. But, this is not entirely clear. It should be possible to build chemical networks that simulate cellular processes operating more quickly than cellular metabolisms do. In addition, it is not clear that a comprehensive understanding of cells would be necessary to achieve biological immortality. Achieving indefinite biological lifespans could be more readily achieved by repairing cellular damage and chemical junk faster than it accumulates than constantly keeping all cells in a state of perpetual youth, which seems to be what Kelly is implying is necessary. In fact, it may be possible to develop therapies for repairing the damages of aging with our current biological knowledge. Since we aren’t superintelligences, it is impossible to tell. But Kelly makes an error when he assumes that keeping all cells in a state of perpetual youth, or total understanding, is required for indefinite lifespans. This shows how even small differences in knowledge between humans can make an all-important difference in research targets and agendas. The difference in knowledge between humans and superintelligences will make the difference larger still.
Considering these factors highlights the earlier point that the perceived difficulty of a given advance, like biological immortality, is strongly influenced by the framing of the necessary prerequisites to achieve that advance, and the intelligence doing the evaluation. Kelly’s framing of the problem is that massive amounts of biological experimentation would be necessary to derive the knowledge to repair the body faster than it breaks down. This may be the case, but it might not be. A higher intelligence might be able to achieve equivalent insights with ten experiments that a lesser intelligence would require a thousand experiments to uncover.
The rate of useful experimentation by superhuman intelligences will depend on factors such as 1) how much data is needed to make a given advance and 2) whether experiments be accelerated, simplified, or made massively parallel.
Research in biology, medicine, and chemistry have exploited highly parallel robotic systems for experiments. This field is called high-throughput screening (HTS). One paper describes a machine that simultaneously introduces 1,536 compounds to 1,536 assay plates, performing 1,536 chemical experiments at once in a completely automated fashion, determining 1,536 dose-response curves per cycle. Only 23 nanoliters of each compound is transferred. This highly miniaturized, highly parallel, high-density mode of experimentation has only begun to be exploited due to advances in robotics. If robotics could be manufactured on a massive scale more cheaply, one can imagine warehouses full of such machines conducting many hundreds of millions of experiments simultaneously.
Another method of accelerating experimentation would be to improve microscale manufacturing and to construct experiments using the minimum possible quantity of matter. For instance, instead of dropping weights off the Leaning Tower of Pisa, construct a microscale vacuum chamber and drop a cell-sized diamond grain in that chamber. Thousands of physics experiments could be conducted in the time it would require to conduct one experiment by the traditional method. With better sensors, you can conduct an experiment on ten cells that with inferior sensors would necessitate a million cells. More fine-grained control of matter can allow an agent to extract much more information from a smaller experiment that costs less and can be run faster and massively parallel. It is conceivable that an advanced Artificial Intelligence could come up with millions of hypotheses and test them all simultaneously in one small building.
In his 1992 paper defining the Singularity, Vernor Vinge called the hypothetical post-Singularity world “a regime as radically different from our human past as we humans are from the lower animals”. Kelly, meanwhile, said that for artificial intelligences to amass scientific knowledge and make breakthroughs (like biological immortality) would require detailed models, and the “testing, vetting and proving of those models” requires “calendar time”. These models will “take years, or months, or at least days, to get results”. Since the comparison between different species is sometimes seen as a model for plausible differences between humans and superintelligences, let’s apply that model to the context of experiments that Kelly is referring to. Do humans create effects in the world faster than squirrels? Yes. Are humans qualitatively better at working towards biological immortality than squirrels? Yes. Do humans have a fundamentally superior understanding of the universe than squirrels do? It would be safe to say that we do.
The comparison with squirrels sounds absurd because concepts like biological immortality and “understanding the universe” are fuzzy at best from the perspective of a squirrel. Analogously, there may be stages in comprehension of reality that are fundamentally more advanced than our own and only accessible to higher intelligences. In this way, the “calendar time” of humans would have no more meaning to a superintelligence than “squirrel time” has relevance to human life. It’s not so much a factor of time — though higher intelligences can do much more in much less time — but also the general category of thoughts which can be processed, objectives which can be imagined, and plans which can be achieved. The objectives and methods of a higher intelligence would be on a completely different level than those of a lower intelligence and are different in kind, not degree.
There are several reasons why it makes sense to assume that qualitatively smarter-than-human intelligence, that is, qualitative differences on the order of difference between humans and squirrels or greater, should be possible. The first reason concerns the relative speed of human neurons relative to artificial computing machinery. Modern computers operate at billions of serial operations per second. Human neurons operate at only a couple hundred serial operations per second. Since most acts of cognition must occur within one second to be evolutionarily useful, and must include redundancy and fault tolerance, the brain is constrained to problem solutions involving 100 serial steps or less. What about the universe of possible solutions to cognitive tasks that require more than 100 serial steps? If the computer you are using had to implement every meaningful operation in 100 serial steps, the vast majority of common algorithms used today would have to be thrown out. In the space of possible algorithms, it quickly becomes obvious that constraining a computer to 100 serial steps is an onerous limitation. Expanding this space by a factor of ten million seems likely to lead to significant qualitative improvements in intelligence.
The reason that qualitatively smarter-than-human intelligence is possible is about neurological hardware and software. There are relatively few hardware differences between humans and chimpanzee brains. The evidence actually supports the notion that primate brains are more distinct from non-primates than humans are from other primates, and that the human brain is merely a primate brain scaled up for a larger body and with an enlarged prefrontal cortex. One quantitative study of human vs. chimpanzee brain cells came to this conclusion:
Despite our ongoing efforts to understand biology under the light of evolution, we have often resorted to considering the human brain as an outlier to justify our cognitive abilities, as if evolution applied to all species except humans. Remarkably, all the characteristics that appeared to single out the human brain as extraordinary, a point off the curve, can now, in retrospect, be understood as stemming from comparisons against body size with the underlying assumptions that all brains are uniformly scaled-up or scaled-down versions of each other and that brain size (and, hence, number of neurons) is tightly coupled to body size. Our recently acquired quantitative data on the cellular composition of the human brain and its comparison to other brains, both primate and nonprimate, strongly indicate that we need to rethink the place that the human brain holds in nature and evolution, and to rewrite some basic concepts that are taught in textbooks. The human brain has just the number of neurons and nonneuronal cells that would be expected for a primate brain of its size, with the same distribution of neurons between its cerebral cortex and cerebellum as in other species, despite the relative enlargement of the former; it costs as much energy as would be expected from its number of neurons; and it may have been a change from a raw diet to a cooked diet that afforded us its remarkable number of neurons, possibly responsible for its remarkable cognitive abilities.
In other words, it appears as if our exceptional cognitive abilities are the direct result of having more neurons rather than neurons in differing arrangements or relative quantities. If this continues to be confirmed in subsequent analyses, it implies, all else equal, that scaling up the number of neurons in the human brain could lead to similar intelligence differentials as those between humans and chimps. Given the evidence above, this should be our default assumption — we would need specific reasoning or evidence to assume otherwise.
A more detailed reason for why qualitatively smarter-than-human intelligence seems possible is that the higher intelligence of humans and primates appears to have something to do with self-awareness and complex self-referential loops in thinking and acting. The evolution of primate general intelligence appears correlated with the evolution of brain structures that control, manipulate, and channel the activity of other brain structures in a contingent way. For instance, a region called the pulvinar was called the brain’s “switchboard operator” in a recent study, though there are dozens of brain areas that could be given this description. Of 52 Brodmann areas in the cortex, at least seven are “hub areas” which lie near the top of a self-reflective control hierarchy: areas 8, 9, 10, 11, 12, 25, and 28. Given that these areas obviously play important roles in what we consider higher intelligence, yet only evolved relatively recently in evolutionary terms and are comparatively poorly developed, it is quite plausible to suggest that there is a lot of room for improvement in these areas and that qualitative intelligence improvements could result.
Imagine a brain that has “hub areas” which can completely reprogram other brain modules on a fine-grained level, the sort of reprogramming and flexibility only currently available in computers. Instead of only being able to reprogram a few percent of the information content of our brains, like we have now, a mind that can reprogram 100 percent of its own information content would allow limitless room for fast, flexible cognitive adaptation. Such a mind could quickly reprogram itself to suit the task at hand. Biological intelligences can only dream of this kind of adaptiveness and versatility. It would open up a vast new space not only for functional cognition but also appreciation of aesthetics and other higher-order mental traits.
Say that we could throw open the hood of the brain and enhance it. How would that work?
To understand how “smarter than human intelligence” would work requires overviewing how the brain works. The brain is a very complicated machine. It operates entirely according to the laws of physics, and includes specific modules designed to handle different tasks. Look at our capabilities of identifying faces; it is clear that our brains have specific neural hardware adapted to rapidly identifying human faces. We don’t have the same hardware for rapidly identifying lizard faces — every lizard is just a lizard. To a lizard, different lizard faces might intuitively appear highly distinct, but to us humans, a species wherein there is no adaptive value in differentiating lizard faces, they all look the same.
The paper “Intelligence Explosion: Evidence and Import” by Luke Muehlhauser and Anna Salamon reviews some features of what Eliezer Yudkowsky calls the “AI Advantage” — inherent advantages that an Artificial Intelligence would have over human thinkers as a natural consequence of its digital properties. Because many of these properties are so key to understanding the “cognitive horsepower” behind claims of “thinkism”, I’ve chosen to excerpt the entire section on “AI Advantages” here, minus references (you can find those in the paper):
Below we list a few AI advantages that may allow AIs to become not only vastly more intelligent than any human, but also more intelligent than all of biological humanity. Many of these are unique to machine intelligence, and that is why we focus on intelligence explosion from AI rather than from biological cognitive enhancement.
Increased computational resources. The human brain uses 85–100 billion neurons. This limit is imposed by evolution-produced constraints on brain volume and metabolism. In contrast, a machine intelligence could use scalable computational resources (imagine a “brain” the size of a warehouse). While algorithms would need to be changed in order to be usefully scaled up, one can perhaps get a rough feel for the potential impact here by noting that humans have about 3.5 times the brain size of chimps, and that brain size and IQ correlate positively in humans, with a correlation coefficient of about 0.35. One study suggested a similar correlation between brain size and cognitive ability in rats and mice.
Communication speed. Axons carry spike signals at 75 meters per second or less. That speed is a fixed consequence of our physiology. In contrast, software minds could be ported to faster hardware, and could therefore process information more rapidly. (Of course, this also depends on the efficiency of the algorithms in use; faster hardware compensates for less efficient software.)
Increased serial depth. Due to neurons’ slow firing speed, the human brain relies on massive parallelization and is incapable of rapidly performing any computation that requires more than about 100 sequential operations. Perhaps there are cognitive tasks that could be performed more efficiently and precisely if the brain’s ability to support parallelizable pattern-matching algorithms were supplemented by support for longer sequential processes. In fact, there are many known algorithms for which the best parallel version uses far more computational resources than the best serial algorithm, due to the overhead of parallelization.
Duplicability. Our research colleague Steve Rayhawk likes to describe AI as “instant intelligence; just add hardware!” What Rayhawk means is that, while it will require extensive research to design the first AI, creating additional AIs is just a matter of copying software. The population of digital minds can thus expand to fill the available hardware base, perhaps rapidly surpassing the population of biological minds. Duplicability also allows the AI population to rapidly become dominated by newly built AIs, with new skills. Since an AI’s skills are stored digitally, its exact current state can be copied, including memories and acquired skills—similar to how a “system state” can be copied by hardware emulation programs or system backup programs. A human who undergoes education increases only his or her own performance, but an AI that becomes 10% better at earning money (per dollar of rentable hardware) than other AIs can be used to replace the others across the hardware base—making each copy 10% more efficient.
Editability. Digitality opens up more parameters for controlled variation than is possible with humans. We can put humans through job-training programs, but we can’t perform precise, replicable neurosurgeries on them. Digital workers would be more editable than human workers are. Consider first the possibilities from whole brain emulation. We know that transcranial magnetic stimulation (TMS) applied to one part of the prefrontal cortex can improve working memory. Since TMS works by temporarily decreasing or increasing the excitability of populations of neurons, it seems plausible that decreasing or increasing the “excitability” parameter of certain populations of (virtual) neurons in a digital mind would improve performance. We could also experimentally modify dozens of other whole brain emulation parameters, such as simulated glucose levels, undifferentiated (virtual) stem cells grafted onto particular brain modules such as the motor cortex, and rapid connections across different parts of the brain. Secondly, a modular, transparent AI could be even more directly editable than a whole brain emulation—possibly via its source code. (Of course, such possibilities raise ethical concerns.)
Goal coordination. Let us call a set of AI copies or near-copies a “copy clan.” Given shared goals, a copy clan would not face certain goal coordination problems that limit human effectiveness. A human cannot use a hundredfold salary increase to purchase a hundredfold increase in productive hours per day. But a copy clan, if its tasks are parallelizable, could do just that. Any gains made by such a copy clan, or by a human or human organization controlling that clan, could potentially be invested in further AI development, allowing initial advantages to compound.
Improved rationality. Some economists model humans as Homo economicus: self-interested rational agents who do what they believe will maximize the fulfillment of their goals. On the basis of behavioral studies, though, Schneider (2010) points out that we are more akin to Homer Simpson: we are irrational beings that lack consistent, stable goals. But imagine if you were an instance of Homo economicus. You could stay on a diet, spend the optimal amount of time learning which activities will achieve your goals, and then follow through on an optimal plan, no matter how tedious it was to execute. Machine intelligences of many types could be written to be vastly more rational than humans, and thereby accrue the benefits of rational thought and action. The rational agent model (using Bayesian probability theory and expected utility theory) is a mature paradigm in current AI design.
It seems likely to me that Kevin Kelly does not really understand the AI advantages of increased computational resources, communication speed, increased serial depth, duplicability, editability, goal coordination, and improved rationality, and how these abilities could be used to accelerate, miniaturize, parallelize, and prioritize experimentation to such a degree that the “calendar time” limitation could be surpassed. The calendar of a powerful AI superintelligence might be measured in microseconds rather than months. Different categories of beings have different calendars to which they are most accustomed. In the time it takes for a single human neuron to fire, a superintelligence might have decades of subjective time to contemplate the mysteries of the universe.
Part of the initial insight that prompted the perspective that Kelly calls “thinkism” was that the brain is a machine which can be accelerated by porting the crucial algorithms on a different substrate, namely a computer, and running them faster. The brain works through algorithms — that is, systematic procedures. For an example, take the visual cortex, the part of the brain that processes what you see. This region of the brain is actually relatively well understood. The first layers capture surface features such as lines, darkness, and light. Deeper layers make out shapes, then motion, then specifics such as which face belongs to which person. It gets so specific that scientists have measured individual neurons that recognize celebrities like Bill Clinton or Marilyn Monroe.
The algorithms that underlie our processing of visual information are understood on a basic level, and it is only a matter of time until all the other cognitive algorithms are understood as well. When they are, they will be implemented on computers and sped up by a factor of thousands or millions. Human neurons fire 200 times every second, computer chips fire 2,000,000,000 times every second.
What would it be like to be a mind running at ten million times human speed? If your mind is really really fast, events on the outside would seem really really slow. All the elapsed time from the founding of Rome to the present day could be experienced subjectively in about two hours. All the time from the emergence of Homo sapiens to the present day could experienced in a week. All the time from the dinosaurs to the present day could be experienced in a mere 2,400 years. Imagine how quickly a mind could accrue profound wisdom running at such an accelerated speed; the “wisdom” of a 90-year old would seem childlike by comparison.
To visualize concretely the kind of arrangement in which these minds could exist, imagine a computer a couple hundred feet across made of dense nanomachinery situated at the bottom of the ocean. Such a computer would have far more computing power than the entire planet today, similar to the way that a modern smartphone has more computing power than the entire world in 1960. Within this computer would exist virtual worlds practically without end; their combined volume far exceeding that of the solar system, or perhaps even the galaxy.
In his post, Kelly seems to acknowledge that minds could be vastly accelerated and magnified in this way: remarkably, he just doesn’t think that this would translate to increased wisdom, performance, ability, or insight significantly beyond the human level. To me, at first impression, the notion that a ten million times speedup would have a negligible effect on scientific innovation or progress seems absurd. It appears obvious that it would have a world-transforming impact. Let’s look at the argument more closely.
The disagreement between Singularitarians such as Vinge and Kurzweil and skeptics such as Kelly seems to be about what sorts of information-acquisition and generation procedures can be imported into this vastly accelerated world and which cannot. In his hard sci-fi book Diaspora, author Greg Egan calls the worlds of these enormously accelerated minds “polises”, which make up the vast bulk of humanity in 2975. Vinge and Kurzweil see the process of knowledge acquisition and creation as being something that can in principle be sped up, brought “within the purview of the polis”, whereas Kelly does not.
Above, I argued how the benefits of experimentation can be accelerated through the processes of running experiments faster, parallelizing them, using less matter, and choosing the right experiments. But what about less controversial information flow from world to polis? To build the polis to begin with, you’d have to be able to emulate — not just simulate — the human mind in detail, that is, copy all of its relevant properties. Since the human brain is one of, if not the most complex object in the universe that we know of, this also implies that a vast variety of less complex objects could be scanned and inputted to the polis in a similar fashion. Trees, for instance, could be mass-inputted into the virtual environment of the polis, consuming thousands or millions of times less computing power than the sentient inhabitants. It goes without saying that nonbiological, inanimate background features such as landscapes could be input into the polis with a bare minimum of challenge or difficulty.
Once a process can be simulated with a reasonable level of computing power, it can be inputted into the polis and run at a factor of tens-of-millions speedup. Newtonian physics, for instance. Today, we use huge computers to perform molecular dynamics simulations on aggregates of a few hundred atoms, simulating a few microseconds of their activity. With futuristic nanocomputers built by superintelligent Artificial Intelligences, macro-scale systems could be simulated for hours of activity for a very affordable cost in computing power. Such simulations would allow these intelligences to extract predictive regularities, or “rules of thumb” which would allow them to avoid simulating these systems in such excruciating detail in the future. Instead of requiring full-resolution molecular dynamics simulations to extrapolate the behavior of large systems, they might resolve a set of several thousand generalities that allow these systems to be predicted and understood with a high degree of confidence. This has essentially been the process of science for hundreds of years, but the “simulations” are instead direct observations. With enough computing power, fast simulations can be “similar enough” to real-life situations that genuine wisdom and insight can be derived from them.
Though real, physical experimentation will be needed to verify the performance of models, those facets of the models that are verified will be quickly internalized by the polis, allowing it to simulate real-world phenomena at millions of times the real-world speed. Once a facet of a real-world system is internalized, understanding it instantly becomes a matter of routine, just as today the design of a massive bridge has become a matter of routine, a factor of running calculations based on the known laws of physics. Though from our current perspective, the complexities of the world of biology seem intimidating, the capability of superintelligences to quickly conduct millions of experiments in parallel and internalize knowledge once it is acquired will quickly dissolve these challenges as our recent ancestors dissolved the challenge of precision engineering.
I have only scratched the surface of the reasons why innovation and progress by superintelligences will predictably surpass the “calendar time” with which humanity has grown so accustomed. As humans routinely perform cognitive feats that bewilder the brightest squirrel or meadow vole, superintelligent scientists and engineers will leave human scientists and engineers in the dust, as if our all prior accomplishments were scarcely worth mentioning. It may be psychologically challenging to come to terms with such a possibility, but it would really just be the latest in an ongoing trend of human vanity being upset by the realities of a godless cosmos.
The Singularity is something that our generation needs to worry about — in fact, it may be the most important task we face. If we are going to create higher intelligence, we want it on our side. The benefits of success would be beyond our capacity to imagine, and will likely include the end of scarcity, war, disease, and suffering of all kinds, and the opening up of a whole new cognitive and experiential universe. The challenge is an intimidating one, but one that our best will rise to meet.
