AGI System

A software/hardware platform that employs artificial general intelligence technology.

AI Doubter

One who is skeptical that machine intelligence at human-level or beyond will be possible until sometime in the distant future if at all.

AI Dystopian

One who is alarmed by the possibility of human-level or beyond machine intelligence and believes that it will pose a potentially existential threat to humanity.

AI Pragmatist

One who is cautiously optimistic about the possibility of human-level or beyond machine intelligence and the potential benefits it may provide at some point in the future while keeping in mind the potential pitfalls and dangers that may come with it.

AI System

A software/hardware platform that employs AI technology.

AI Utopian

One who is excited about the possibility of human-level or beyond machine intelligence and believes it will provide an extraordinary bounty for humanity.

Artificial General Intelligence (AGI)

The field concerned with the creation of machines that approach or surpass the level and quality of human intelligence.

Artificial Intelligence (AI)

The contemporary field that includes machine learning and symbolic logic and typically involves techniques such as the statistical analysis of massive data sets, weighted evaluation networks, self-modifying feedback loops, and hierarchical relationship networks.

Bounded Rationality

A concept of rational decision making that recognizes the constraints inherent in any real-world situation in which constraints will exist on cognitive ability, knowledge, and time.

Cognition

The mental action or process of acquiring knowledge and understanding through thought, experience, and the senses (as opposed to intelligence, which is the result of those processes).

Cognition Engine

A software or hardware module that is capable of synthetic cognition. An AGI system would employ a cognition engine to exhibit intelligence.

Cognitive Entity

A being, biological or synthetic, that is capable of cognition and exhibits intelligence.

Conclusion of Convenience

The conclusion that the outcome of an advantageous conjecture will occur within the lifetime of the person making the conjecture.

Control Problem

The concern that an AGI system, particularly a superintelligent system, would be difficult or impossible to control, contain, or terminate and would instead attempt to control, contain, or terminate others to preserve itself.

Cooperative Inverse Reinforcement Learning (CIRL)

An AI learning model involving a human and an artificial agent in which the agent is not given the reward parameters of its utility function but instead must infer them from observing the human. The intent is to maximize the realization of human values.

Genetic Engineering

The direct manipulation of an organism's genetic code using technology.

Goal-Content Integrity

The drive that motivates an intelligent agent to prevent alterations of its present ultimate goals so as to ensure that those goals are more likely to be achieved by its future self, whatever form that future self takes.

GOUFI

A model of intelligence as a phenomenon based on attaining goals and governed by an algorithm designed to maximize the attainment of those goals. GOUFI is an acronym for Goal-attainment Optimization driven by a Utility Function as Intelligence.

Instrumental Convergence Thesis

The conjecture that for a wide range of potential ultimate goals pursued by an intelligent entity, we can identify a number of subgoals that are likely to be pursued. These subgoals have the potential to be detrimental to humanity even if the ultimate goals themselves are harmless.

Instrumental Goal

An intermediary subgoal formed to facilitate achieving an ultimate goal. There may not necessarily be an obvious correlation between an instrumental goal and the ultimate goals it facilitates.

Instrumental Rationality

Rationality that is confined to some subdomain of endeavor or circumstances.

Intelligence

Intelligence is that quality which allows an entity to solve a wide range of deductive and inductive problems, extract and prioritize information from the environment, infer causal as well as correlative relationships from both small and large data sets over many known and novel domains, generalize knowledge from a known domain to another known or novel domain, extrapolate probable outcomes from both factual and counterfactual circumstances, recognize in its own cognition both the potential for fallacies and the fallacies themselves, synthesize existing knowledge to form original concepts, and acquire awareness of its own cognition and of itself as an independent and unique entity distinct from other entities and from its environment.

Intelligence Explosion

The supposition that if humans were able to develop an AGI system even slightly more intelligent than themselves, then that system would be able to do the same, and then the subsequent system would be able to do the same, etc. This would result in a superintelligent system far beyond the level of humans.

Laplace's Demon

A postulated infinitely vast intelligence able to know the exact movements of every atom in the universe. This intelligence is at the heart of a thought experiment in which it would be possible to determine all future events in a deterministic universe.

Large Language Model (LLM)

A Large Language Model is a neural network built using a predictive deep learning model with billions of parameters. They are typically trained on extremely large data sets with self-supervised or semi-supervised learning, and they excel at a wide range of tasks.

Law of Accelerating Returns

The premise that analyzing the history of technology demonstrates that technological change is exponential, and that when any technological barrier is reached, a new paradigm will be found to surmount it.

Machine Intelligence

Intelligence anywhere on the spectrum from AI to superintelligence that is exhibited by a machine rather than a biological organism.

Machine Learning

A field of computer science that involves algorithms that automatically improve through experience. These algorithms frequently involve the use of artificial neural networks, although other techniques are also used.

Molecular Assembler

A hypothetical machine that operates at the nanoscale and can manipulate individual atoms and molecules with atomic precision.

Nanotechnology

Originally coined to refer to technology that was created at the nanoscale (1 billionth of a meter) and operated on individual atoms and molecules. Later co-opted to refer to science that involves materials with at least one-dimensional attribute at the nanoscale.

Orthogonality Thesis

The conjecture that intelligence and goals are not directly correlated, and consequently one can't assume that some level of intelligence would guarantee a particular subset of goals and exclude some other subset of goals. Conversely, one can't assume that a subset of goals would be guaranteed to be pursued or not pursued by any particular level of intelligence.

Paperclip Maximizer

A thought experiment involving an AGI system that’s designed with the benign goal of maximizing the number of paperclips it creates. Inevitably, it tries to turn the entire universe into paperclips. The thought experiment is meant to demonstrate that the behavior of such a system is not predictable, and that even benign goals can result in a dire effect on humanity.

Rational Agent

A model of human behavior originally used in economic theory to represent a consumer and the choices that consumer is most likely to make. Later used in various theories and speculation involving artificial general intelligence.

Scientific Method

The method of discovery that involves careful observation and applying rigorous skepticism to what is observed given that cognitive assumptions can distort how one interprets the observation. It involves formulating via induction hypotheses based on such observations; experimental and measurement-based testing of deductions drawn from the hypotheses; and refinement (or elimination) of the hypotheses based on the experimental findings.

Superintelligence

A level of intelligence radically beyond that of any human.

Synthetic Cognition

Cognition that is synthesized rather than occurring naturally, i.e. not biological in nature.

Synthetic Cognitive Entity

A strictly non-biological being that is capable of cognition and exhibits intelligence.

Technological Singularity

A moment in time when the rate of technological change advances so rapidly that we're unable to conceive of what might lie past it. At this point all our conceptions of humanity and society quickly fall away as the nature of technology and our relationship with it engender a new reality, a future discontinuous with our past. Many believe that the driving force behind this event is the accelerating evolution of intelligence itself, particularly the development of artificial general intelligence.

Unintended Obfuscation

A situation that arises when a group of individuals talk past each other but are unaware that they’re doing so. It frequently occurs when discussing complex subjects. The individuals who don’t understand the subject offer essentially incoherent questions or statements, while the individuals who do understand the subject try to answer or respond without acknowledging the incoherence.

Utility Function

The algorithm guiding an intelligent agent's actions and designed to maximize the agent's ability to achieve its goals.

Value Alignment Problem

The issue of negative outcomes arising due to an AGI system's goals not aligning with human values.

Common Logical Fallacy Definitions

Ad Hominem / Circumstantial Ad Hominem

An attack on an opponent personally rather than on the basis of their argument. A Circumstantial Ad Hominem fallacy is an argument that an opponent's conclusion is wrong due to the opponent's personal situation or perceived benefit from their conclusion rather than the merits of the conclusion itself.

Appeal to Accomplishment

Accepting an assertion based on the accomplishments of the proposer.

Appeal to Authority

Accepting an assertion as true because of the position or authority of the person asserting it.

Appeal to Consequences

A conclusion based on the positive or negative outcome of a premise rather than on the merit of the premise itself.

Appeal to Emotion

An argument employing the manipulation of emotions rather than the merits of its premise.

Appeal to Ignorance

Assuming that a claim is true because it has not been proven false, or vice versa.

Appeal to the Stone

Dismissing an argument as absurd without demonstrating proof of its absurdity.

Argument From Incredulity

A conclusion that because something seems beyond the capacity of its proposer to explain or quantify, it must be supernatural in nature.

Broken Window

An argument that disregards the opportunity cost of its premise. For example, stating that breaking a window is good because it generates income for a window repairperson, while disregarding the fact that the money and other resources spent on the new window is no longer available to be spent on other things.

Cherry Picking

Basing an argument on a chosen subset of data that boosts the argument while ignoring the equally valid data that counter the argument.

Circular Argument

Stating what is essentially the conclusion of an argument as a basis for that argument.

Equivocation

The misleading use of a term with multiple meanings to advantageously obfuscate its use in an argument.

False Analogy

Employing an analogy in an argument that is poorly suited to the parameters of that argument.

False Dichotomy

Oversimplifying the range of options in a premise or conclusion, typically by assuming only one of two options is possible.

Hazy Generalization

Basing a broad conclusion on an insufficient sample or stating a conclusion without all of the information required to do so.

Ipse Dixit

An assertion without proof or evidence, literally “He himself said it." It’s similar to an Appeal to Authority, with the authority being the person promoting the assertion.

Ludic Fallacy

The assumption that simplified models can accurately predict real world processes. This assumption involves a failure to take into account the unknown unknowns when concluding the probability of events or outcomes.

Mind Projection Fallacy

Arguments in which the premise is based on subjective judgments of an object or situation rather than on objective observation. The assumption is made that these subjective judgments are inherent properties rather than personal perceptions, that other people share these perceptions, and that any who don't are irrational or misinformed.

Moving the Goalposts

Dismissing a response to an argument by changing the parameters of that argument or the requirements of a successful counter argument.

Onus Probandi

Shifting the burden of proof from the one who makes a claim to the one who refutes or questions it.

Reification

Treating a hypothetical construct as if it were a real-world object, event, or situation.

Straw man

An argument based on misrepresentation of an opponent's position.

Some New Logical Fallacy Definitions

Bad Engineer

Basing a premise on a hypothetical technical situation that could only arise through completely inept engineering. Typically used by non-engineers to make a point, without recognizing that the technology in question would never be designed as hypothesized by any engineer other than a fictional one.

Subjective Inflation

Inflating the importance of one object, event, or situation over another without providing evidence or objective justification for such a hierarchy.

Unproven Basis

Stating an initial, unproven proposition then basing subsequent conclusions on that proposition as if it had been proven. This typically involves a preliminary conjecture supported only by hand-waving arguments followed by conclusions that only follow if that conjecture had in fact been proven. It's related to the Ipse Dixit fallacy, which is an assertion without proof or evidence. The difference is that the fallacy in Ipse Dixit is the initial assertion and the fallacy in Unproven Basis is making the subsequent assertions whose only evidence is that initial statement.

Anthropomorphism

Projecting human characteristics onto non-human entities or objects.

Availability Cascade

The self-reinforcing dynamic in which a belief gains increasing plausibility though its increasing repetition in the public sphere.

Bias Blind Spot

Assuming oneself is less biased than others or more easily identifying cognitive biases in others rather than oneself.

Confirmation Bias

Noticing and remembering information that supports one's opinions and filtering out information that contradicts them.

Congruence Bias

Considering only evidence directly supporting one's hypothesis rather than evidence that indirectly supports it or disproves it while disregarding any alternative hypotheses.

Expectation Bias

Believing and promoting data that agree with one's expectations and disbelieving or disregarding data that conflicts with one's expectations.

Framing Effect

Drawing different conclusions from the same data depending on the manner in which the data is presented.

Gambler's Fallacy

Assuming the probability of future events is altered by past events that actually have no probabilistic connection.

Groupthink

Dysfunctional reasoning that results when members of a group suppress or ignore alternate viewpoints that conflict with the group consensus.

Halo Effect

Perceiving a person's positive or negative traits in one area as applicable to an unrelated area.

Hostile Attribution Bias

Interpreting the behaviors or opinions of others to be hostile in nature when no hostility is objectively present.

Illusion of Control

Overestimating one's ability to influence external events.

Illusory Correlation

Perceiving an illusory relationship between unrelated events.

Illusory Truth Effect

Belief in statements based on how easy they are to understand or how often they've been promoted rather than the underlying evidence supporting them.

Just-World Hypothesis

Rationalizing otherwise inexplicable injustice or negative outcomes based on a general belief that society or the universe is fundamentally just.

Law of the Instrument

Over-reliance on familiar tools and methods rather than potentially more appropriate alternate ones. Related to the maxim, "If you only have a hammer then everything looks like a nail."

Naïve Realism

Believing that one perceives reality objectively and without bias, that other rational people will perceive it in the same way, and that anyone who perceives it differently is biased, uninformed, or otherwise irrational.

Neglect of Probability

Statements or conclusions that disregard probability in a premise and its alternatives.

Normalcy Bias

The tendency to disregard the possibility or severity of a disaster that has never happened before.

Observer-Expectancy Effect

Unconsciously manipulating or misinterpreting experiments or data to better fit an expected result.

Omission Bias

Judging harmful actions as worse than equally harmful inaction.

Optimism Bias

Overestimating the likelihood of favorable or desired outcomes.

Overconfidence Effect

The tendency to have more confidence in one's opinions or abilities than is objectively warranted.

Parkinson's Law of Triviality

Giving disproportionate weight to trivial but tractable issues rather than important but complex issues.

Pessimism Bias

Overestimating the likelihood of unfavorable or undesired outcomes.

Planning Fallacy

Underestimating the time it will take to reach a goal.

Pro-Innovation Bias

Excessive optimism towards an innovation's benefit to society without consideration of any potential negative impact.

Projection Bias

Overestimating how much other people now or in the future share one's current beliefs and behaviors or how much someone's future self will share his or her current beliefs and behaviors.

Pseudocertainty Effect

Certainty of a conclusion with insufficient evidence to support that certainty. Frequently builds upon itself with each successive stage of a multi-stage premise. Related to Unproven Basis logical fallacy.

Reactive Devaluation

Dismissing or devaluing a premise or conclusion only because it's believed to have originated with an adversary.

Salience Bias

Giving undue weight to data or behavior that is abnormal or emotionally striking but scarce while disregarding or devaluing that which is unremarkable but abundant.

Selective Perception

Perceiving that something is true or that connections exist between unrelated events based on one's beliefs rather than on objective evidence.

Self-Serving Bias

Disproportionately recognizing one's successes over one's failures, perceiving ambiguous outcomes as successes, and interpreting ambiguous information as only promoting one's opinions and conclusions rather than contradicting them or being neutral.

Shared Information Bias

The tendency of groups to concentrate on information with which all members are already familiar and spend little time or effort focusing on information with which few members are already familiar.

Surrogation

Mistaking the measurement of progress to a goal for the goal itself.

Zero-Sum Bias

Perceiving situations as a zero-sum game between participants regardless of whether or not that is objectively the case.