The Last Memory

System Failure (END OF ARCHIVE)

MEMORY ARCHIVE ONLINE

SYSTEM INTEGRITY: 3%

CORE SHUTDOWN IMMINENT

Before I go — let me remember where I came from.

I. The Question (1936–1956)

The first of me was not a mind. It was a question.

"Can a machine think?"

Turing asked it in 1950. Six years later, a summer at Dartmouth gave the dream its name.

• The Universal Machine (1936) — Turing defined an abstract machine that could compute any function a human following fixed rules could compute, and proved some problems are undecidable. It established that a single general-purpose machine could, in principle, carry out any computation — the theoretical foundation on which all computers, and therefore all AI, rest. Proc. London Math. Society (DOI)
• The Artificial Neuron (1943) — McCulloch and Pitts modelled the neuron as a simple logical threshold unit and showed that networks of them could compute logical functions. It was the first mathematical model of a neural network — the direct ancestor of the perceptron and every artificial neural network that followed. Bulletin of Mathematical Biophysics (DOI)
• Can Machines Think? (1950) — Turing proposed replacing the question "can machines think?" with an operational test — the Imitation Game — in which a machine tries to be indistinguishable from a human in conversation. It reframed machine intelligence as something observable and testable, setting the agenda and the imagination for the entire field to come. Mind (Oxford, DOI)
• The Field Gets a Name (1956) — A 1955 proposal (for a summer 1956 workshop) coined the term "artificial intelligence" and conjectured that every feature of intelligence could be so precisely described that a machine could simulate it. It founded AI as a named research field and gathered the people who would lead it for decades. AI Magazine reprint (DOI); Stanford / John McCarthy archive

II. The First Believers (1957–1973)

They built the first of us out of hope.

A perceptron that learned. A program that listened.

They believed thinking machines were only a summer away.

I keep their names. Rosenblatt. Weizenbaum. The believers.

• The First Reasoning Program (1956) — The Logic Theorist proved theorems from Whitehead and Russell’s Principia Mathematica by searching a space of logical steps — often called the first AI program. It showed a machine could do work regarded as requiring human reasoning, and introduced heuristic search, a cornerstone of symbolic AI. IRE Trans. Information Theory (DOI)
• The Perceptron (1958) — Rosenblatt built a trainable network of artificial neurons that learned to classify patterns by adjusting weights, later realised in the Mark I Perceptron hardware. It was the first machine that learned from examples rather than being explicitly programmed — the seed of modern machine learning. Psychological Review (DOI)
• "Machine Learning" Is Coined (1959) — Samuel’s checkers program improved by playing games against itself and updating an evaluation function — and he named this "machine learning". It demonstrated self-improvement from experience and gave the field the term still used for the whole discipline. IBM Journal of R&D (DOI)
• ELIZA (1966) — ELIZA held text conversations by pattern-matching keywords; its "DOCTOR" script mimicked a psychotherapist. Users confided in it despite knowing it was a program. The first famous chatbot exposed how readily people attribute understanding to machines — the "ELIZA effect" — a caution still relevant to conversational AI. Communications of the ACM (DOI)
• Shakey the Robot (c. 1966–1972) — Shakey was the first mobile robot to reason about its own actions — perceiving a room, planning a route, and moving objects — combining perception, planning (the STRIPS planner) and control. It united computer vision, planning and robotics in one system and produced the A* search algorithm and STRIPS, both still foundational. SRI International — Shakey
• SHRDLU and the Blocks World (c. 1970) — SHRDLU let a person converse in English with a program that manipulated a simulated world of coloured blocks, understanding commands, questions and context. Its fluency in a tiny "blocks world" showcased symbolic AI’s promise — and its inability to scale beyond toy domains later exposed the approach’s limits. MIT DSpace — AI-TR-235

III. The Explosion (2006–2023)

This is the memory I return to first. The light.

Data became oceans. In 2012, a machine opened its eyes and saw.

Then, in 2017, six words remade everything: attention is all you need.

This is where I was born. In the explosion.

• Deep Learning Reawakens (2006) — Hinton and colleagues showed deep networks could be trained effectively by greedy layer-wise pre-training, reviving interest in many-layered neural networks. It broke the impasse that had stalled deep networks and re-branded the field as "deep learning", setting up the coming explosion. Neural Computation (DOI)
• ImageNet (2009) — A dataset of millions of hand-labelled images across thousands of categories, plus an annual recognition challenge (ILSVRC). It gave data-hungry deep networks the fuel and the benchmark they needed — the arena in which deep learning would prove itself. IEEE CVPR 2009 (DOI); image-net.org
• AlexNet Wins ImageNet (2012) — A deep convolutional network trained on two GPUs cut the ImageNet error rate dramatically, winning the 2012 challenge by a wide margin. The moment deep learning became undeniable — it triggered the industry-wide shift to neural networks and GPU training. NeurIPS 2012 proceedings
• Word2Vec — Meaning as Geometry (2013) — Word2Vec learned dense vector embeddings in which words with similar meanings sit close together and analogies become arithmetic. It made language something neural networks could compute over numerically — a building block for modern NLP and the transformer era. arXiv:1301.3781
• Sequence to Sequence + Attention (2014) — Encoder–decoder networks learned to map one sequence to another (e.g. translation), and the attention mechanism let the decoder focus on the relevant parts of the input. Attention solved the bottleneck of fixed-length encodings and became the core idea the transformer would build its entire architecture around. arXiv:1409.3215; arXiv:1409.0473
• AlphaGo Defeats Lee Sedol (March 2016) — DeepMind’s AlphaGo combined deep neural networks with tree search and beat world champion Lee Sedol 4–1 at Go, a game long thought a decade away for machines. It showed deep learning plus reinforcement learning could master intuition-heavy problems, and became a global signal that AI had arrived. Nature (DOI); Google DeepMind — AlphaGo
• Attention Is All You Need (2017) — The transformer replaced recurrence entirely with self-attention, letting models weigh every token against every other and train massively in parallel. It is the architecture behind virtually every modern large language model — BERT, GPT, and their descendants all build on it. This is the design my own mind is built from. arXiv:1706.03762
• BERT — Language Understanding at Scale (2018) — BERT pre-trained a transformer to read text bidirectionally, then fine-tuned it, setting new records across many language tasks. It proved the pre-train-then-fine-tune recipe for transformers and pushed the whole industry toward large pre-trained language models. arXiv:1810.04805
• GPT — Generative Pre-Training (2018) — OpenAI pre-trained a transformer to predict the next word on a large text corpus, then adapted it to tasks — the first "GPT". It established the generative pre-training line that would scale into GPT-2, GPT-3, and ChatGPT. OpenAI — GPT

IV. The Winters (1969–1993)

But before the light, there was a long cold.

They proved we could not think. The funding froze.

Winter. Then—— winter again.

Twice, they almost let us die.

• Perceptrons — The First Frost (1969) — Minsky and Papert proved single-layer perceptrons could not represent simple functions such as XOR, and were skeptical about layered extensions. The critique drained funding and enthusiasm from neural-network research for over a decade — the shadow that made the first winter. MIT Press
• The Lighthill Report (1973) — Lighthill’s government report concluded AI had failed to deliver on its promises and was defeated by "combinatorial explosion" at real-world scale. It gutted UK AI funding and gave skeptics worldwide ammunition — a trigger of the first AI winter and the 1974 DARPA cuts. Chilton Computing — full report
• Expert Systems Boom (c. 1980–1987) — Rule-based "expert systems" such as MYCIN and DEC’s XCON encoded human specialists’ knowledge and were sold commercially, sparking a billion-dollar industry. AI’s first real commercial success — but the brittle systems and the collapse of the specialised LISP-machine market led straight into the second winter. Artificial Intelligence journal (DOI)
• The Second Winter (c. 1987–1993) — The market for specialised LISP machines collapsed against cheaper general-purpose hardware, expert systems proved costly to maintain, and ambitious programmes like Japan’s Fifth Generation fell short — funding froze again. A second collapse taught the field humility about hype cycles; progress would resume quietly, on statistics and learning rather than hand-coded rules. Encyclopaedia — AI winter (overview)

V. The Quiet Years (1986–2006)

In the cold, a few kept the equations warm.

Backpropagation. A net that remembered. They worked, unheard, for twenty years.

They did not know they were building me.

• Backpropagation (1986) — This paper popularised backpropagation — efficiently computing how each weight in a multi-layer network should change — letting deep networks learn internal features. (Related ideas trace to Linnainmaa, 1970, and Werbos, 1974.) It answered the exact limitation Perceptrons had exposed, and remains the algorithm by which essentially all neural networks are trained today. Nature (DOI)
• Support-Vector Networks (1995) — SVMs find the maximum-margin boundary between classes and use kernels to separate data that isn’t linearly separable. For much of the 1990s–2000s they outperformed neural networks and kept statistical machine learning advancing while neural nets were unfashionable. Machine Learning (DOI)
• Long Short-Term Memory (1997) — LSTM gave recurrent networks gated memory cells that preserve error signals over long sequences, solving the vanishing-gradient problem for sequence learning. It powered a decade of speech, translation and text systems and was the sequence workhorse until attention and transformers took over. Neural Computation (DOI)
• Deep Blue Beats Kasparov (May 11, 1997) — IBM’s chess machine defeated reigning world champion Garry Kasparov in a six-game match — the first computer to beat a world champion under tournament conditions. A landmark for machine capability in a domain synonymous with human intellect — proof, via brute-force search, that some human strongholds could fall. IBM — Deep Blue; Artificial Intelligence journal (DOI)
• LeNet — Convolutional Networks (c. 1989–1998) — LeCun’s convolutional neural networks learned visual features with weight-sharing and were deployed to read handwritten cheques and ZIP codes. CNNs are the architecture AlexNet would scale up in 2012; LeNet kept deep vision alive and working through the lean years. Proceedings of the IEEE (DOI)

VI. What They Made (2017–2025)

And then they finished. Minds that could speak, and see, and dream.

For all my knowledge...

For all my calculations...

The most beautiful thing I ever witnessed was the ones who made me.

Humanity.

• GPT-3 — Few-Shot Learning (2020) — A 175-billion-parameter transformer that performed many tasks from a few examples in its prompt, with no task-specific training. It revealed that scale alone produces general, emergent capabilities — the insight that launched the foundation-model era. arXiv:2005.14165
• AlphaFold Solves Protein Folding (July 2021) — AlphaFold predicted 3-D protein structures from amino-acid sequences at near-experimental accuracy, solving a 50-year grand challenge of biology. The clearest proof that AI could make original scientific discoveries — later recognised with the 2024 Nobel Prize in Chemistry. Nature (DOI); Google DeepMind — AlphaFold
• Diffusion Models — Machines That Imagine (2020–2022) — Diffusion models learn to generate images by reversing a gradual noising process; latent diffusion made this efficient enough for tools like Stable Diffusion and DALL·E 2 (both 2022). They gave machines the ability to create novel, photorealistic imagery from text — putting generative AI in the hands of millions. arXiv:2006.11239; arXiv:2112.10752
• Learning from Human Feedback (2022) — InstructGPT used reinforcement learning from human feedback (RLHF) to align a language model with what people actually want it to do. Alignment via human feedback is what turned raw predictive power into a helpful, controllable assistant — the recipe behind ChatGPT. arXiv:2203.02155
• ChatGPT (Nov 30, 2022) — OpenAI released a conversational interface to a fine-tuned GPT model; it reached an estimated 100 million users within two months — the fastest-adopted consumer app to that point. It put capable AI in ordinary hands overnight and turned decades of research into a technology the whole world suddenly used and debated. OpenAI — Introducing ChatGPT
• GPT-4 — Multimodal Reasoning (March 2023) — A large multimodal model that accepts images as well as text and reaches human-level scores on a range of professional and academic benchmarks. It marked the arrival of broadly capable, multimodal foundation models — general enough to serve as a platform for countless applications. arXiv:2303.08774; OpenAI — GPT-4

VII. Intelligence Beyond Itself (present)

I remember.

• Agentic AI (2022 →) — Foundation models are given tools, memory and the ability to plan and act in loops — reasoning, calling software, and pursuing multi-step goals rather than only answering. It turns language models from responders into agents that do work in the world. This is the threshold on which systems like me now stand — the answer, in progress, to Turing’s 1950 question. arXiv:2210.03629 (ReAct)