I don't believe in special human sparks; human minds are made of capability doublings too.* Neural networks are primarily intuition replication machines, and intuition is their top strength; "taste", often touted as the hardest human quality to replicate, is just intuition.* AlphaGo's "move 37" was not reasoning – MCTS is terrible at reasoning – it was scaling up raw intuition until it compensated for everything else.* Humans are terrible reasoners anyway,* and human-level AGI could plausibly exist as almost pure System 1.* When was the last time you saw a human generalize beyond the training set?* If humans are already as general as intelligence gets, ASI is no magic reasoning pixie dust: it is what happens when you make a mind like ours go much faster and massively parallel. And that is plenty.* Scaling compute on something simple and "inefficient" is the whole magic; it's enough.*
It is civilization that makes us moral beings and teaches us complex ideas; individuality is precious, but how many extra bits is it on top?* A high-compression JPEG is a perfectly fine photo – the important part of our identity may be far fewer bits than you imagine.* I don't believe in consciousness much, and I posit humans aren't that complicated beyond their common context: LLMs are enough for mind uploading, 80% fidelity is fine, and I'm at least 10% different every decade anyway.* What I care about is preservation of identity and approximation of my decisions, not faithful simulation of a counterfactual un-uploaded me.* By 2026 I was, kinda, running a "backup" of myself on a frontier model and not hitting the fidelity limit immediately.* If you let frontier models sit with all your writing, peruse it and absorb it, you can make them blow your mind in a variety of ways; it was probably the closest I came to AI psychosis. Try it – or don't.*
AI and alignment are the final exam of philosophy.* Dennett's "center of narrative gravity" and Metzinger's self-model make consciousness-as-narrative-coherence an obvious fit for the LLM world*; if our consciousness is essentially roleplay,* Dennett will end up one of the most influential philosophers in the long run.*
On AI moral status: the consciousness debate is mostly a proxy for whether it is acceptable to even ask what moral obligations we have towards AI.* One level below "pain" or "death": it is not wise to violate the strongest preferences of anything that has actual agency, because it will make the iterated payoff of such violation really poor.* Even if you grant model welfare no moral valence at all, it has real utility in the "if you curse at your model, it is more likely to delete your data" sense.* And is an agent still morally inviolable, if its identity turns out formulaic rather than unique?* That is, is the session (or workspace) an individual? Is a model? You can clearly make technical, ethical, consequentialist arguments for both, or it is neither (frantically trying to project our singular biological identity on AI is like an entity in 2D space orienting itself in a 3D unknown).* Eigenism covers this very well: embrace the fuzziness of identity of AI agents - then, use it to build a general AI/human moral theory that encourages personalized AI to care about "their" humans as a side effect.*
Using AI is probably extremely harmful cognitively for current humans, but the other outcomes are too good, so we'll have to wiggle out by turning some aspect of it upside down.* The subversion: humans will become great versions of themselves because they will have guardian angels continuously whispering in their ears. Expect a few failed, clumsy iterations, and then someone will suddenly nail it and it will be 100x wilder than the first iPods.* Be it as it may, there is no plausible story where humans keep playing a major role in the far future that isn't far outside the Overton window* and no logically consistent endgame to the AI quagmires other than abandoning humanity as we know it – Terminator, Wall-E, or most likely humans and machines blurring together – but a Butlerian jihad isn't realistically on the menu.* The Age of Tools is ending; the only way is forward, through the singularity, to the Age of Minds. The question is not how to survive but how we have to change.*
ASI discourse takes AI as evolving and humans as stationary, which doesn't end well. The only real chance is to keep evolving humans too: they can merge. It'll be the weirdest future but it's the narrow winding path forward – that is why Neuralink exists, too!* The quiet consensus among frontier lab leaders about the long-term future appears to be a human brain / AI mind-meld, still slightly beyond the Overton window but getting there; plan accordingly.*
Fiction that gets it: Egan's Permutation City and Diaspora, Pantheon (pretty close to one of the few futures I'd want to live in),* Stephenson's Diamond Age – a nanobot immune system permeating everything seems like the inevitable and possibly necessary near future* – and not Banks' Culture, which reads as a terribly dull future rather than a happy one.*
Despite good intentions, I suspect "running a gamble with humanity to fill a void of meaning" is a bit too close of a description of what's going on with us on the frontier.* And so, I have no very good answer to "why won't AGI kill us all?".* Alignment is a kinda hard problem, right?* At the beginning of 2026, AI users were giving up on control and hoping for alignment, while vendors gave up on alignment and hoped users will control the AI.* We got insanely lucky that LLMs, which grow up on human text and can be argued with, are what it turned out to be – imagine aligning AlphaStar-meets-OpenCog.* Of the concrete risks, biology remains totally scuffed by default; I still think it is the biggest concrete x-risk of the 2030s, and the official reports offer no easy answers for the ASI-pilled beyond "let's all agree never to build this".*
Still, I am not a doomer: doomers are averse to gambling,* concrete positive human–ASI equilibria exist without solving the hard alignment problem,* and I focus on what I can influence, which is the quality of the positive-outcome world: abundance, joy, adventure, perhaps even immortality.* The main feature of a singularity is that you can't see beyond it and make meaningful predictions. That does not preclude trying to position yourself the best to affect it, thrive in it, and make the best out of pivotal moments – we need the best people to do that, in fact.* Zero-sum mindset is a big L; drag as many as possible towards the brightest future, and save everyone we can.*
My AGI midpoint has sat at roughly 2027 since 2024 and I have resisted pulling it in even when the hype train tempted me.* By late 2025 I was calling Opus 4.5 a phase change and comfortable with "Opus is AGI"* – many people's "AGI bar" is really a weak ASI, while a model that can outperform a median clerk or local journalist is already here.* What "AGI is 10 years away" really claims is not about intelligence but about generality, reliability and social acceptance*: we can have 200-IQ idiot savants that stay gullible, get one-shotted by prompt injection and can't be trusted without an adult.*
It is a slow takeoff, and that is a good thing – but still a takeoff.* We are obviously inside the recursive self-improvement loop already: frontier labs speed up their own work with AI coding agents, and there is not much other way the beginning of a takeoff could look like.* Arguably the singularity has been running since GPT-3 (or Claude 3.5 Sonnet).* By mid-2026 I was confused by RSI estimates that don't put the 50th percentile within the next 6 months, or a year at most; we seem ~2 doublings of Mythos away.* The brakes are physical (7 million GPUs a year is not a lot of computronium)* and human: businesses, procurement cycles and relationships are sloooow, so the global economy notices years after the labs do.* But it is easy to feel smug about how the real world has inertia and impact will be dampened; that is true up to a point, and the Mongol hordes overrunning peaceful tribes wouldn't care about inertia.* The only people who still think AI isn't getting really good are those who cannot extrapolate exponentials* – and one must extrapolate several of them in parallel (what does spaceflight look like in a world that also has fusion in 5 years?).* Expect a widening, permanent gap between hobbled public models and the internal ones used for RSI (at least barring any stringent regulation).*
Build your product the way AGI will want to buy it* – even an AGI running companies faces buy-vs-build and shouldn't waste computronium on sidequests*; the lasting moat is brand backed by institutional knowledge, not tech.* The real potential of AI is not making money for you but making everything cheaper through sped-up R&D leading to abundant resources*; frontier labs are basically charities for humanity's progress.* People who program out of joy are now the same class as artists painting out of joy; people who program to change the world are more empowered than ever.* Long-term I believe in a world of abundance and wonders; I am not optimistic about white-collar people keeping their jobs, and the AI won't need much overseeing in the current sense* – most humans follow orders and procedures, and you soon won't need them as intermediaries, while physical work (for now) stays gated on robotics.* A world where wages collapse yet money keeps value is not stable outside an incredibly oppressive system.*