AI Firms Digitize Rare Books Before They Vanish

I stopped buying physical books years ago—not because I didn’t care for them, but because I couldn’t justify the space. Then I saw the news last week: AI companies are now buying up secondhand books by the pallet load, scanning them, and tossing the originals. Not just any books—rare ones, first editions, even out-of-print titles. The worst part? They’re using intermediaries to hide the fact that the books are being destroyed.

Here’s the kicker: since the start of 2025, more than half of all new content on the internet has been AI-generated. If this trend keeps up, we’re staring down a future where most of what we read wasn’t written by humans at all. The question isn’t just whether we’ll notice the difference—it’s whether we’ll still have the physical books to compare against.

Technical Overview

The most interesting thing about this system isn’t its speed or scale—it’s that it’s untouched by machines. Most AI systems today rely on either pre-trained models or fine-tuned versions of existing ones, but this isn’t just another variant of Llama or Mistral. That Hacker News thread you linked describes a system built from scratch, with no reliance on existing model weights or architectures. The developers started with raw transformer blocks and built the rest themselves, which means every decision about attention mechanisms, normalization, and tokenization is their own.

The architecture follows the standard transformer design but diverges in key places. For example:

• Positional embeddings use a learned absolute scheme instead of rotary or ALiBi.
• Activation functions swap GELU for SwiGLU, with a specific β value of 1.0 that isn’t commonly used.
• Normalization is done with RMSNorm instead of LayerNorm, and it’s applied before the attention block rather than after.

Here’s the minimal config that reproduces their setup:

model:
  name: "scratch-transformer"
  dim: 2048
  n_layers: 16
  n_heads: 16
  head_dim: 128
  ff_dim: 8192
  activation: "swiglu"
  norm: "rmsnorm"
  pos_encoding: "learned_absolute"
  vocab_size: 50257

The training pipeline is equally non-standard. They use a custom tokenizer trained on a filtered subset of the Pile, with a vocabulary size of 50,257 tokens. The dataset is deduplicated and then sharded into 128 files, each processed in parallel with a custom C++ tool that strips HTML and normalizes whitespace before tokenization.

The interesting part here isn’t just the novelty—it’s the tradeoffs. RMSNorm is faster but less stable during training. SwiGLU with β=1.0 works well empirically but lacks the theoretical backing of its original paper. And a learned absolute positional embedding scales poorly beyond context lengths of 2,048 tokens. Whether these choices are worth it depends entirely on your use case. If you’re prototyping, this level of control is overkill. If you’re trying to push the limits of efficiency, it’s worth studying—but don’t expect plug-and-play results.

Industry Impact

I’m not convinced we’ve actually crossed the point where 51% of new internet content is machine-generated. The methodologies used to claim that number are usually narrow—counting blog posts scraped from RSS feeds, or public code repositories, or papers uploaded to arXiv. They miss the long tail: emails, instant messages, private documentation, private repos, local files, and the vast amount of content that never makes it to the open web. Even if the headline number is directionally correct, it tells us more about what we can measure than what actually matters.

That said, the real anxiety isn’t volume; it’s signal dilution. If everything looks like a trained-from-corpora response, then surface-level coherence becomes the dominant heuristic for trust. That’s already happening in code review tools and drafting assistants: people accept output because it compiles, not because it’s original or insightful. The next step isn’t mass deception; it’s quiet normalization. We’ll stop noticing when a paper is synthetic because its prose matches the expected style of a mid-tier journal. We’ll assume a blog post is human unless it cites an impossible number of niche sources. The system will adapt to the noise by lowering the standard for what we consider “human.”

On HN, the thread (49383026) fixates on the “eventual unreadability” angle—people fretting that students, journalists, and researchers will ingest AI content uncritically. But the more immediate risk is professional atrophy: engineers who never write from scratch, analysts who can’t structure a blank spreadsheet, writers who treat their first draft as the final submission. That’s not a detection problem; it’s a skill problem. The tools will keep improving, so the gap between “good enough” AI output and “truly original” work will widen. Within two years, we’ll see job postings that explicitly ask for “original thought,” as if it’s a rare elective rather than the default expectation.

Conclusion

Two years ago, AI companies still marketed their book-scanning projects as preservation efforts. Now they’re openly admitting the real goal is exclusivity—digitize everything before competitors can, then let the physical copies rot on someone else’s dime. What’s left after that is a library of one: a single corporate repository whose contents no one outside the AI cartel can audit, cite, or even access without paying. We already have the technology to build open archives; what we lack is the will to enforce it.

I’m still not sure what to make of the volunteers who keep feeding these scans into Anna’s Archive under fire. They’re preserving what corporations want to erase, but they’re also building a shadow collection that could outlast its creators. Maybe that’s the only kind of library left now—the one that exists in defiance.