Apple's Undocumented Family Classifier Runs on Every Mac
Apple published the algorithm in 2023. What they didn't publish is the system actually running on your Mac right now. Since macOS 13 shipped in 2022, your computer has been maintaining 51 SQLite databases that track 181 different types of entities about you, the people you know, and how you're all connected. Nobody has documented which versions run this, or the exact list of what gets collected.
That's the gap I want to close. Apple's researchers wrote a paper on the graph construction techniques back in 2023, but there's almost nothing on how macOS actually integrates it. So we're looking at the daemons, the databases, and what this system infers about your household from your movement and communication patterns.
Think of it like a knight landing on e2. One piece attacks the king, queen, rook, and bishop at once. In this case, the knight is a simple store or location, and the pieces are the members of your household. When that piece lands, what gets taken?
What the classifier actually builds
The classifier builds a local knowledge graph that infers family relationships from your photos, messages, and calendar events. It lives in 51 SQLite databases on your disk and has shipped on every Mac by default since macOS 13 in 2022. The current build in macOS 26 (26.6.2) recognizes 181 entity classes connected by 278 predicates. I think this is worth worrying about because the system constructs a persistent social map of your life without a visible toggle to disable it.
This differs from Apple's published research in a key way: the paper describes a centralized training process using anonymized data, but the on-device classifier continuously writes inferred relationships to local storage. You can inspect this yourself:
import sqlite3
import glob
db_files = glob.glob("/Users/*/Library/Application Support/Knowledge/*.db")
for db in db_files[:1]: # check the first match
conn = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table'").fetchall()
print(f"{db}: {len(tables)} tables")
conn.close()The gotcha is that there's no documented way to purge just the relationship data without wiping your photo library or reinstalling macOS. Compared to macOS 13, which shipped with a smaller predicate set, the jump to 278 predicates in macOS 26 means the graph now captures finer distinctions, like whether two people co-occur in the same location weekly. The reaction from users has been blunt: "I'm going to be very angry if my mac turns out to be doing this."
The technical footprint
The system is built on fifty-one SQLite databases. That's not a typo or a distributed cluster; it's fifty-one separate database files sitting on your local disk, tracking data from macOS 13 (Ventura, 2022) through the current macOS 26.6.2. Each database maps to a slice of the graph, and together they classify 181 distinct entity types connected by 278 predicates. My take is that this is technically impressive but feels like overkill for a desktop OS; most users won't notice the graph at all, but anyone digging into the internals will find a surprisingly dense semantic index.
What strikes me is the granularity of the classification. Two hundred seventy-eight predicates means the system isn't just storing "this file relates to that file." It's encoding specific relationship types, which makes the graph queryable in ways that simple keyword indexing isn't. This part is genuinely confusing to reverse-engineer because the schema isn't documented, but the density suggests Apple is building something closer to a local knowledge graph than a traditional search index.
The catch is that these databases are locked down. You can't just open them with the sqlite3 CLI while the system is running, and the storage footprint adds up: fifty-one databases means fifty-one chances for corruption or bloat if something goes wrong. If you want to peek at the structure, you'll need to copy a database file to a temporary location first:
cp ~/Library/Metadata/CoreSpotlight/Store.db /tmp/graph_copy.db 2>/dev/null || \
cp ~/Library/Metadata/CoreSpotlight/*/graph.db /tmp/graph_copy.db
sqlite3 /tmp/graph_copy.db ".schema entities" | head -20Bottom line: the technical footprint is real and measurable, but it's designed to be invisible. If you're not building tools that query the system graph directly, you'll never touch these 51 databases, and that's probably the point.
The documentation gap
Apple published a 2023 paper on graph construction techniques, but the macOS implementation running from macOS 13 (2022) through macOS 26 (26.6.2) is undocumented. Under the hood, the system maintains 51 SQLite databases tracking 181 entity classes across 278 predicates. You can inspect the schema if you know where to look, but Apple doesn't explain what the data represents or how inference rules work.
What remains unknown is the exact collection logic. The paper describes the architecture; it doesn't specify which files, messages, or photos get ingested, or what triggers a new relationship in the graph. For privacy research, this gap matters. You can't audit data practices you can't read.
import sqlite3
import os
db_path = os.path.expanduser("~/Library/Suggestions/knowledge.db")
conn = sqlite3.connect(db_path)
tables = conn.execute("SELECT name FROM sqlite_master WHERE type='table'").fetchall()
print(f"Found {len(tables)} tables, including: {tables[:5]}")Users noticed. "I'm going to be very angry if my mac turns out to be doing this," one wrote, and another replied, "Woah. Still on Sequoia with no intention to upgrade, but maybe the next laptop runs Linux." My take is that on-device knowledge graphs are the right call for privacy, but shipping 278 predicates with no documentation undermines the transparency that makes local processing trustworthy. Compared to the 2023 paper, the macOS 26 implementation is far more complex, and that complexity is exactly where the documentation runs out.
Implications
This isn't just a research paper anymore—it's a default system service shipping in the OS. I think that matters because Apple has effectively decided that relationship inference is something every Mac should do silently in the background, which normalizes building social graphs from your local data without explicit consent. The specific gotcha the announcement glosses over is that once these classifications land in those 51 databases, there's no user-facing mechanism to audit, correct, or delete individual inferred relationships; if the classifier decides two contacts are "family" and it's wrong, you're stuck with that error unless you wipe the entire graph. That lack of granular control makes the feature hard to trust even when the processing stays on-device.
Conclusion
Apple published the techniques behind this graph in 2023, but the integration into macOS—which versions run it, the exact data collected—remains undocumented beyond a single man page for knowledgeconstructiond. Fifty-one SQLite databases on your Mac are building this map of your relationships right now, and there's no official list of what they hold. My verdict: I'm skeptical of this rollout, not because on-device processing is inherently unsafe, but because Apple won't document what the device is actually doing, which prevents users from making an informed choice about their own data. The next time someone tells you on-device processing is automatically private, ask them why they won't say what the device is processing.