How music is indexed on BitTorrent, and why a pointer beats a copy

The instinct when building a music catalog is to collect the music. Rip it, store it, tag it, back it up. That instinct is expensive and, for an index, unnecessary. What a catalog actually needs is an address.

The forty character address

Every torrent is identified by an infohash: a SHA-1 digest of the torrent's metadata, written as forty hexadecimal characters. It looks like this:

fa5e338bbe3e50696c1027bd61f5f86c8918b084

That string is not a link to a server. It is the identity of a specific set of files, and any peer holding those files will recognise it. Give it to the network and the network can find the content, with no central host in the middle.

Pointing at one song inside an album

A torrent usually holds a whole album, sometimes an entire discography. Addressing a single track needs one more number: the index of the file inside the torrent.

fa5e338bbe3e50696c1027bd61f5f86c8918b084 : 4
                infohash                   file

Those two values are the complete pointer to one song. Around a kilobyte of database row stands in for a thirty megabyte file, and the thirty megabytes never touch our disks.

What that buys you

• Storage stops mattering. A catalog of five million songs would be well over a hundred terabytes as audio. As pointers it fits comfortably in an ordinary database.
• Indexing gets fast. Recording where a song lives is far quicker than downloading it, so the catalog grows at the speed of metadata rather than the speed of your connection.
• The same song can have many sources. A popular track exists in dozens of releases. Each is another pointer to the same song, which means redundancy without duplication.

The catch, and the fix

A pointer is only as good as the swarm behind it. A torrent nobody seeds is an address to an empty house. That is why source health belongs in the index itself: seeder counts, last verification, and a fallback order. When a song has eight sources and the healthiest one stops answering, the next takes over and the listener notices nothing.

An index that stores copies is a library. An index that stores addresses is a map. Maps are smaller, they update faster, and they never run out of shelf space.

The trade is real: a library owns its contents, a map depends on the territory. Which is why the interesting engineering in a pointer catalog is not the pointing. It is everything you do to make sure the address still leads somewhere.