Rockbox is a legendary open-source audio firmware originally written for portable hardware players like the iPod. It is a deeply embedded system: written in C, designed for flash storage and tiny CPUs, with no operating system underneath it. Every track it has ever played in its two-decade history lived on a local filesystem.
This post documents how I extended Rockbox — running as a hosted desktop application for Linux/MacOS — to transparently stream audio over HTTP, while keeping the codec layer completely unaware that anything changed.
The Problem
Rockbox's audio pipeline is built around POSIX file descriptors. Every codec, every metadata reader, every buffering thread operates on open() / read() / lseek() / close(). That's the contract. No network, no streaming, no async I/O — just files.
We needed to break that contract at the thinnest possible seam: the point where a file path becomes an open file descriptor.
The Architecture at a Glance
Five layers. The codec has no idea which path it takes.
Layer 1: The Rust Netstream Crate
The first thing I needed was a reliable HTTP client that speaks the same language as a POSIX file. The crates/netstream crate exposes exactly six C-compatible functions:
rb_net_open(url) → handle id (i32)
rb_net_read(h, buf, n) → bytes read (i64)
rb_net_lseek(h, off, whence) → new position (i64)
rb_net_len(h) → content length (i64)
rb_net_content_type(h, buf, n) → bytes written (i64)
rb_net_close(h)This is the POSIX I/O contract, translated to HTTP. The implementation uses reqwest with a global blocking HTTP client backed by rustls.
Internal State
Each open stream is tracked in a global HashMap<i32, Arc<Mutex<StreamState>>>:
struct StreamState {
url: String,
pos: u64,
content_length: Option<u64>,
content_type: Option<String>,
response: Option<reqwest::blocking::Response>,
}
Handle IDs are monotonically increasing integers dispensed by an atomic counter. The global map is locked only long enough to clone the Arc — actual I/O happens outside the global lock.
Seeking: The Tricky Part
lseek is where the illusion of a file gets expensive. HTTP responses are forward-only streams. To seek backwards, you have to open a new connection with a Range header. To seek forward, you have to discard bytes. rb_net_lseek tries the smart path first:
seek(offset)
│
├─ new offset == current pos? → no-op, return pos
│
├─ send Range: bytes={offset}-
│ │
│ ├─ server returns 206 Partial Content?
│ │ → validate Content-Range, update pos ✓
│ │
│ └─ server returns 200 (ignored Range)?
│ → fall back: reopen stream from 0,
│ skip (offset) bytes by reading and discarding
│
└─ SEEK_END: use content_length to compute absolute offset,
then follow same logic above
This handles both seekable servers (CDNs, S3, typical file servers with Accept-Ranges: bytes) and non-seekable servers (some live stream proxies that return 200 regardless).
Unknown File Size
When a server omits Content-Length (chunked encoding, live streams), rb_net_len returns -1. The C bridge layer turns that into 0x7FFFFFFF (~2 GiB). Rockbox's buffering layer treats that as a very large file and truncates naturally when read() returns 0 at the end of the stream — the standard EOF contract.
Layer 2: The C Bridge (streamfd.c / streamfd.h)
The bridge is thin on purpose. Its only job is to decide which path to take and encode the result into a form the rest of Rockbox can hold in an int.
File Descriptor Encoding
Normal POSIX file descriptors are non-negative integers. We steal the negative space:
fd ≥ 0 → real file descriptor (open, read, lseek, close)
fd == -1 → error / unset sentinel
fd ≤ -1000 → HTTP stream handle
handle_id = -1000 - fdA Rust handle ID h becomes C fd (-1000 - h). Converting back: handle_id = -1000 - fd. This encoding costs zero memory and zero allocations — the handle is embedded in the integer.
Examples:
| Rust handle | C fd |
|-------------|--------|
| 0 | -1000 |
| 1 | -1001 |
| 42 | -1042 |
The Dispatch Functions
Each function is a two-branch switch on stream_is_http_fd(fd):
ssize_t stream_read(int fd, void *buf, size_t n)
{
if (stream_is_http_fd(fd)) {
return (ssize_t) rb_net_read(http_fd_to_handle(fd), buf, n);
}
return read(fd, buf, n);
}
stream_open additionally detects the URL prefix:
int stream_open(const char *path, int flags)
{
if (strncmp(path, "http://", 7) == 0 ||
strncmp(path, "https://", 8) == 0) {
int32_t h = rb_net_open(path);
if (h < 0) return -1;
return STREAM_HTTP_FD_BASE - (int)h; // encode as negative fd
}
return open(path, flags);
}
Zero Cost on Embedded Builds
The header guards the entire implementation behind #ifdef STREAM_HTTP_ENABLED, which is only set for SIMULATOR and APPLICATION builds. On any real embedded target, every stream_* symbol reduces to a preprocessor macro that calls the native Rockbox function:
#define stream_open(path, flags) open((path), (flags))
#define stream_read(fd, buf, n) read((fd), (buf), (n))
#define stream_lseek(fd, off, whence) lseek((fd), (off), (whence))
// ...
No binary overhead. No behavior change. The same caller code compiles to optimal native I/O on device.
Layer 3: The Playlist System
Before any audio plays, a track has to be added to a playlist. Rockbox playlists are .m3u-style control files stored on disk. Each entry is a path relative to the playlist directory, or an absolute path.
The format_track_path Hack
apps/playlist.c contains a function called format_track_path that normalizes every path written into the control file. Without intervention, it would prepend the playlist directory to whatever string you hand it — which would turn http://localhost:6062/tracks/abc123 into /home/user/http://localhost:6062/tracks/abc123, which is completely useless.
I added an early-exit for URLs:
/* HTTP(S) URLs are absolute — copy them as-is.
* Also strip a spurious leading '/' that an older buggy build may
* have written into the control file. */
const char *url_src = src;
if (*url_src == '/') {
if (strncmp(url_src + 1, "http://", 7) == 0 ||
strncmp(url_src + 1, "https://", 8) == 0)
url_src++; /* skip the spurious slash */
}
if (strncmp(url_src, "http://", 7) == 0 ||
strncmp(url_src, "https://", 8) == 0) {
strlcpy(dest, url_src, buf_length);
return (ssize_t)strlen(dest);
}
URL detected → written verbatim. Everything else follows the original relative/absolute path logic.
Playlist Directory for HTTP Tracks
Rockbox's playlist_create() writes its control file into a directory you provide. For local tracks, the natural choice is the directory containing the music files. For HTTP URLs, there is no such directory — and using / as a fallback fails on macOS because the filesystem root is not writable.
The fix: use $HOME (or /tmp) when the playlist contains HTTP URLs.
let dir = if first.starts_with("http://") || first.starts_with("https://") {
std::env::var("HOME").unwrap_or("/tmp".to_string())
} else {
// parent directory of the first local track
parent_dir(first)
};
rb::playlist::create(&dir, None);
rb::playlist::build_playlist(tracks, 0, tracks.len());
Layer 4: The Audio File Server
For tracks already in the local library, the client doesn't send a raw file path over the network — it sends an internal URL like http://localhost:6062/tracks/{uuid}. The uuid is the track's database primary key (a CUID).
The HTTP server (port 6062) handles GET /tracks/{id} and HEAD /tracks/{id}:
async fn index_file(req: HttpRequest) -> Result<NamedFile, Error> {
let id = req.match_info().get("id").unwrap();
let pool = req.app_data::<Pool<Sqlite>>().unwrap();
let track = repo::track::find(pool.clone(), id).await?
.ok_or_else(|| actix_web::error::ErrorNotFound("track not found"))?;
Ok(NamedFile::open(track.path)?)
}track.path is the local file path stored in SQLite. Actix serves the file with full support for Range requests — which means the netstream crate's seek implementation gets real HTTP 206 responses and can seek in O(1) rather than having to skip bytes.
Album art is served separately from GET /covers/{hash}.{ext}, pointing directly at pre-extracted image files in ~/.config/rockbox.org/covers/.
Layer 5: Remote Track Metadata
Before a remote track can be displayed correctly in the UI (title, artist, album, album art), its metadata has to be saved to SQLite. We can't wait for the entire file to download — some tracks are hundreds of megabytes.
Partial Download Probe
download_partial_remote_file fetches the first 8 MB using a Range: bytes=0-8388607 header, writes it to a temp file, and returns that path:
URL
│
├─ GET {url} Range: bytes=0-8388607
│
├─ 206 Partial Content? → write up to 8 MB, done
│ └─ Content-Type header → pick file extension
│
└─ 200 OK (server ignored Range)? → same, write up to 8 MB
└─ extension_from_url_path() as fallback
→ /tmp/rockbox-remote-probe-{md5(url)}.mp3 (auto-deleted on Drop)
8 MB is enough for the ID3 tag, Vorbis comment, or MP4 metadata box of any real audio file — plus embedded album art. Rockbox's get_metadata(-1, path) reads this temp file exactly as it would a local file, extracts all fields, and returns an Mp3Entry.
The track is then saved to SQLite with md5 = md5(original_url) as the lookup key. Subsequent lookups use find_by_md5(md5(url)).
Album Art Fallback
Because the URL-keyed record was saved from the HTTP stream (which may have no embedded art depending on file format or server range handling), album_art can be None even when the local copy has art. The fix is a two-step lookup:
find_by_md5(md5(url)) // finds by URL hash
│
├─ album_art is Some? → use it ✓
│
└─ album_art is None?
│
└─ URL matches /tracks/{uuid}?
│
└─ find(pool, uuid) // look up local track by primary key
│
└─ use local_track.album_art ✓
This is applied in three places: the HTTP current_track endpoint, the real-time gRPC stream_current_track broker, and the playlist entries broker.
Putting It All Together: Playing a Track
Here is the complete flow for a single play_track("http://localhost:6062/tracks/abc123") call, from client request to audio output:
Client
│
│ gRPC: PlayTrack { path: "http://localhost:6062/tracks/abc123" }
▼
crates/rpc/src/playback.rs ::play_track()
│
│ POST /playlists { tracks: ["http://localhost:6062/tracks/abc123"] }
▼
crates/server/src/handlers/playlists.rs ::create_playlist()
│
├─ detects URL → dir = $HOME
├─ rb::playlist::create($HOME, None) // write control file to $HOME
└─ rb::playlist::build_playlist(...) // add URL to playlist
│
│ C FFI → apps/playlist.c
│ format_track_path("http://localhost:6062/tracks/abc123")
│ → URL detected, stored verbatim in .m3u control file
│
│ PUT /playlists/start
▼
crates/server/src/handlers/playlists.rs ::start_playlist()
│
│ rb::playlist::start(...)
▼
Rockbox audio engine (apps/playback.c)
│
│ opens current playlist entry: "http://localhost:6062/tracks/abc123"
▼
apps/streamfd.c ::stream_open("http://localhost:6062/tracks/abc123")
│
├─ detects "http://" prefix
├─ rb_net_open("http://localhost:6062/tracks/abc123") → handle = 0
└─ returns fd = -1000 - 0 = -1000
│
│ Rust netstream crate
│ GET http://localhost:6062/tracks/abc123
▼
crates/graphql/src/server.rs ::index_file()
│
├─ repo::track::find(pool, "abc123") → track.path = "/music/song.mp3"
└─ NamedFile::open("/music/song.mp3") with Range support
│
│ HTTP 200 (or 206 for Range requests)
▼
Rust netstream (StreamState { pos: 0, content_length: Some(8200000) })
│
│ returns handle 0 → fd = -1000 to codec
│
│ codec calls stream_read(-1000, buf, 4096)
│ stream_lseek(-1000, 0, SEEK_SET)
│ stream_content_type(-1000, type_buf, 64)
│ ...
▼
Audio decoded and played 🔊
Local Filesystem Flow (for comparison)
When the path is a local file like /music/song.mp3, the flow is simpler and unchanged from the original Rockbox behavior:
Client
│
│ gRPC: PlayTrack { path: "/music/song.mp3" }
▼
create_playlist()
│
├─ dir = "/music" (parent of first track)
└─ rb::playlist::build_playlist(["/music/song.mp3"], ...)
│
│ format_track_path("/music/song.mp3")
│ → regular path, stored as absolute path
▼
stream_open("/music/song.mp3")
│
├─ no "http://" prefix
└─ open("/music/song.mp3", O_RDONLY) → fd = 7 (normal fd)
│
│ codec calls read(7, buf, 4096)
│ lseek(7, 0, SEEK_SET)
│ ...
▼
Audio decoded and played 🔊The difference: one branch in stream_open. The rest of the codec is identical.
The Real-Time Metadata Pipeline
While audio plays, the server runs a broker loop (start_broker) that publishes live updates over gRPC subscriptions. These are consumed by clients to show current track info, elapsed time, playlist state, and album art.
start_broker loop (every ~500ms)
│
├─ rb::playback::current_track() → Mp3Entry { path, elapsed, ... }
│
├─ rb::playlist::get_track_info(index).filename
│ └─ URL or file path from playlist control file
│
├─ find_by_md5(md5(filename)) → db_metadata
│ ├─ album_art Some? → use it
│ └─ album_art None? → find(pool, uuid_from_url) → local track art
│
├─ SimpleBroker::publish(Track { album_art, elapsed, ... })
│ └─ consumed by stream_current_track gRPC subscription
│
└─ SimpleBroker::publish(Playlist { entries with album_art })
└─ consumed by stream_playlist gRPC subscription
One subtlety: rb::playback::current_track() calls Rockbox's audio_current_track() from the audio engine, which can return a path that differs from the playlist entry for HTTP stream tracks (depending on internal buffering state). The broker always uses playlist_get_track_info(index).filename — the ground-truth path from the playlist control file — as the key for DB lookups.
Why This Design Works
The codec is untouched. The entire audio decoding stack — MP3, FLAC, AAC, Vorbis, WavPack, a dozen more — calls stream_open / stream_read / stream_lseek just like it always called open / read / lseek. No codec code was modified.
The seam is at the right level. By intercepting at the file descriptor boundary rather than at a higher abstraction (e.g., virtual filesystem), we get all codec seek patterns for free. Codecs that seek to the end to find ID3 tags, codecs that seek backward to reparse Vorbis headers, codecs that skip forward to decode random-access frames — all of them work through the same two-branch dispatch.
HTTP Range requests map cleanly to lseek. The POSIX seek semantics (SEEK_SET, SEEK_CUR, SEEK_END) translate directly to Range: bytes=N- headers. The fallback (skip bytes on non-206 response) handles poorly-behaved servers without breaking playback.
Zero cost on the original platform. The preprocessor macros ensure that embedded Rockbox builds on real hardware are completely unaffected. The entire streaming layer compiles away.
Limitations and Future Work
True live streams (infinite chunked responses, no
Content-Length) work for sequential playback but seeking is undefined behavior. A separate streaming mode that disables seeking would be cleaner.TLS certificate validation is enabled (rustls defaults), but certificate pinning for internal server-to-server communication is not implemented.
The metadata probe downloads up to 8 MB per track on first scan. For large libraries of remote tracks, a queue-based background scanner with rate limiting would be more polite.
Re-encoding to a single format on the server side would allow the codec to always receive a known-good stream, avoiding the current complexity of codec detection from HTTP Content-Type headers.
Conclusion
The hack is about 600 lines of Rust, 130 lines of C, and a 15-line change to playlist.c. Everything else — twenty years of audio codec development, every format Rockbox supports, every playback feature — continues working without knowing anything changed.
The key insight: find the thinnest seam in a system that you can cut without disturbing either side. In Rockbox, that seam was the moment a path string becomes an open file descriptor. Everything above it thinks in terms of tracks and playlists. Everything below it thinks in terms of bytes. We just taught the seam to speak HTTP.
The full implementation is available at github.com/tsirysndr/rockbox-zig.