Rust mmap Memory-Mapped I/O: Why Reading Files Can Be as Fast as Accessing Memory

Today let’s talk about something interesting: Rust mmap memory-mapped I/O.

Sounds fancy, right? “Memory-mapped I/O”, “mmap”, “zero-copy”… feels like stuff only OS kernel developers need to understand.

But actually, if you’ve ever done Rust file reading performance optimization, or used certain databases, you’ve probably been using it already without knowing.

Let Me Tell You a Story

Imagine this scenario: you’re a librarian, and someone comes to borrow a book.

The traditional way works like this:

Customer says they want to see page 42. You run to the bookshelf, find the book, flip to page 42, copy the content onto a piece of paper, then hand the paper to the customer.

Customer finishes and wants page 43. You run back, copy again, hand it over.

Exhausting? Absolutely exhausting.

The memory-mapped way works like this:

You just plop the book on the customer’s desk: “Here’s the book, flip through it yourself.”

Customer can read any page they want without you running back and forth.

This is the core idea of Rust mmap memory-mapped I/O: stop shuttling data back and forth, let the program access file contents directly, just like accessing memory.

What’s Wrong with Traditional Rust File Reading?

Let’s see how normal file reading works:

use std::fs::File;
use std::io::Read;

let mut file = File::open("data.log")?;
let mut buffer = Vec::new();
file.read_to_end(&mut buffer)?;
println!("{}", buffer[0]);

Looks pretty clean, right? But what’s happening under the hood:

Disk → Kernel buffer → User buffer → Your program

Data gets copied twice. Once from disk to kernel, once from kernel to your program.

What if the file is 5GB? Congratulations, that 5GB gets copied twice, eating up double the time and memory. This is the pain point of traditional Rust file reading.

How Does Rust mmap Memory-Mapped I/O Solve This?

The idea behind memory-mapped I/O is simple: since the kernel already loaded the data into memory, why copy it again for me? Can’t I just use it directly? This is the core idea of zero-copy technology.

Yes, you actually can.

use memmap2::MmapOptions;
use std::fs::File;

let file = File::open("data.log")?;
let mmap = unsafe { MmapOptions::new().map(&file)? };
println!("{}", mmap[0]);

That’s it? That’s it. Implementing Rust mmap with the memmap2 library is just that simple.

No explicit read operations, no buffer allocation. You just use the file like an array and you’re done - that’s the magic of zero-copy.

What happens under the hood becomes:

Disk → Kernel page cache → Your program accesses directly

One copy eliminated, zero-copy achieved. For large files, this savings is massive - this is real Rust performance optimization.

What Actually Happens Under the Hood with Rust mmap?

Let me draw you a diagram:

┌────────────────────────────┐
│       Your Program         │
│  mmap() → Virtual Address  │
└──────────┬─────────────────┘
           │
           ▼
┌────────────────────────────┐
│       Page Cache           │
│  OS stores file content    │
└──────────┬─────────────────┘
           │
           ▼
┌────────────────────────────┐
│       Disk File            │
│  data.log on your drive    │
└────────────────────────────┘

When you access mmap[1000], the CPU might find that this memory region hasn’t been loaded yet. Then the OS secretly loads the corresponding 4KB of data from disk.

This is called a “page fault”, but you don’t notice at all - to you, it’s like that array was always in memory.

This is why people call Rust mmap memory-mapped I/O “black magic”. You never called read, but data just comes out.

Wait, Why Does Rust mmap Make Me Write unsafe?

Sharp-eyed readers might have noticed there’s an unsafe keyword in the code.

This is Rust reminding you: “Buddy, you’re playing with fire here.”

Why playing with fire? Because memory mapping has a hidden pitfall: what if someone else modifies the file while you’re using it?

Imagine this: you’re reading a file, and another process truncates it. In C, your program might crash outright, or read garbage data without even knowing.

Rust uses unsafe to explicitly tell you: this is beyond my protection, be careful.

This isn’t Rust shirking responsibility - it’s Rust educating you. It lets you know where the risk boundaries are.

A Real Example: Scanning Log Files with Rust mmap

Say you have a 5GB log file and want to count how many lines contain “ERROR”. How big is the performance gap between traditional Rust file reading and Rust mmap memory-mapped I/O?

Traditional Rust file reading approach:

use std::io::{BufRead, BufReader};

let file = File::open("server.log")?;
let reader = BufReader::new(file);
let count = reader.lines()
    .filter(|l| l.as_ref().unwrap().contains("ERROR"))
    .count();
println!("Found {} errors", count);

It works, but it’s slow. For multi-GB files, might take 20+ seconds. Want Rust performance optimization? Keep reading.

Rust mmap memory-mapped I/O approach (using memmap2):

use memmap2::MmapOptions;
use std::str;

let file = File::open("server.log")?;
let mmap = unsafe { MmapOptions::new().map(&file)? };
let data = str::from_utf8(&mmap)?;
let count = data.matches("ERROR").count();
println!("Found {} errors", count);

No loop reading, no buffer allocation. Direct string search in memory - zero-copy technology makes everything simple.

On modern SSDs, how big is the gap between traditional Rust file reading and Rust mmap? Here’s the data:

Over 7x difference. And memory usage is even lower because the OS intelligently manages which pages need to stay in memory. This is the power of Rust performance optimization.

Why Do Databases Love Rust mmap Memory-Mapped I/O?

You might have heard that databases like SQLite and LMDB use memory mapping extensively. Why?

Because the core need of databases is: fast random access to large amounts of data.

With traditional I/O, every record read goes through the “system call → kernel copy → user space” process.

With Rust mmap memory-mapped I/O, the database file directly becomes a giant memory array. Read record 1000? Just mmap[offset], done - zero-copy straight to the point.

This is why people say: Rust mmap memory-mapped I/O lets databases treat the file itself as an in-memory data structure, not dead data sitting on disk.

How to Safely Use Rust mmap Memory-Mapped I/O?

Since unsafe sounds scary, is there a safer way?

Yes, just wrap it:

use memmap2::MmapOptions;
use std::fs::File;

pub struct SafeMmap {
    mmap: memmap2::Mmap,
}

impl SafeMmap {
    pub fn open(path: &str) -> std::io::Result<Self> {
        let file = File::open(path)?;
        let mmap = unsafe { MmapOptions::new().map(&file)? };
        Ok(Self { mmap })
    }

    pub fn as_bytes(&self) -> &[u8] {
        &self.mmap
    }
}

Now your business code can look like this:

let mapped = SafeMmap::open("data.db")?;
println!("First byte: {}", mapped.as_bytes()[0]);

The unsafe is isolated inside SafeMmap::open, everywhere else is safe Rust code.

This is the idiomatic approach in the Rust community when using memmap2: encapsulate dangerous operations and expose clean APIs.

Summing Up Rust mmap Memory-Mapped I/O

Rust mmap memory-mapped I/O is basically:

1. Turn files into memory arrays - zero-copy, zero read loops
2. Rust honestly exposes risks - unsafe isn’t a flaw, it’s education
3. Great for large files - databases, log analysis, compilers all love it - a weapon for Rust performance optimization
4. Can be safely wrapped - isolate unsafe, keep business code elegant

After using Rust mmap memory-mapped I/O once, when you look back at traditional Rust file reading, you’ll feel like “how did I ever put up with that?”

Don’t believe me? Try memmap2 yourself.

Alright, that’s it for today.

By the way, if you have friends still foolishly reading large files line by line in a loop, share this article with them. Save who you can.

Got any thoughts after reading, or hit any pitfalls when actually using it? Let’s chat in the comments.