What happens during IO
Table of Contents
What happens when you call read()?
When your program calls read(), control is transferred into the kernel through a system call. The kernel first identifies the file object associated with the provided file descriptor. At this point, the operating system checks the kernel's page cache—a region of memory in RAM where recently accessed file data is stored.
If the requested data is already in the page cache (for example, if it was read earlier or pre-fetched by the kernel), the kernel simply copies it directly from the cache into the buffer you provided in user space. This makes the read operation very fast. However, if the data is not cached, the kernel must issue a disk I/O request to retrieve the required blocks from storage. Once the data arrives, it is placed in the page cache for future use, and then copied into your buffer.
Regardless of whether the data came from the cache or from disk, it always passes through kernel space first. The final step is copying the data from the kernel's memory into the user-space buffer you supplied, making it accessible to your program.
What is a buffer?
A buffer is a temporary region of memory, usually located in RAM, that is used to hold data while it is being transferred between two components that operate at different speeds. Buffers are particularly important in file I/O, where the CPU and RAM can process data much faster than storage devices like disks.
Without buffering, the CPU would be forced to wait every time data is fetched from or written to disk. By introducing a buffer, the operating system reads larger chunks of data (such as 4KB or 8KB blocks) from disk into memory in a single operation. The CPU can then consume data from this buffer at full speed, rather than stalling for every byte. This dramatically reduces the performance gap between slow disk access and fast CPU execution.
Buffers also reduce the overhead of system calls. Each call to read()or write() requires a costly context switch into the kernel. Without buffering, reading just one byte would require one system call per byte. With a buffer, however, the operating system can fetch or write thousands of bytes in a single call, and your program can consume or produce the data incrementally.
Finally, buffering improves efficiency on the storage device itself. Disks, especially traditional HDDs, are optimized for sequential and block-aligned access. Writing many small chunks directly would result in fragmented, inefficient writes. By using a buffer, the operating system can batch these small writes together and flush them as a larger, properly aligned block, improving overall performance and reducing wear on the storage medium.
User-space buffer
A user-space buffer is a region of memory allocated within the virtual address space of a user-mode application or process. It acts as temporary storage for data that the application needs to process or exchange. Unlike kernel buffers, which live in the privileged kernel memory space, user-space buffers are owned entirely by the application itself.
These buffers are commonly used to hold data that applications read from input devices, prepare for output to files or network sockets, or simply manipulate internally. For example, when a C program reads a line of text into a character array, that array is a user-space buffer. Similarly, when an application prepares data to send over a network using the send() function, the data typically resides in a user-space buffer until the kernel copies it into its own buffers for transmission.
A key aspect of user-space buffers is the strict separation enforced by the operating system between user space and kernel space. User applications cannot directly access kernel memory, and the kernel cannot directly manipulate user-space memory. Instead, data must be explicitly transferred between the two via mechanisms such as copy_from_user() or copy_to_user(). This design ensures stability and security by preventing user processes from corrupting kernel data or accessing resources they should not control.
In I/O operations, user-space buffers serve as the staging ground for data. When an application issues a read system call, the kernel first fetches the requested data (often through the page cache) and then copies it into the user-space buffer provided by the program. Conversely, when writing data, the application fills a user-space buffer, and the kernel copies that data into its own buffers before sending it to the device. This buffering model balances efficiency with security while maintaining a clear boundary between application-level memory and kernel-managed resources.
What happens when you call write?
When an application calls the write() system call, the kernel first copies the data from the application's user-space buffer into the kernel's page cache. At this point, the call usually returns immediately, giving the impression that the write has completed. In reality, the data now resides in kernel memory, but it has not yet reached the physical disk.
Once the data is in the page cache, the kernel marks the relevant memory pages as “dirty.” This is the kernel's way of keeping track of which parts of memory still need to be synchronized with the storage device. The system does not block the application while this is happening, which is why writes often feel fast from the perspective of the program.
The actual transfer of data to disk occurs later. This can happen automatically through background flush mechanisms that periodically clean dirty pages, or explicitly when the application calls functions such as fsync(). At that moment, the kernel ensures that all dirty pages associated with the file are written to persistent storage, making the changes durable.
What are memory-mapped files?
A memory-mapped file allows a file (or part of it) to be mapped directly into a process's virtual address space. When you call mmap(), the operating system sets up page table entries so that memory accesses in that region correspond directly to portions of the file on disk.
When your program first accesses an address in the mapped region, a page fault occurs. The OS responds by loading the corresponding file page from disk into RAM (via the page cache). After that, the file contents can be read and written as if they were part of a normal array in memory—no explicit read() or write() system calls are needed.
By contrast, when you call read(fd, buf, size), the kernel copies data from the file (or from the page cache if it is already loaded) into the user-provided buffer. If the data is not in memory yet, the OS performs I/O to fetch the blocks into the page cache and then copies them into your buffer. After the read(), the data exists in two places: in the kernel's page cache and in your application's buffer.
In short, mmap() avoids the extra copy step. It lets processes access file contents directly through memory, while read()always involves copying data into a separate user buffer. This makes memory-mapped files especially useful for working with very large files and random access patterns, or when multiple processes need to share access to the same data efficiently.
What is Kernel Bypass
Kernel bypass is a technique that allows user-space applications to access hardware devices directly, such as network interface cards (NICs) or storage controllers, without routing data through the operating system's kernel. This approach is commonly used in high-performance and low-latency systems where even small delays can have a significant impact.
Under the traditional model, when an application sends or receives data, the request travels through the kernel's networking stack. The kernel performs various operations such as routing, validation, copying data between kernel space and user space, and managing buffers. Although this ensures safety and abstraction, it adds overhead through context switches and memory copies, which increase latency and CPU usage.
Kernel bypass avoids these steps by giving applications direct access to hardware from user space. Instead of relying on the kernel to process packets, the application interacts with the NIC through specialized libraries or drivers that map the device's memory into user space. This allows packets to be read or written directly without transitioning into the kernel.
By eliminating context switches, packet copying, and kernel-level processing, kernel bypass significantly reduces latency and frees CPU resources. It also allows applications to implement highly optimized, application-specific packet handling logic. Technologies such as DPDK (Data Plane Development Kit), RDMA (Remote Direct Memory Access), and io_uring are built around this idea and are widely used in systems requiring ultra-low-latency networking and high throughput.
Because kernel bypass trades abstraction for speed, it requires careful design and often direct management of memory, buffers, and hardware queues. However, in performance-critical systems such as financial trading platforms, network appliances, and real-time data processing systems, the gains in throughput and latency make kernel bypass a powerful technique.