The History of Virtualization

The History of Virtualization

Virtualization is the fundamental abstraction layer that decouples physical computing hardware from the operating systems and applications that execute upon it. By inserting a specialized software layer, typically a hypervisor or a container engine, between the physical components and the software stack, a single physical machine can be partitioned into multiple isolated, independent environments. This technology solved fundamental economic and technical constraints in computer science, transforming how data centers operate, scale, and distribute resources.

The history of virtualization spans over six decades.^[19] It began with the massive, prohibitively expensive mainframes of the 1960s, where the primary goal was to maximize the utilization of multimillion dollar hardware through time-sharing. It evolved through a period of dormancy during the microcomputer revolution of the 1980s, only to experience a massive resurgence in the late 1990s. During this period, x86 server sprawl forced the industry to find ways to consolidate underutilized hardware, leading to the development of dynamic binary translation and hardware-assisted virtual machines.

Today, virtualization extends far beyond the central processing unit (CPU). It encompasses network virtualization, storage virtualization, and operating system-level containerization. It is the foundational technology of cloud computing, enabling the dynamic, on-demand allocation of compute, memory, and storage resources across global data centers.^[20] Without virtualization, the modern internet, software as a service (SaaS), and hyperscale cloud infrastructure would be physically and economically impossible.

Origins in Time-Sharing and Mainframes (1960s to 1970s)

In the late 1950s and early 1960s, computers such as the IBM 7090 were massive, expensive installations. Organizations relied on batch processing, meaning the computer executed one sequential task at a time. This approach was highly inefficient; the CPU often sat idle waiting for slow mechanical input/output operations, such as reading punch cards or magnetic tape, to complete. To solve this inefficiency, researchers developed time-sharing, a concept allowing multiple users to interact with the computer simultaneously by rapidly switching the CPU context between tasks. The Massachusetts Institute of Technology (MIT) pioneered this approach with the Compatible Time-Sharing System (CTSS)^[1], first demonstrated in 1961 on an IBM 709 and later expanded under Project MAC. CTSS allowed multiple users at remote terminals to share the mainframe processing time, proving that interactive computing was viable.

IBM recognized the commercial necessity of maximizing hardware utilization and began experimenting with deeper abstraction layers. Between 1964 and 1965, IBM researchers developed the M44/44X project, which used an IBM 7044 to simulate multiple virtual 7044 machines. This project introduced experimental paging and virtual memory concepts. Concurrently, researchers at the IBM Cambridge Scientific Center began work on the CP-40 project^[2], an experimental time-sharing system designed for a heavily modified IBM System/360 Model 40.^[21] The hardware modifications included a custom associative memory system to support dynamic address translation (DAT), allowing the physical memory to be mapped to virtual addresses. This meant the operating system could use non-contiguous blocks of physical memory as if they were a single contiguous address space. CP-40 successfully created multiple virtual machines, each running its own operating system.

This research quickly evolved into CP-67, designed for the IBM System/360 Model 67. The Model 67 was specifically engineered with DAT hardware to support time-sharing. CP/CMS^[3] was distributed as a free, unsupported Type-III library; VM/370 (released in 1972) was IBM's first commercial hypervisor. The Control Program ran in supervisor state and managed the hardware, intercepting privileged instructions from the virtual machines. CMS provided a lightweight, single-user operating system for each virtual machine, giving each user the illusion of a dedicated mainframe.

The success of CP-67 led IBM to formalize virtualization in its mainline enterprise products. In August 1972, IBM announced the System/370 Advanced Function, which included standard hardware support for virtual memory across the product line, and released the VM/370^[4] operating system. VM/370 cemented the virtual machine as a standard enterprise technology. It allowed organizations to run legacy applications, test new software, and support multiple users on a single mainframe without conflict, establishing a paradigm that dominated enterprise computing for a decade.

The Formalization of Virtualization (1974)

As virtualization gained traction in the mainframe market, computer scientists sought to define the exact hardware requirements necessary to support it efficiently. In July 1974, Gerald J. Popek of the University of California, Los Angeles, and Robert P. Goldberg of Harvard University published a paper in Communications of the ACM titled "Formal Requirements for Virtualizable Third Generation Architectures"^[5]. This paper established the theoretical foundation for virtualization that the industry still references today.

Popek and Goldberg defined a Virtual Machine Monitor (VMM) as a software layer that exhibits three specific properties. First, equivalence (or fidelity): a program running under the VMM must exhibit a behavior identical to that demonstrated if it were running directly on the bare hardware, with the only permissible exception being differences in timing caused by VMM intervention. Second, resource control (or safety): the VMM must be in complete control of the virtualized resources, preventing any virtual machine from accessing hardware not explicitly allocated to it, and ensuring that a crash in one virtual machine does not affect the host or other guests. Third, efficiency (or performance): a statistically dominant subset of the virtual processor instructions must be executed directly by the physical hardware, without software intervention by the VMM.

To determine if a hardware architecture could support a VMM with these properties, they classified CPU instructions into specific categories. Privileged instructions are those that trap to the VMM if executed in a user-level state. Sensitive instructions are those that attempt to read or modify the configuration of resources, or alter the processor mode or state.

The Popek and Goldberg virtualization theorem^[6] stated that a computer architecture is strictly virtualizable if and only if the set of sensitive instructions is a subset of the set of privileged instructions. In a strictly virtualizable architecture, the VMM runs in the highest privilege mode, and the guest operating system runs in a lower privilege mode. When the guest attempts a sensitive operation, the hardware automatically traps to the VMM, which simulates the instruction and returns control to the guest. This theorem became the benchmark for evaluating processor architectures and highlighted a critical architectural flaw in the microprocessors that would soon dominate the computing industry.

The Microcomputer Era and the x86 Challenge (1980s to 1990s)

During the 1980s and early 1990s, the computing industry shifted away from centralized mainframes toward the client server model, powered by inexpensive microprocessors like the Intel x86 family and the Motorola 68000 series. Because this commodity hardware was cheap, organizations adopted a strict one application per server model. Operating systems of the era, such as Microsoft Windows NT, Novell NetWare, and OS/2, were designed with the assumption that they possessed absolute, exclusive control over the physical hardware. Consequently, virtualization was largely ignored in the microcomputer space; the performance overhead of a hypervisor was deemed unnecessary when physical machines were highly affordable.

However, as data centers grew rapidly during the dot-com boom, this model created severe physical and economic constraints. Servers were vastly underutilized, often running at 5 to 15 percent of their total CPU capacity. Data centers ran out of floor space, power, and cooling capacity. The industry desperately needed to consolidate these workloads, but virtualizing the x86 architecture^[7] presented a massive technical hurdle.^[22]

The x86 architecture, originating with the Intel 8086 and evolving through the 80286 and 80386 lines, failed the Popek and Goldberg theorem. The architecture utilized a ring-based privilege system to protect memory and hardware, with Ring 0 reserved for the operating system kernel and Rings 1 through 3 for device drivers and user applications. To virtualize x86, the VMM needed to run in Ring 0, forcing the guest operating system into a less privileged ring, a technique known as ring deprivileging.

The problem was that 17 instructions in the x86 instruction set were sensitive but not privileged. Instructions like POPF (Pop to Flags), PUSHF (Push Flags), SGDT (Store Global Descriptor Table), and SIDT (Store Interrupt Descriptor Table) behaved differently depending on the privilege ring. For example, if a guest operating system in Ring 1 attempted to disable interrupts using a sensitive instruction, the x86 hardware would not trap to the hypervisor in Ring 0. Instead, it would silently ignore the command, leading the guest operating system to assume interrupts were disabled when they were not, inevitably crashing the virtual machine.

Early attempts to run multiple operating systems on x86 relied on full software emulation, such as Connectix Virtual PC^[8] and Insignia SoftPC. Emulators translated every single instruction through software, which bypassed the architectural flaws but resulted in severe performance degradation. This made emulation entirely unsuitable for enterprise server environments, leaving the x86 platform fundamentally unvirtualizable by traditional standards.

The x86 Renaissance and Binary Translation (1998 to 2001)

The solution to the x86 virtualization problem emerged in the late 1990s. In 1998, Diane Greene, Mendel Rosenblum, Scott Devine, Edward Wang, and Edouard Bugnion founded VMware^[9]. They developed a software-based approach to virtualize the x86 architecture without the massive performance penalty of full emulation, effectively bypassing the hardware limitations.

VMware breakthrough was dynamic binary translation. The VMware hypervisor allowed the vast majority of safe, non-sensitive instructions to execute directly on the physical CPU, satisfying Popek and Goldberg efficiency requirement. However, the hypervisor dynamically intercepted the instruction stream, identified the 17 problematic non-virtualizable instructions, and translated them on the fly into safe, virtualizable instruction sequences. The VMM analyzed the guest code in basic blocks, translated the sensitive instructions into safe equivalents that trapped to the VMM correctly, and stored the translated code in a translation cache to ensure that subsequent executions incurred minimal overhead.

In May 1999, the company released VMware Workstation, a Type 2 hypervisor that ran on top of a host operating system like Windows or Linux.^[23] This proved the binary translation technology worked, but enterprise consolidation required a more efficient, dedicated architecture.

In 2001, VMware released ESX Server, a Type 1 bare-metal hypervisor.^[23] Unlike a Type 2 hypervisor, ESX Server installed directly on the physical hardware, eliminating the overhead of a host operating system. It provided a dedicated, highly optimized kernel, known as the vmkernel, to manage CPU scheduling, memory paging, and disk I/O among multiple concurrent virtual machines. This release triggered a massive wave of server consolidation across the IT industry. Companies could shrink their physical footprint by running dozens of virtual servers on a single physical host. In 2003, VMware introduced vMotion, a technology that allowed a running virtual machine to be moved from one physical server to another without downtime, fundamentally changing how data center maintenance and load balancing were performed.

Paravirtualization and Hardware Assistance (2003 to 2008)

While dynamic binary translation solved the x86 problem, it still incurred computational overhead, particularly for memory management and system calls. Researchers at the University of Cambridge sought a faster alternative, leading to the creation of the Xen^[10] project. In 2003, Paul Barham and his colleagues published "Xen and the Art of Virtualization"^[11] at the ACM Symposium on Operating Systems Principles.

Xen introduced paravirtualization to the x86 ecosystem. Instead of tricking an unmodified guest operating system into thinking it was running on bare metal, paravirtualization required modifying the guest operating system kernel to be explicitly aware of the hypervisor. The problematic x86 instructions were replaced with hypercalls, direct software requests to the Xen hypervisor, analogous to system calls made by user applications to an operating system. Xen utilized a specialized management virtual machine called Domain 0 (Dom0) to handle hardware drivers and administrative tasks, while user virtual machines ran in Domain U (DomU). This eliminated the need for binary translation and achieved near-native performance. However, it required access to the guest operating system source code, making it highly effective for open-source systems like Linux but difficult for proprietary systems like Microsoft Windows.

Recognizing that virtualization was now a permanent fixture of the data center, hardware vendors finally updated the x86 architecture to support it natively. In November 2005, Intel introduced Intel Virtualization Technology^[12] (VT-x, codenamed Vanderpool). VT-x added a new set of instructions called Virtual Machine Extensions (VMX) and two new operational modes: VMX root operation for the hypervisor and VMX non-root operation for the guest. It also introduced the Virtual Machine Control Structure (VMCS), a data structure stored in memory that manages the state of the guest and host during transitions (VM entries and VM exits). This allowed unmodified guest operating systems to run in Ring 0 within the non-root mode, causing the problematic instructions to trap cleanly to the hypervisor.

AMD followed in 2006 with AMD Virtualization (AMD-V, codenamed Pacifica), utilizing a similar structure called the Virtual Machine Control Block (VMCB). While CPU virtualization was solved, memory virtualization remained a bottleneck due to the overhead of maintaining shadow page tables in software. AMD introduced RVI in 2007 with its Barcelona processors, while Intel introduced EPT in 2008.^[13] SLAT moved memory virtualization from software to hardware, drastically reducing the performance overhead of virtual machines and making hardware-assisted virtualization the undisputed industry standard.

The Rise of Cloud Computing and the Hypervisor Wars (2006 to 2010s)

The ability to decouple operating systems from physical hardware laid the foundation for cloud computing. If a virtual machine was simply a set of files managed by a hypervisor, it could be provisioned, duplicated, backed up, and migrated across a network in seconds via an application programming interface (API).

In August 2006, Amazon Web Services (AWS) launched the beta of Amazon Elastic Compute Cloud (EC2)^[14]. Initially built on the Xen hypervisor, EC2 allowed developers to rent virtual machines (called instances) by the hour, starting with the m1.small instance type. This shifted IT infrastructure from a capital expenditure (buying physical servers) to an operational expenditure (renting virtual capacity), fundamentally altering the economics of the software industry.

As cloud computing grew, the hypervisor market became highly competitive. Avi Kivity developed KVM^[15] in mid-2006, and it was merged into the mainline Linux kernel in early 2007. Merged into the mainline Linux kernel in version 2.6.20, KVM turned the Linux kernel itself into a Type 1 hypervisor.^[24] Because KVM was integrated directly into Linux, it benefited from all the standard kernel features, including memory management, process scheduling, and hardware drivers, without requiring a separate, monolithic hypervisor layer. KVM operated in kernel space, while user-space programs, primarily QEMU (Quick Emulator), utilized the /dev/kvm interface to set up the guest virtual machine address space and feed it instructions. QEMU handled the emulation of hardware devices such as disk controllers and network cards, while KVM executed the guest code directly on the host CPU using the hardware virtualization extensions. This architecture made KVM highly efficient and modular. It eventually became the default virtualization technology for many open-source cloud platforms, including OpenStack, and replaced Xen as the underlying engine for AWS EC2 instances.

Simultaneously, Microsoft recognized the strategic necessity of owning the virtualization layer in Windows environments. In 2008, Microsoft released Hyper-V^[16] (originally codenamed Viridian) as a standalone product and as a feature within Windows Server 2008. Hyper-V utilized a microkernelized architecture, where the hypervisor ran directly on the hardware and the host operating system ran within a special management partition called the parent partition. This parent partition hosted the virtualization stack and provided hardware drivers for the child partitions (the virtual machines). The introduction of Hyper-V created a fierce rivalry with VMware, driving rapid innovation in live migration, high availability, and dynamic memory allocation across the industry.

Operating System-Level Virtualization and Containers (1979 to Present)

While hardware virtualization partitioned physical servers into multiple virtual machines, each virtual machine still required its own complete operating system. This resulted in significant overhead, as each guest OS consumed memory, CPU cycles, and storage space just to maintain its own kernel and background processes. To address this inefficiency, the industry developed operating system-level virtualization, commonly known as containerization.

The conceptual roots of containerization date back to 1979 with the introduction of the chroot system call in Version 7 Unix. The chroot command allowed a system administrator to change the apparent root directory for a specific running process and its children, isolating them from the rest of the file system. While primarily a security mechanism, it introduced the idea of process isolation.

In 2000, Poul-Henning Kamp and Robert Watson significantly advanced this concept by introducing FreeBSD Jails^[17]. Jails expanded upon chroot by isolating not just the file system, but also the process ID namespace, the network subsystem, and user accounts. A process running inside a jail could not interact with processes outside of it, providing a lightweight, secure environment that behaved like a separate machine but shared the same underlying FreeBSD kernel.

Sun Microsystems followed in 2005 with Solaris Zones^[18], which provided a similar application container environment for the Solaris operating system. Zones allowed administrators to partition a single Solaris instance into multiple isolated execution environments, each with its own virtual network interfaces and resource limits, without the overhead of a hypervisor.

In the Linux ecosystem, containerization matured with the development of Linux Containers (LXC) in 2008. LXC utilized two critical Linux kernel features: namespaces and cgroups (control groups). Namespaces provided isolation by ensuring that a group of processes could only see their own specific network interfaces, process IDs, and mount points. Control groups, originally developed by engineers at Google, provided resource management by limiting and monitoring the CPU, memory, and disk I/O usage of a group of processes.