The short answer
Quick answer: A computer usually has far more runnable tasks than CPU cores. The operating system's scheduler shares the cores among them by letting each task run for a short time slice, typically a few milliseconds, then pausing it and running another. A hardware timer forces the switch, so no program can hog the CPU. The scheduler chooses who runs next using priorities and a fairness rule: tasks that have had less CPU time recently go first, which keeps interactive programs responsive.
The problem
Open a task manager and you will see hundreds of threads. A laptop might have eight cores. Each core can run exactly one thread at any instant, so something has to decide:
- Which thread runs on which core?
- For how long?
- What happens when a more important thread wakes up?
The scheduler makes these decisions thousands of times per second. Technically it schedules threads, not whole programs; see processes vs threads.
Thread states
Each thread is in one of a few states:
| State | Meaning |
|---|---|
| Running | Currently executing on a core |
| Ready (runnable) | Could run, waiting for a core |
| Blocked (sleeping) | Waiting for something: disk, network, a timer, a lock, a key press |
Most threads are blocked most of the time. Your text editor spends nearly all its life waiting for you to type. The scheduler only chooses among the ready threads.
Preemption: nobody gets to hog the CPU
Early systems used cooperative multitasking: a program ran until it volunteered to give up the CPU. One badly written program could freeze the whole machine.
Modern systems use preemptive multitasking. A hardware timer interrupts the CPU at regular intervals. On each interrupt the kernel regains control and can decide to switch threads, whether the running program likes it or not.
A thread also gives up the CPU voluntarily whenever it blocks, for example by making a system call that has to wait for the disk.
The context switch
Switching from thread A to thread B means:
- Save A's CPU registers, program counter and stack pointer.
- Pick B.
- If B belongs to a different process, switch to its memory mapping.
- Restore B's saved registers and continue where it left off.
The switch itself takes on the order of a microsecond. The larger cost is indirect: B's data is probably not in the CPU cache, so it runs slowly at first. That is why schedulers avoid switching more often than necessary.
What makes a good scheduler
Schedulers juggle goals that pull against each other:
- Responsiveness. A key press or mouse click should be handled immediately.
- Throughput. Get as much total work done as possible, which favours fewer switches.
- Fairness. Nobody should starve.
- Energy. Idle cores should sleep, and small tasks should run on efficient cores.
A video encoder wants long uninterrupted runs. A text editor wants a few microseconds of CPU immediately. The same policy must serve both.
Classic algorithms
| Algorithm | Idea | Weakness |
|---|---|---|
| First come, first served | Run tasks in arrival order to completion | A long task makes everyone wait |
| Shortest job first | Run the quickest task first | Needs to know the future; long tasks can starve |
| Round robin | Give each task a fixed time slice in turn | Treats interactive and batch tasks the same |
| Priority scheduling | Always run the highest priority task | Low priority tasks can starve |
| Multilevel feedback queue | Demote tasks that use full slices, promote those that block often | Many tuning knobs |
The multilevel feedback queue captures an important insight: tasks that often block are probably interactive, so they should run quickly when they wake. Tasks that always use their full slice are CPU-bound and can wait a little.
How Linux does it
For many years Linux used the Completely Fair Scheduler (CFS). Its idea is simple: track how much CPU time each task has received, as a value called virtual runtime, and always run the task with the least. Tasks are kept in a sorted tree so the next one is found quickly. The kernel's CFS documentation describes it as modelling an "ideal, precise multi-tasking CPU".
An interactive task that sleeps most of the time accumulates little virtual runtime, so when it wakes it naturally jumps the queue.
Since kernel 6.6, Linux has been moving to EEVDF (Earliest Eligible Virtual Deadline First). It keeps the fairness accounting but also gives each task a virtual deadline, which lets latency-sensitive tasks be served sooner without extra heuristics. The EEVDF documentation explains the details.
Priorities and "nice"
On Unix-like systems each task has a nice value from -20 to 19. A higher value means the task is "nicer" to others and gets a smaller share of CPU. It is a weight, not a strict ranking: a nice 19 task still runs, just less.
nice -n 10 ./long-backup.sh # start with lower priority
renice -n 5 -p 12345 # change a running process
Separate real-time scheduling classes exist for audio, robotics and similar work, where a task must run within a deadline. These always take precedence over normal tasks.
Multiple cores
With several cores, the scheduler keeps a run queue per core and balances load between them. Two extra considerations appear:
- Cache affinity. A thread runs faster on the core it used last, because its data may still be cached there. Schedulers avoid moving threads unnecessarily.
- Different core types. Many modern chips mix fast performance cores with slower efficient ones. The scheduler tries to put demanding work on the fast cores and background work on the efficient ones.
In containers and cloud systems, control groups add another layer, limiting how much CPU a whole group of processes may use.
Frequently asked questions
What is a time slice?
The maximum time a thread may run before the scheduler considers switching to another. It is usually a few milliseconds and often varies with load.
Why is my computer responsive even at 100% CPU?
Fair schedulers favour tasks that have used little CPU recently. Your interactive apps mostly sleep, so they get served promptly when they wake, even while a heavy job uses the rest.
What does CPU usage of 100% mean?
That there was always at least one ready thread for each core during the measurement period. It says nothing about whether the work was useful.
Can I control which core a program uses?
Yes. This is called CPU affinity (taskset on Linux). It is occasionally useful for benchmarking or latency-critical software, but the scheduler usually does better on its own.
Conclusion
The scheduler creates the illusion that everything runs at once by switching between tasks faster than you can notice. Preemption guarantees no program can take over, fairness accounting keeps interactive tasks snappy, and cache-aware placement keeps the switching cheap. It is one of the busiest and least visible parts of any operating system.
Related articles
- Processes vs Threads: What's the Real Difference?
- What Actually Happens When You Run a Program
- What Is a System Call and Why Is It Expensive?
- How the CPU Cache Makes Code Fast (and How to Write Cache-Friendly Code)
Sources and further reading
