You spend an undergraduate semester writing #pragma omp parallel for on a Sun UltraSPARC, thinking it’s a relic you’ll never touch again. Twenty-five years later, you’re deploying ONNX Runtime on Linux — and OpenMP is still there, quietly running half the numerical world.

This happened to me last week.

I was debugging a CPU-bound ONNX inference pipeline on a Linux server. The model was a straightforward BERT variant — nothing exotic. But the CPU usage pattern caught my eye: 3200% on a 32-core machine. That’s not a coincidence. That’s parallel dispatch.

A quick ldd on the ONNX Runtime shared libraries confirmed it. libiomp5.so (Intel’s OpenMP runtime). Then libgomp.so.1 (GCC’s OpenMP runtime). Then OpenBLAS with its own OpenMP linkage. Three different OpenMP runtimes in one process, all trying to manage thread pools independently.

I stared at the terminal for a moment. The last time I wrote an OpenMP pragma was 1998.

The 90s Flashback

Our Parallel Computing course in the late 90s was taught on Sun UltraSPARC workstations — those beige towers with the Sun logo that weighed as much as a small car. The lab had maybe a dozen of them, and we’d book time slots like it was a mainframe. We wrote C programs that multiplied matrices, computed Mandelbrot sets, and ran N-body simulations. The magic incantation was always the same:

#pragma omp parallel for
for (int i = 0; i < N; i++) {
    /* ... do the thing ... */
}

At the time, parallelism felt exotic. A 4-core machine was a big deal. OpenMP was the easy way to make use of those cores without becoming a pthreads expert. We learned it, used it for the semester, and moved on. I assumed it was a teaching tool — something universities used because it was simple enough for undergraduates.

I never encountered it professionally. Somewhere between telecom software, enterprise .NET systems and data centre infrastructures. OpenMP simply vanished from my world. Not because parallelism disappeared—it was everywhere—but because it was increasingly handled by frameworks, runtimes, and libraries. Threading was always someone else’s problem - the problems we worked on moved higher up the stack, and OpenMP became one of those technologies I remembered from university and quietly assumed had been left behind.

I was wrong about OpenMP. It wasn’t a teaching tool that faded away. It was a foundation that everyone kept building on.

Why OpenMP Still Matters

OpenMP solves one problem better than anything that came after it: shared-memory parallelism with minimal ceremony.

If you have a CPU with 8–128 cores, a large loop, and work that is embarrassingly parallel, the problem has always been:

“How do I split this across threads without rewriting my program?”

OpenMP’s answer is a single line:

#pragma omp parallel for
for (int i = 0; i < N; i++) {
    work(i);
}

That’s it. The compiler and runtime handle thread creation, work distribution, load balancing, and cleanup. The engineer describes the what. The implementation handles the how.

This ratio — productivity to performance — is why OpenMP survived. Not because it’s elegant (though it is in its own way), but because a single pragma can turn a single-core program into a 128-core program without restructuring the code.

Why Nothing Replaced It

Many things tried. Here is what happened to each:

Technology What happened
POSIX Threads Too low-level. pthread_create/pthread_join everywhere is how you write bugs, not programs. OpenMP sits on top of thread runtimes and gives a better model.
Intel TBB / oneTBB C++ only, not ISO standardized. Scientific computing already had massive OpenMP codebases. Became complementary, not a replacement.
C++ Threads / std::async Great for task parallelism. Terrible for “I have 100 million iterations” data parallelism. OpenMP still wins on ease of expressing parallel loops.
Cilk Plus Academically loved. Intel abandoned it. OpenMP absorbed its ideas (tasking, work stealing).
Go goroutines Beautiful model. Different language. Not dropping into a million-line Fortran climate model.
Rust / Rayon Strongest contender. Arguably more elegant than OpenMP. But Rust is young in HPC, and scientific computing has enormous C/C++/Fortran investments that work today.

The pattern is clear. New languages and frameworks solve parallel programming beautifully — for their own ecosystems. They cannot replace the accumulated decades of C, C++, and Fortran code running on every supercomputer and server room on the planet.

The Real Reason OpenMP Survives

The HPC ecosystem is conservative for good economic reasons.

A weather model may cost billions to develop, run on supercomputers for twenty years, and contain millions of lines of Fortran. When a team responsible for that code asks about parallelism, they do not ask “what is the coolest threading framework?” They ask “can I get 20% more performance without rewriting 5 million lines?”

OpenMP is excellent at that.

The question is not technical preference. It is capital preservation. OpenMP lets you incrementally parallelize existing code — one pragma at a time, one file at a time, one subroutine at a time — without changing the architecture.

The 2026 OpenMP Is Not Your 90s OpenMP

If you think OpenMP is still just #pragma omp parallel for, you are missing twenty years of evolution.

The modern spec includes:

#pragma omp simd                      // Vectorization hint (pure SIMD)
#pragma omp parallel for simd         // Threads + SIMD combined
#pragma omp task                      // Dynamic task spawning
#pragma omp target teams distribute   // GPU offloading

Here is a progression that shows how far it has come:

Scalar:

for (int i = 0; i < n; i++)
    y[i] = a*x[i] + y[i];

SIMD only:

#pragma omp simd
for (int i = 0; i < n; i++)
    y[i] = a*x[i] + y[i];

SIMD + multicore:

#pragma omp parallel for simd
for (int i = 0; i < n; i++)
    y[i] = a*x[i] + y[i];

Same algorithm. Three levels of performance intent. The compiler and runtime handle the hardware mapping.

And this is not just CPU anymore:

#pragma omp target teams distribute parallel for
for (int i = 0; i < n; i++)
    a[i] = b[i] + c[i];

This can run on GPUs. OpenMP quietly grew into a portable language for expressing parallel intent across CPUs, GPUs, and accelerators — all through the same pragma interface from the 90s.

The Stack You Did Not Know You Had

Here is the part that surprised me most. You do not need to use ONNX Runtime to encounter OpenMP. You had been using it for years without knowing.

A Python developer writes:

import numpy as np
C = A @ B

Looks innocent. But underneath:

Python
  ↓
NumPy
  ↓
BLAS
  ↓
OpenBLAS / MKL / BLIS
  ↓
OpenMP or pthread worker threads
  ↓
All CPU cores light up

The Python code never mentions threads, locks, synchronization, or OpenMP. Yet 32 cores may suddenly start working. The matrix multiplication escaped Python entirely and is happening inside highly optimized native code — and that native code often uses OpenMP to split work across cores.

The NumPy docs explicitly note that high-performance linear algebra is delegated to BLAS backends such as OpenBLAS or Intel MKL, and those libraries use multiple threads controlled by — you guessed it — OMP_NUM_THREADS.

Many data scientists have tuned OpenMP without realizing it:

OMP_NUM_THREADS=8 python train.py

or

export OMP_NUM_THREADS=4

That environment variable comes straight out of the OpenMP ecosystem. Imagine telling a Python ML engineer: “You configured OpenMP last year. You just did not know its name.”

The same pattern repeats across the stack. XGBoost uses OpenMP. PyTorch’s CPU backend uses OpenMP through MKL. Pandas operations that hit NumPy underneath use it. Even .NET’s TensorPrimitives — SIMD-accelerated tensor operations in the BCL — is philosophically the same thing: tell the runtime “this work is independent,” let it exploit the hardware.

OpenMP and SIMD: Workers and Shovels

A common mental model separates threading (OpenMP) from vectorization (AVX512) as if they are competing technologies. They are not. They solve different dimensions of the same problem:

OpenMP: How many workers?
AVX512: How much work per worker?

The analogy I like:

  • OpenMP = hiring more workers
  • SIMD/AVX512 = giving each worker a bigger shovel

If you only hire more workers: 32 workers, one shovel each. Good.

If you only give a bigger shovel: one worker, a 16× bigger shovel. Also good.

The real win is both: 32 workers with 16× bigger shovels. That is what modern HPC, ML inference, MKL, OpenBLAS, and CPU-based AI workloads are doing — threads × SIMD width.

A matrix multiplication today might run across 32 CPU cores via OpenMP, with each core processing 16 floats per AVX512 instruction. The effective throughput is:

Threads (32) × SIMD width (16) = 512× scalar throughput

That is not one technology winning over the other. It is both working together.

The Bridge to Modern Developers

OpenMP was doing declarative parallelism before most modern frameworks existed. Today we praise Rayon (Rust), Parallel LINQ (C#), Java Parallel Streams, TPL, ForkJoin, Dask, and Spark because they let us say “here is work” instead of “here is how to create and manage 37 worker threads.” OpenMP was already saying that in the 1990s.

Ecosystem What you write What you are really saying
OpenMP #pragma omp parallel for “Each iteration is independent.”
C# Parallel.For(...) “Split this work across cores.”
Java parallelStream() “Process these items concurrently.”
Rust par_iter() “Parallelize this collection.”
Python executor.map(...) “Run these tasks independently.”
CUDA kernel<<<...>>>() “Run this many times in parallel.”

Different ecosystems. Same idea. Describe independent work. Let the runtime schedule the workers.

A C# developer writing Parallel.For is doing the same conceptual thing as a Fortran developer writing !$OMP PARALLEL DO in 1998. The abstraction is higher, the syntax is different, but the underlying question has not changed in thirty years.

A Useful Rule of Thumb

OpenMP is not a technology. It is a promise:

If your machine gets more CPU cores tomorrow, your code can use them tomorrow.

Without OpenMP, the loop says “do one million jobs.” With OpenMP, it says “do one million independent jobs.” The difference is a single line that remains true whether you are running on a 4-core laptop from 2005 or a 128-core server in 2026.

The deeper reason it survived thirty years: CPUs kept changing. 1 core → 2 → 4 → 8 → 64 → 128. The idea that “this loop can run independently” never changed. OpenMP captured that idea in a way that is still valid today.

And enormous parts of modern computing are still standing on that stack — even when the person writing the code thinks they are “just using Python” or “just deploying ONNX Runtime.”

The Loop Closes

A young engineer using TensorPrimitives.Add(...) in 2026 is pursuing the exact same goal as an engineer writing !$OMP PARALLEL DO in Fortran in 1998:

“I have a mountain of independent work. How do I make every core and every vector unit in this CPU earn its paycheck?”

OpenMP answered the core part. SIMD/AVX512 answered the vector part. Modern frameworks are mostly sophisticated ways of combining both.

I learned OpenMP on a Sun UltraSPARC in 1998, never touched it for two decades, and rediscovered it in 2026 inside ONNX Runtime — exactly where it has been all along, quietly making things fast while higher abstractions took the credit.

The surprising thing is not that OpenMP is still here. It is that it never left.