I keep telling people Nivara isn’t an ML library. I’m not sure I believe it anymore.

Its not NumPy for .NET, not a tensor library, not a vector math library, not an embedding similarity engine and not an AutoDiff framework.

I wrote that list. And, with a great deal of help, I wrote the code the list disowns — the dot products, the cosine similarities, the norms, all of it. There was a version of me that wanted Nivara to be the thing the list says it isn’t.

This is the story of how I argued myself out of my own code.

I should be clear about who’s telling this story. I’m a C# engineer, not an ML expert. I have never derived a backprop rule from memory, and I won’t pretend otherwise. People like me avoid this math on purpose. It’s the wall that keeps seasoned engineers out of ML.

Nivara started because .NET shipped tensors. The platform put the math in the base class library, right next to List<T>, and that changed the equation for me — the math was no longer something I had to go find; it was already in the language. What I still didn’t have was a translation of the scary parts: the gradients, the loss functions, the training loops. That’s what coding agents did, the way I wrote about in LLMs as Equalizers. The math didn’t get easy. It became translatable.

The Shape

It starts with the simplest question, the one I keep circling back to.

What is a tensor?

The mathematical definition is a multilinear map over vector spaces, which I have never once needed. The practical definition is one line.

A tensor is an N-dimensional array of numbers.

That’s it. A spreadsheet is a 2D tensor. A color image is a 3D tensor — height, width, channels. A batch of images is a 4D tensor — add the batch dimension. Text is a 2D tensor of token IDs; add embeddings and it becomes three: batch, sequence, embedding.

The word doing all the work is shape.

Six numbers are six numbers. Six numbers arranged as [2, 3] read as a grid. Reshape is the operation that makes this visible.

Tensor<int> flat = Tensor.Create([1, 2, 3, 4, 5, 6], [6]);
Tensor<int> shaped = flat.Reshape(2, 3);
// [[1, 2, 3],
//  [4, 5, 6]]

Same numbers. Same memory. Different shape. Everything in ML is this negotiation with shape — what counts as a row, what counts as a channel, what counts as a batch. Get the shape wrong and the numbers mean nothing.

For years, that was the extent of my mental model, and I was fine with it. Past the shape was math I had spent a career successfully ignoring. Tensors are arrays with good public relations.

The Floor

So I built Nivara — the dataframe layer that could hand its columns to that new primitive. Tensors gave the library a reason to exist.

My first move was asking a coding agent to bring the tensor operations I kept seeing in PyTorch notebooks into .NET. A dot product. Cosine similarity. A norm. Matrix multiply. They worked. They were also a lie in a very specific way — every one of them was a wrapper around code that already existed, or was about to exist, in the .NET base class library.

.NET 9 shipped Tensor<T> and TensorPrimitives as the cornerstone data structure of AI in .NET, with SIMD-accelerated operations and zero-copy interop with ML.NET, TorchSharp, and ONNX Runtime. .NET 10 made the tensor APIs stable — no longer experimental. The platform had absorbed the exact layer we had just built.

So I did the honest thing. I annotated my own work for deletion.

[Obsolete("Use TensorPrimitives.Add on spans obtained via TryGetSpan", false)]
public static NivaraSeries<T> AddTensor<T>(this NivaraSeries<T> left, NivaraSeries<T> right)

ReLU, the simplest useful activation, stopped being a loop and became a noun.

TensorPrimitives.Max(span, T.Zero, result);

That’s the whole function. One call, SIMD-accelerated, generic over every numeric type, maintained by the same team that maintains the JIT.

But here’s the part I have to be honest about, because it’s the reason this post exists. The layer the platform absorbed was the abstraction — the tensor type, the arithmetic. And it was exactly the layer I was willing to give up, because it was the layer that never needed to be mine.

The layers below it did.

The And Yet

Then I decided Nivara needed to train models, because you cannot train a model through dataframe operations. I had never built an autograd engine. I wanted one to exist in .NET, so I asked the coding agents to translate one — reverse-mode autograd, a computation graph, Adam, convolutions, transformer blocks. About fifty files that make Nivara, in the most awkward possible way, a deep learning framework.

The first wall the engine hit was its own null handling.

Every operation carried two execution paths: one for columns with nulls, one without. HasNulls checks. WithoutNulls copies. Null-mask propagation. Forty percent of every operation’s code was the mask, and every operation had two ways to be wrong.

The fix was a decision document. ADR-001. AutoDiff is a non-nullable domain. Strip the nulls once at the boundary, before a value ever enters the graph, and every operation runs on a single path.

And the point of the decision was not elegance. It was the floor.

And this part was mine. Not because it was ML — because it was data engineering. The null mask is a storage concern, and storage is the language I actually speak.

Null-free data means TryGetSpan(out var span) always succeeds. And if you can get a span, you can hand the numbers to TensorPrimitives — SIMD, one path, no fallback branch. The null mask wasn’t just a correctness burden. It was standing between the code and the performance.

This is what a kernel looks like after that decision.

a.Data.TryGetSpan(out var aSpan);
b.Data.TryGetSpan(out var bSpan);
var result = new T[aRows * bCols];
TensorsHelper.MultiplyCore(aSpan, bSpan, result, aRows, aCols, bCols);

The tensor keeps its shape as metadata. The data lives flat, inside the column. The moment math happens, everything drops to spans.

The kernels were naive. The matmul transposes one operand and dots rows in parallel. The convolutions went through im2col. The embedding layer traded a one-hot matrix multiply for a gather, and the benchmark showed the gap. None of this math was mine — the agents translated it from PyTorch, and my job was to make it fast and make it correct. Naive — but they ran flat, on the SIMD floor.

And they gave some fight to Torch.

The parity exercise trained an identical three-layer network in Nivara and in PyTorch. The loss curves matched within 0.04%. That test was my ground truth. I can’t derive a gradient by hand — but I can compare two loss curves until I trust one. MiniLM, a real distilled BERT, runs end to end in pure C# on these kernels. Nobody would mistake it for a GPU training stack. But the gradient math is right, the optimizer is right, and the loops ride the same hardware the platform optimizes for.

The Adam optimizer in that engine rents its state buffers from ArrayPool — the exact buffers I wrote about in Borrowed Memory. The optimizer whose first training run printed NaN was already living on this floor.

Where the platform lacked an operation, I wrote it at the span level. A sigmoid hand-built from four TensorPrimitives calls, with a comment in the source that says, in effect, delete this when .NET 11 lands. It’s the same discipline as the [Obsolete] attribute, pointed the other way. The abstraction was debt. The floor was where the performance was.

What Nivara Owns

So I went back to the document, and I asked it the question the repo now opens with.

What does Nivara own that .NET 10 does not?

The answer is the table. Columns with schemas. Null masks with SQL-like semantics. Joins, grouping, query planning. The shape of the data — what the columns mean, what belongs together, what’s missing — that’s the layer nobody else is going to build for me, and it’s the layer where a dataframe earns its keep.

The tensor math is not the product. It’s interop.

using var products = NivaraFrame.Create(
    ("ProductId", NivaraColumn<string>.CreateForReferenceType(["A", "B", "C"])),
    ("Embedding", NivaraColumn<float[]>.CreateForReferenceType([
        [0.9f, 0.2f, 0.5f, 0.4f],
        [0.1f, 0.9f, 0.2f, 0.7f],
        [0.7f, 0.1f, 0.8f, 0.2f],
    ]))
);

float[] query = [0.8f, 0.1f, 0.6f, 0.3f];

var ids = products.GetColumn<string>("ProductId");
var embeddings = products.GetColumn<float[]>("Embedding");

var ranked = Enumerable.Range(0, products.RowCount)
    .Select(i => new
    {
        ProductId = ids[i],
        Score = TensorPrimitives.CosineSimilarity(embeddings[i], query)
    })
    .OrderByDescending(x => x.Score)
    .Take(3);

Nivara owns the table. The base class library owns the math. The seam between them is a span.

In LLMs as Equalizers, I argued that an idea translated across ecosystems stops being a copy and becomes its own thing. That’s what this engine is. I didn’t port PyTorch to .NET — the agents translated its intent, and it grew its own bones: columnar, null-aware, span-driven. And then the platform absorbed the tensor layer anyway.

I wrote, in The Evolution of Data Science, that ML had stopped being frontier engineering and become infrastructure. I don’t think I understood my own sentence until Nivara taught it to me. The tensor became infrastructure the moment the platform absorbed it. And that wasn’t a loss — it was what let me stop competing with the platform and start owning what it doesn’t.

A shape is metadata. A schema is a shape with opinions. Tensors are the infrastructure under the shapes — and shapes are the infrastructure under the data.

More from Nivara

More from The C# Diaries