DocumentationArchitecture

GetAttentionShape

A zoomed-in view of one operation in Piro’s stateful inference loop.

Architecture detail

GetAttentionShape

Deferred CTM architecture: derives attention head dimensions for a specialized retrieval stack.

GetAttentionShape(weights):
    modelWidth = weights.attention.modelWidth
    headCount = weights.attention.headCount
    if modelWidth % headCount != 0:
        return Error("attention width is not divisible by head count")
    d_head = modelWidth / headCount
    return modelWidth, headCount, d_head

What it does

Reads modelWidth and headCount from weights.attention, verifies divisibility, and derives d_head = modelWidth / headCount.

Why it exists

Scaled dot-product attention must know each head’s dimensionality, so the source of d_head is explicit and reviewable.

← Back to architecture