DocumentationArchitecture

Observation

A zoomed-in view of one operation in Piro’s stateful inference loop.

Architecture detail

Observation

The current multimodal input presented to Piro’s stateful model. PiroInput is the implementation-level data object behind this boundary.

← Full model

PiroInput = Observation()

Proposed request contract

The serving adapter identifies which external state value to continue. The model itself receives the value, not the key used to store it.

POST /v1/sessions/{session_id}/observe

{
  "parts": [
    { "type": "text", "text": "What is happening here?" },
    { "type": "image", "uri": "blob://...", "mime_type": "image/png" }
  ],
  "metadata": {
    "source": "ios",
    "captured_at": "2026-07-22T12:00:00Z"
  }
}

Accepted parts

texttyped user input

imagephoto, screenshot, camera frame

audiospeech or sound recording

videoshort temporal visual input

filePDF, code, or document

jsonstructured environment data

← Back to architecture