RmjF7@dZddlmZddlmZddlmZddlmZddlm Z ddl m Z m Z ddl m Zdd l mZdd l mZdd l mZdd l mZdd l mZerddlmZddl mZddlmZmZdddddddd-dZddddddd.d#Z dddddddd$d/d'Z!dddddddd$d0d)Z"d*Z#d+Z$gd,Z%dS)1aLLM lifecycle helpers for non-streaming and streaming calls. This module is the LLM analogue of ``nemo_relay.tools``. It manages emitted events, global middleware, optional request codecs for annotated intercepts, and optional response codecs for structured end-event annotations. Example:: import nemo_relay request = nemo_relay.LLMRequest( {}, {"messages": [{"role": "user", "content": "hello"}], "model": "demo-model"}, ) async def impl(req): return {"id": "r1", "choices": [{"message": {"role": "assistant", "content": "hi"}}]} result = await nemo_relay.llm.execute( "demo-provider", request, impl, response_codec=nemo_relay.codecs.OpenAIChatCodec(), ) ) annotations)Mapping)datetime) TYPE_CHECKING)ensure_scope_stack) LLMRequest LlmStream)llm_call) llm_call_end)llm_call_execute)llm_conditional_execution)llm_request_intercepts)llm_stream_call_execute)Json)AnnotatedLLMResponse)LlmCodecLlmResponseCodecNhandle attributesdatametadata model_name timestampnamestrrequestrr str | Nonerdatetime | Nonec Ltt||||||||S)a(Start a manual LLM span and return its ``LLMHandle``. Args: name: Provider or logical call name recorded on emitted events. request: Raw ``LLMRequest`` to associate with the call. handle: Optional parent scope handle. When omitted, the current scope becomes the parent. attributes: Optional native LLM attributes attached to the start event. data: Optional JSON application payload stored on the LLM handle. metadata: Optional JSON metadata recorded on the emitted start event. model_name: Optional normalized model name to record separately from the provider-specific request payload. timestamp: Optional timezone-aware ``datetime`` recorded as the handle start time and on the emitted start event. When omitted, the current runtime time is used. Returns: LLMHandle: Handle used to finish the manual span with ``call_end()``. Notes: This starts only the manual LLM lifecycle span. It applies sanitize-request guardrails to the emitted start-event payload but does not run request or execution intercepts. ``timestamp`` must be a timezone-aware ``datetime``; strings and naive datetimes are rejected. Example:: import nemo_relay request = nemo_relay.LLMRequest({}, {"messages": [], "model": "demo-model"}) handle = nemo_relay.llm.call( "demo-provider", request, handle=None, attributes=None, data={"attempt": 1}, metadata={"path": "manual"}, model_name="demo-model", ) nemo_relay.llm.call_end( handle, {"ok": True}, data={"cached": False}, metadata={"status": "success"}, ) r)r_native_llm_call)rrrrrrrrs V/home/thesage/.hermes/hermes-agent/venv/lib/python3.11/site-packages/nemo_relay/llm.pycallr#Bs@r       rrannotated_responseresponse_codecrr&0AnnotatedLLMResponse | Mapping[str, Json] | Noner'LlmResponseCodec | NonereturnNonec Jtt|||||||S)aFinish a manual LLM span started by ``call()``. Args: handle: LLM handle returned by ``call()``. response: Raw JSON-compatible response to record on the end event. data: Optional JSON payload used when the sanitized ``response`` is JSON null. metadata: Optional JSON metadata recorded on the emitted end event. annotated_response: Optional normalized response annotation attached to the emitted end event. Accepts an ``AnnotatedLLMResponse`` returned by a codec, or a JSON-compatible mapping matching that schema. response_codec: Optional response codec used to derive ``annotated_response`` from the sanitized end-event payload for observability. Ignored when ``annotated_response`` is provided. timestamp: Optional timezone-aware ``datetime`` recorded on the emitted end event. When omitted, the runtime default end timestamp is used. Returns: None: This function returns after the end event has been recorded. Notes: ``call_end()`` applies sanitize-response guardrails to the emitted end-event payload. ``response_codec`` and ``annotated_response`` enrich observability output only and do not rewrite the recorded response. Response codec failures are raised after the end event is emitted without an annotation. ``timestamp`` must be a timezone-aware ``datetime``; strings and naive datetimes are rejected. r%)r_native_llm_call_end)rresponserrr&r'rs r"call_endr/s=L  -%   r$rrrrrcodecr'r1LlmCodec | Nonec Ptt||||||||||  S)a Run an LLM call through the managed middleware pipeline. Pipeline order: 1. LLM conditional-execution guardrails 2. LLM request intercepts 3. LLM sanitize-request guardrails for emitted start events 4. LLM execution intercepts 5. ``func(request)`` 6. LLM sanitize-response guardrails for emitted end events Args: name: Provider or logical call name recorded on emitted events. request: Raw ``LLMRequest`` passed through guardrails, intercepts, and then into ``func``. func: Provider callback invoked as ``func(request)`` after middleware has finished processing the request. handle: Optional parent scope handle. When omitted, the current scope becomes the parent. attributes: Optional native LLM attributes attached to the start event. data: Optional JSON application payload stored on the managed LLM handle. metadata: Optional JSON metadata recorded on the emitted start event. model_name: Optional normalized model name to record separately from the provider-specific request payload. codec: Optional request codec used to provide ``AnnotatedLLMRequest`` values to request intercepts. response_codec: Optional response codec used to attach a normalized response to the emitted ``LLMEnd`` event for observability. Returns: Json: The raw JSON-compatible value returned by ``func`` or by an execution intercept. Notes: ``codec`` enables annotated request intercepts. ``response_codec`` decodes the raw response for observability only and does not change the value returned to the caller. Example:: import nemo_relay request = nemo_relay.LLMRequest( {}, {"messages": [{"role": "user", "content": "hi"}], "model": "demo-model"}, ) async def impl(req): return {"id": "r1", "choices": [{"message": {"role": "assistant", "content": "hello"}}]} result = await nemo_relay.llm.execute( "demo-provider", request, impl, handle=None, attributes=None, data={"path": "managed"}, metadata={"request_id": "req-1"}, model_name="demo-model", codec=None, response_codec=nemo_relay.codecs.OpenAIChatCodec(), ) r0)r_native_llm_call_execute) rrfuncrrrrrr1r's r"executer6sFX #   %    r$r cTtt|||||||||| | |  S)a Run a streaming LLM call through the managed middleware pipeline. Args: name: Provider or logical call name recorded on emitted events. request: Raw ``LLMRequest`` passed through guardrails and intercepts. func: Provider callback invoked as ``func(request)`` that yields raw JSON chunks. collector: Callback invoked for each chunk after streaming intercepts run. It typically accumulates state for ``finalizer``. finalizer: Callback invoked after the stream completes to build the final JSON-compatible response recorded on the ``LLMEnd`` event. handle: Optional parent scope handle. When omitted, the current scope becomes the parent. attributes: Optional native LLM attributes attached to the start event. data: Optional JSON application payload stored on the managed LLM handle. metadata: Optional JSON metadata recorded on the emitted start event. model_name: Optional normalized model name to record separately from the provider-specific request payload. codec: Optional request codec used to provide ``AnnotatedLLMRequest`` values to request intercepts. response_codec: Optional response codec used to attach a normalized final response to the emitted ``LLMEnd`` event for observability. Returns: LlmStream: Async iterator that yields the streamed JSON chunks. Notes: ``collector`` observes the post-intercept chunk values. ``finalizer`` runs once at natural stream completion or explicit close and should return a representation of the full response, not the final chunk. If the caller stops consuming early, call ``await stream.aclose()`` to cancel the producer, finalize the partial response, and release native stream resources. Example:: import nemo_relay request = nemo_relay.LLMRequest( {}, {"messages": [{"role": "user", "content": "hi"}], "model": "demo-model"}, ) collected = [] async def impl(req): yield {"token": "hel"} yield {"token": "lo"} def collect(chunk): collected.append(chunk) def finalize(): return {"text": "".join(chunk["token"] for chunk in collected)} stream = await nemo_relay.llm.stream_execute( "demo-provider", request, impl, collect, finalize, handle=None, attributes=None, data={"path": "stream"}, metadata={"request_id": "req-2"}, model_name="demo-model", codec=None, response_codec=None, ) async for chunk in stream: print(chunk) r0)r_native_llm_stream_call_execute) rrr5 collector finalizerrrrrrr1r's r"stream_executer;sLl *   %    r$c>tt||S)a&Apply global LLM request intercepts to ``request``. Args: name: Provider or logical call name used when evaluating intercepts. request: Raw ``LLMRequest`` to pass through the registered request intercept chain. Returns: LLMRequestInterceptOutcome: The complete request, annotation, and pending-mark outcome produced by the intercept chain. Notes: This runs only the request-intercept chain. It does not execute guardrails, codecs, provider callbacks, or stream handling. )r_native_llm_request_intercepts)rrs r"request_interceptsr>|s  )$ 8 88r$c<tt|S)aRun LLM conditional-execution guardrails for ``request``. Args: request: Raw ``LLMRequest`` to validate against registered conditional-execution guardrails. Returns: str | None: A rejection message if execution should be blocked, otherwise ``None``. Notes: This helper evaluates only conditional-execution guardrails and does not invoke request intercepts, codecs, or provider execution. )r!_native_llm_conditional_execution)rs r"conditional_executionrAs ,W 5 55r$)r#r/r6r;r>rA)rrrrrrrr)r&r(r'r)rrr*r+) rrrrrrr1r2r'r)) rrrrrrr1r2r'r)r*r )&__doc__ __future__rcollections.abcrrtypingrnemo_relay._contextrnemo_relay._nativerr r r!r r-r r4r r@rr=rr8 nemo_relayrrnemo_relay.codecsrrr#r/r6r;r>rA__all__r$r"rLs4#"""""###### 222222=777777<<<<<<<<  !!%CCCCCCT  KO.2!%//////n  !!.2XXXXXXD  !!.2ddddddN999(666&   r$