litellm_tracer
ChatCompletion
Bases: ModelResponse
Wrapper of litellm.types.utils.ModelResponse
with ImpactsOutput
ChatCompletionChunk
Bases: ModelResponse
Wrapper of litellm.types.utils.ModelResponse
with ImpactsOutput
LiteLLMInstrumentor()
Instrumentor initialized by EcoLogits to automatically wrap all LiteLLM calls
Source code in ecologits/tracers/litellm_tracer.py
litellm_match_model(model_name)
Match according provider and model from a litellm model_name.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
model_name |
str
|
Name of the model as used in litellm. |
required |
Returns:
Type | Description |
---|---|
Optional[tuple[str, str]]
|
A tuple (provider, model_name) matching a record of the ModelRepository. |
Source code in ecologits/tracers/litellm_tracer.py
litellm_chat_wrapper(wrapped, instance, args, kwargs)
Function that wraps a LiteLLM answer with computed impacts
Parameters:
Name | Type | Description | Default |
---|---|---|---|
wrapped |
Callable
|
Callable that returns the LLM response |
required |
instance |
Completions
|
Never used - for compatibility with |
required |
args |
Any
|
Arguments of the callable |
required |
kwargs |
Any
|
Keyword arguments of the callable |
required |
Returns:
Type | Description |
---|---|
Union[ChatCompletion, CustomStreamWrapper]
|
A wrapped |
Source code in ecologits/tracers/litellm_tracer.py
litellm_async_chat_wrapper(wrapped, instance, args, kwargs)
async
Function that wraps a LiteLLM answer with computed impacts in async mode
Parameters:
Name | Type | Description | Default |
---|---|---|---|
wrapped |
Callable
|
Async callable that returns the LLM response |
required |
instance |
AsyncCompletions
|
Never used - for compatibility with |
required |
args |
Any
|
Arguments of the callable |
required |
kwargs |
Any
|
Keyword arguments of the callable |
required |
Returns:
Type | Description |
---|---|
Union[ChatCompletion, CustomStreamWrapper]
|
A wrapped |