huggingface_tracer
ChatCompletionOutput(impacts=None)
dataclass
Bases: ChatCompletionOutput
Wrapper of huggingface_hub.ChatCompletionOutput
with ImpactsOutput
ChatCompletionStreamOutput(impacts=None)
dataclass
Bases: ChatCompletionStreamOutput
Wrapper of huggingface_hub.ChatCompletionStreamOutput
with ImpactsOutput
HuggingfaceInstrumentor()
Instrumentor initialized by EcoLogits to automatically wrap all HuggingFace Hub calls
Source code in ecologits/tracers/huggingface_tracer.py
huggingface_chat_wrapper(wrapped, instance, args, kwargs)
Function that wraps a HuggingFace Hub answer with computed impacts
Parameters:
Name | Type | Description | Default |
---|---|---|---|
wrapped |
Callable
|
Callable that returns the LLM response |
required |
instance |
InferenceClient
|
Never used - for compatibility with |
required |
args |
Any
|
Arguments of the callable |
required |
kwargs |
Any
|
Keyword arguments of the callable |
required |
Returns:
Type | Description |
---|---|
Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]
|
A wrapped |
Source code in ecologits/tracers/huggingface_tracer.py
huggingface_async_chat_wrapper(wrapped, instance, args, kwargs)
async
Function that wraps a HuggingFace Hub answer with computed impacts in async mode
Parameters:
Name | Type | Description | Default |
---|---|---|---|
wrapped |
Callable
|
Async callable that returns the LLM response |
required |
instance |
AsyncInferenceClient
|
Never used - for compatibility with |
required |
args |
Any
|
Arguments of the callable |
required |
kwargs |
Any
|
Keyword arguments of the callable |
required |
Returns:
Type | Description |
---|---|
Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]
|
A wrapped |