Skip to content

utils

llm_impacts(provider, model_name, output_token_count, request_latency, electricity_mix_zone='WOR')

High-level function to compute the impacts of an LLM generation request.

Parameters:

Name Type Description Default
provider str

Name of the provider.

 required
model_name str

Name of the LLM used.

 required
output_token_count int

Number of generated tokens.

 required
request_latency float

Measured request latency in seconds.

 required
electricity_mix_zone Optional[str]

ISO 3166-1 alpha-3 code of the electricity mix zone (WOR by default).

 'WOR'

Returns:

Type Description
Optional[Impacts]

The impacts of an LLM generation request.
Source code in ecologits/tracers/utils.py 
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
def llm_impacts(
    provider: str,
    model_name: str,
    output_token_count: int,
    request_latency: float,
    electricity_mix_zone: Optional[str] = "WOR",
) -> Optional[Impacts]:
    """
    High-level function to compute the impacts of an LLM generation request.

    Args:
        provider: Name of the provider.
        model_name: Name of the LLM used.
        output_token_count: Number of generated tokens.
        request_latency: Measured request latency in seconds.
        electricity_mix_zone: ISO 3166-1 alpha-3 code of the electricity mix zone (WOR by default).

    Returns:
        The impacts of an LLM generation request.
    """

    model = models.find_model(provider=provider, model_name=model_name)
    if model is None:
        logger.debug(f"Could not find model `{model_name}` for {provider} provider.")
        return None

    if model.architecture.type == ArchitectureTypes.MOE:
        model_total_params = model.architecture.parameters.total
        model_active_params = model.architecture.parameters.active
    else:
        model_total_params = model.architecture.parameters
        model_active_params = model.architecture.parameters

    electricity_mix = electricity_mixes.find_electricity_mix(zone=electricity_mix_zone)
    if electricity_mix is None:
        logger.debug(f"Could not find electricity mix `{electricity_mix_zone}` in the ADEME database")
        return None
    if_electricity_mix_adpe=electricity_mix.adpe
    if_electricity_mix_pe=electricity_mix.pe
    if_electricity_mix_gwp=electricity_mix.gwp
    return compute_llm_impacts(
        model_active_parameter_count=model_active_params,
        model_total_parameter_count=model_total_params,
        output_token_count=output_token_count,
        request_latency=request_latency,
        if_electricity_mix_adpe=if_electricity_mix_adpe,
        if_electricity_mix_pe=if_electricity_mix_pe,
        if_electricity_mix_gwp=if_electricity_mix_gwp,
    )