Endpoint

(dedicated.endpoint)

Overview

Available Operations

wandb_artifact_create - Create endpoint from W&B artifact
create - Create a new endpoint
list - List all endpoints
get_spec - Get endpoint specification
update - Update endpoint spec
delete - Delete endpoint
get_version_history - Get endpoint version history
get_status - Get endpoint status
sleep - Sleep endpoint
wake - Wake endpoint
terminate - Terminate endpoint
restart - Restart endpoint

wandb_artifact_create

Create an endpoint from Weights & Biases artifact. If the idempotency key is provided, the API will check if the endpoint already exists, and rollout the existing endpoint if it does. In such cases, the project id must be provided.

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.wandb_artifact_create(
        wandb_artifact_version_name="org/registry/name:v0"
    )

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`wandb_artifact_version_name`	str	✔️	The specific model artifact version from Weights & Biases. The referred artifact will be used to create a new endpoint in Friendli Dedicated Endpoints or rollout an existing one.
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`accelerator`	OptionalNullable[models.AcceleratorRequirement]	➖	Specifies the instance type for the endpoint.
`autoscaling_policy`	OptionalNullable[models.AutoscalingPolicy]	➖	Defines autoscaling settings for the endpoint.
`idempotency_key`	OptionalNullable[str]	➖	Used by Friendli Dedicated Endpoints to track which webhook automation triggered an endpoint rollout. If the `idempotencyKey` is provided, the API will check if the endpoint already exists, and rollout the existing endpoint if it does. In such cases, the `projectId` must be provided. Any unique value can be used.
`name`	OptionalNullable[str]	➖	Specifies the name of your endpoint. If not provided, a name will be automatically generated for you.
`project_id`	OptionalNullable[str]	➖	Specifies where endpoint will be created in your Friendli Dedicated Endpoints. If not provided, a new project will be created within your default team.
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointWandbArtifactCreateResponse

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

create

Create a new endpoint and return its status

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.create(
        advanced={
            "tokenizer_add_special_tokens": False,
            "tokenizer_skip_special_tokens": False,
        },
        hf_model_repo="<value>",
        instance_option_id="<id>",
        name="<value>",
        project_id="<id>",
    )

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`advanced`	models.EndpointAdvancedConfig	✔️	Endpoint advanced config.
`hf_model_repo`	str	✔️	HF ID of the model.
`instance_option_id`	str	✔️	The ID of the instance option.
`name`	str	✔️	The name of the endpoint.
`project_id`	str	✔️	The ID of the project that owns the endpoint.
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`autoscaling_policy`	OptionalNullable[models.AutoscalingPolicy]	➖	The auto scaling configuration of the endpoint.
`hf_model_repo_revision`	OptionalNullable[str]	➖	HF commit hash of the model.
`initial_version_comment`	OptionalNullable[str]	➖	The comment for the initial version.
`simplescale`	OptionalNullable[models.EndpointSimplescaleConfig]	➖	The simple scaling configuration of the endpoint.
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

list

List all endpoint statuses

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.list(project_id="", limit=20)

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`project_id`	OptionalNullable[str]	➖	The ID of the project. If omitted, query all endpoints under the team.
`cursor`	OptionalNullable[Union[bytes, IO[bytes], io.BufferedReader]]	➖	Cursor for pagination
`limit`	OptionalNullable[int]	➖	Limit of items per page
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointListResponse

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

get_spec

Get the specification of an endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.get_spec(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointSpec

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

update

Update the specification of a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.update(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`advanced`	OptionalNullable[models.EndpointAdvancedConfig]	➖	The advanced configuration of the endpoint.
`autoscaling_policy`	OptionalNullable[models.AutoscalingPolicy]	➖	The auto scaling configuration of the endpoint.
`hf_model_repo`	OptionalNullable[str]	➖	HF ID of the model.
`hf_model_repo_revision`	OptionalNullable[str]	➖	HF commit hash of the model.
`instance_option_id`	OptionalNullable[str]	➖	The ID of the instance option.
`name`	OptionalNullable[str]	➖	The name of the endpoint.
`new_version_comment`	OptionalNullable[str]	➖	Comment for the new version.
`simplescale`	OptionalNullable[models.EndpointSimplescaleConfig]	➖	The simple scaling configuration of the endpoint.
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointSpec

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

delete

Delete a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.delete(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

Any

Errors

Error Type	Status Code	Content Type
models.HTTPValidationError	422	application/json
models.SDKError	4XX, 5XX	/

get_version_history

Get version history of a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.get_version_history(endpoint_id="<id>", limit=20)

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`cursor`	OptionalNullable[Union[bytes, IO[bytes], io.BufferedReader]]	➖	Cursor for pagination
`limit`	OptionalNullable[int]	➖	Limit of items per page
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointVersionHistoryResponse

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

get_status

Get the status of a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.get_status(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

sleep

Put a specific endpoint to sleep

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.sleep(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

wake

Wake up a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.wake(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

terminate

Terminate a specific endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.terminate(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

restart

Restart a FAILED or TERMINATED endpoint

Example Usage

import os

from friendli import SyncFriendli

with SyncFriendli(
    token=os.getenv("FRIENDLI_TOKEN", ""),
) as friendli:
    res = friendli.dedicated.endpoint.restart(endpoint_id="<id>")

    # Handle response
    print(res)

Parameters

Parameter	Type	Required	Description
`endpoint_id`	str	✔️	The ID of the endpoint
`x_friendli_team`	OptionalNullable[str]	➖	ID of team to run requests as (optional parameter).
`retries`	Optional[utils.RetryConfig]	➖	Configuration to override the default retry behavior of the client.

Response

models.DedicatedEndpointStatus

Errors

Error Type	Status Code	Content Type
models.SDKError	4XX, 5XX	/

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Endpoint

Overview

Available Operations

wandb_artifact_create

Example Usage

Parameters

Response

Errors

create

Example Usage

Parameters

Response

Errors

list

Example Usage

Parameters

Response

Errors

get_spec

Example Usage

Parameters

Response

Errors

update

Example Usage

Parameters

Response

Errors

delete

Example Usage

Parameters

Response

Errors

get_version_history

Example Usage

Parameters

Response

Errors

get_status

Example Usage

Parameters

Response

Errors

sleep

Example Usage

Parameters

Response

Errors

wake

Example Usage

Parameters

Response

Errors

terminate

Example Usage

Parameters

Response

Errors

restart

Example Usage

Parameters

Response

Errors