live avatar

The resources (image, video, voice) generated by our API are valid for 7 days. Please save the relevant resources as soon as possible to prevent expiration.

Create session

POST https://openapi.akool.com/api/open/v3/liveAvatar/session/create

Request Headers

Parameter	Value	Description
Authorization	Bearer token	Your API Key used for request authorization. You can get from https://openapi.akool.com/api/open/v3/getToken api.

Body Attributes

Parameter	Type	Value	Description
avatar_id	String		Digital human model in real-time avatar, The current system provides four options: _“Kai_1080P_back”、“Katha_1080P_back”、“Olivia_1080P_back”、“who_1080P_back”. If you want to use a custom uploaded video, you need to call the https://openapi.akool.com/api/open/v3/avatar/create interface to create a template. This process takes some time to process. You can check the processing status through the interface https://openapi.akool.com/api/open/v3/avatar/detail. When status=3, you can use the avatar_id field to pass it in.
background_url	String		The user can provide a background image to replace the avatar background in the video. If it is not passed or is empty, the avatar default background will be used.

Response Attributes

Parameter	Type	Value	Description
code	int	1000	Interface returns business status code(1000:success)
msg	String		Interface returns status information
data	Object	`{ "_id": "", "status": 1, "stream_urls": { "client_pull_stream_url":"", "client_chat_room_url":"", "server_chat_room_url":"" } }`	`_id`: Interface returns data, status: the status of session： [1:queueing, 2:processing, 3:completed, 4:failed], stream_urls: the url of live avatar, client_pull_stream_url: Client streaming address, client_chat_room_url: Client websocket link address, server_chat_room_url: server websocket link address, The address suffix such as server_8898 is the object processed by the server algorithm

Example Body

{
  "_id": "6698c9d69cf7b0d61d1b6420",
  "status": 1,
  "task_id": "6698c9d69b4c131d2cf15f2d",
  "app_name": "openapi_live",
  "stream_name": "20240718075254093_8898",
  "stream_urls": {
    "client_pull_stream_url": "https://stream-open-test.akool.io:8443/final_openapi_live/20240718075254093_8898.flv", // Client streaming address
    "client_chat_room_url": "wss://stream-open-test.akool.io:8443/final_20240718075254093_8898/client_8898.chat",     // Client websocket link address
    "server_chat_room_url": "wss://stream-open-test.akool.io:8443/final_20240718075254093_8898/server_8898.chat"      // server websocket link address, The address suffix such as server_8898 is the object processed by the server algorithm
  }
}

Request

curl --location 'https://openapi.akool.com/api/open/v3/liveAvatar/session/create' \
--header 'authorization: Bearer token' \
--header 'content-type: application/json' \
--data '{
    "avatar_id": "Kai_1080P_back",
    "background_url": "https://static.website-files.org/assets/images/generator/text2image/1716867976184-c698621e-9bf3-4924-8d79-0ba1856669f2-6178_thumbnail.webp",
    "webhookUrl":"https://openapitest.akool.com/api/open/v3/test/webhook"
}'

Response

{
    "code": 1000,
    "msg": "OK",
    "data": {
        "_id": "6698c9d69cf7b0d61d1b6420",
        "uid": 100010,
        "type": 1,
        "status": 1,
        "task_id": "6698c9d69b4c131d2cf15f2d",
        "app_name": "openapi_live",
        "stream_name": "20240718075254093_8898",
        "stream_urls": {
            "client_pull_stream_url": "https://stream-open-test.akool.io:8443/final_openapi_live/20240718075254093_8898.flv",   // Client streaming address
            "client_chat_room_url": "wss://stream-open-test.akool.io:8443/final_20240718075254093_8898/client_8898.chat",   // Client websocket link address
            "server_chat_room_url": "wss://stream-open-test.akool.io:8443/final_20240718075254093_8898/server_8898.chat" // server websocket link address, The address suffix such as server_8898 is the object processed by the server algorithm
        }
    }
}

Get Session Info Result

GET https://openapi.akool.com/api/open/v3/liveAvatar/session/detail?id=6698c9d69cf7b0d61d1b6420

Request Headers

Parameter	Value	Description
Authorization	Bearer token	Your API Key used for request authorization. You can get from https://openapi.akool.com/api/open/v3/getToken api.

Query Attributes

Parameter	Type	Value	Description
id	String	NULL	video db id: You can get it based on the `_id` field returned by https://openapi.akool.com/api/open/v3/liveAvatar/session/create .

Response Attributes

Parameter	Type	Value	Description
code	int	1000	Interface returns business status code(1000:success)
msg	String	OK	Interface returns status information
data	Object	`{ status:1, _id:"", stream_urls:{} }`	status: the status of live avatar：【1:queueing, 2:processing, 3:completed, 4:failed】 stream_urls: the url of live avatar, _id: Interface returns data

Example Request

curl --location 'http://openapi.akool.com/api/open/v3/liveAvatar/session/detail?id=6698c9d69cf7b0d61d1b6420' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK",
  "data": {
    "_id": "6698c9d69cf7b0d61d1b6420",
    "uid": 100010,
    "type": 1,
    "status": 3,
    "task_id": "6698c9d69b4c131d2cf15f2d",
    "stream_urls": {
        "client_pull_stream_url": "https://stream-open-test.akool.io:8443/final_openapi_live/20240718075254093_8898.flv"
    }
  }
}

Close Session

POST https://openapi.akool.com/api/open/v3/liveAvatar/session/close

Request Headers

Parameter	Value	Description
Authorization	Bearer token	Your API Key used for request authorization. You can get from https://openapi.akool.com/api/open/v3/getToken api.

Body Attributes

Parameter	Type	Value	Description
id	String	NULL	session id: You can get it based on the `_id` field returned by https://openapi.akool.com/api/open/v3/liveAvatar/session/create .

Response Attributes

Parameter	Type	Value	Description
code	int	1000	Interface returns business status code(1000:success)
msg	String	OK	Interface returns status information

Example Request

curl --location 'http://openapi.akool.com/api/open/v3/liveAvatar/session/close' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK"
}

Pause Session

POST https://openapi.akool.com/api/open/v3/liveAvatar/session/pause

Calling this interface will pause the streaming, and the processing of data may be delayed by 0.5s to several seconds. Resending the websocket message will continue streaming. Please note that if you do not send messages for a long time after pausing, the connection will time out and you will need to restart the session.

Request Headers

Parameter	Value	Description
Authorization	Bearer token	Your API Key used for request authorization. You can get from https://openapi.akool.com/api/open/v3/getToken api.

Body Attributes

Parameter	Type	Value	Description
id	String	NULL	session id: You can get it based on the `_id` field returned by https://openapi.akool.com/api/open/v3/liveAvatar/session/create .

Response Attributes

Parameter	Type	Value	Description
code	int	1000	Interface returns business status code(1000:success)
msg	String	OK	Interface returns status information

Example Request

curl --location 'http://openapi.akool.com/api/open/v3/liveAvatar/session/pause' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK"
}

Get Session List

GET https://openapi.akool.com/api/open/v3/liveAvatar/session/list?page=1&size=10&status=1

Request Headers

Parameter	Value	Description
Authorization	Bearer token	Your API Key used for request authorization. You can get from https://openapi.akool.com/api/open/v3/getToken api.

Query Attributes

Parameter	Type	Value	Description
page	Number	1	Current number of pages，Default is 1.
size	Number	10	Current number of returns per page，Default is 100.
status	Number	NULL	session status: 【1:queueing, 2:processing, 3:completed, 4:failed】.

Response Attributes

Parameter	Type	Value	Description
code	int	1000	Interface returns business status code(1000:success)
msg	String	OK	Interface returns status information
data	Array	`{count： 1, data: [{ task_id: "xx", stream_urls: {} }] }`	task_id: task id of session. url: the url of live avatar.

Example Request

curl --location 'http://openapi.akool.com/api/open/v3/liveAvatar/session/list?page=1&size=10&status=1' \
--header 'Authorization: Bearer token'

Response

{
  "code": 1000,
  "msg": "OK",
  "data": {
    "count": 18,
    "result": [
      {
        "stream_urls": {
          "client_pull_stream_url": "https://stream-open.akool.io:8443/final_openapi_live/20240615060910789_3712.flv",
          "client_push_stream_url": "https://stream-open.akool.io:8443/openapi_live/20240615060910789_3712.whip"
        },
        "_id": "666d3006247f07725af0f884",
        "uid": 100010,
        "type": 1,
        "status": 1,
        "task_id": "666d300635d473f81c8be7b1"
      }
    ]
    }
}

Live Avatar WebSocket

WSS client_chat_room_url

you can get the url client_chat_room_url from the interface 【Create session】. like wss://stream-open.akool.io:8443/final_open_live/client_3835.chat

Receive Data

Parameter	Type	Value	Description
type	String	chat	default chat.
payload	String		To pass in content, you need to serialize the json content.
payload.message_id	String		Message id. The same id indicates that the incoming data belongs to the same message.
payload.voice_id	String		Voice model id, you can get the field voice_id from the interface 【Get Voice List Result】.
payload.language	String		Language code, you can get the field lang_code from the interface 【Get Language List Result】.
payload.mode_type	String	1	default 1.
payload.question	String		The text content passed in, such as ‘Hello’.
payload.prompt	Object		Regarding the custom description of the digital person.
payload.prompt.from	String	text	the value range of from is 【text, url】. The default type=“text”
payload.prompt.content	String		When type is text, content is the text description of the avatar role. When type is url, content is the pdf link of the description of the avatar role. The link size limit is 50M. The default type=“text” and content=""
to	String		Server ID, you can get from server_chat_room_url link address, The address suffix such as server_8898 is the server id.

Example Send Data

The payload here is not serialized. When using it, you need to use JSON.stringify() to convert the object into a json string. If it is other types, you can convert it to base64 first.

{
    "type": "chat",                         //  default 'chat'.
    "payload": {
      "message_id":"msg-1723629433573",     //  Message id. The same id indicates that the incoming data belongs to the same message.
      "voice_id":"21m00Tcm4TlvDq8ikWAM",    //  Voice model id, you can get the field *voice_id* from the interface 【Get Voice List Result】.
      "language":"en",                      //  Language code, you can get the field 'lang_code' from the interface 【Get Language List Result】
      "mode_type":1,                        //  1 Retelling 2 Dialogue， default *1*.
      "prompt":{
        "from": "text", 
        "content": ""
      },
      "question":"Hello"                    //  The text content passed in, such as 'Hello'.
    },
    "to": "server_8898"   //  Server ID, you can get from *server_chat_room_url* link address, The address suffix such as server_8898 is the server id.
}

Receive Data

The payload has been parsed here. In actual use, you need to perform JSON.parse() to restore it first.

{
    "type": "chat",                       //  default 'chat'.
    "from": "server_8898",                //  Current server id.
    "to": "client_8898",                  //  Current client id.
    "payload": {
      "message_id": "msg-1723629433573",  //  Message id. The same id indicates that the incoming data belongs to the same message.
      "voice_id": "21m00Tcm4TlvDq8ikWAM", //  Voice model id, you can get the field *voice_id* from the interface 【Get Voice List Result】.
      "language": "English",              //  Language
      "mode_type": 1,                     //  default *1*.
      "prompt": {
        "from": "text", 
        "content": ""
      },
      "question": "",                     //  Current input question
      "answer": "Hello!  How can I assist you today? "  // avatar's answer
    }
}

  const WebSocket = require('ws')
  const ws = new WebSocket('wss://stream-open-test.akool.io:8443/final_[stream_name]/client_[stream_name].chat');

  ws.on('open', function open() {
    let message = {
      "type": "chat",
      "payload": JSON.stringify({
          "message_id":"1",
          "voice_id":"21m00Tcm4TlvDq8ikWAM",
          "language":"en",
          "mode_type": 1,
          "prompt":{
            "from": "text", 
            "content": "" 
          },
          "question": "hello"
      }),
      "to": "server_8898"
    };
    ws.send(JSON.stringify(message));
  });

  ws.on('message', function message(data) {
      console.log('received: %s', data);
  });

Response Code Description

Please note that if the value of the response code is not equal to 1000, the request is failed or wrong

Parameter	Value	Description
code	1000	Success
code	1003	Parameter error or Parameter can not beempty
code	1008	The content you get does not exist
code	1009	Youdo not have permission to operate
code	1101	Invalid authorization or Therequest token has expired
code	1102	Authorization cannot be empty
code	1200	The account has been banned
code	1201	create audio error, pleasetry again later
code	1202	The same video cannot be translated lipSync inthe same language more than 1 times
code	1203	video should be with audio
code	1204	Your video duration is exceed 60s!
code	1205	Create videoerror, please try again later
code	1207	The video you are using exceeds thesize limit allowed by the system by 300M
code	1209	Please upload a videoin another encoding format
code	1210	The video you are using exceeds thevalue allowed by the system by 60fp
code	1211	Create lipsync error, pleasetry again later

Authentication

AI Tools Suite

Create session

Get Session Info Result

Close Session

Pause Session

Get Session List

Live Avatar WebSocket

Authentication

AI Tools Suite

​Create session

​Get Session Info Result

​Close Session

​Pause Session

​Get Session List

​Live Avatar WebSocket

Create session

Get Session Info Result

Close Session

Pause Session

Get Session List

Live Avatar WebSocket