Body

• chunking_settings object

  Chunking configuration object

  Hide chunking_settings attributes Show chunking_settings attributes object

  • max_chunk_size number

    The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

  • overlap number

    The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

  • sentence_overlap number

    The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

  • strategy string

    The chunking strategy: sentence or word.

• service string Required

  Value is cohere.

• service_settings object Required
  Hide service_settings attributes Show service_settings attributes object

  • api_key string Required

    A valid API key for your Cohere account. You can find or create your Cohere API keys on the Cohere API key settings page.

    IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

    External documentation
  • embedding_type string

    Values are binary, bit, byte, float, or int8.

  • model_id string

    For a completion, rerank, or text_embedding task, the name of the model to use for the inference task.

    • For the available completion models, refer to the Cohere command docs.
    • For the available rerank models, refer to the Cohere rerank docs.
    • For the available text_embedding models, refer to Cohere embed docs.

    The default value for a text embedding task is embed-english-v2.0.

  • rate_limit object

    This setting helps to minimize the number of rate limit errors returned from the service.

    Hide rate_limit attribute Show rate_limit attribute object

    • requests_per_minute number

      The number of requests allowed per minute. By default, the number of requests allowed per minute is set by each service as follows:

      • alibabacloud-ai-search service: 1000
      • anthropic service: 50
      • azureaistudio service: 240
      • azureopenai service and task type text_embedding: 1440
      • azureopenai service and task type completion: 120
      • cohere service: 10000
      • elastic service and task type chat_completion: 240
      • googleaistudio service: 360
      • googlevertexai service: 30000
      • hugging_face service: 3000
      • jinaai service: 2000
      • mistral service: 240
      • openai service and task type text_embedding: 3000
      • openai service and task type completion: 500
      • voyageai service: 2000
      • watsonxai service: 120
  • similarity string

    Values are cosine, dot_product, or l2_norm.

• task_settings object
  Hide task_settings attributes Show task_settings attributes object

  • input_type string

    Values are classification, clustering, ingest, or search.

  • return_documents boolean

    For a rerank task, return doc text within the results.

  • top_n number

    For a rerank task, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in a text_similarity_reranker retriever query and top_n is set, it must be greater than or equal to rank_window_size in the query.

  • truncate string

    Values are END, NONE, or START.