Documentation for cogito:70b

Used for chat completions via the OpenAI-compatible /api/v1/chat/completions endpoint. It can be used for various tasks like text generation, question answering, summarization, coding, and mathematical reasoning, depending on the model's specialization.

This endpoint mirrors the OpenAI API structure, allowing you to use existing OpenAI SDKs by changing the baseURL. Refer to the full documentation for the /api/v1/chat/completions endpoint for details on headers, parameters (stream, temperature, max_tokens, tools/functions, response_format, etc.), error handling, and response format (including streaming).

Sample Request (cURL - Non-streaming)

# Replace <your-orchestrator-host> and <API_KEY> with your actual values
curl https://<your-orchestrator-host>/api/v1/chat/completions \
  -H "Authorization: Bearer <API_KEY>" \
  -H "X-User-Id: your_user_id_123" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "cogito:70b",
        "messages": [
          { "role": "system", "content": "You are a helpful assistant." },
          { "role": "user", "content": "Explain the concept of transformer models in simple terms." }
        ],
        "temperature": 0.7,
        "max_tokens": 150
      }'

Sample Request (Python SDK)

from openai import OpenAI

client = OpenAI(
    api_key="<API_KEY>", # Replace with your key
    base_url="https://<your-orchestrator-host>/api/v1", # Replace with your host
    default_headers={
        "X-User-Id": "your_user_id_123"
    }
)

try:
    response = client.chat.completions.create(
        model="cogito:70b",
        messages=[
            {"role": "system", "content": "You are a helpful coding assistant."},
            {"role": "user", "content": "Write a python function to calculate factorial."}
            # For vision models, add image content if supported by your SDK/setup
            # {"role": "user", "content": [
            #     {"type": "text", "text": "Describe this image"},
            #     {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
            # ]}
        ],
        temperature=0.5,
        stream=False # Set to True for streaming response
    )
    print(response.choices[0].message.content)
except Exception as e:
    print(f"An error occurred: {e}")