# Models and usage

## Configure a model

Open **Canopy** in LiteLLM and select **Settings**.

1. Select **Sync models** to load your LiteLLM model groups and prices.
2. Choose an existing model group, or select **Add model** to create a new model name.
3. Choose a Canopy catalog model or an organization preset as the target.
4. Review **Competing deployments**. Matching deployments with known prices are selected
   initially. Keep the providers you want Canopy to compete with, or clear all selections to
   route directly to Canopy.
5. Enable the mapping and select **Save configuration**.

Workers check for changes every ten seconds. Allow time for LiteLLM to load the change, then
refresh Settings to check the reported loaded revision. That report confirms one worker, not
every replica.

**Test saved mapping** obtains and releases a quote without running inference. Use a real
request to check the complete flow.

### If a deployment cannot compete

Disabled checkboxes explain missing model identities, unknown prices, or model mismatches.
Manage ordinary providers and their prices in LiteLLM, then select **Sync models** again.

Keep stable `model_info.id` values. If LiteLLM cannot identify the underlying model, declare its
Canopy catalog ID in that deployment's YAML entry:

```yaml
model_info:
  id: coding-openai
  canopy_model_id: openai/gpt-4.1-mini
```

This declares which model the deployment already serves. It does not change the provider's model.

## Send a request

Keep using your LiteLLM endpoint and credentials. Set `model` to the mapped LiteLLM model name,
not the Canopy target ID. For a mapping named `coding`:

```sh
curl "$LITELLM_BASE_URL/v1/chat/completions" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"coding","messages":[{"role":"user","content":"Hello"}],"stream":true}'
```

Set `LITELLM_BASE_URL` to your proxy origin and use a LiteLLM key with access to that model.

Canopy does not add an output limit. Set `max_tokens` or `max_completion_tokens` if you want one.
Cost comparisons use estimated usage; those estimates do not truncate the response. Your output
limit can reduce the credits reserved for a request. Settlement charges actual usage and releases
unused reserved credits.

## View routing and savings

Open **Routing & savings** to see routing attempts and cost estimates. A failed Canopy attempt
followed by an ordinary fallback appears as two attempts, not one.

Savings compare the selected ordinary deployment's recorded rates with Canopy's settled charge.
Before settlement, the report uses Canopy's quoted rates. Missing usage can also require estimates.
Direct Canopy routes have no competing cost or savings estimate, and negative savings remain visible.

Reports contain no prompts or completions. Delivery is best effort, so use them to review routing,
not as a complete billing record.

## Change or remove a mapping

Use the ellipsis menu beside **Enabled** to edit or remove a mapping, then select
**Save configuration**. **Discard changes** restores the saved settings.

Renaming changes the model name clients must request and clears competing selections. Removing
Canopy from an existing group leaves its ordinary providers in place. Removing a Canopy-only
model removes that route.

## If something goes wrong

* **The panel rejects access:** open it through LiteLLM as a proxy administrator. Check that the
  plugin URL matches the external LiteLLM origin.
* **A saved change has not applied:** check the loaded revision, database configuration, and shared
  connection directory. Workers keep their last applied configuration if a refresh fails and retry.
* **Canopy is not selected:** check the enabled mapping, selected competitors, model identities,
  and prices. Ordinary routing can win the cost comparison or remain in use when no offer is available.
* **A direct route fails:** check Canopy credits and provider availability. Direct routing does not
  fall back to unselected providers in the group.

Contact the Canopy team if the issue continues. Do not share connection files, API keys, or prompts.