> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wednesdayai.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Groq

# Groq

Groq provides ultra-fast LLM inference using their custom Language Processing Unit (LPU) hardware. It supports a range of open-source models including Llama, DeepSeek, Qwen, and Kimi, all served through an OpenAI-compatible API.

## Requirements

* A Groq API key from [console.groq.com](https://console.groq.com)

## Quickstart

```bash theme={"dark"}
export GROQ_API_KEY="gsk_..."
wednesdayai onboard --auth-choice apiKey --token-provider groq --token "$GROQ_API_KEY"
```

Then set a default model:

```json5 theme={"dark"}
{
  agents: {
    defaults: {
      model: { primary: "groq/llama-3.3-70b-versatile" },
    },
  },
}
```

## Configuration

### Environment variable

```bash theme={"dark"}
export GROQ_API_KEY="gsk_..."
```

WednesdayAI reads `GROQ_API_KEY` automatically and enables the `groq` provider.

### Config snippet

```json5 theme={"dark"}
{
  env: { GROQ_API_KEY: "gsk_..." },
  agents: {
    defaults: {
      model: { primary: "groq/llama-3.3-70b-versatile" },
    },
  },
}
```

### Provider settings

| Setting | Value |
| - | - |
| Provider ID | `groq` |
| Base URL | `https://api.groq.com/openai/v1` |
| API | OpenAI-compatible (`openai-completions`) |
| Auth env var | `GROQ_API_KEY` |

## Available models

| Model ID | Name | Context | Notes |
| - | - | - | - |
| `llama-3.3-70b-versatile` | Llama 3.3 70B Versatile | 131K | |
| `llama-3.1-8b-instant` | Llama 3.1 8B Instant | 131K | Fast |
| `llama3-70b-8192` | Llama 3 70B | 8K | |
| `llama3-8b-8192` | Llama 3 8B | 8K | |
| `meta-llama/llama-4-maverick-17b-128e-instruct` | Llama 4 Maverick 17B 128E Instruct | 131K | |
| `meta-llama/llama-4-scout-17b-16e-instruct` | Llama 4 Scout 17B 16E Instruct | 131K | |
| `deepseek-r1-distill-llama-70b` | DeepSeek R1 Distill Llama 70B | 131K | Reasoning |
| `moonshotai/kimi-k2-instruct` | Kimi K2 Instruct | 131K | |
| `moonshotai/kimi-k2-instruct-0905` | Kimi K2-Instruct 0905 | 262K | |
| `openai/gpt-oss-120b` | GPT-OSS 120B | 131K | Reasoning |
| `openai/gpt-oss-20b` | GPT-OSS 20B | 131K | Reasoning |
| `qwen-qwq-32b` | Qwen QwQ 32B | 131K | Reasoning |
| `qwen/qwen3-32b` | Qwen3 32B | 131K | Reasoning |
| `gemma2-9b-it` | Gemma 2 9B IT | 8K | |
| `mistral-saba-24b` | Mistral Saba 24B | 32K | |

Use model IDs as `groq/<model-id>` (example: `groq/llama-3.3-70b-versatile`).

## Troubleshooting

**Rate limit errors**

Groq imposes per-minute and per-day request limits by tier. If you hit rate limits:

* Switch to a smaller model (for example `groq/llama-3.1-8b-instant`) for lower-rate tasks.
* Add multiple API keys as auth profiles for rotation: `wednesdayai models auth add-key --provider groq`.
* Check your current usage at [console.groq.com](https://console.groq.com).

**401 Unauthorized**

* Verify your key starts with `gsk_` and is copied in full.
* Re-run onboarding: `wednesdayai onboard --auth-choice apiKey --token-provider groq --token "$GROQ_API_KEY"`.

**No models listed**

* Confirm `GROQ_API_KEY` is set in the environment where the gateway runs.
* Check `wednesdayai models status` to see if the `groq` provider is detected.

## Related

* [Model providers](https://docs.wednesdayai.dev/next/developers/concepts/model-providers.md)
* [Models](https://docs.wednesdayai.dev/next/developers/concepts/models.md)
* [Model failover](https://docs.wednesdayai.dev/next/developers/concepts/model-failover.md)
* [Gateway configuration](https://docs.wednesdayai.dev/next/admin/gateway/configuration.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.