Command R+: Complete Benchmark Performance Guide

cohere r-plus 2024-04-04 CC-BY-NC-4.0

In-depth analysis of Command R+ performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorcohere
Versionr-plus
Release Date2024-04-04
Context Window128000 tokens
Input Modalitiestext
Output Modalitiestext
LicenseCC-BY-NC-4.0
Documentationhttps://docs.cohere.com/docs/command-r-plus

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU75%2024-04-045-shotview
HUMANEVAL70.7pass@12024-04-04—view
GSM8K68.4%2024-04-040-shot CoTview
MATH35.6%2024-04-040-shot CoTview
BBH66.1%2024-04-043-shot CoTview

Pricing

TierPriceCurrency
Input$2.5 / MtokUSD
Output$10 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://cohere.com/pricing · as of 2024-08-01

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Command R+

Modellöversikt

Cohere Command R+ 是面向企业的 104B 参数模型，128K 上下文窗口，针对检索增强生成（RAG）与工具调用优化，支持 SOC2/GDPR/ISO 27001 合规，适合企业级生产部署。

Kärnspecifikationer

LeverantörVersionReleasedatumKontextfönsterInmatningsmodaliteterUtmatningsmodaliteterLicens
Coherer-plus2024-04-04128KtexttextCC-BY-NC-4.0

Benchmarkprestanda

BenchmarkPoängEnhetAnteckningar
MMLU (Massive Multitask Language Understanding)75.0%5-shot
HumanEval70.7pass@1—
GSM8K (Grade School Math 8K)68.4%0-shot CoT
MATH35.6%0-shot CoT
BBH (BIG-Bench Hard)66.1%3-shot CoT

Priser

InmatningUtmatningCacheläsningCacheskrivning
————

per miljon tokens

Styrkor

  • 企业级合规认证。

Svagheter

  • 闭源专有模型，不支持自托管。

Användningsområden

  • 代码生成与调试
  • 企业客户支持

Referenser