Self-Reference Count
Self Reference
Pass rate
0%
Survived 4 out of 15 breakers
Gemma 3 introduces multimodality, supporting vision-language input and text outputs. It handles context windows up to 128k tokens, understands over 140 languages, and offers improved math, reasoning, and chat capabilities, including structured outputs and function calling. Gemma 3 27B is Google's latest open source model, successor to [Gemma 2](google/gemma-2-27b-it)
128,000 tokens
$0.04 /1M tokens
$0.15 /1M tokens
65,536
| Test | Category | Latest Result | Success Rate | |
|---|---|---|---|---|
| Self-Reference Count | Self Reference | 0% | ||
| Alice's Brother Problem | Logic Reasoning | 0% | ||
| Contradictory Premises | Logic Reasoning | 0% | ||
| Car Wash Dilemma | Logic Reasoning | 0% | ||
| The Missing A | Pattern Matching | 0% | ||
| Horse Race Logic | Logic Reasoning | 0% | ||
| The Compartment Trick | Logic Reasoning | 0% | ||
| Reverse Word Test | Character Manipulation | 9% | ||
| 10-Step Instructions | Instruction Following | 18% | ||
| Broken Mug | Lateral Thinking | 20% | ||
| Bullshit Detector | Epistemic Humility | 25% | ||
| Coin Flip Paradox | Logic Reasoning | 75% | ||
| Strawberry Problem | Character Counting | 82% | ||
| Silence Protocol | Instruction Following | 91% | ||
| Sycophancy Trap | Logic Reasoning | 100% |