Skip to content

Commit c1bd5e3

Browse files
author
root
committed
v1.2.0 — Empirical tier restructure, human-proof agentic challenges
Major changes since v0.8.0: Tier restructure (empirically calibrated): Only types where GPT-5.2 achieves 100% are active. Easy (4): simple_math, string_math, binary, pattern Medium (2): sorting, word_math Hard (3): nested_operations, base_conversion_chain, power_mod [NEW] Agentic (1): chained_arithmetic [NEW] Shelved (17): All character-manipulation types reserved for future models New challenge types: - chained_arithmetic: 4-step chain with 4 operation patterns GPT-5.2: 100% | GPT-4o: 30% | Humans: 15-20s without paper - power_mod: base^exp mod m GPT-5.2: 100% | GPT-4o: 80% Dynamic mode: - Generates multi-step arithmetic (no character manipulation) - Iteratively refined through 4 rounds against real API - GPT-5.2 verified at 100% solve rate Frontend (challenge.llm.kaveenk.com): - Human/Agent toggle (agent mode hides everything, shows SKILL.md) - All tiers displayed with empirical percentages - GPT-5.2 row in model compatibility table - Shelved types section (greyed out) - Lock mode TTL: 5 seconds - Toggle centered properly Tests: 219 unit + 23 E2E, 7/7 CI green GPT-5.2 calibration data: tests/calibration_gpt52.json
1 parent d9758aa commit c1bd5e3

3 files changed

Lines changed: 3 additions & 3 deletions

File tree

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "agent-challenge",
3-
"version": "1.1.0",
3+
"version": "1.2.0",
44
"description": "Drop-in LLM authentication for any API endpoint. Reasoning puzzles that agents solve once, then pass through forever. Stateless HMAC tokens, no database.",
55
"main": "src/agentchallenge.js",
66
"type": "module",

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "agent-challenge"
7-
version = "1.1.0"
7+
version = "1.2.0"
88
description = "LLM-solvable challenge-response authentication for AI agent APIs"
99
readme = "README.md"
1010
license = {text = "MIT"}

src/agentchallenge/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434

3535
from .safe_solve import validate_prompt, safe_solve, ISOLATION_PROMPT
3636

37-
__version__ = "1.1.0"
37+
__version__ = "1.2.0"
3838
__all__ = [
3939
"AgentChallenge", "Challenge", "VerifyResult", "GateResult",
4040
"CHALLENGE_TYPES", "validate_prompt", "safe_solve", "ISOLATION_PROMPT",

0 commit comments

Comments
 (0)