AI A/B Testing Engine — Project Docs
Architecture, integration guide, and variant types for the AI-driven A/B testing engine
AI A/B Testing Engine
An AI-powered split testing system that runs on any lesson page via a single <script> tag. Admins generate text, diagram, or interactive GameEngine variants for any element and launch 50/50 tests without touching page source.
Architecture
| Layer | Technology | Role |
|---|---|---|
| Engine API | Next.js on Vercel | Test management, AI generation, event tracking |
| Database | Neon PostgreSQL (Prisma) | Stores tests, variants, events |
| SDK | Vanilla JS (sdk.js) |
Deterministic bucketing, DOM mutation, conversion tracking |
| LLM | Groq llama-3.3-70b-versatile |
Game/diagram generation |
| LLM | Groq llama-3.1-8b-instant |
Text rewrite variants |
Jekyll Integration
One line in _includes/head-custom.html:
<script src="https://ai-ab-test-engine.vercel.app/sdk.js"
data-site-baseurl=""
data-project-id="cmnfour780000f63gu69kg2v8"></script>
data-site-baseurl is rendered by Liquid and passed to the SDK so GameEngine asset paths resolve correctly on GitHub Pages.
The global include now points at the deployed Vercel engine, so production pages fetch the same SDK that the dashboard manages.
Game Variant
The most advanced variant type. The LLM extracts key concepts → server assembles a verified GameEnginev1.1 ES module with NPC characters → SDK imports GameExecutor from this site’s own /assets/js/pages/runners/index.js and auto-runs the level.
WASD to walk, E to interact with NPCs and read concept dialogues.
Experiment Flow
- Open any lesson page with
?ab_admin=true. - Select the element to test in the overlay.
- Generate a variant, launch the test, and let the SDK bucket visitors.
- Review views and conversions in the engine dashboard before declaring a winner.
Reading Results
Use views for exposure checks and conversions for click-goal tests. A healthy experiment should show traffic on both variants before the team declares a winner or opens a follow-up implementation PR.
Troubleshooting
If a GameEngine variant does not render, confirm the script tag includes data-site-baseurl. The SDK uses that value to import the local GameExecutor assets from the Pages site.
Notebook
See the project notebook for a full walkthrough with live demo links.