REINFORCEMENT LEARNING // PRAXIS LAB

Strategic Learning
Through Action.

An experimental reinforcement learning system exploring how strategic capability emerges from self-play, deep search loops, and state-space representation.

Search Scale
10,000+
MCTS branches / step

Parallel Monte Carlo Tree Search evaluating candidate action branches with neural priors.

Baseline Win Rate
84.6%
against Alpha-Beta heuristics

Outperforming traditional evaluation heuristics via self-play representation learning.

Convergence Timeline
< 24h
distributed training time

Rapid value function stabilization using asynchronous gradient updates.

01 — CORE MECHANICS

How Praxis Learns

SEARCH DYNAMICS & REINFORCEMENT

[01]SEARCH DYNAMICS

Monte Carlo Tree Search with Neural Priors

Praxis combines neural policy priors with parallel tree exploration. Rather than relying entirely on immediate feed-forward output, the agent explores tens of thousands of future board trajectories before selecting an optimal action.

ALGORITHMIC IMPLEMENTATION
Parallel rollout queue with PUCT exploration constant (c_puct = 1.25)
Action prior probability distribution: Softmax(logits / T)
Virtual loss injection for multi-threaded tree traversal
[02]FEEDBACK GENERATION

Autonomous Self-Play Reinforcement Loops

The agent generates its own training data by playing millions of games against previous iterations of itself. Win/loss outcomes iteratively adjust policy weights, continually identifying weaknesses and developing counter-strategies.

ALGORITHMIC IMPLEMENTATION
Replay buffer tracking 2,500,000 active state transitions
Prioritized experience replay sampling with dynamic weighting
Dual value loss convergence: MSE(z, v) + CrossEntropy(pi, p)
[03]REPRESENTATION LEARNING

Dual-Head Policy & Value Networks

A shared convolutional and transformer backbone extracts game state representations, splitting into two distinct heads: a Policy Head predicting the move distribution and a Value Head predicting expected game outcome (-1 to +1).

ALGORITHMIC IMPLEMENTATION
Backbone: 24 ResNet blocks + 4 Transformer attention layers
Policy Head: Action space shape (4672 moves)
Value Head: Scalar tanh output in continuous range [-1.0, 1.0]
PUBLICATIONS & LOGS

Read the Praxis Research Log

Explore published findings, training trajectories, and benchmark evaluations on strategic agent learning.