This post is short. It is mainly a reminder of the parallels between HFT and the quantification of biology.
Every great industrial revolution faced bottlenecks for achieving its eventual success. An area of technical accomplishment that is oftentimes overlooked, mainly because of emotional perspective, is high frequency trading.
One of the most successful companies in history, Citadel, had to overcome lots of these engineering challenges to actually achieve scale. They have become differentiated from others, like Renaissance, in that they extended past trading to become a platform for other hedge funds to trade through. Though they started as a quantitative hedge fund, they ultimately became one of the most important financial infrastructure companies in the world.
Transaction Infrastructure
Main problem: latency of transaction execution
Inventions:
- microwave trading networks to cut fiber-optic latency (40% faster)
- order book modeling to predict market movements milliseconds-nanoseconds ahead
- custom-built optimal placement algorithms
Mathematical Models for Market Prediction and Risk Management
Main problem: separate signal from noise for better decision making
Inventions:
- statistical arbitrage models
- Hidden Markov Models to predict order flow and market-maker behavior
- self-correcting alpha models, hedging models
High Dimensional Optimization and Compute Constraints
Main problem: scale without increasing costs or information loss
Inventions:
- parallelized compute allowed compressing petabytes of financial information in real time
- private fiber-optic networks directly linked to stock exchanges