game theory game theory n. A branch of mathematics that deals with strategies for maximizing gains or minimizing losses in competitive situations having defined constraints and involving random factors.