Conservative Decision Making Gets You Beat