We didn't have the compute required (GPUs powerful enough to parallelize forward and backward pass). This compute is what allows us to train from human knowledge or distillation.