Training Llama-2 Model on AWS Trainium
Training Llama-2 Model on AWS Trainium In this blog we will run multi-node training jobs using AWS Trainium accelerators in Amazon EKS. Specifically, you will pretrain Llama-2-7b on 4 AWS EC2 trn1.32xlarge instances using a subset of the RedPajama dataset. Selecting the Right Llama-2 Model Size Choosing the appropriate model size of Llama-2 depends on […]