AWS Batch Overview
AWS Batch is a fully managed service for running jobs across multiple availability zones within a region. With AWS Batch you set up job queues, and then create job definitions that specify which shell scripts, Linux executables or Docker container images run your jobs. When a job on a queue is started, it runs in a compute environment: which can be Fargate, or on specified instance types. You can also specify the minimum, desired, and maximum number of vCPUs for the environment, along with the amount you are willing to pay for a Spot Instance as a percentage of the On-Demand Instance price, and AWS Batch will manage that environment
AWS Batch Benefits
- removes the need to manage the infrastructure required to execute the batch
- efficiently provisions resources
- eliminates capacity constraints
- reduces compute costs
- delivers results quickly
AWS Batch Features
- support multiple compute environments (
EC2,SPOT,FARGATEandFARGATE_SPOT) - priority job queues where 10 is the highest priority and 1 the lowest
- multiple job queues – where each job queue is assocaited with only one type of compute environment
- job queue tagging – used for billing reporting
- multi-node parallel job definition – allows the job to be run across multiple nodes on EC2 only. An AWS Batch multi-node parallel job is compatible with any framework that supports IP-based, internode communication, such as Apache MXNet, TensorFlow, Caffe2, or Message Passing Interface (MPI).To run multi-node parallel jobs on AWS Batch, your application code must contain the frameworks and libraries necessary for distributed communication.
AWS Batch Security
AWS Batch makes calls to other AWS services on your behalf to manage the resources that you use with the service. Before you can use the service, you must have an IAM policy and role that provides the necessary permissions to AWS Batch.
The AWSBatchServiceRole policy sets up access to a few dozen EC2, autoscaling, ECS, logs and IAM actions. This role i set up automatically the first time you access the service.
AWS Batch Pricing
There is no explicit charge for the AWS Batch service. You are charged on a pay as you go basis for the EC2, Lambda and Fargate instances the batch jobs use. You can use your Reserved Instances, Savings Plan, EC2 Spot Instances, and Fargate. Any discounts will be applied at billing time.
