The rough idea behind the cloud integration is to package our code into a docker container and use AWS Batch to run thousands of experiments concurrently.
- Terraform (see installation tutorial here)
We use Terraform to define our infrastructure with AWS Batch, which you can spin up as follows
# assuming you are at the root of the CleanRL project poetry install --with cloud cd cloud python -m awscli configure terraform init export AWS_DEFAULT_REGION=$(aws configure get region --profile default) terraform apply
Don't worry about the cost of spining up these AWS Batch compute environments and job queues. They are completely free and you are only charged when you submit experiments.
Then your AWS Batch console should look like
Uninstalling/Deleting the infrastructure is pretty straightforward:
export AWS_DEFAULT_REGION=$(aws configure get region --profile default) terraform destroy