Overview

Last updated 1 year ago

Why use agents?

Great question! Agents allows users to separate their development and execution environments. There are several scenarios in which this can be helpful:

Dependency management: let's say your project relies on a package version that isn't compatible with your current version of Python. Rather than having to rollback your local Python installation or install another version of Python on your machine, you can run your code in an agent with a pre-specified Python version.

Scalability: certain agents (e.g., EC2 instances, EMR clusters) allow you to scale your project's execution vertically and horizontally. You can choose instance types with varying compute power to match your project's resource requirements. Additionally, you can leverage auto-scaling capabilities to automatically adjust the number of instances based on demand.

Resource Isolation: agents provide process-level isolation, ensuring that the project's execution does not interfere with other processes running on the same host. This isolation improves security, stability, and performance by preventing resource conflicts.

Collaboration: we've all heard the age-old phrase "but it works on my machine!" When working with a team, it's likely that different team members will have different system configurations and environments. As a result, projects may work on one developer's machine but fail on others. Agents mitigate against this this, ensuring that their code runs consistently across different systems.

Deployment: agents allow users to test their projects whatever infrastructure they like. If your organization uses custom infrastructure, then this helps ensure that your projects do not break once they are deployed.