When setting up a new project from the ground up, there are many details to consider. In this article, I document my experience with starting a project in AWS from scratch. I’ll be creating a new organization, called PrototypeTopia, that will serve as a personal space where I can develop prototypes and explore side projects.
The initial step involves configuring AWS accounts for development. Recently, I've embarked on a learning journey with a tool named org-formation. Although I was familiar with it, this was my first hands-on experience. The setup for AWS environments, user access, service access, and other necessary configurations can be complex. Enter org-formation.
What is Org-formation?
Org-formation is a tool designed to manage AWS Organizations resources using infrastructure as code. This means that, rather than manually configuring AWS accounts, organizational units, and service control policies, you can define these as code templates. By doing so, AWS setups become more consistent, repeatable, and traceable.
Before diving deeper, I'd like to acknowledge Michael Bahr for his invaluable insights on starting with org-formation. His article provided a strong foundation and guidance. I wholeheartedly recommend checking it out.
Step-by-Step Guide to Setting Up org-formation
Step 1: Setting up the AWS Account
Start by creating a new AWS account. This account will serve as the AWS Organization's root. This means it will own all organizational units and sub-accounts within the AWS Organization.
Step 2: Configuring the AWS Root Account
Log into the AWS account designated as the Organizations root management account. Create a new IAM user and grant it the AdministratorAccess policy. This is crucial for initializing org-formation. Once org-formation is set, you can revoke this user due to its high access privilege.
After creating the root account:
Log into the AWS account designated as the Organizations root management account.
Create a new IAM user.
Assign the AdministratorAccess policy to this user. This will be essential for our initial setup with org-formation. Remember, we'll be deleting this user down the line as it possesses excessive privileges.
To track and analyze AWS costs, enable Cost Explorer. We’ll also need this later to set up budget alerts.
Enable Cost Explorer in AWS:
Navigate to the AWS Management Console.
Open the Billing and Cost Management dashboard.
In the navigation pane, choose Cost Explorer.
Choose “Enable Cost Explorer.”
Step 3: Manually Create an AWS Organization
As of now, org-formation requires users to set up an AWS Organization manually.
How to Create an AWS Organization:
Sign in to the AWS Management Console in the AWS root account.
Go to the AWS Organizations console.
In the navigation pane, choose "Create organization."
Follow the on-screen instructions.
Step 4: Installing org-formation
Follow instructions in https://github.com/org-formation/org-formation-cli#installation to install the org-formation cli tool.
Next, install git-remote-codecommit to be able to work with AWS CodeCommit repositories from your local machine.
Step 5: Initialize org-formation
To ensure an organized and automated workflow, we'll employ org-formation's codepipeline setup.
Use the admin profile made in Step 2. Although this isn't a long-term solution, it aids in the initial setup. Future configurations will transition to short-lived authentication sessions following the integration of Single Sign-On (SSO).
Execute the following:
export AWS_PROFILE=admin-profile org-formation init-pipeline --region <YOUR-REGION>This creates a CodeCommit repository and a CodePipeline. By default, this command uses the AWS default profile, unless you specify a different one using AWS_PROFILE.
Viewing the CodeCommit Repository:
Open the AWS Management Console.
Navigate to the AWS CodeCommit dashboard.
Here, you can see and manage your repositories.
Step 6: Cloning the org-formation Repository
Clone the repository using the HTTPS (GRC) option and the administrator profile with the following command:
export AWS_PROFILE=admin-profile
git clone codecommit::us-east-1://organization-formation
Step 7: Adding Your First Account to the Organization with org-formation
Understanding org-formation's Organizational Units and Accounts
At its core, org-formation uses a defined structure to manage AWS organizations. It employs a file, typically named organization.yml, as the blueprint. Within this file, users can define Organizational Units (OUs) and accounts.
Organizational Units (OUs): These are hierarchical, container-like structures that allow for the grouping of AWS accounts. OUs can help impose standardized controls over accounts and offer a logical method to manage and categorize them.
Accounts: An AWS account is a standalone entity that holds AWS resources. In the context of org-formation, accounts are tied to specific OUs, offering an extra layer of organizational clarity.
Adding the Organizational Unit and Test Account
To add our first test organizational unit and associated account, add the following code in organization.yml:
Organization:
ManagementAccount:
# ...
OrganizationRoot:
# ...
TestOU:
Type: OC::ORG::OrganizationalUnit
Properties:
OrganizationalUnitName: Test
Accounts:
- !Ref TestAccount1
TestAccount1:
Type: OC::ORG::Account
Properties:
AccountName: Test Account 1
RootEmail: email@example.com
Leveraging Email Aliases for Unique Email Addresses
AWS requires a unique email address for each account. If you're using an email provider like Gmail, you can use email aliases for this purpose. For instance, if your main email is username@gmail.com, you can register an AWS account with username+testaccount1@gmail.com and still receive emails at your primary address. This tactic makes it efficient to manage multiple AWS accounts without needing separate email addresses.
Deploying the New Organization and Account
After defining your test OU and account in the organization.yml file:
1. Commit and push your changes:
git add organization.yml
git commit -m "Added TestOU and TestAccount1"
git push
2. Allow the pipeline to execute. This operation will take care of deploying the new organizational structure and account.
3. Await an email notification. AWS will send an email to the specified address (in our example, email@example.com) to confirm the account creation.
4. Validate the Changes: Navigate to the AWS Organizations console to confirm the new organizational unit and associated account's presence.
By following these steps, you've now added an initial layer of structure to your AWS Organization using org-formation. This foundational layer offers a taste of what's achievable as you delve deeper into organizing and optimizing your AWS setup.
Conclusion
Setting up AWS Organizations and accounts can initially seem daunting, but tools like org-formation streamline the process, allowing developers to manage AWS resources efficiently. This article showcased the power of org-formation, from establishing a new organization to adding your first organizational unit and account, highlighting the tool's potential for effective AWS management.
As we progress, our next focus will be on further leveraging org-formation. In part two, we'll discuss setting up budget alarms for all accounts, ensuring financial oversight across our projects. Stay tuned!
References
Managing AWS Organizations using the open source org-formation tool — Part 1
Org-Formation docs: Organization resources
Org-Formation docs: org-formation templates