As of yet, no belief system on the planet (or beyond) will keep you safer than following a sound security plan. Image from: https://theoriginalunderground.com/products/i-want-to-believe-postcard

☁️ How to Begin Securing Your Amazon Web Services Accounts ☁️

Getting from “whoa thats a 🕳 lotta 🕳 holes 🕳” to something that will at least let you sleep at night 😴 🛌

After having worked intensely with Amazon Web Services (AWS) for the last year, I henceforth divide my wishes into the following compartments:

  1. Things I wish I knew
  2. Things I wish I did
  3. Things I wish AWS had released

I hope you will get something in all those three categories here, with a finger pointing also to some great new additions to AWS.

The below is a compilation of things I have worked with, things I think I kind of understand, things I remember (gosh darn, all of the things I can’t pull from the dark recesses of the mind) and a lot of this has been vetted (implementation-wise) by third-party security consultants. Then there’s also a dash of hygiene recommendations and best practices. And the services that I wish existed when I began working with my latest client.

It is not a full inventory (if that can ever exist), nor an exhaustive list of items to check off that will take you from 0 to 100. I probably forgot a lot! It’s also necessarily less about architecture and more about guard rails, service settings, and account-level considerations. I believe that you should find at least a few items that may be worth acting on and start small, beginning with easy wins. Never get too cocky in this domain and don’t be afraid to let others look at and discuss your security posture.

You will always want external validation, always be on-top of AWS best practices and current advice, and you probably should bring in AWS to talk and look at your setup. They are good, nice people too!

AWS has collected security-related documentation at https://docs.aws.amazon.com/security/. That should be your prime resource to make decisions when it comes to security. It may overlap some of the material here, but absolutely bookmark that one and at least glance at the top services that you use today.

By the way: In the bottom of this article I list a number of tools. If you want scripts to get you up and running with some tooling (works with AWS, Google Cloud, and Azure) then hit the link below.

Know your responsibility and what the common threats are

For many coming to the cloud, there seems to be a low knowledge of what their own responsibility is for security. I encourage architects, security specialists and tech leads to read about that:

You’ll find a pretty easy-on-the-eyes chart.

AWS Shared Responsibility Model

AWS’s responsibility is “Security of the Cloud”:

AWS is responsible for protecting the infrastructure that runs all of the services offered in the AWS Cloud. This infrastructure is composed of the hardware, software, networking, and facilities that run AWS Cloud services.

The customer—that’s you!—has responsibility for “Security in the Cloud”. This translates into concerns like data, encryption, safe deployment, network traffic encryption and configuring services correctly and safely. The more managed the services you pick for your architecture, the more you leave stuff to AWS. Don’t underestimate the money you save by not being breached or having to care just as much about security. That’s my #1 tip, actually: Use managed services if you can.

Then, get a hang of the OWASP Top 10. It’s an authoritative list of wide-scale web application problems/concerns that has stayed relevant (and virtually unchanged) for years. It speaks tons about how prevalent certain types of issues are.

Philosophy (also known as “TL;DR”)

If I have to condense AWS security and how you should approach it, then I think the below summarizes it, though of course a bit forcibly:

  • Use infrastructure-as-code as a core pattern. Avoid manual work at all costs, both because of toil/waste of time, but even more because of the possibility to version control changes to your overall systems, avoid drift between environments, and also to deploy new, complete environments in a matter of minutes.
  • Never allow users to have non-expiring access.
  • Look at testing regularly with different cadences and scopes: maybe system-level scan once a day; penetration-testing (“pentesting”) each application when new features get added; adding automated alarms and remediations for events that should be auto-handled in the future…
  • Externally validate security from time to time so you stay “real” and don’t become a prisoner of your own pipe dream.
  • Shift left on security so it’s an ongoing activity shared with developers. Secure a solid build pipeline, and do at least the minimum (easy) steps like auditing dependencies (Snyk, Github’s built-in Dependabot, or even as simple as npm/yarn audit).
  • Be humble about how much time and energy and patience will need to be spent to ensure understanding, empathy and knowledge from developers, product owners, business analysts and all else involved when shifting responsibility onto them. Give time, slack, and possibility to do training.
  • Use encryption for every single thing and every service. If you don’t have a known and good reason to do otherwise, always rely on latest standards.
  • Never have secrets (such as sensitive environment variables) in plain text; instead use Secrets Manager or Parameter Store to fetch the values.
  • Don’t get yourself into a bind with making it impossible to rotate keys since non-rotated keys will inevitably become a security issue at some point. Never specify them hardcoded or passed-in anywhere even if it can be very convenient to place them in (for example) Bitbucket deployment variables.
  • Assume a whitelisting policy (rather than a “blacklisting” policy) where you explicitly allow actions, and therefore implicitly deny any other actions. This is a lot safer than specifying what a user can’t do, but this approach can also add a bit more management overhead. In short, this means your standard approach will be:
Trust no one. Deny everything.
From https://www.keepcalmandposters.com

Remember: “It’s not paranoia if they’re really after you.”

Man, I saw this film too many times as a 12-year old. Is that the real reason I wrote this article? 🤔

Hardening your AWS account security

General

  • Create resilience and buffers between systems (ex. use SQS) so systems can’t fail with domino effect.
  • Separation of concerns/segregation of duties wherever you can.
  • Setup automated alerts and minimize need for manual monitoring of anything that can break. AWS is a cloud so everything emits an event that can be handled, you just need to figure out what those events are.
  • Have clear processes for onboarding/offboarding so users can be added/removed. No, your processes will not be best-in-class in the first year or two, if you’re a young company, so spend the time wisely. In a big company with more process-orientation, you may instead need to loosen yourself from old manners and ways-of-working if those hurt you.
  • Use tagging to know what resources (like databases or storage buckets) contain different types of data, so you can comply with information requests and GDPR etc.
  • Never allow blanket star (“*”) access unless it makes absolute sense. Try to always give systems or users only the minimum required access.

Account settings

Enforce a strict password policy, such as:

  • Expire after a number of days
  • Use 16 or more characters
  • Require lower and upper case letters, and numbers
  • Require special characters
  • Deny last 24 passwords
Example of a strict password policy (AWS IAM).
  • Lazy? Set up a security baseline, for example with Terraform. Regardless, consider packaging networking and security guidelines in something like Terraform so it can be used across several accounts.
  • Rotate access credentials (access key and secret access key) for users/roles that are exposed or sensitive or important.
  • Use Secrets Manager and rotate keys that are safe to auto-rotate, i.e keys that are “masters” with Secrets Manager. Fetch secrets from Secrets Manager in programmatic scenarios.

AWS Organizations

  • Use AWS Organizations to create and invite accounts, and synchronize account-level settings across them.
  • Turn on all features so you can set up security later with Security Hub etc.
  • Set up Service Control Policies.
  • Set up Tag policies and enforce them with the above SCPs.
  • Ideally create a separate “logging” account where you can funnel all (or at least most) logs mentioned in this article.

Security tools (Security Hub, GuardDuty, Config…)

There are a LOT of tools. Read up on all of them and decide which ones make sense for you and your usage. I highly recommend Security Hub, Config, and GuardDuty at minimum.

There are even more tools that aren’t in the strict security category, like AWS Config, but which will absolutely help you reach a more secure cloud.
  • Configure Security Hub (and any required tools) to work cross-account and feed your master account with all the data, so you can get a consolidated view.
  • Right now, the available standards that you can activate are CIS Foundations and PCI-DSS. I recommend that you turn on both, and then deactivate any specific checks that are not relevant in your case.
  • Also set up CloudTrail. Funnel logs into a super-secure logging account.
  • Use Service Control Policy to disallow tampering with any of the logs.

IAM

  • Only use “deployment user” type system users, or other absolutely necessary users.
  • Try to let (for example) deployment systems have a basically naked role at first, and let it assume (for the minimum duration) a role with more privileges while it does its work. No user should have permanent or non-expiring access.
  • IAM users should not have policies directly attached, if you set up IAM users. Instead let them be in groups with policies attached to those.

SSO

  • Use AWS SSO for user management: it’s a LOT easier, faster, and more convenient than using regular IAM for flesh-and-blood people. Allow user sessions to be max 10 hours (or so) so they need to log in once every day.
  • Use groups to separate users. Developers, for example, could be further segmented into their respective product team.
  • Force MFA for all SSO users and disallow email login.
  • Use same password policy as above (16+ characters etc.).

Networking and VPC

  • Close down default VPCs and security groups as far as possible. Remove all inbound and outbound rules from default security groups so those restrict all traffic.
  • Your own Network Access Control Lists and Security Groups should allow only minimum required ports, both ingoing and outgoing.
  • Never actually use the default security group; always use custom ones per application use-case. Else disallow or make it as hard as possible to use the AWS-provided default security group.

KMS

  • For anything that needs to be encrypted and shared with external users/accounts, you will want to have customer-managed keys.
  • I’ve found no real reason to not use AWS-managed KMS for most cases. Unless you will need to share access—or if regulation and procedure dictates you create and manage your own key—the regular AWS-managed one will do just fine.

S3

  • Buckets should have some kind of same-region (or cross-region) replication for added redundancy, probably in completely different accounts.
  • Buckets should have encryption. Always encrypt, starting with standard AWS-provided keys is super simple. Just do it!
  • Always block all public access for anything that is not absolutely meant to be public. Even then you could use Origin Access Identity with CloudFront (for sites, ex.) to lock down usage.
  • Don’t use Access Control Lists for S3, just go with bucket policies. (There MAY be cases where you have to use ACL, but it’s practically deprecated now)
  • One strategy worth attempting if you don’t (only) use us-east1 is to replicate and adding versioning for sensitive content (with Cross-Region Replication; avoid doing this manually!) and running AWS Macie on the bucket to ensure it does not contain things it shouldn’t (credentials, secrets, credit card numbers etc.).
  • Set MFA Delete to be required for anything that you absolutely don’t want to delete, such as customer data.
  • Only allow HTTPS to S3, using a bucket policy (could be done with Service Control Policy as well, I suppose).
  • Turn on object versioning if it makes sense.
  • Turn on access logging if it makes sense.
  • Set account-wide “block all public access” if possible.

WAF

  • Regional WAFs can be used with API Gateways. Try to cover OWASP 10. Beware of both low-volume (password field hacking) and high-volume threats (DDOS).
  • Web Application Firewall (global) works for CloudFront. Make changes that are relevant for a higher volume of traffic than the API ones.
  • Consider starting with a package like AWS WAF Security Automations.

Also see this resource for more information:

CloudFront

  • Always use HTTPS, and use TLS 1.2/1.3 if you don’t absolutely require legacy compatibility for certain clients.
  • Use Origin Access Identity with S3 if running S3 for frontends.

Lambda and Lambda@Edge

  • Provide Serverless Framework (or whatever you run) plugins like iam-roles-per-function (i.e. granular function permissions).
  • Don’t expose private functions/services, use invokeLambda() functionality instead to keep private functions hidden and unavailable.

API Gateway

  • Request mapping if you can do it.
  • Set up rate limiting.
  • Use API keys if necessary (though never rely on API keys for any real security, just as a tiny layer of obfuscation).
  • Use Cognito or Lambda authorizers to authenticate users.
  • If using Serverless Framework (or similar), make sure functions are only accessible through an API and not also through direct invocation.

EC2

  • Lock down VPC, NACL and Security Groups.
  • Never directly expose your EC2 instances; put them behind some security and/or obscurity. A good choice would be an Application Load Balancer or Global Accelerator.

DynamoDB

  • Always use encryption.
  • Set up backups, maybe point-in-time recovery on a schedule.
  • You may want to look into Global Tables as it gives you added resiliency.
  • Disallow scan actions. This can be done easiest in a Service Control Policy on any accounts where this is likely to be used.
  • Set up on-demand capacity instead of provisioned capacity (if it makes performance and financial sense), since it’s going to be a lot harder to take your systems down if someone overloads your database.

SQS/SNS

  • Encrypt them and any Dead Letter Queues.
  • Use SQS and SNS as buffering layers for any potentially high-volume traffic.

Improving processes to include security

I’ve found that simply hoarding platforms and tooling and knowledge in a tiny group may absolutely help an organization, but any modern theory on how to do development will shine the spotlight on the individual developer and his/her/their contribution. Investigate how your org could take on something like:

I recently wrote my own article on how hard it is to get from “legacy/traditional” operations to DevOps or anything like it. Might be worth 10 minutes of your time?

Testing your security posture

Time to test your security posture? Done with the AWS-provided stuff? In that case, it’s high time to set up some custom tooling!

Pentesting

The above repository collects the below popular (and relatively easy-to-use) tools:

Other tooling and resources you may want to use:

  • OWASP ZAP—”The world’s most popular free, open source web security tool.”
  • Artillery — Load testing tool; community edition is free. Use it to pressure-test your systems.
  • Gremlin—Chaos testing tool. Free version offers basic capabilities that still may give you a fair bit of leverage.
  • DNSDumpster — Free domain research tool. Useful when trying to map your origins and domains for infiltration points, and when you want to know what information you might leak through DNS.
  • TunnelBear — Native application for Mac, Windows or Mobile; free 500 MB per month. Use it when you quickly need to test connecting to resources from other locations/countries.

And a couple of services that might be interesting to you:

  • Detectify—Automated security and asset monitoring for all teams. Also have white-hat hackers on staff to break your things.
  • Prisma Cloud—Previously PureSec, one of few serverless security companies, if that’s how you roll.

Further resources

Considerations when working with serverless

Conduct regular Well-Architected Framework reviews

Only recently did I actually do a Well-Architected Framework review by using the Well-Architected Tool. It has a number of “lenses” which is basically a specific area with its own set of concerns. Just recently, the serverless lens was released, which is one of the reasons I did it. There is free training from AWS available for it, as well. Could be worth doing it once every 3/6 months, or for product teams to do it every once in a while.

Keep on fighting the bogeyman every day

I discussed recently with a seasoned, senior security expert, and I complained a bit that it seems that it’s sometimes a struggle to quantify the importance of security. He almost scoffed as he said (and I paraphrase):

Marriott got hacked recently. It was a data breach of relatively non-sensitive information. It cost them over a hundred million dollars. British Airways was also hacked, they got about the same percentage fine, resulting in even bigger fines. There is also bad-will and bad publicity and distrust as an effect of this. Security can absolutely be quantified. Security is what ensures you do not make mistakes that force you to pay that kind of money.

For me, security is one of those areas where hard tech and soft processes and people skills go together, which is why I’ve spent considerable time working recently with cloud security. No technology can save you from a malevolent employee, either. We haven’t even touched that kind of security here!

Fighting the bogeyman requires a lot of new tools, skills, and ways of working. Hope you got something out of this short overview, and best of luck in defending your castle from your own barbarians!