I've used the ASEA to get a number of organizations setup. I prefer it to Control Tower (it can be installed on top of CT). The ASEA is open source and written in AWS cdk so it can be forked and modified if needed.
The guy or gal that's been working on it the last two years has slowly been working through my bucket list.
If I can just get guardrails that configure the basic AWS security foundation stuff like password policies, I'll be satisfied. And oddly enough, the CloudFormation coverage for this stuff is abysmal. We don't even allow IAM users in the member accounts, but we really need to check off this compliance box.
Seems relevant to me. The article is from your organization, and that information is something I wouldn't have known if not for the prior commenter's comment.
If they had disclosed their involvement in their profile at least I could give them the benefit of the doubt but in this case, like the other commenters, I assumed he had used the tool as a customer and had a positive experience, not that he was literally the founder of the group making the tool.
That’s not to say his opinion is not wanted, just that the potential bias should be made transparent.
There aren‘t any DMs on Hacker News, and if he didn’t post that comment I would have thought it was a disinterested commenter recommending something they had used, not somebody who works on the project. The cultural norm here is to disclose when you are recommending your own product and it’s not childish to point out when people fail to do that, it’s reinforcing that cultural norm.
Good thing they... (checks notes) ...know exactly who made the post and can reach out to them on LinkedIn or email them.
Cultural norms are reinforced by good examples, there's nothing good about snippy public comments. Reaching out to them privately isn't hard, and, dare I say it: is more inclusive. I don't think I need to explain the origin of the word assume.
It's impolite to assume on someone's behalf in public.
Could you explain how messaging in private is so hard to do if you have no problem making the comment in public...?
You seem to have a really hard time grasping that this entire comment thread we are part of wouldn't exist if OP had reached out about their concerns in private.
> It's impolite to assume on someone's behalf in public.
They asked a question, they didn’t state an assumption.
And why is it impolite.
> Could you explain how messaging in private is so hard to do if you have no problem making the comment in public...?
we’re not talking about difficulty, we’re talking about politeness.
> You seem to have a really hard time grasping that this entire comment thread we are part of wouldn't exist if OP had reached out about their concerns in private.
i don’t think i’d be here if it weren’t for you calling him impolite.
in fact, i’m quite surprised you would assume i have a hard time grasping why this thread exists in public rather than messaging me in private. would you mind messaging me on another social media platform directly before you do that? i hear that’s the polite thing to do here.
And yet you haven't sent me any emails, hatware. My door is always open, why not practice what you preach rather than be repeatedly harsh and make a scene over existing community norms? Crowley subsequently disclosed the affiliation, we're good.
Maybe you're newer here; It's a courteous social more of the HN community to be actively transparent about potential conflicts of interest.
I'm actually a fan of Crowley, he's got quite a brain.
You get what you give. I don't think I need to send emails to the one who has no problem calling others out in public.
It's pretty simple, I requested that concerns about conflict of interest are taken offline. But, here you are, making snide remarks in public at another person. Zero for two.
For what it's worth, I spoke up on this because I would be quite annoyed if someone did not give me the courtesy of correcting a mistake in private before broadcasting publicly about it. It doesn't matter if it comes from a stranger or a trusted friend. I can tell you aren't picking this up, but I'm happy to explain it ad nauseum so you can be a better individual to your peers.
>It's a courteous social more of the HN community
From what I can tell, the "norm" is to gang up on new folks without thinking critically about it. It's reddit with less complexity and more ego.
You get what you give.
> why not practice what you preach
From your profile: "I subscribe to the ideology of live and let live."
There is a lot of value in the CoI being visible quickly. While with an email there could be an unbound delay in the CoI being visible. And often visibility for such things drops off quite hard.
If 90% of the impressions happen within an hour of the post happening and it takes 2 hours for the CoI being visible, then 90% of people probably weren't aware of it.
I regularily don't check emails for hours at a time, and would really apprciate someone publicly "calling me out" if I were to forget disclosing a CoI. And I am sure we could as @dang for how quickly impressions drop off on HN specifically. Reddit is the most similar platform and there, a topic usually is active for a few hours at most, so after that window, the disclosure would be practically useless.
And how is stating a disagreement bullying? You can just disengage if tgis is that unpleasant to you.
Sure, it'd be nicer to privately do it, but IMO in such cases, informing the public is the more important part rather than being nice. (There were no insults, no accusations of it being intentional or such, which I'd personally count as nice enough)
Or do you believe we have to coddle everyone online? (I am being serious, and do not intend to troll, but can see how it can be taken as trolling)
If config can go this long with half-assed implementation I don’t see why control tower is going to fare better with more adoption. Most large enterprises are going to want to roll their own anyway.
Centralized management and application of IAM policy with the goal of giving teams the freedom to manage their own account, including account security, while still protecting the organization as a whole.
When customers request single tenancy in the cloud, where single tenancy is referring to an AWS account, being able to automate account management will be important when trying to scale.
Short explanation: it allows you to access a Linux instance via SSH using SSM as an IAM controlled proxy or use RDP for Windows.
You don’t need ingress access to your instance or even egress internet access if your security policies mandate it as long as you set up the correct service endpoints.
Also, just use Control Tower and federate it with your IDP - Active Directory, Okta, etc.
A lot of us are busy solving business needs in smaller companies/startups and don't have the time nor expertise to learn every single AWS service and come up with a justification for utilizing it.
A lot of these tools actually make your life easier and faster in the long run.
ControlTower for example. Takes about 30 mins to setup on normal AWS (on GovCloud it was much more complicated, took me half a day). But then setting up new accounts is one click and it’s preconfigured with correct restrictions and security measures, which individually would take several hours per account to do without controltower. So it’s an easy savings from the beginning. The only real cost is the cost of AWS config. So if you’re using that already (for SecurityHub for example) then it’s nothing additional.
IAM Identity Center makes user management not only more secure but faster and easier. It will take half a day to maybe a full day to setup the first time. But now every new user will be a few clicks with access across multiple AWS accounts. You can remove them in one click across all accounts. So these are just really simple additions to your workflow that save you time and improve security.
SSM is another example. It’s adding a policy to your instance role and checking a box (or adding a flag in Terraform or CLI) and it’s enabled. It’s no additional cost. It saves you time because you don’t need to manage user accounts on the server anymore (they are managed broadly through IAM or PermissionSets). No more copying around SSH keys or rotating them when people leave. It improves security and saves you time.
There’s little (if any at all) downside to any of these things. It’s all upside. For the most part, these don’t even have any significant costs associated with them. They are generally provided for free where you’re only cost is the underlying resources that you’re managing, which of course your paying for regardless.
I should add. Which identity center (previously called AWS SSO) you can tie it into your G-Suite or Microsoft 365 and just have it create AWS accounts for new hires as you onboard them by making email accounts. When they leave it automatically removes their access.
Not to mention the quality of life on this tool is incredible. When you truly have tens or hundreds of AWS accounts, the SSO tool makes it so nice to jump between them as an actual user. And I’m actually a huge fan of the CLI integration to get CLI access to any of them with a simple command on the AWS CLI. It’s super slick and will save you probably 5 hours the first week you use it.
We started using it a year ago and it’s been a game changer at our organization. As a user I don’t ever want to go back to normal IAM. Such a pain.
I understand your frustrations with AWS, and I get that solutions for enterprises don't always work for smaller companies, but when someone describes a solution that reduces operational complexity while increasing security, they should get thanks. It's not their fault that AWS has too many services.
I think this is very important and generally poorly understood:
Scaling problems exist both up and down.
In exactly the same way there are solutions that work well in the small but become disproportionately expensive when you scale them up, there are solutions that are cost-effective on a large scale that become prohibitively expensive on a smaller scale.
The latter category includes a large chunk of enterprise-y cloud solutions.
This is an example of why the basic AWS Cloud Architect Associate exam is a good idea. It's how I learned about ControlTower (after using AWS for years)
I agree to an extent, though SSH may make sense for break-glass usage. SSM was unavailable during the large scale control plane outage last December, and I saw clients that would have been dead in the water without SSH as an alternative. That said, it's important to really think through how to provide that secondary layer without weakening the overall security posture. It's possible, but it does take work.
So this is the basic problem: it doesn't give strong isolation for many users. Something like goteleport.com is better if you need lots of people to remote into random machines.
These type of articles make me wanna quit doing anything in IT.
It's not a criticism of the author, more the current state of technology in AWS.
I'd really like to have just 1 AWS account where I can see and do everything there and not keep switching and think about account IDs or which account has what S3 bucket/server whatever.
It's always a tradeoff between blast radius and (in)convenience.
With everything in one account someone might accidentally destroy your production environment. That'll be awkward to explain. And with IAC, it could be as simple as one change to a VPC config that recreates instead of updates the VPC.
In normal IT (not Twitter) you usually have a production and test environment at least. The inconvenience more than makes up for accidentally destroying something in production. And before you say anything, if you haven't done that you haven't been in IT for long enough.
However, if you want to do it your way you should become the director and make it that way. You'll be sorry, but that'll be on your head.
I'm not sure if that's possible, but I haven't checked.
Same. My current company has the same problem, which is that they continually layer additional complexity onto their previous architectural mistakes to try and mitigate them. This results in a complicated system where different independent parts all interact with each other like a rube goldberg machine. The dynamics of the resulting system become sort of perilous and unknowable, and it paralyzes future changes because nobody can predict the nth order effects.
Then do it. Why does your activity in IT depend on what someone on the internet says? For the rest of us, multi-account setups offer a large number of advantages that outweigh the inconveniences and we have built tooling to deal with most of those anyway.
The really frustrating things is that while AWS make the "Account" their best isolation layer they make managing those accounts with their tooling absolutely awful.
Very nice write up. As a billing guy I especially liked the tip about using resource policies to enable cross-account access and save on KMS request costs.
There's one issue with companies using hundreds of AWS accounts if you're a vendor to them: integrating services. Some folks here may be interested in a technique called "CloudFormation StackSets" which can deploy bits of infrastructure to multiple AWS accounts in one command. Vantage uses this to setup our billing integration and we wrote up the method here, https://www.vantage.sh/blog/using-cloudformation-stacksets-t...
For an IdP it seems like Dex combined with an LDAP server would be the simplest and most flexible solution. For reliability, I'm curious about throwing together a really simple LDAP server that stores records in AWS S3. That way your IdP can be trivially replicated with as much reliability as you want and nearly no maintenance. (Dex's storage can be Etcd, but I would also look to implement S3 storage)
Honestly, having lived a parallel life to the Windows ecosystem, TIL about “red forest.” I do think, though, that cross-account AWS actions are much more first-class than it sounds like jumping between forests ever was.
> Don’t do this! Any principal in your management account, by default, is able to assume the OrganizationAccountAccessRole in each and every one of the accounts created using the organizations:CreateAccount API.
I should note that if you use AWS Control Tower Account Factory to create the member accounts then this role does not get created.
The "Audit" account that is created by Control Tower is probably the best one to serve as the Administrative Access Account
> Any principal in your management account, by default, is able to assume the OrganizationAccountAccessRole in each and every one of the accounts created using the organizations:CreateAccount API.
This is an untrue statement. For a principal in the management account to assume OrganizationAccountAccessRole, they need to have a principal-based policy that gives sts:AssumeRole permissions for it. Otherwise, great article. We use this pattern at $DAYJOb
What's this like on Google Cloud? Would you create a project to get into other projects and would that achieve most of what this achieves? And would you use a GSuite address so you don't log into the console just by logging into the email?
Everything in GCP is built atop the Google Auth system and tied to a GSuite domain. It’s Org->Folder->Project hierarchy is very similar to AWS Orgs. However, it’s far easier from there. IAM is tied to your gsuite email, and service accounts are also email addresses. One never needs to login with different creds to access another project. You just use your Google login or activate a service account.
Projects are a really flexible abstraction. My company has one for every stage for every team plus specialized projects for net/VPC, GCR, logs, etc, hundreds in total. We’re about halfway through the F500 list. Projects are a nice abstraction and getting the setup OP described is a lot more idiomatic to GCP.
There is a lot you cannot do, or are forced to do in specific ways. A shit-ton of GCP features require project-level access, and often it's impossible to make permissions more fine-grained. For other things, you have to modify or apply a policy at the Org level. It's really broken. You basically have to abandon GCP features if you want strong isolation guarantees for most of their features. AWS is not even close to as braindead with their design.
But you need to keep going back to the SSO console to switch accounts because only one can be active at once. With GCP you can have multiple tabs open with different accounts.
The best part is that the account is in the URL so you can just link to specific resources in different accounts. So many of our runbooks for GCP are like "click this link" whereas for AWS it is "make sure you are looked into {specific-account} then click this link". The latter is much more error prone and can break your workflow if you were doing something in a different account previously.
This* is a feature, not a bug for me – I use separate profiles in Chrome or Container Tabs in Firefox!
Edit: I realized you are talking about switching accounts twitter style. I don't mess with that - I use a separate Chrome profile. Also you are arguing for GCP, I thought you were arguing against it.
* Having separate accounts, not being able to have separate accounts - I know it's possible with AWS as AWS doesn't force you to use a single account.
You can’t have nested projects, but for the purposes of organizations there is folders and orgs, which are container of containers.
GCP’s IAM somewhat addresses the isolation and scope problem mentioned in the article. Not all GCP apis, atleast with respect to OAuth2, properly utilize IAM, insofar that they require overly power OAuth2 scopes. For example, to list cloud functions you need permissions to create and edit, too. That’s broken.
In GCP, many orgs find themselves proliferating in projects because GCP’s billing is abstruse. People isolate resources to projects so they know how much specific services actually cost.
This in turn presents another problem. GCP Web Console’s search doesn’t index well. For example, substring search doesn’t work on far too many resources. VMs are the exception, but we think this is inverted. Substring search should work on every resource, not be exceptional. Historically it didn’t even do cross project search. This is frankly not acceptable for a search company.
We created a second GSuite for GCP at Slack because we didn’t want email and other corp IT assets to be mixed into what could’ve (but didn’t) become production infrastructure.
For enterprises it seems this is already baked-in, ie. when you're a Google Workspace (previously GSuite) user, your project selector has an inherent hierarchy stemming from the domain, ie. example.com -> project1, project2, etc. and, in my limited experience, switching between accounts on the command line is pretty good. But this article still makes a good point about keeping different environments in different siloed projects.
It seems that with isolation between projects on gcloud the number of separate accounts needed is less, which is good because it's also harder and more expensive to create multiple accounts. If gsuite is used very carefully, 1 is enough, but I think 2 would be better for most.
I think you're misunderstanding AWS accounts; they're not talking about AWS "logins", they're talking about actual "accounts" which are the entities that house resources like GCP projects. You can have an "organization" that has many accounts under it with sensible IAM, although it's less clean than GCP.
Yeah, that's what I'm saying. There's a lot of overlap with people who are using GSuite for things like email and people who are using GCP for production systems. It isn't great that the login/2FA for email automatically give access to GCP. Email is used so often, it's hard to be as cautious with it all the time as one can be with a something used less often.
What tool do people here use to search across AWS accounts?
Disclaimer: we are building a search engine to search for resources across “workspaces”. In AWS, this unit is the Account. In GCP, this unit is the Project.
Yeah except having to navigate between the two can be tricky. Also that means we have multiple dynamos and multiple cognitos (in our case, test and prod), which is a pain.
I use Firefox for the prod console and Chrome for the test console. Obviously that system doesn't scale past 3-4 accounts. ;)
I would say that once you get beyond those additional two-three accounts, you have tackled the mess and scaling to 1500 gets easier. And yes it's mind blowing.
We couldn’t build from source the last time we tried, but it is open source. We just made a decision today to open source our UI, so let us know if you’re interested
Same in the previous company I worked for. All in one account, spending millions of dolards on it, terrible architecture. From the top of my head: Unecessary peerings everywhere, bad subnet configuration that provoked network conflicts in certain cases, no autoscaling, IAM access based exclusively on keys, no VPN etc And if that wasn't enough, everything was "done" with one of the most terrible examples of "automation" I've ever seen in my whole life: A huge Terraform mono repo with hundreds of files with hundreds of duplicated resource calls (because they didn't even use modules). It took them at least a week for just adding a simple IAM user without breaking anything else and the CD would take hours to finish. An absolute nightmare. As no one else in the company had experience with the cloud, the team managing all I mention was threated as some kind of supreme elite as everyone else though they were doing a great job. Beyond me.
If we all started using Control Tower perhaps they'd get funded enough to continue to build it out and make it awesome.