The "API server" only runs when requests come in. Otherwise Modal scales everything down to 0, and you don't pay for anything. It's similar to putting Lambda behind an API gateway.
> assuming you have enough expertise in-house to run it.
Most small to mid-sized companies would almost always opt to pay a bit more for a managed offering and not have to bring in expertise in-house for things that are not their core competency.
For a small company, that's sensible. For a mid-size company, I think it's short-sighted. I'd argue that mid-size tech companies need the expertise that comes with running more of their own stack. If you don't have the expertise, then you're going to be dealing more with support engineers to solve your problems, and you won't have as much insight into your tech stack as you'd want.
There's a big difference between the experience of opening a support ticket to fix a problem with your database and paging one of your salaried engineers to fix it--someone who actually runs the service to begin with.
The way I've heard it phrased best is the way (I think) Allen Ward put it.
If a component is high cost, tightly integrated with the rest of the system, and difficult to design, then you need to be an expert on it in order to evaluate suppliers and make a good purchasing decision.
To become an expert, you have to build it for a while yourself. And once you have built it for a while and become an expert, you might as well keep that up, unless there's a strong economic incentive not to.
I've seen engineers build and run a clustered database and make a total pig's ear out of it. Every day was another outage (or the continuation of yesterday's outage). Several times we were a hair's breadth from actual irrevocable customer data loss. Just because you got a few of your salaried engineers to build a DB doesn't mean that they had the correct knowledge and respect for data to do it properly.
I’ve seen people make a mess out of SaaS offerings that are supposed to be foolproof. If we assume that we’re hiring incompetent engineers, we can make a mess out of any situation.
You’re both right: most companies want to hire the least competent (least costly) engineer capable of doing the job. Then they have mixed luck in hiring and wind up with a distribution of engineers, including some plain incompetent ones. They barely have competent enough engineers to build their core product, let alone perform undifferentiated technical operations on self-managed infrastructure.
I agree with you that if we assume all our engineers are strictly competent, then that gives us a major advantage over our vendors, and tilts the scales to build over buy.
I don't think it's strictly about competence and more about specialism. In my personal experience, which you can take with a large grain of salt, what happens is that the engineering manager with a certain type of career history tends to look down on any type of operations work and anyone who does it. I have heard things that come off sounding like this: "We hire world class computer scientists from prestigious Ivy League schools and FAANG experience, so why do I need to hire some one-trick-pony DB guy, when my engineers could build their own databases from scratch?"
How are mid-sized companies going to afford to retain that expertise, once they've trained it?
Or, to put it another way, mid-sized company X can never pay as much as hyperscale company Y for Z expertise. Because utilization at mid will always be less than at hyper.
I'm all for retaining inhouse expertise, but it's fundamentally a return-on-capital optimization problem, albeit one where your capital asset has the ability to walk out the door. So inhousing core, using managed everything else is a more reasonable bargain.
The expertise in question isn’t really used at “hyperscale” companies. Not everyone wants to work at large companies anyway. And the expertise isn’t that difficult to train or acquire (compared to getting another software engineer).
> Or, to put it another way, mid-sized company X can never pay as much as hyperscale company Y for Z expertise. Because utilization at mid will always be less than at hyper.
Being able to run a PostgreSQL cluster is a handy skill but won’t get you hired at Google.
Google can afford to pay an engineer to make cool stuff like a faster malloc implementation or analyze the best possible way to encode data to be stored on disk. That’s because making Google’s malloc 1% faster will pay someone’s salary many times over, or likewise for saving 1% CPU in your disk servers.
But running your own database cluster and container orchestration ain’t exactly rocket science these days, and running your own stack means you have an expert in-house when the shit hits the fan—and that’s when your expert earns their salary, many times over.
One root cause I've seen over during database outages is insufficient IO. It's caused by the other most common root cause: lack of actual expertise. Until you've been through the wringer, it's easy to lack respect for data: how important it is, how fucking large it is, and how long it takes to do basically anything with a few TB of it. If you didn't hire a real expert, and your DB-for-a-day guy miscalculated the spec for your bare metal clusters, you'll find it extremely hard to magic up faster hardware during a 4am full-outage when Europe wakes up. If you're on cloud, at least you have the flexibility to increase block storage IOPS or quickly reboot a node on beefier hardware or faster network connections. But suddenly the "cheap" DIY database is a lot more expensive than you budgeted for, and the phrase "nobody needs a dedicated DB guy" starts sounding really short sighted.
I’m talking about managing your own software on IaaS as an alternative to running off SaaS, you seem to think I’m talking about ditching the cloud entirely!
> Because utilization at mid will always be less than at hyper.
Will it?
Economies of scale have diminishing returns. If you have one engineer who is 20% busy, your engineer cost per unit will be five times higher than a company five times your size who keeps that one person 100% busy.
If you have four engineers who are all 100% busy, compared to a company 250 times your size with 1000 engineers, your cost per unit is equivalent.
Hey, your math makes sense, and is a good illustrative example, but in the real world if you schedule has your engineers 100% busy, then you have no slack to handle emergencies/unexpected events.
I'd target 60-70% utilization of engineer time. The remaining 30-40% is for stuff which can be dropped when needed (refactoring, non-critical/experimental projects, personal learning, ...). And if you have good engineers, they probably have a really good idea of how to prioritize for filling that 30-40% so as to move the company forward.
There is an inverse relationship between resource utilization and queue length, under reasonable assumptions. If you think of low CPU utilization as "wasted" CPU and try to fill it up, you can completely kill your service's ability to quickly respond to requests.
I like to make this analogy when talking to people who feel like they are not working hard enough. If you fill up an engineer's time with high-priority work, you get the same problem as if you fill up a machine's CPU with high-priority tasks... you get a system that cannot respond quickly, a system that spends a lot of time overloaded.
It's a fun exercise to try and calculate the relationship between utilization and expected queue length.
Plus, those batch jobs are still important, they're just not as time-sensitive.
Even for a small company in the software space, running your own self-hosted OSS stack isn't a big ask. I've spent ~80 hours over the past year setting up and running our stack. It's nothing too crazy[1] but it covers our needs with no ongoing fees and, more importantly, no outside dependencies to rely on.
[1] Proxmox hosting a virtual server running Ubuntu, providing Kimai, WeKan, Mattermost, Jitsi, Mediawiki, OpenVPN access, some Samba shares, source control, and comprehensive backups.
Ribbonfarm's take on kairos vs chronos: "internal and external clocks [...] individual time and social time, sensed time and read time [...] Bergsonian vs Einsteinian time." And now things that used be synchronized are becoming async, chronological time is bleeding into kairological time.
The fact is that you need people to have this collective delusion to work really hard and care about something, before value can be created. This kind of work ethic and commitment is what virtually every company that you admire today has required to get where it is today.
I personally don't think this is "dehumanizing" or unfair, since the whole point is to explicitly opt in for such a life. There are enough people out there who want to dedicate a lot of their time to working on a team or mission they care about, and OP's narrative is great at selecting for people like that.
I've learned the hard way that perfect abstractions don't exist. The mathematician in me wants to find the "most elegant representation" of a given problem, but when I give in to that urge, I often end up with a god function that takes `n` boolean flags that toggle the behavior slightly for different cases.
Why does this happen? I think a partial explanation is in this Nietzsche essay [1], where he says, "every concept arises from the equation of unequal things" — in other words, the abstractions of the world were built bottom up in our heads, and the Platonic essence of things is just a fairy tale we tell young programmers so they can sleep easily at night.
A key insight can be to abstract behaviors, rather than state.
It is typically much more possible to cleanly abstract the behavior for a single interaction/ role, than for the entirety of an entity's state. If you have multiple interactions you might want to use more than one interface.
Once you have clean behavioral APIs for your interactions, you may also be able to use composition, wrapping and delegation to implement/ enhance behavior. This is the 'Strategy pattern'.
The one thing you lose here is object identity -- you can no longer assume a delegate is the entity itself. This is no big loss given the clarity & flexibility that are gained.
Useful, but not universal. Efficiency is not the point of storytelling. Some details might not appear to have led anywhere in the story, but could have been there for texture, subtext or symbolism.
Chekhov's minimal, efficient, realist style is unique to him and complements his tone and content, but it's not a necessary condition for good writing. Consider Nabokov's short story Symbols and Signs (https://www.newyorker.com/magazine/1948/05/15/symbols-and-si...) which is beautiful purely because of subtext and texture.
It seems more relevant for movies and tv, where the camera often points to the gun very noticeably. I enjoy showing off as a prophet simply paying attention to that.
> The trouble with frameworks is that to use it, you have to match your application to the language of the API. If you’re not careful, you end up coupling your system to the framework. This means that you create a cycle between yourself and the framework by architecting your component as an interpreter from F_o to F_i.
You can avoid this cyclic dependency by marginalizing the framework at the edges of your application, as suggested in the post, but that strikes me as much easier to do with libraries.
1) there is no viable search competitor to Google, therefore
2) search must be a much harder problem that takes more engineers than we expect, therefore
3) it's hard to reproduce Google's success because you can't actually do what 10^n engineers do in a weekend.
I think that's plausible, but it needs to be emphasized that search could very well be not that hard, but no one has overtaken Google because they have inferior marketing, distribution, present too high of a switching cost, etc.
Just because Google has a bunch of engineers solving that problem (and reproducing what Google does seems complex for correct reasons pointed out in the article) doesn't mean that the engineers are actually adding that much marginal value. My guess is that even if Google invested less in search engineering, and didn't solve real time indexing and some other edge cases this well, it would still be leading the field.
Part of the challenge of competing, is that it's so easy for Google to adopt good features from other search engines.
I'm not sure what to call these, but you know the rich results Google searches provide these days? Like showing the definition of a word, or a snippet of data from Wikipedia off to the side of the main search results. It's a useful feature, and it was first developed by DuckDuckGo not Google. But it didn't matter, because it was so easy for Google to adopt a feature like that into their own product.
