You need multiple services whenever the scaling requirements of two components of your system are significantly different. That's pretty much it. These are often called micro services, but they don't have to actually be "micro"

That's the most nonsensical reason to adopt microservices imo.

Consider this: every API call (or function call) in your application has different scaling requirements. Every LOC in your application has different scaling requirements. What difference does it make whether you scale it all "together" as a monolith or separately? One step further, I'd argue it's better to scale everything together because the total breathing room available to any one function experiencing unusual load is higher than if you deployed everything separately. Not to mention intra- and inter-process comm being much cheaper than network calls.

The "correct" reasons for going microservices are exclusively human -- walling off too much complexity for one person or one team to grapple with. Some hypothetical big brain alien species would draw the line between microservices at completely different levels of complexity.

At this point, I'm convinced that too many people simply haven't built software in a way that isn't super Kubernetes-ified, so they don't know that it's possible. This is the field where developers think 32 GB of RAM isn't enough in their laptop, when we went to the moon with like... 4K. There is no historical or cultural memory in software anymore, so people graduate not understanding that you can actually handle 10,000 connections per second now on a five-year-old server.

Many developers started their career during the ZIRP era where none of the typical constraints of "engineering" (cost control, etc) actually applied and complexity & high cloud bills were seen as a good thing, so no wonder.

Splitting up responsibilities into separate services should in fact primarily be done to reduce costs.

You’re focusing on the theoretical and ignoring cost. That’s incompetent engineering.

There is in fact software that consumes large amounts of resources for fundamental reasons that saves real dollars when it is split into different units for the purpose of scaling those units independently. Most of that software is not primarily dealing with the number of connections it has.

What are you talking about.

When people talk about scaling requirements they are not referring to minutiae like "this function needs X CPU per request and this other function needs Y CPU per request", they are talking about whether particular endpoints are primarily constrained by different things (i.e. CPU vs memory vs disk). This is important because if I need to scale up machines for one endpoint that requires X CPU but the same service has another endpoint requiring Y memory whereas my original service only needs Z memory and Y is significantly larger than Z then suddenly you have to pay a bunch of extra money to scale up your CPU-bound endpoint because you are co-hosting it with a memory-bound endpoint.

If all your endpoints just do some different logic and all hit a few redis endpoints, run a few postgres queries, and assemble results then keep them all together!!!

EDIT: my original post even included the phrase "significantly different" to describe when you should split a service!!! It's like you decided to have an argument with someone else you've met in your life, but directed it at me

I think I get your point: e.g. some part of your application absolutely needs the world's fastest NVMe as its local disk. Another part of your application just needs more cores from horizontal scaling. If you horiz scale to get more cores, you're also paying to put the world's fastest NVMe on each new machine.

Eh... I think you can achieve this in a simpler way with asymmetric scaling groups and smartly routing workloads to the right group. I feel like it's an ops/infra problem. There's no reason to constrain the underlying infra to be homogenous.

Another case I’ve seen is to separate a specific part of the system which has regulatory or compliance requirements that might be challenging to support for the rest of the larger system, eg HIPAA compliance, PCI compliance, etc.

(To clarify, I’m not disagreeing with you!)

Good point, I was primarily thinking of engineering reasons, but alas there are indeed other concerns.

It has other advantages.

Operationally, it is very nice to be able to update one discrete function on its own in a patch cycle. You can try to persuade yourself you will pull it off with a modular monolith but the physical isolation of separate services provides guarantees that no amount of testing / review / good intentions can.

However, it's equally an argument for SOA as it is for microservices.

There are some other benefits like having different release cycles for core infrastructure that must never go down vs a service that greatly benefits from a fast pace of iteration.

Literally one function per service though is certainly overkill though unless you're pretty small and trying to avoid managing any servers for your application.

I came here to say the same. If you’re arguing either for or against microservices you’re probably not thinking about the problem correctly. Running one big service may make sense if your resource needs are pretty uniform. Even if they’re not you need to weight the cost of adding complexity vs the cost of scaling some things prematurely or unnecessarily. Often this is an acceptable precursor to splitting up a process.

You can still horizontally scale a monolith and distribute requests equally or route certain requests to certain instances; the only downside is that those instances would technically waste a few hundred MBs of RAM holding code for endpoints they will never serve; however RAM is cheap compared to the labor cost of a microservices environment.

For those numbers, yeah you should absolutely do that. But you might want to host your database on different machines than your application because the numbers will likely differ by much more than a few hundred MBs.

Seems like that would be pretty huge.

Except Apple is the one making file sharing hard between non-Apple devices and they aren't making most of their money on ads

There's a weird gleefulness about AI being a bubble that'll pop any day now both in this thread and in the world at large. It's kinda weird and I find most of the predictions about the result of such a bubble popping to sound highly exaggerated.

I think it’s because a lot of people feel like AI is being pushed from the top a lot with all kinds of spectacular claims / expectations, while it’s actually still difficult to apply to a lot of problems.

The AI bubble bursting would be somewhat of an “I told you so!” moment for a lot of people.

And there’s also a large group that’s genuinely afraid to lose their job because of it, so for that group it’s very much understandable.

Whether or not the predictions of the bubble popping are exaggerated or not, I cannot tell; it feels like most companies investing in AI know that the expectations are huge and potentially unrealistic (AGI/ASI in the next few years), but they have money to burn and the cost of missing out on this opportunity would be tremendous. I know that this is the position that both Meta and Google shared with their investors.

So it all seems to be a very well calculated risk.

I do agree that there seems to be a bubble, imo it's largely in the application space with the likes of cursor being valued at $23B+, but I don't see GPU purchases going down anytime soon and I don't even see usage going down. If these overhyped apps fail then it seems like something else will take their place. The power that LLMs provide is just too promising. It seems like those predicting things like a global economic crisis or the new big model-providers like OpenAI going to 0 just think that AI is like NFTs with no real intrinsic value.

> I don’t even see usage going down

Depends on the price to use these cloud LLMs.

I work for a company making AI related products. I've asked multiple Executive VP's if we are in an AI bubble. They all dance around the question.

I started working in 1997 and saw the dot com bubble and collapse. If you had asked me in 1999 I thought it would go on forever. I used to get 5 recruiter calls a day. In 2001 I got zero recruiter calls the entire year and it took me 10 months to find a new job.

I saw people at Cisco that had exercised stock options and held on to it at $80 and then it dropped to $20 and they had huge AMT tax bills that they couldn't pay even if they sold everything they had including their house.

My warnings of the AI bubble popping are not just an "I told you so" but also a warning to people to diversify out of their company stock and diversify in general.

>There's a weird gleefulness about AI being a bubble that'll pop any day now both in this thread and in the world at large

All i want is some cheap RAM for my PC.

I'd say the most mundane thing I use chat GPT for is to tell me which deeply nested menu some option is in for software I don't use very often.

I feel like most people would get value out of that.

Most recent example from a few days ago 'How do I change settings around spacing for "heading 2" in a document?'

If you're talking about all of AI with your statement I think you may need to reconcile that opinion with the fact that chat GPT alone has almost 1 billion daily users. Clearly lots of people derive enormous value from AI.

If there's something more specific and different you were referring to I'd love to hear what it is.

I’m probably an outlier: I use chatgpt/gemini for specific purposes, but ai summaries on eg google search or youtube gives me negative value (I never read them and they take up space).

I agree about the summaries! I think AI is applied in a lot of bad ways ATM although TBH I've heard some people like the summaries

I can't say I find them 100% useless - though I'd rather they not pop up by default - and I understand why people like them so much. They let people type in a question and get a confident and definitive answer all in natural language, which is what it seems like the average person has tried to do all along with search engines. The issue is that they think whatever it spits out is 100% true and factual, which is scary.

> Clearly lots of people derive enormous value from AI.

I don’t really see this. Lots of people like freebies, but the value aspect is less clear. AFAIK, none of these chatbots are profitable. You would not see nearly as many users if they had to actually pay for the thing.

There are hundreds of thousands of developers here including me who would vouch that AI does make them vastly more productive.

And AI skeptics are waiting to see the proof in the pudding. If we have a new tool that makes hundreds of thousands of devs vastly more productive, I expect to see the results of that in new, improved software. So far, I'm just seeing more churn and more bugs. It may well be the case that in a couple years we'll see the fruits of AI productivity gains, but talk is cheap.

The proof is in feature velocity of devs/teams that use it and in the layoffs due to efficiency gains.

I think it's very hard to convince AI skeptics since for some reason they feel more threatened by it than rest. It's counterproductive and would hinder them professionally but then it's their choice.

Without rigorous, controlled study I'm not ready to accept claims of velocity, efficiency, etc. I'm a professional software engineer, I have tried various AI tools in the workplace both for code review and development. I found personally that they were more harmful than effective. But I don't think my personal experience is really important data here. Just like I don't think yours is. What matters is whether these tools actually do something or whether instead they just make some users feel something.

The studies I've seen--and there are very few--seem to indicate the effect is more placebo than pharmacological.

Regardless, breathless claims that I'm somehow damaging my career by wondering whether these tools actually work are going to do nothing to persuade me. I'm quite secure in my career prospects, thank you kindly.

I do admit I don't much like being labeled an "AI skeptic" either. I've been following developments in machine learning for like 2 decades and I'm familiar with results in the field going back to the 1950s. You have the opportunity here to convince me, I want to believe there is some merit to this latest AI summer. But I am not seeing the evidence for it.

You say you've used AI tools for code review and deploys, but do you ever just use chat GPT as a faster version of Google for things like understanding a language you aren't familiar with, finding bugs in existing code, or generating boilerplate?

Really I only use chat GPT and sometimes Claude code, I haven't used these special-cased AI tools

> If you want rigorous controlled studies you'll get it in due time.

I hope so, because the alternative is grim. But to be quite honest I don't expect it'll happen, based on what I've seen so far. Obviously your experience is different, and you probably don't agree--which is fine. That's the great thing about science. When done properly it transcends personal experience, "common sense", faith, and other imprecise ways of thinking. It obviates the need to agree--you have a result and if the methodology is sound in the famous words of Dr. Malcolm "well, there it is." The reason I think we won't get results showing AI tooling meaningfully impacts worker productivity are twofold:

(1) Early results indicate it doesn't. Experiences differ of course but overall it doesn't seem like the tools are measurably moving the needle one way or the other. That could change over time.

(2) It would be extremely favorably in the interests of companies selling AI dev tools to show clearly and inarguably that the things they're selling actually do something. Quantifying this value would help them set prices. They must be analyzing this problem, but they're not publishing or otherwise communicating their findings. Why? I can only conclude it's because they're not favorable.

So given these two indications at this point in time, a placebo like effect seems most likely. That would not inspire me to sign a purchase agreement. This makes me afraid for the economy.

months worth of credits? What does that mean?

Productive how? If you’re not measuring the thing, how can you tell that the thing improved?

Feature velocity has increase manifold. And this is despite decreased team size.

A small startup now has the chance to compete with incumbents without raising from VCs.

Not talking about you, but I think cynics just look at it with too much pessimism and therefore can't see it.

It's not really about optimism or pessimism, it's effect vs no effect. Self reported anecdotes like yours abound, but as far as I'm aware the effect isn't real. That is, it's not in fact true that if a business buys AI tools for its developers their output will increase in some way that impacts the business meaningfully. So while you may feel more productive using AI tooling, in fact you probably aren't, actually.

No. If you're trying to make a causal link between some layoffs and AI tooling you need to bring the receipts. Show that the layoffs were caused by AI tooling, don't just assume it. I don't think you can, or that anyone has.

Most public companies aren't attributing it to AI yet due to fear of backlash, but it's obvious to most of us.

And Startups and SMB's are more open to attributing it to efficiency gains due to AI.

If you aren't seeing it then probably it's because you are in a niche vertical. Which is great for you, but not representative on the whole.

I am very much not an AI skeptic, I use AI every day for work, and it's quite clear to me that most of the layoffs of the past few years are correcting for the absurd over hiring from the Covid era. Every software company really convinced themselves that they needed like 2-3x the workforce they actually did because "the world changed". Then it became clear that the world in fact did not fundamentally change in the ways they thought.

Chat GPT just happened to come out around the same time so we get all this misattribution

I agree, the first thing I asked copilot to write me was a feature estimator:

def mult2(points): return 2xpoints

Its going to be really interesting what happens to this labor pool once you've finally sunk all the ships youre let aboard.

> Feature velocity has increase manifold.

And yet, every time somebody not paid by the AI companies tries to measure it, they don't find it.

From what I hear, many doctors pass X-ray and MRI scans to Chatgpt for a ...second opinion :-)

As in they didn't ask leetcode-style questions or you solved the same sort of questions no problem?

These were real-world type problems etc. Maybe I just use cursor too much and I've forgotten the short-hand they were looking for.

Prior I never had to do leetcode screens for interviews because of my network - don't really regret not burning tons of hours grinding leetcode.

My best guess is they thought my skills weren't advanced enough for my age or found it weird I was bald.

IMO it's always been a red flag if a company doesn't do coding screens, even if someone at the company says they're good. There's just too many awful programmers out there to be willing to move forward with a candidate on word of mouth alone.

It really sounds like you just need to brush up on your interviewing skills, maybe some leetcode practice, maybe you just need someone to stare at you while you solve problems to get over the nerves of the situation.

I used to practice in an environment without even basic autocomplete because I felt like it was more effective, maybe something like that would help you.

Couldn't agree more - I'd even be open to an unpaid contract period before an offer goes fulltime.

> Prior I never had to do leetcode screens for interviews because of my network - don't really regret not burning tons of hours grinding leetcode.

I hate to be the one to break this to you but this is what it comes down to. For those without great networks there is only leetcode hell. I don't have to tell you how dumb it is, everyone hates it for a reason, but that's the game. If i was in your shoes i would try to level up before changing course, i.e banging out 1-2 leetcode hards everyday. Best of luck which ever path you take.

says "or found it weird I was bald."

You're bald! That explains everything! Why didn't you mention this earlier?

First off, shave your head. Now you're no longer bald and you're signaling that you are in control. Use your baldness to dominate instead of lurking.

Secondly, you seem to object to boomers, but boomers are the best thing that happened to you: boomer music, boomer drugs, boomer sex is the best! You know the boomer ethics so adopt them, take Boom to heart: become a boomer and use your baldness to pound it into your opponents!

"Boom, boom, boom

let's go back to my room,

So we can do it all night,

and you can make me feel alright!"

https://www.youtube.com/watch?v=xXMrDu7374Y

Are you only applying to senior roles? Idk the specifics of your background, but it sounds like you haven't been somewhere long enough to get promoted and have largely worked on your own projects.

To me that doesn't really say that you have senior engineer skills, i.e. designing scalable systems (both in compute and development-wise), leading multi-person projects, considering trade-offs, etc.

I built a bespoke IOT stack that currently runs over 10k devices and a full production line interface / deployment for a factory we used in china. I'm not going to define what "senior engineer" is, but this was done largely solo with engineers I managed to maintain it.

Sounds impressive enough to probably get a senior role somewhere. Maybe you aren't selling yourself well in your resume?

hard to say - just trying to figure it out

Based on your responses, I think your soft skills / people skills are letting you down. I would recommend working diligently on how you interact and communicate so that you do not come across as arrogant, dismissive, or out of touch.

Too down to earth, OP wants to vent his frustrations.

> This is why any EE/ECE grads are much better developers than CS grads, because once you understand fundamentals

This is largely not the case in my experience. They probably understand the lower level details of manipulating memory better, but there's a lot more to developing software than understanding that memory is a specific place.

>but there's a lot more to developing software than understanding that memory is a specific place.

Yep, and all of that is derivative from how to organize memory. Classes are just more fancy structs. Object creation is memory initialization. Processing flow and code reuse is recognizing memory access patterns. ETC and so on.

All of software engineering is not derivative of memory organization. Algorithm efficiency for example is entirely independent of physical memory implementations. Just because an algorithm may use a set or an array does not make it derivative.

> Algorithm efficiency for example is entirely independent of physical memory implementations

This is wrong. Lots of algorithms behave completely differently based on memory layout. There is typically an inflection point where big O effects start coming into play, but at most scales, memory locality tends to be way more important in determining how efficiently something runs.

That's why I said algorithm efficiency and not wall-clock time or something else implying that I was talking about real-time.

If your algorithm is poorly designed and does significantly more work than necessary then that is independent of physical memory.

It's like you are trying to misunderstand me.

There are plenty of non political conversations on Reddit, it's a really big site.