Somehow people feel that if they won't do a generic solution for a problem at hand they failed.

In reality the opposite is often true, when people try to make generic solution they fail to make something simple, quick and easy to understand for others. Let alone the idea that abstraction will make system flexible and easier to change in the future. Where they don't know the future and then always comes a plot twist which does not fit into "perfect architecture". So I agree with that idea that abstraction is not always best response to complex system. Sometimes making copy, paste and change is better approach.

The take-away I took from it is that as developers, we love to solve problems using technical solutions. Sometimes, the real problem is one of narration. As we evolve our languages, better technical abstractions become available. But that's not going to prevent 'enterprisey' code from making things look and sound more difficult. Just look at any other field where abstractions aren't limited by technicalities: the same overcomplicated mess forms. Bad narrators narrate poorly, even when they are not limited.

Abstraction can be used to evoke category theoretical things, but more often it’s used to avoid making a decision. It’s fear based. It’s overbuilding the bridge to avoid problems we don’t know or understand. And that is not Engineering.

I find sometimes that it helps me to think of it as a fighter or martial artist might. It is only necessary to not be where the strike is, when the strike happens. Anything more might take you farther from your goal. Cut off your options.

Or a horticulturist: here is a list of things this plant requires, and they cannot all happen at once, so we will do these two now, and assuming plans don’t change, we will do the rest next year. But plans always change, and sometimes for the better.

In Chess, in Go, in Jazz, in coaching gymnastics, even in gun safety, there are things that could happen. You have to be in the right spot in case they do, but you hope they don’t. And if they don’t, you still did the right thing. Just enough, but not too much.

What is hard for outsiders to see is all the preparation for actions that never happen. We talk about mindfulness as if it hasn’t been there all along, in every skilled trade, taking up the space that looks like waste. People learn about the preparation and follow through, instead of waiting. Waiting doesn’t look impressive.

So that analogy works against your point.

In Go there are patterns that are probably safe, but that safety only comes if you know the counters. In a handicapped game, it’s not at all uncommon for white to probe live or mostly live groups to see if the student knows their sequences. You see the same in games between beginners.

Professional players don’t do this to each other. They can and will “come sideways” at a problem (aji) if it can still be turned into a different one, but they don’t probe when the outcome is clear. In a tournament it inflicts an opportunity cost on the eventual winner, and it is considered rude or petty. They concede when hope is lost.

They still invested the energy, but now it comes mostly from rote memorization.

I’m self taught so the former has been more challenging than it might be if I’d gone through a rigorous CS program, but I’ve benefited from learning among peers who had that talent in spades. The latter talent is one I find unfortunately lacking in many engineers regardless of their experience.

I’m also coming from a perspective where I started frontend and moved full stack til I was basically backend, but I never lost touch with my instinct to put user intent front and center. When designing a system, it’s been indispensable for anticipating abstraction opportunities.

I’m not saying it’s a perfect recipe, I certainly get astronaut credits from time to time, but more often than not I have a good instinct for “this should be generalized” vs “this should be domain specific and direct” because I make a point to know where the domain has common patterns and I make a point to go learn the fundamentals if I haven’t already.

I also have a lot of experience starting with very lo-fi and manual scripting prototypes to validate user needs and run a process like release management or db admin, which would then need to be wrapped in some light abstractions to hide some of the messy details to share with non-maintainers.

Problem is, I've noticed that more junior developers tend to look at a complex prototype that hits all the user cases, and see it as being complicated. Then they go shopping for some shiny toy that can only support a fraction of the necessary cases, and then I have to spend an inordinate amount of time explaining why it's not sufficient and that all the past work should be leverages with a little bit of abstraction if they don't like the number of steps in the prototype.

So, not-generic can also end up failing from a team dynamic perspective. Unless everyone can understand the complexity, somebody is going to come along and massively oversimplify the problem, which is a siren song. Queue the tech debt and rewrite circle of life.

In true it’s an optimisation problem where both under abs over abstracting, or choosing a the wrong abstractions lead to less optimal outcomes.

To get more optimal out comes it helps to know what your optimisation targets are: less code, faster compilation, lower maintenance costs, performance, ease of code review, adapting quirky to market demands, passing legally required risk evaluations, or any number of others.

So understand your target, and choose your abstractions with your eyes open.

I’ve dealt with copy paste hell and inheritance hell. Better is the middle way.

I often remember that old joke:

When asked to pass you the salt, 1% of developers will actually give it to you, 70% will build a machine to pass you a small object (with a XML configuration file to request the salt), and the rest will build a machine to generate machines that can pass any small object from voice command - the latter being bootstrapped by passing itself to other machines.

Also makes me remember the old saying

- junior programmer find complex solutions to simple problems

- senior programmers find simple solutions to simple problems, and complex solutions to complex problems

- great programmers find simple solutions to complex problems

To refocus on the original question, I often find the following misconceptions/traps in even senior programmers architecture:

1) a complex problem can be solved with a declarative form of the problem + a solving engine (i.e. a framework approach). People think that complexity can be hidden in the engine, while the simple declarative DSL/configuration that the user will input will keep things apparently simple.

End result:

The system becomes opaque for the user which has no way to understand how things work.

The abstraction quickly leaks in the worst possible way, the configuration file soon requires 100 obscure parameters, the DSL becomes a Turing complete language.

2) We have to plan for future use cases, and abstract general concepts in the implementation.

End result:

The abstraction cost is not worth it. You are dealing with a complex implementation for no reason since the potential future use cases of the system are not implemented yet.

3) We should factor out as much code as possible to avoid duplication.

End result:

Overly factored code is very hard to read and follow. There is a sane threshold that should not be reached in the amount of factorization. Otherwise the system becomes so spaghetti that understanding a small part requires untangling dozens and dozens of 3 lines functions.

---

When I have to argue about these topics with other developers, I often make them remember the worst codebase they had to work on.

Most of the time, if you work on a codebase that is _too_ simplistic and you need to add a new feature to it, it's a breeze.

The hard part, is when you have an already complex system and you need to make a new feature fit in there.

I'd rather work on a codebase that's too simple rather that too complex.

1) When gradually most of your implementation is happening in a DSL / graph based system all of your best tools for debugging and optimizing are useless.

2) So often I've seen people make an 'engine' before they make anything that uses the engine and in practice the design suffers from needless complexity and is difficult to use because of practical matters not considered or forseen during the engine creation. Usually much work has been spent on tackling problems that are never encountered but add needless complexity and difficulty in debugging. Please design with debugging having an equal seat at the table!

3) Overly factored code is almost indistinguishable from assembly language. - Galloway

I'm pretty good at fighting off features that add too much complexity, but the abstraction trap has gotten me more than once. Usually, a moderate amount of abstraction works great. I've even done well with some really clever abstractions.

Abstraction can be seductive, because it can have a big payoff in reducing complexity. So it's often hard to draw the line, particularly when working in a language with a type of abstraction I've not worked much with before.

Often the danger point comes when you understand how to use an abstraction competently, but you don't yet have the experience needed to be an expert at it.

This may be a price worth paying: transformations that actually reduce complexity are much easier to perform on an abstraction of a program than on the whole mess with all its gory details. It's just something to keep in mind.

[0] https://news.ycombinator.com/item?id=22601623

Also, it's pointless to claim that transformations on an abstraction can reduce complexity, but abstractions themselves can not. Abstractions are required for that reduction in complexity.

- default parameter values

- union types

- names arguments

I would also throw in list and map literals, to do away with something like varargs.

All of these are much simpler, implementation-wise, than method overloading. None would require anywhere near the compiler or bytecode-level support that method overloading does. It just has a very low power to weight ratio, when other langauge features are considered. And, unfortunately, it makes implementing all those other features (useful on their own) extremely difficult.

You need default values (and ideally named parameters) to really make them equivalent, but yes, both address a similar set of developer needs.

The java implementation, where it bleeds into bytecode and method scoring in insane ways, is particularly unfortunate.

Without overloading lookup would just be via name.

I don not think that method overloading is necessarily required to not make a mess.

But unfortunately, it is way overused. It takes a long time to develop good judgement. I'm still working on it.

I don't consider myself to be an exceptional developer, but this alone has launched my career much faster than it would if I was purely, technically competent. Ultimately, this is a sense of business understanding. The more senior/ranking you are at a company, the more important it is for you to have this tune in well.

It can be really, really hard to say no at first, but over time the people ask you to build things adapt. Features become smaller, use cases become stronger, and teams generally operate happier. It's much better to build one really strong feature and fill in the gaps with small enhancements than it is to build everything. Eventually, you might build "everything", but you certainly don't need it now. If your product can't exist without "everything", you don't have a strong enough business proposition.

----

Note: No, doesn't mean literally "I'm/we're not building this". It can mean two things:

* Forcing a priority. This is the easiest way to say no and people won't even notice it. Force a priority for your next sprint. Build a bunch of stuff in a sprint. Force a priority for another sprint. Almost inevitably, new features will be prioritized over the unimportant left overs. On a 9 month project, I have a 3 month backlog of things that simply became less of a priority. We may build them, but there's a good chance nobody is missing them. Even if we build half of them, that still puts my team 1.5 months head. For a full year, that's almost like getting 2 additional months of build time.

* Suggesting an easier alternative. Designers have good hearts and intentions, but don't always know how technically difficult something will be. I'm very aggressive about proposing 80/20 features - aka, we can accomplish almost this in a much cheaper way. Do this on 1 to 3 features a sprint and suddenly, you're churning out noticeably more value.

I'm not surprised; that and a lot of the Java "culture" in general seems to revolve around creating solutions to problems which are either self-inflicted or don't actually exist in practice, ultimately being oriented towards extracting the most (personal) profit for a given task. In other words: why make simple solutions when more complex ones will allow developers to spend more time and thus be paid more for the solution? When questioned, they can always point to an answer laced with popular buzzwords like "maintainability", "reusability", "extensibility", etc.

You don't have to go all C++ insane with it and allow overloading of everything, but just being able to do arithmetic with non-primitive types would be very nice.

A couple of operator overloading abominations in C++ are iostreams, and a regex DSL (where operators are re-purposed to kinda sorta look like regex expressions).

One in-between option I have kicked around w/ people is offering interfaces that allow you to implement operator overloading (e.g. Numeric). Then you wouldn't have one-off or cutesy operator overloading, but would rather need to meet some minimum for the feature. (Or at least feel bad that you have a lot of UnsupportedOperationExceptions)

Java had/has a ton of potential, but they kept/keep adding features that make no sense to me, making the language much more complex without some obvious day-to-day stuff like list literals, map literals, map access syntax, etc.

Oh well.

    cout << "Hello World";

It looks like they did that just because they can.

The thing is, people thought it was a good idea at the time, and had good reasons. It took maybe a decade before experience showed otherwise. Even today it takes a while to convince newer programmers that it is indeed a bad idea, and even then they don't really believe it.

An even more perniciously bad language feature is macros. I'm pretty sure that once I step down as D's BDFL or am forcibly removed D will get macros :-/

All three features, useful on their own, could be added to java at maybe 10% of the complexity of method overloading. With method overloading, they are all exponentially more complicated.

It's crazy how many places method overloading ends up rearing its ugly head if you are dealing with the JVM.

Gets to be lots of fun when you throw autoboxing and, especially, generics, into the mix.

I've implemented both, I disagree. Unless you are talking about default arguments in the presence of method overloading, which is insane, and which I have also implemented.

I mean, the java runtime has all sorts of crazy stuff going on w/ a security manager, to almost zero benefit. Seems easy enough to make something like this work.

The function definition looks like this:

  (defun foo (&optional (a 1 a-passed))
    (if a-passed (print a) 
       (print "a not passed"))

  > (foo 10)
  10
  > (foo 1)
  1
  > (foo)
  a not passed

You can also have better behavior between multiple defaults if you know whether someone gave a value or not.

Basically, it is another tool compared to overloading. It should be no surprise that there can be preference on how to use between them.

  >>> x = Foo(a=1, b=2, c=3)
  >>> x._replace(a=1000, c=3000)
  Foo(a=1000, b=2, c=3000)

It does: set a unique `object()` instance as the default. It will only compare equal to itself, but it's even clearer to read if you check for identity with `is`. You'll need to define it outside of the function definition.

It's not the most concise, but it works, and it's an established pattern used in many high quality open source projects.