I don't think you realize just how painfully expensive Oracle is. The reason they only went with an active/passive server layout vs. going active/active/active RAC is because they would've had to move to enterprise licensing. Then you're talking (at what I imagine their scale is) 7-8 figures a year.
So if you look at just enterprise licensing, it's $47k per "processor" - which in their world, a single intel core counts as 0.5 processors. So if you've got a 36 core machine (which I think they mentioned in the video) that's 18 "processors". Or $846,000 for one box. You want clustering (RAC) - that's another $315,000. Partitioning? $207,000. So now we're up to ~$1.4 million for a single server. YAY!
Obviously these are list prices, so you should be paying less than that, but it gets ugly FAST.
Judging by the salaries from job sites and my yandex interview, the cost would most certainly top up at $0.5 million (that's a very generous guess, probably half or two thirds that, realistically). Not sure how much Oracle costs at Yandex scale (a LOT more?).
Am I missing something? How do you figure 10 man years = $500K ? Or is Yandex super stingy? DBAs and Developers worth their salt (who could pull off a multi-TB migration/etc without significant outage), would easily each go for $140K/yr minimum (so, 180-200K fully loaded with benefits and such).
Still though: your point is valid in that, Oracle RAC cluster licensing and support is serious bucks.
it wouldn't have been 10 man years if they had less PL/SQL. I don't understand why people keep falling into that trap of putting complex logic in stored procedures.
No, you are not right. In video (Q&A section in the end) you can hear that rewriting stored logic took a couple of weeks of one developer. Maybe a couple of month with good tests.
It took much more time to rewrite backends logic because of lots of legacy code in many applications without using abstraction libraries.
Oh, cool. Can you expand on that? What kind of logic had to change? I mean, SQL queries are SQL queries? Is that because you were using low level C Oracle drivers of some sort?
Also if you could say a couple of words about the MySQL failure it would be rather helpful because I might have a friend who might be currently deciding on whether to migrate to MySQL or Postgres from Oracle.
We had several kinds of stored logic in Oracle:
1. for logical consistency of data (i.e. when you store new message you should increment counter for folder, thread, etc.). It seems correct to have such logic in the database rather than in the application.
2. for common code (i.e. moving a message from one folder to another may be done from web backend, mobile backend, imap backend or pop3 backend). This kind of logic is easier to have in one place than in all backends.
3. some business logic (i.e. finding thread for a new incoming message was done in PL/SQL).
First type is still in the database. Some part of type 2 was moved to our abstraction library (that is used by all applications). Logic of thrird type was moved to our applications and it greatly reduced stored code size.
About failed attempt of moving to MySQL. It was more than 10 years ago and I wasn't working at Yandex at that time but AFAIK the main problems were not around technical problems, they were organizational - not all members of the team agreed on moving to MySQL.
I'm a postgres fan and booster myself but I'd need a good reason to migrate a data backend on any project I'm managing.