> RAG is infinitely more accessible and cheaper than finetuning.

it depends on your data access pattern. If some text goes through LLM input many times, it is more efficient for LLM to be finetuned on it once.

This assumes the team deploying the RAG-based solution has equal ability to either engineer a RAG-based system or to finetune an LLM. Those are different skillsets and even selecting which LLM should be finetuned is a complex question, let alone aligning it, deploying it, optimizing inference etc.

The budget question comes into play as well. Even if text is repetitively fed to the LLM, that might happen over a long enough time compared to finetuning which is a sort of capex that it is financially more accessible.

Now bear in mind, I'm a big proponent of finetuning where applicable and I try to raise awareness to the possibilities it opens. But one cannot deny RAG is a lot more accessible to teams which are likely developers / AI engineers compared to ML engineers/researchers.

> But one cannot deny RAG is a lot more accessible to teams which are likely developers / AI engineers compared to ML engineers/researchers.

It looks like major vendors provide simple API for fine-tuning, so you don't need ML engineers/researchers: https://platform.openai.com/docs/guides/fine-tuning

Setting RAG infra is likely more complicated than that.

You are certainly right, managed platforms make finetuning much easier. But managed/closed model finetuning is pretty limited and in fact should be named “distribution modeling” or something.

Results with this method are significantly more limited compared to all the power open-weight finetuning gives you (and the skillset needed in return).

And in either case don’t forget alignment and evals.

> Results with this method are significantly more limited compared to all the power open-weight finetuning gives you (and the skillset needed in return).

I am not sure I understand why you are so certain that finetuned top market models, built by top researchers will be significantly worse than whatever open source model you pick.

> I partly disagree, software still sucks and it's a great time to build.

you can build great software, but the bigger challenge is to defeat network effect of preoccupied/monopolized market.

I agree, the monopolistic practices are concerning, especially against the open internet.

they want speed check in California, which is major market, and automakers likely will comply.

Its just it is not easy to come into specific pre-occupied space for outsiders.

But clickbench doesn't have joins..

interesting advice: monitor how often ClickHouse crashes with segfaults. Doesn't sound like production grade quality product.

I don't think is going to happen often, but definitely happens. We see that more than anyone would do because we run a lot of different workload types, any possible combination of types/sql/engines...

Running postgres in my previous company we saw a few of them as well and I'd consider postgres a "production grade" system.

maybe one line of defense is that Chrome is good for society, but no one wants to buy and support it..

Solution could be to mix RL training with foundational knowledge training, so LLM can refresh memory and not forget things.

Did you run the code?

No, but even if it is buggy, it is usual LLM cycle:

- you: there is no such function

- LLM: you are absolutely right, here is fixed code