Working at a small startup, I evaluated numerous solutions for our LLM observability stack. That was early this year (IIRC Langfuse was not open source then) and Phoenix was the only solution that worked out of the box and seemed to have the right 'mindset', i.e. using Otel and integrating with Python and JS/Langchain. Wasted lots of time with others, some solutions did not even boot.
I suppose it depends on the way you approach your work. It's designed with an experimental mindset so it makes it very easy to keep stuff organized, separate, and integrate with the rest of my experimental stack.
If you come from an ops background, other tools like SigNoz or LangFuse might feel more natural, I guess it's just a matter of perspective.
sure, just starting to get some up on HF. A good example might be GSM8K as this shows the structured output where every result is strictly formatted - I am using this right now to train models and managaing to get a small qwen model up in the 60% range, which wildly is higher then llama2 and xAI Grok 1
I cant keep track of these agent platforms (or whatever you call it.. is there even a term?) They seem to be popping out of nowhere and getting eye popping funding, that too in rounds that shouldnt be happening so soon. Am I going mad or is the bubble super frothy?
This information is very useful to the open source community. Whats the rationale in not "building in the public"? Is Ollama turning its back on the open source community? Also why should we believe ollama web search is better than my locally run searxng server?
Oh yes! that is why I want to provide the names of the providers we use. I do believe in building in the open. The web search functionality has a very generous free tier (it is behind Ollama's free account to prevent abuse) that allows you to give it a try comparing to running a searxng server locally.
On making the search functionality locally -- we made considerations and gave it a try but had trouble around result quality and websites blocking Ollama for making a crawler. Using a hosted API, we can get results for users much faster. I'd want us to revisit this at some point. I believe in having the power of local.
"..words are a terrible straitjacket. It's interesting how many prisoners of that straitjacket resent its being loosened or taken off."
Stanley Kubrick
Do not mock what you do not understand. This person (persons? collective consciousness?) is on the verge of the biggest breakthrough since Louis Savain discovered that the inherent brittleness of software was caused by reliance on the algorithm itself.
Developing a bypass to language is hardly a breakthrough. It's been underway 20K years. We just got sidetracked by symbols. You engineers are so reliant on math, you can't see a way around it. That was the only way, math was a trick to specificity. It can't work.
whats your solution to bypass language? I do see the point that its a lossy compression medium but I also dont see how we can directly hook up our latent spaces
And no we didn't need a subscription reminder every 10s of interaction