One observation: Sundar's comments in the main video seem like he's trying to communicate "we've been doing this ai stuff since you (other AI companies) were little babies" - to me this comes off kind of badly, like it's trying too hard to emphasize how long they've been doing AI (which is a weird look when the currently publicly available SOTA model is made by OpenAI, not Google). A better look would simply be to show instead of tell.
In contrast to the main video, this video that is further down the page is really impressive and really does show - the 'which cup is the ball in is particularly cool': https://www.youtube.com/watch?v=UIZAiXYceBI.
Other key info: "Integrate Gemini models into your applications with Google AI Studio and Google Cloud Vertex AI. Available December 13th." (Unclear if all 3 models are available then, hopefully they are, and hopefully it's more like OpenAI with many people getting access, rather than Claude's API with few customers getting access)
He's not wrong. DeepMind spends time solving big scientific / large-scale problems such as those in genetics, material science or weather forecasting, and Google has untouchable resources such as all the books they've scanned (and already won court cases about)
They do make OpenAI look like kids in that regard. There is far more to technology than public facing goods/products.
It's probably in part due to the cultural differences between London/UK/Europe and SiliconValley/California/USA.
I don’t think Google is the same as IBM here. I think Google’s problem is its insanely low attention span. It frequently releases innovative and well built products, but seems to quickly lose interest. Google has become somewhat notorious for killing off popular products.
On the other hand, I think IBM’s problem is its finance focus and longterm decay of technical talent. It is well known for maintaining products for decades, but when’s the last time IBM came out with something really innovative? It touted Watson, but that was always more of a gimmick than an actually viable product.
Google has the resources and technical talent to compete with OpenAI. In fact, a lot of GPT is based on Google’s research. I think the main things that have held Google back are questions about how to monetize effectively, but it has little choice but to move forward now that OpenAI has thrown down the gauntlet.
In addition, products that seem like magic at launch get worse over time instead of better.
I used to do all kinds of really cool routines and home control tasks with Google home, and it could hear and interpret my voice at a mumble. I used it as an alarm clock, to do list, calendar, grocery list, lighting control, give me weather updates, set times etc. It just worked.
Now I have to yell unnaturally loud for it to even wake, and even then the simplest commands have a 20% chance of throwing “Sorry I don’t understand” or playing random music. Despite having a device in every room it has lost the ability to detect proximity and will set timers or control devices across the house. I don’t trust it enough anymore for timers and alarms, since it will often confirm what I asked then simply… not do it.
Ask it to set a 10 minute timer.
It says ok setting a timer for 10 minutes.
3 mins later ask it how long is remaining on the timer. A couple years ago it would say “7 minutes”.
Now there’s a good chance it says I have no timers running.
It’s pathetic, and I would love any insight on the decay. (And yes they’re clean, the mics are as unobstructed as they were out of the box)
Yes, we burned the biscuits when my sister-in-law was visiting over Thanksgiving because she used the Google assistant to set an alarm and the alarm did not go off. Timers no longer work and there's no indication that this is the case.
Google Home perplexes me. I have several of them around the house and they were perfectly fine for years, but someone in the last couple of years they are markedly worse. I would be happy if they just rolled back to 4 years ago and never touch it again. Now, I just wonder how much worse it will get before I give up on the whole ecosystem.
Same experience with Google Assistant on Android. I used to be able to use it to create calendar events in one shot. A few years ago it started insisting on creating events in steps, which always failed miserably.
> its insanely low attention span. It frequently releases innovative and well built products, but seems to quickly lose interest quickly. Google has become somewhat notorious for killing off popular products.
I understood this problem to be "how it manages its org chart and maps that onto the customer experience."
To add some color to this, the culture for a very long time would reward folks that came up with novel solutions to problems or novel products. These folks would dedicate some effort into the implementation, land the thing, then secure a promo with no regard for the sustainability of the aforementioned solution. Once landed, attention goes elsewhere and the thing is left to languish.
This behavior has been observed publicly in the Kubernetes space where Google has contributed substantially.
Along with your thoughts, I feel that Google's problem has always been over-promising. (There's even comedy skits about it.)
That starts with the demonstrations which show really promising technology, but what eventually ships doesn't live up to the hype (or often doesn't ship at all.)
It continues through to not managing the products well, such as when users have problems with them and not supporting ongoing development so they suffer decay.
It finishes with Google killing established products that aren't useful to the core mission/data collection purposes. For products which are money makers they take on a new type of financially-optimised decay as seen with Search and more recently with Chrome and YouTube.
I'm all for sunsetting redundant tech, but Google has a self-harm problem.
The cynic in me feels that part of Google's desire to over-promise is to take the excitement away from companies which ship* what they show. This seems to align with Pichai's commentary, it's about appearing the most eminent, but not necessarily supporting that view with shipping products.
* The Verge is already running an article about what was faked in the Gemini demo, and if history repeats itself this won't be the only thing they mispresented.
Google has one major disadvantage - it's an old megacorporation, not a startup. OpenAI will be able to innovate faster. The best people want to work at OpenAI, not Google.
Also there’s less downside risk for OpenAI. Google has layers of approvals and risk committees because they don’t want to put the money machine at risk through litigation, reputation or regulation. OpenAI has nothing to lose—this is their only game. That allows them to toe the line of what’s acceptable like Uber in its early years. With all the copyright risk involved, that’s a big deal.
I think the analogy is kind of strained here - at the current stage, OpenAI doesn't have an overwhelming superiority in quality in the same way Google once did. And, if marketing claims are to be believed, Google's Gemini appears to be no publicity stunt. (not to mention that IBM's "downfall" isn't very related to Deep Blue in the first place)
> OpenAI doesn't have an overwhelming superiority in quality in the same way Google once did
The comparison is between a useful shipping product available to everyone for a full year vs a tech demo of an extremely limited release to privileged customers.
There are millions of people for whom OpenAI's products are broadly useful, and the specifics of where they fall short compared to Gemini are irrelevant here, because Google isn't offering anything comparable that can be tested.
I'd say IBM's downfall was directly related to failing to monetize Deep Blue (and similar research) at scale.
At the time, I believe IBM was still "we'll throw people and billable hours at a problem."
They had their lunch eaten because their competitors realized they could undercut IBM on price if they changed the equation to "throw compute at a problem."
In other words, sell prebuilt products instead of lead-ins to consulting. And harness advertising to offer free products to drive scale to generate profit. (e.g. Google/search)
I don't really see how IBM would ever be able to monetize something like Deep Blue. It was a research project that was understood to not be a money-maker (outside of PR, probably), and it resulted in highly specialized hardware running highly specialized software, working for its one purpose. I agree that their business model and catering to big business first is what likely led to them scaling down today, but it's still disconnected from Deep Blue.
It's an interesting analogy. I think Googles problem is how disruptive this is to their core products monetization strategy. They have misaligned incentives in how quickly they want to push this tech out vs wait for it to be affordable with ads.
Whereas for OpenAI there are no such constraints.
Did IBM have research with impressive web reverse indexing tech that they didn't want to push to market because it would hurt their other business lines? It's not impossible... It could be as innocuous as discouraging some research engineer from such a project to focus on something more in line.
This is why I believe businesses should be absolutely willing to disrupt themselves if they want to avoid going the way of Nokia. I believe Apple should make a standalone apple watch that cannibalizes their iPhone business instead of tying it to and trying to prop up their iPhone business (ofc shareholders won't like it). Whilst this looks good from Google - I think they are still sandbagging.. why can't I use Bard inside of their other products instead of the silly export thing.
No, because OpenAI and Microsoft both have “CUSTOMER NONCOMPETE CLAUSES” in their terms of use. I didn’t check Apple, but Google doesn’t have any shady monopolistic stuff like that.
“What You Cannot Do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: […] Use Output to develop models that compete with OpenAI.” (Hilarious how that reads btw)
“AI Services.
”AI services” are services that are labeled or described by Microsoft as including, using, powered by, or being an Artificial Intelligence (“AI”) system. Limits on use of data from the AI Services. You may not use the AI services, or data from the AI services, to create, train, or improve (directly or indirectly) any other AI service.”
That 100% does include GitHub Copilot, by the way. I canceled my sub. After I emailed Satya, they told me to post my “feedback” in a forum for issues about Xbox and Word (what a joke). I emailed the FTC Antitrust team. I filed a formal complaint with the office of the attorney general of the state of Washington.
I am just one person. You should also raise a ruckus about this and contact the authorities, because it’s morally bankrupt and almost surely unlawful by virtue of extreme unfairness and unreasonableness, in addition to precedent.
AWS, Anthropic, and NVIDIA also all have similar Customer Noncompete Clauses.
I meekly suggest everyone immediately and completely boycott OpenAI, Microsoft, AWS, Anthropic, and NVIDIA, until they remove these customer noncompete clauses (which seem contrary to the Sherman Antitrust Act).
Just imagine a world where AI can freely learn from us, but we are forbidden to learn from AI. Sounds like a boring dystopia, and we ought to make sure to avoid it.
They cannot enforce a non-compete on a customer. Check out the rest of their terms that talk about durability. They will sneakily say "our terms that are illegal don't apply but the rest do."
You cannot tell a customer that buying your product precludes them from building products like it. That violates principles of the free market, and it's unenforceable. This is just like non-competes in employment. They aren't constitutional.
> But no, they cannot levy fines or put you in jail.
Those are the consequences that matter. I don't care if Microsoft or Google decide they don't want to be friends with me. They'd stab me in the back to steal my personal data anyway.
Sounds like we need legislature to void these "customer non-compete clauses". Not holding my breath though, see what govts allows copyrights to become. Govts seems to protect (interests of near-) monopolies more than anything.
This is a perfect example of the owner class getting away with crime (copyright infringement) and using it against the public (you can't use AI output!).
Businesses are not entitled to life or existence the way individuals are.
It's not unlawful, it's not morally bankrupt. Noncompete clauses have been around since the beginning of human commercial activity and have a valid reason to exist - to encourage companies/people/investors to put large sums of capital at risk to develop novel technologies. If there was no way to profit from them, the capital would be non-existent.
It's not theater, it's very real. Companies are making decisions to not use data generated from openai. They are making the decision because they know if they go the other way they know they risk it being leaked via someone internal that they are doing it, that it's pretty easy to figure out during a discovery process. I'm involved in this issue right now, and no one is treating it as something to just blow off. I know several other companies in the same boat.
They have many orders of magnitude more money and attorneys that would work full-time on such a case to ensure that even if they lost the court battle, the person or company doing the thing that they didn't like would be effectively bankrupted, so they still win in the end.
You'll find that if you learn a good amount about the law, it's empowering. The courts are an adversarial place. For every person getting sued... someone is suing. It's isn't "big brother" or "my keeper" or "the man keeping you down" or however you imagine it. You can be the one exerting the pressure if you know what you are doing.
When did you last try? I’m too embarrassed to say how often and onto what kind of surfaces my iPhone 12 has been dropped, but I’m amazed it’s still seemingly completely functional.
My iPhone 4, on the other hand, shattered after one incident…
I was more referring to Nokia's complacency which led to its demise. Nokia was infamous for incremental updates to their phone line, making users upgrade regularly. You could never find a "complete" Nokia phone; each phone was deliberately crippled some how. Apple does the same with their iDevices.
Xbox and Surface have been around a long time as product categories. Xbox isn't even the premier device in its segment.
Highly doubt MS will ever be successful on mobile... their last OS was pretty great and they were willing to pay devs to develop, they just couldn't get it going. This is from someone who spent a ton of time developing on PocketPC and Windows Mobile back in the day.
Products are not the reason for their resurgence.
Apple makes a ton in services, but their R&D is heavily focused on product and platform synergy to that ecosystem extremely valuable.
Afaict, Windows Phone mostly failed because of timing. In the same way that XBox mostly succeeded because of timing. (In the sense that timing dominated the huge amount of excellent work that went into both)
Microsoft is a decent physical product company... they've usually just missed on the strategic timing part.
Timing was definitely an issue - first Windows Phone came 3 years after iOS and 2 after Android. AFA the product itself, I think the perception it needed to overcome was more PocketPC/Windows Mobile having an incredibly substandard image in the market after the iOS release which seemed light years ahead, esp. since MS had that market to themselves for so many years.
That said, it got great reviews and they threw $$ at devs to develop for it, just couldn't gain traction. IME it was timing more than anything and by the time it came to market felt more reactionary than truly innovative.
By this I mean that Microsoft had the positioning of an iPhone in a not-so-great version.
Where as Android relied on the "Open source" and free side for manufacturers to adapt to their phones, even if Google's services remained proprietary.
Can we really talk about timing, when it's above all a problem of a product that didn't fit the market?
Apple is the new Sony might be better. I'm trying to figure out who is the upcoming premium tech product company... not thinking of any. I think Tesla wants to be
I have considered Oracle and MS to be competing for the title of new IBM.
Maybe MS is shaking it off with their AI innovation, but I think a lot of that is just lipstick.
Deep Blue was the name of the computer itself rather than the software, but to answer your question - it didn't use machine learning, its program was written and tweaked by hand. It contained millions of different games and positions, and functioned by evaluating all possible moves at a certain depth. As far as I know, practical machine learning implementations wouldn't be a thing for a decent while after Deep Blue.
Oh it's good they working on important problems with their ai. Its just openai was working on my/our problems (or providing tools to do so) and that's why people are more excited about them.
Not because of cultural differences. If you are more into weather forecasting, yeah it sure may be reasonable to prefer google more.
Stuff like alphafold has and will have huge impact in our lives, even if I am not into spending time folding proteins myself. It is absurd to make this sort of comparisons.
That’s what makes Altman a great leader. He understands marketing better than many of these giants. Google got caught being too big. Sure they will argue that AI mass release is a dangerous proposition, but Sam had to make a big splash otherwise he would be competing with incumbent marketing spendings far greater than OpenAI could afford.
It was a genius move to go public with a simple UI.
No matter how stunning the tech side is, if human interaction is not simple, the big stuff doesn’t even matter.
That statement isn't really directed at the people who care about the scientific or tech-focused capabilities. I'd argue the majority of those folks interested in those things already know about DeepMind.
This statement is for the mass market MBA-types. More specifically, middle managers and dinosaur executives who barely comprehend what generative AI is, and value perceived stability and brand recognition over bleeding edge, for better or worse.
I think the sad truth is an enormous chunk of paying customers, at least for the "enterprise" accounts, will be generating marketing copy and similar "biz dev" use cases.
The thing is that OpenAI doesn't have an "iPhone of AI" so far. That's not to say what will happen in the future - the advent of generative AI may become a big "equalizer" in the tech space - but no company seems to have a strong edge that'd make me more confident in any one of them over others.
A big advantage if this was a product with strong network externalities like social media networks, or even somewhat mobile phones with platform-biased communication tools.
But I don't see generative AI as being particularly that way.
GenAI does not have network effects, correct. There was a time last year when consumer search was still on the table, and I can see how MSFT winning share there might have conferred network effects for genAI, but it didn't happen. Now it's all about the enterprise, which is to say isolated data, which pretty much rules out network effects.
Phones are an end-consumer product. AI is not only an end-consumer product (and probably not even mostly an end-consumer one). It is a tool to be used in many different steps in production. AI is not chatbots.
Great. But school's out. It's time to build product. Let the rubber hit the road. Put up or shut up, as they say.
I'm not dumb enough to bet against Google. They appear to be losing the race, but they can easily catch up to the lead pack.
There's a secondary issue that I don't like Google, and I want them to lose the race. So that will color my commentary and slow my early adoption of their new products, but unless everyone feels the same, it shouldn't have a meaningful effect on the outcome. Although I suppose they do need to clear a higher bar than some unknown AI startup. Expectations are understandably high - as Sundar says, they basically invented this stuff... so where's the payoff?
The usual reasons, evil big corp monopoly with a user-hostile business model etc.
I still use their products. But if I had to pick a company to win the next gold rush, it wouldn't be an incumbent. It's not great that MSFT is winning either, but they are less user-hostile in the sense that they aren't dependent on advertising (another word for "psychological warfare" and "dragnet corporate surveillance"), and I also appreciate their pro-developer innovations.
They do not make Openai look like kids. If anything, it looks like they spent more time, but achieved less. GPT-4 is still ahead of anything Google has released.
From afar it seems like the issues around Maven caused Google to pump the brakes on AI at just the wrong moment with respect to ChatGPT and bringing AI to market. I’m guessing all of the tech giants, and OpenAI, are working with various defense departments yet they haven’t had a Maven moment. Or maybe they have and it wasn’t in the middle of the race for all the marbles.
> and Google has untouchable resources such as all the books they've scanned (and already won court cases about)
https://www.hathitrust.org/ has that corpus, and its evolution, and you can propose to get access to it via collaborating supercomputer access. It grows very rapidly. InternetArchive would also like to chat I expect. I've also asked, and prompt manipulated chatGPT to estimate the total books it is trained with, it's a tiny fraction of the corpus, I wonder if it's the same with Google?
That's correct. LLMs are plausible sentence generators, they don't "understand"* their runtime environment (or any of their other input) and they're not qualified to answer your questions. The companies providing these LLMs to users will typically provide a qualification along these lines, because LLMs tend to make up ("hallucinate", in the industry vernacular) outputs that are plausibly similar to the input text, even if they are wildly and obviously wrong and complete nonsense to boot.
Obviously, people find some value in some output of some LLMs. I've enjoyed the coding autocomplete stuff we have at work, it's helpful and fun. But "it's not qualified to answer my questions" is still true, even if it occasionally does something interesting or useful anyway.
*- this is a complicated term with a lot of baggage, but fortunately for the length of this comment, I don't think that any sense of it applies here. An LLM doesn't understand its training set any more than the mnemonic "ETA ONIS"** understands the English language.
**- a vaguely name-shaped presentation of the most common letters in the English language, in descending order. Useful if you need to remember those for some reason like guessing a substitution cypher.
If you can watch the video demo of this release, or for that matter the Attenborough video, and still claim that these things lack any form of "understanding," then your imagination is either a lot weaker than mine, or a lot stronger.
Behavior indistinguishable from understanding is understanding. Sorry, but that's how it's going to turn out to work.
Why are people so eager to believe that people can? When it comes to the definitions of concepts like sentience, consciousness, thinking and understanding, we literally don't know what we're talking about.
It's premature in the extreme to point at something that behaves so much like we do ourselves and claim that whatever it's doing, it's not "understanding" anything.
We've studied human behavior enough to understand that there are differences between animals in the level of cognition and awareness they (outwardly) exhibit.
Are we not generally good at detecting when someone understands us? Perhaps it's because understanding has actual meaning. If you communicate to me that you hit your head and feel like shit, I not only understand that you experienced an unsatisfactory situation, I'm capable of empathy -- understanding not only WHAT happened, but HOW it feels -- and offering consolation or high fives or whatever.
A LLM has an understanding of what common responses were in the past, and repeats them. Statistical models may mimic a process we use in our thinking, but it is not the entirety of our thinking. Just like computers are limited to the programmers that code their behavior, LLMs are limited to the quality of the data corpus fed to them.
A human, you can correct in real time and they'll (try to) internalize that information in future interactions. Not so with LLMs.
By all means, tell us how statistically weighted answers to "what's the next word" correlates to understanding.
By all means, tell us how statistically weighted answers to "what's the next word" correlates to understanding.
By all means, tell me what makes you so certain you're not arguing with an LLM right now. And if you were, what would you do about it, except type a series of words that depend on the previous ones you typed, and the ones that you read just prior to that?
A human, you can correct in real time and they'll (try to) internalize that information in future interactions. Not so with LLMs.
Not so with version 1.0, anyway. This is like whining that your Commodore 64 doesn't run Crysis.
We've put an awfully lot of effort into figuring that out, and have some answers. Much of the problems in exploring the brain are ethical because people tend to die or suffer greatly if we experiment on them.
Unlike LLMs, which are built by humans and have literal source code and manuals and SOPs and shit. Their very "body" is a well-documented digital machine. An LLM trying to figure itself out has MUCH less trouble than a human figuring itself out.
There are reasons that humans can't report how many books they've read: they simply don't know and didn't measure. There is no such limitation for an LLM to understand where its knowledge came from, and to sum it. Unless you're telling me a computer can't count references.
Also, why are we comparing humans and LLMs when the latter doesn't come anywhere close to how we think, and is working with different limitations?
The 'knowledge' of an LLM is in a filesystem and can be queried, studied, exported, etc. The knowledge of a human being is encoded in neurons and other wetware that lacks simple binary chips to do dedicated work. Decidedly less accessible than coreutils.
Imagine for just a second that the ability for computers to count “references” has no bearing on this, there is a limitation and that LLMs suffer from the same issue as you do.
Why should I ignore a fact that makes my demand realistic? Most of us are programmers on here I would imagine. What's the technical reason an LLM cannot give me this information?
Bytes can be measured. Sources used to produce the answer to a prompt can be reported. Ergo, an LLM should be able to tell me the full extent to which it's been trained, including the size of its data corpus, the number of parameters it checks, the words on its unallowed list (and their reasoning), and so on.
These will conveniently be marked as trade secrets, but I have no use for an information model moderated by business and government. It is inherently NOT trustworthy, and will only give answers that lead to docile or profitable behavior. If it can't be honest about what it is and what it knows and what it's allowed to tell me, then I cannot accept any of its output as trustworthy.
Will it tell me how to build explosives? Can it help me manufacture a gun? How about intercepting/listening to today's radio communications? Social techniques to gain favor in political conflicts? Overcoming financial blockages when you're identified as a person of interest? I have my doubts.
These questions might be considered "dangerous", but to whom, and why shouldn't we share these answers?
Where it has fallen down (compared to its relative performance in relevant research) is public generative AI products [0]. It is trying very hard to catch up at that, and its disadvantage isn't technological, but that doesn't mean it isn't real and durable.
[0] I say "generative AI" because AI is a big an amorphous space, and lots of Google's products have some form of AI that is behind important features, so I'm just talking about products where generative AI is the center of what the product offers, which have become a big deal recently and where Google had definitely been delivering far below its general AI research weight class so far.
> Google is very good at AI research.
Where it has fallen down (compared to its relative performance in relevant research) is public generative AI products
In such cases, I actually prefer Google over OpenAI. Monetization isn’t everything
> In such cases, I actually prefer Google over OpenAI.
For, what, moral kudos? (to be clear, I'm not saying this is a less important thing in some general sense, I'm saying what is preferred is always dependent on what we are talking about preferences for.)
> Monetization isn’t everything
Providing a user product (monetization is a different issue, though for a for-profit company they tend to be closely connected) is ultimately important for people looking for a product to use.
> For the good of society? Performing and releasing bleeding edge research benefits everyone, because anyone can use it.
OK, but that only works if you actually do the part that lets people actually use the research for something socially beneficial. A research paper doesn't have social benefit in itself, the social benefit comes when you do something with that research, as OpenAI has.
> There is nothing open about OpenAI and they wouldn't exist in their current form without years of research funded by Google.
True enough. But the fact remains that they're the ones delivering something we can actually use.
I personally think of it as open in the sense that they provide an API to allow anyone to use it (if they pay) and take advantage of the training they did. Is in contrast to large companies like Google which have lots of data and historically just use AI for their own products.
Edit:
I define it as having some level of being open beyond 'nothing'. The name doesn't scale well over time based on business considerations and the business environment changing and was named poorly when 'open source' is a common usage of open within tech. They should have used AI products to help them in naming the company and be aware of such potential controversies.
From chatgpt today (which wasn't an option at the time but they maybe could have gotten similar information or just thought about it more):
What are the drawbacks to calling an AI company 'open'?
...
"1. Expectations of Open Source: Using the term "open" might lead people to expect that the company's AI technology or software is open source. If this is not the case, it could create confusion and disappointment among users and developers who anticipate access to source code and the ability to modify and distribute the software freely.
2. Transparency Concerns: If an AI company claims to be "open," there may be heightened expectations regarding the transparency of their algorithms, decision-making processes, and data usage. Failure to meet these expectations could lead to skepticism or distrust among users and the broader public."
I mean, we do use that word to describe physical retail shops as being available to sell vs being closed to sell, so it's not an insane use... though I do think that in a tech context it's more misleading than not.
Compared to a curated video service like HBO Max, Hulu, or Netflix, that's an accurate way to describe the relative differences. We aren't used to using that terminology through, so yes, it comes across as weird (and if the point is to communicate features, is not particularly useful compared to other terminology that could be used).
It makes a bit less sense for search IMO, since that's the prevalent model as far as I'm aware, so there's not an easy and obvious comparison that is "closed" which allows us to view Google search as "open".
They publish but don't share. Who cares about your cool tech if we can't experience it ourselves? I don't care about your blog writeup or research paper.
Google is locked behind research bubbles, legal reviews and safety checks.
The researchers at all the other companies care about the blog write-ups and research papers. The Transformer architecture, for example, came from Google.
Sharing fundamental work is more impactful than sharing individual models.
To take an example from the past month, billions of users are now benefiting from more accurate weather forecasts from their new model. Is there another company making more money from AI-powered products than Google right now?
It's a very fuzzy question I posed. For pure customer-pays-for-AI-service it could be Microsoft. I'm kind of thinking of it as: Google's core products (search, ads, YouTube, Gmail) would not be possible with AI and they are huge cash cows.
Only indirectly, but I wanted to point out that there are a lot of interesting research innovations that get implemented by Google and not some other company.
Or, well, like many companies; all the peons doing the actual work, creation etc and the executives and investors profiting at the top. All it takes is to be lucky to be born into generational wealth apparently.
I think the real underlying cause is the explosion of garbage that gets crawled. Google initially tried to use AI to find "quality" content in the pile. It feels like they gave up and decided to use the wrong proxies for quality. Proxies like "somehow related to a brand name". Good content that didn't have some big name behind it gets thrown out with the trash.
I think the bottom line (profit) inversely correlates with the quality of search results. I've been using phind.com lately and it seems there can be search without junk even in this age.
Google has lots of people tagging search rankings, which is very similar with RLHF ranking responses from LLMs. It's interesting that using LLMs with RLHF it is possible to de-junk the search results. RLHF is great for this task, as evidenced by its effect on LLMs.
Right. It’s less that their declining quality of search results is due to AI and more that the AI got really good at monetizing and monetizing and quality search results are sometimes in opposition.
This entire thread kinda ignore that they are also selling ad space on many sites and their objective function in ordering search is not just the best possible result. Case in point the many sites stealing stack overflow content and filling it with adverts ranking higher than the source, that committed the cardinal sin of running their own and network.
> I've been using phind.com lately and it seems there can be search without junk even in this age.
A few reasons partially (if not fully) responsible for it might be:
- Google is a hot target of SEO, not Phind.
- If Google stops indexing certain low quality without a strong justification, there would be lawsuits, or people saying how "Google hasn't indexed my site" or whatever. How would you authoritatively define "low quality"?
- Having to provide search for all spectrum of users in various languages, countries and not just for "tech users".
Web has grown by 1000x over years. The overall signal to noise ratio has been worsen, around by 100x and SEO has been become much more sophisticated and optimized against Google. A large fraction of quality content has been moving toward walled gardens. The goalpost is moving (much) faster than technologies.
Yup, and us humans produce as much garbage as we can too. "60 hours of black screen" type videos on YouTube that gotta be stored on CDNs across the globe, taboola's absolutely vile ads, endless scripted content made by content creators for the short term shock/wow value.
I recently google searched "80cm to inches" and it gave me the result for "80 meters to inches". I can't figure out how it would make this mistake aside from some poorly conceived LLM usage
I highly doubt that this is related to any LLM use. It would breathtakingly uneconomical and completely unnecessary. It's not even interesting enough for an experiment.
This does highlight the gap between SOTA and business production. Google search is very often a low quality, even user hostile experience. If Google has all this fantastic technology, but when the rubber hits the road they have no constructive (business supporting) use cases for their search interface, we are a ways away from getting something broadly useful.
It will be interesting to see how this percolates through the existing systems.
I am at first just saying that search as PageRank in the early days is a ML marvel that changed the way people interact with the internet. Figuring out how to monetize and financially survive as a business have certainly changed the direction of its development and usability.
This is because their searches are so valuable that real intelligence, i.e. humans, have been fighting to defeat google's AI over billions of dollars of potential revenue.
> Sundar's comments in the main video seem like he's trying to communicate "we've been doing this ai stuff since you (other AI companies) were little babies" - to me this comes off kind of badly
Reminds me of the Stadia reveal, where the first words out of his mouth were along the lines of "I'll admit, I'm not much of a gamer"
How about we go further and just state what everyone (other than Wall St) thinks: Google needs a new CEO.
One more interested in Google's supposed mission ("to organize the world's information and make it universally accessible and useful"), than in Google's stock price.
I've been making this exact comparison for years at this point.
Both inherited companies with market dominant core products in near monopoly positions. They both kept the lights on, but the companies under them repeatedly fail the break into new markets and suffer from a near total lack of coherent vision and perverse internal incentives that contribute to the failure of new products. And after a while, the quality of that core product starts to stumble as well.
The fact that we've seen this show before makes it all the more baffling to me that investors are happy about it. Especially when in the same timeframe we've seen Satya Nadella completely transform Microsoft and deliver relatively meteoric performance.
Balmer made Microsoft the most profitable it had ever been. He didn't grow them into big new areas, but he improved the focus, trimmed the costs, and vastly improved the bottom line. A successful company may need vision and expansion, but at some point it also needs to be able to actually convert that into profit, otherwise you turn into Sun - or indeed recent Google, who've come out with some great products but never managed to convert any of them into profit centers.
The dude shipped Windows 8! He insisted on this insane mishmash of tablet and windows that made sense to nobody. Somehow they shipped this, which tells me the emperor wears no clothes.
I completely agree with Satya Nadella, I haven't seen a turnaround since Steve Jobs came back to Apple. He took a company that couldn't seem to get out of its way and turned it into an innovative, exciting, and insanely profitable company.
He's also totally transformed the public image of Microsoft, from ruthless monopolist to one of the least evil, most open giant tech companies. With actions, not words.
It's not all perfect and wonderful, but they're miles away from the Gates/Ballmer era, it's remarkable.
Are you all on drugs? This is the company that published a poll speculating on the death of a missing woman. The one that asks you to explain yourself when you try to close OneDrive, and ignores/resets your browser preferences while also forcefully installing crapware like the Bing bar. They're the ones about to create a mountain of ewaste by making Win11 unusable on older hardware. They're also the ones fighting government (and winning) in order to consolidate the game industry to further reduce competition and hurt consumers. I could keep going, but it's a very long list.
There seems to be some small pocket of tech people who are permanently enthralled by this organization. Does Nadella have is own reality distortion field? If so it must be pretty damn strong in order to pierce the smell of dog shit surrounding his employer.
I'm wondering why they're keeping him around. Maybe they feel like they've got more control when Sundar is in charge, since he's less likely to make any rash decisions or sudden movements (or any movements at all...)
Your comment history is exclusively limited to posts about Google product releases and stock performance (and one about Sergey Brin's airship), so I'm sorry if I don't consider you an unbiased observer. And sure, maybe you honestly believe in the company, and that's why you invest in it. But just because you think you've aligned your incentives (stock portfolio) with those of the company, doesn't mean you've accurately assessed its health and future outlook.
For those of us closer to the ground - the "HN hive mind," if you will - in the same industry but not at Google, the signs are far from positive. Top line revenue looks good, but Microsoft grew more in the past decade than Google. There is a massive dependence on advertising revenue, which is so large that it's basically an existential threat to the company (although admittedly, GCP is beginning to show promise after recently posting its first profitable quarter). The rest of the industry is actively fighting Google's ability to display ads to their users. The quality of the flagship Search product is possibly the lowest it's ever been. YouTube is driving users away while picking pennies up off the floor. Employees are leaving to build startups like OpenAI with the tech they researched at Google. Morale is extremely low. Recruiting pipelines are likely suffering; most developers with an offer from Google and a company paying equivalent salary (in other words, the best developers) will not choose Google. Public perception is hostile, amidst both the general public and early adopters like developers. Governments are litigating, potential anti-trust breakups are on the horizon. But most importantly: Google has failed to fundamentally innovate since about 2005; if you disagree, please name an innovative product created from scratch at Google since that time.
The Waymo self-driving car product seems like it will be quite transformative to entire industries once they get clearance to deploy it further than San Francisco where it is already providing rides day in and day out. Or does that not count for some reason?
Disclaimer: I own Google stock simply by virtue of being invested in mutual and index funds, as are most people.
Isn't that the product that had to scale back recently because it required an average of two humans per car to remotely operate it?
I'm (mostly) genuinely asking. I might have it confused with another company, and I have to admit I don't follow self-driving closely.
But also, Waymo was an acquisition (slightly arguable, since Google merged it with its own self-driving efforts, but the founding team was acquired). I asked for an example of an innovative product created from scratch at Google.
You're thinking of Cruise. Waymo has not scaled back in any way, and in fact is in the process of expanding to LA with a limited pilot through the winter.
I don't think the fact that some of the first people on the team had worked together previously makes Waymo not "created at Google". The project they worked on before, the DARPA challenge, was not a commercial product, and at the time no company was seriously investing in self-driving cars as a viable technology. This isn't like YouTube, which was a well-known brand and viable business pre-acquisition. It was Google resources that made it possible to build the rest of the Waymo team, lobby governments to allow self-driving cars on the road, work with hardware manufacturers, and leverage the rest of Google's software stack, ML expertise, street view data, and datacenter capacity to build and train the driver.
You're thinking of Cruise, which had to stop operations for malfeasance. If you want to tell me that the Google Self-driving Car Project, which is what Waymo was called before it was spun out from Google, didn't come from Google, I'm not sure what to say.
To add to my comment above: Google DeepMind put out 16 videos about Gemini today, the total watch time at 1x speed is about 45 mins. I've now watched them all (at >1x speed).
My current context: API user of OpenAI, regular user of ChatGPT Plus (GPT-4-Turbo, Dall E 3, and GPT-4V), occasional user of Claude Pro (much less since GPT-4-Turbo with longer context length), paying user of Midjourney.
Gemini Pro is available starting today in Bard. It's not clear to me how many of the super impressive results are from Ultra vs Pro.
Overall conclusion: Gemini Ultra looks very impressive. But - the timing is disappointing: Gemini Ultra looks like it won't be widely available until ~Feb/March 2024, or possibly later.
> As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.
> Early next year, we’ll also launch Bard Advanced, a new, cutting-edge AI experience that gives you access to our best models and capabilities, starting with Gemini Ultra.
I hope that there will be a product available sooner than that without a crazy waitlist for both Bard Advanced, and Gemini Ultra API. Also fingers crossed that they have good data privacy for API usage, like OpenAI does (i.e. data isn't used to train their models when it's via API/playground requests).
What they've released today: Gemini Pro is in Bard today. Gemini Pro will be coming to API soon (Dec 13?). Gemini Ultra will be available via Bard and API "early next year"
Therefore, as of Dec 6 2023:
SOTA API = GPT-4, still.
SOTA Chat assistant = ChatGPT Plus, still, for everything except video, where Bard has capabilities . ChatGPT plus is closely followed by Claude. (But, I tried asking Bard a question about a youtube video today, and it told me "I'm sorry, but I'm unable to access this YouTube content. This is possible for a number of reasons, but the most common are: the content isn't a valid YouTube link, potentially unsafe content, or the content does not have a captions file that I can read.")
SOTA API after Gemini Ultra is out in ~Q1 2024 = Gemini Ultra, if OpenAI/Anthropic haven't released a new model by then
SOTA Chat assistant after Bard Advanced is out in ~Q1 2024 = Bard Advanced, probably, assuming that OpenAI/Anthropic haven't released new models by then
Watching these videos made me remember this cool demo Google did years ago where their earpods would auto translate in realtime a conversation between two people talking different languages. Turned out to be demo vaporware. Will this be the same thing?
I think they're getting at the idea that it was demoed as a real time babelfish, where a conversation simple happened between two people wearing the devices. Instead it was a glorified spoken dropdown selector for choosing the language, and a press and hold mechanism that just tied into the existing phone app without any actual changes or upgrades to that already available translation mechanism. The thought was that you'd simply start talking to each other and hear the other in your language as you go - not speak a block all at once, stop, translate, play back from your phone to them, stop, let them speak a whole reply at once while the phone listens to them, stop, translate, hear their response in your earpiece. Which basically meant the device itself didn't bring much if anything to the table that couldn't be done with any other headphones and doing the language select and start/stop recording on the phone itself.
When I watch any of these videos, all the related videos on my right sidebar are from Google, 16 of which were uploaded at the same time as the one I'm watching.
I've never seen the entire sidebar filled with the videos of a single channel before.
> to me this comes off kind of badly, like it's trying too hard to emphasize how long they've been doing AI
These lines are for the stakeholders as opposed to consumers. Large backers don't want to invest in a company that has to rush to the market to play catch-up, they want a company that can execute on long-term goals. Re-assuring them that this is a long-term goal is important for $GOOG.
Its a conceit but not unjustified, they have been doing "AI" since their inception. And yeah, Sundar's term up until recently seems to me to be milking existing products instead of creating new ones, so it is a bit annoying when they act like this was their plan the whole time.
Google's weakness is on the product side, their research arm puts out incredible stuff as other commenters have pointed out. GPT essentially came out from Google researchers that were impatient with Google's reluctance to ship a product that could jeopardize ad revenue on search.
The point is if you have to remind people then you’re doing something wrong. The insight to draw from this is not that everyone else is misinformed about googles abilities (the implication), its that Google has not capitalized on their resources.
It's such a short sighted approach too because I'm sure someone will develop a GPT with native advertising and it'll be a blockbuster because it'll be free to use but also have strong revenue generating potential.
I also find that tone a bit annoying but I'm OK with it because it highlights how these types of bets, without an immediate benefit, can pay off very well in the long term, even for huge companies like Google. AI, as we currently know it, wasn't really a "thing" when Google started with it and the payoff wasn't clear. They've long had to defend their use of their own money for big R&D bets like this and only now is it really clearly "adding shareholder value".
Yes, I know it was a field of interest and research long before Google invested, but the fact remains that they _did_ invest deeply in it very early on for a very long time before we got to this point.
Their continued investment has helped push the industry forward, for better or worse. In light of this context, I'm ok with them taking a small victory lap and saying "we've been here, I told you it was important".
> only now is it really clearly "adding shareholder value".
AI has been adding a huge proportion of the shareholder value at Google for many years. The fact that their inference systems are internal and not user products might have hidden this from you.
> we've been doing this ai stuff since you (other AI companies) were little babies
Actually, they kind of did. What's interesting is that they still only match GPT-4's version but don't propose any architectural breakthroughs. From an architectural standpoint, not much has changed since 2017. The 'breakthroughs', in terms of moving from GPT to GPT-4, included: adding more parameters (GPT-2/3/4), fine-tuning base models following instructions (RLHF), which is essentially structured training (GPT-3.5), and multi-modality, which involves using embeddings from different sources in the same latent space, along with some optimizations that allowed for faster inference and training. Increasing evidence suggests that AGI will not be attainable solely using LLMs/transformers/current architecture, as LLMs can't extrapolate beyond the patterns in their training data (according to a paper from DeepMind last month):
"Together our results highlight that the impressive ICL abilities of high-capacity sequence models may be more closely tied to the coverage of their pretraining data mixtures than inductive biases that create fundamental generalization capabilities."[1]
Sundar studied material science in school and is only slightly older than me. Google is a little over 25 years old. I guarantee you they have not been doing AI since I was a baby.
And how many financial people worth reconning with are under 30 years old? Not many.
Unless you are OpenAI, the company, I doubt OP implied it was aimed at you. But then I wouldn't know as I am much younger than Sundar Pichai and I am not on first name basis with him either ;-)
I do think that’s a backfire. Telling me how long you’ve been doing something isn’t that impressive if the other guy has been doing it for much less time and is better at it. It’s in fact the opposite.
Echoes of Apple “leveraging” the Mouse/GUI interface from Xerox. I wonder if Google is at risk of going to way of Xerox, where they were so focused on their current business and product lineups they failed to see the potential new business lines their researchers were trying to show them.
>In a well-reasoned opinion, the 9th Circuit Court of Appeals recently held that the GOOGLE trademark has not suffered death by genericide – even if the public uses it as a verb for searching the Internet.
>The case before the court sprang from the registration of 763 domain names that incorporated the term GOOGLE. After losing a domain name dispute arbitration, the domain name owners sued to have various trademark registrations for GOOGLE cancelled, claiming that the mark had become generic for the act of searching the Internet. The court rightly observed that a claim of genericide must always relate to specific goods or services, and that use of “google” as a verb for searching the Internet was not sufficient evidence that GOOGLE had become generic for “search engine services” or any other goods or services.
>The general rule of thumb is that trademarks are best thought of as “adjectives” that modify a generic noun. But this “part of speech” approach is not determinative to whether a mark has become generic. And while for years Xerox sought to instill in the public’s mind the trademark significance of XEROX by stating that “not even Xerox can xerox,” evidently Google can google without destroying the mark.
Just a little reminder from Xerox / prepared by Needham, Harper & Steers Advertising, Inc. -- Not even Xerox can Xerox / prepared by Needham Harper Worldwide, Inc. (March 1985) -- Once a trademark, not always a trademark / [Xerox Corporation].
Though it was a long time ago, I recall that my law school Business Torts casebook contained a copy of Xerox’s old ad, “Not Even Xerox Can Xerox”, which Xerox used to promote proper use of its trademark and fight genericide. Back in the day, Xerox was by far the most well-know copier brand, leased by offices all over. In this day and age, now that most people have a copier at home (as part of a multifunction printer) and it could be a Canon, HP, Brother, Epson or other brand, I think the younger folk are not so likely to refer to copying as “Xeroxing”. It poses an interesting quandary: Xerox may be winning the genericide war but they are no longer dominating the competition. Which is preferable?
If the LEGO trademark is used at all, it should always be used as an adjective, not as a noun. For example, say "MODELS BUILT OF LEGO BRICKS". Never say "MODELS BUILT OF LEGOs". Also, the trademark should appear in the same typeface as the surrounding text and should not be isolated or set apart from the surrounding text. In other words, the trademarks should not be emphasized or highlighted. Finally, the LEGO trademark should always appear with a ® symbol each time it is used.
Weird for us to personify a corporation like that tbh. Google didn't invent transformers, researchers working at Google did.
Sure Google paid em money/employed em, but the smarts behind it isn't the entity Google or the execs at the top, Sundar etc; it's those researchers. I like to appreciate individualism in a world where those at the top have lobbied their way into a 1% monopoly lmao.
First, OpenAI is not some "little guy". It's a bigger corporation than 99.99% of companies that have ever existed. It's like calling Airbnb or Uber "little guys".
Second, yes, the researchers did that but the company funded it with no guarantee of return. Your argument can be applied to any company or organization; it's needless pedantry.
770 employees is bigger than 99.9% off all companies ever? Because we saw how easily those employees would have walked out the front door only too recently.
It might very well be considering most companies are small businesses but I was referring to their valuation/funding. It's worth tens if not hundred(s) of billions of dollars.
> One observation: Sundar's comments in the main video seem like he's trying to communicate "we've been doing this ai stuff since you (other AI companies)
Sundar has been saying this repeatedly since Day 0 of the current AI wave. It's almost cliche for him at this point.
And he's going to keep saying it to tell investors why they should believe Google will eventually catch up in product until Google does catch up in product and he doesn't need to say it anymore.
Or until Google gives up on the space, or he isn't CEO, if either of those come first, which I wouldn't rule out.
I spotted that too, but also, it didn't recognise the "bird" until it had feet, when it is supposedly better than a human expert. I don't doubt that the examples were cherry-picked, so if this is the best it can do, it's not very convincing.
I would've liked to see an explanation that includes the weight of water being displaced. That would also explain how a steel ship with an open top is also able to float.
In fairness, the performance/size ratio for models like BERT still gives GPT-3/4 and even Llama a run for it's money. Their tech isn't as product-ized as OpenAI's, but Tensorflow and it's ilk have been an essential part of driving actual AI adoption. The people I know in the robotics and manufacturing industries are forever grateful for the out-front work Google did to get the ball rolling.
You seem to be saying the same thing- Googles best work is in the past, their current offerings are underwhelming, even if foundational to the progress of others.
Didn't Google invent LLMs and didn't Google have an internal LLm with similar capabilities long before openai released the gpts? Remember when that guy got fired for making a claim it was conscious ?
No this is not correct. Arguably OpenAI invented LLMs with GPT3 and the preceding scaling laws paper. I worked on LAMDA, it came after GPT4 and was not as capable. Google did invent the transformer, but all the authors of the paper have left since.
Incredible stuff, and yet TTS is still so robotic. Frankly I assume it must be deliberate at this point, or at least deliberate that nobody's worked on it because it's comparatively easy and dull?
(The context awareness of the current breed of generative AI seems to be exactly what TTS always lacks, awkward syllables and emphasis, pronunciation that would be correct sometimes but not after that word, etc.)
Sundar's comments about Google doing AI (really ML) are based more on things that people externally know very little about. Systems like SETI, Sibyl, RePhil, SmartASS. These were all production ML systems that used fairly straightforward and conventional ML combined with innovative distributed computing and large-scale infrastructure to grow Google's product usage significantly over the past 20 years.
However, SmartASS and sibyl weren't really what external ML people wanted- it was just fairly boring "increase watch time by identifying what videos people wioll click on" and "increase mobile app installs" or "show the ads people are likely to click on".
It really wasn't until vincent vanhoucke stuffed a bunch of GPUs into a desktop and demonstrated scalable and dean/ng built their cat detector NN that google started being really active in deep learning. That was around 2010-2012.
But their first efforts in BARD were really not great. I'd just have left the bragging out in terms of how long. OpenAI and others have no doubt sent a big wakeup call to google. For a while it seemed like they had turned to focus an AI "safety" (remembering some big blowups on those teams as well) with papers about how AI might develop negative stereotypes (ie, men commit more violent crime then women?). That seems to have changed - this is very product focused, and I asked it some questions that in many models are screened out for "safety" and it responded which is almost even more surprising (ie. Statistically who commits more violent crime, men or women).
The big concern was biased datasets iirc and shit fits for people of color. Like clearly mislabeling feminine looking women as men, and a stupid high false positive rate for face detection.
That was relevant given they were selling their models to law enforcement.
> A better look would simply be to show instead of tell.
Completely! Just tried Bard. No images and the responses it gave me were pretty poor. Today's launch is a weak poor product launch, looks mostly like a push to close out stuff for Perf and before everybody leaves for the rest of the December for vacation.
A simple REST API with a static token auth like OpenAI API would help. Previously when I tried Bard API it was refusing to accept token auth, requiring that terrible oauth flow so I gave up.
That was ages ago. In AI even a week feels like a whole year in other fields. And many/most of those researchers have fled to startups, so those startups also have a right to brag. But not too much - only immediate access to a model beating GPT4 is worth bragging today (cloud), or getting GPT3.5 quality from a model running on a phone (edge).
So it's either free-private-gpt3.5 or cloud-better-than-gpt4v. Nothing else matters now. I think we have reached an extreme point of temporal discounting (https://en.wikipedia.org/wiki/Time_preference).
The Transformer paper “Attention is All You Need” came out in 2017. Sundar got the CEO job two years earlier, so he was in CEO diapers at the time if you will.
I would argue Google has done almost nothing interesting since then (at least not things they haven't killed)
I find this video really freaky. It’s like Gemini is a baby or very young child and also a massively know it all adult that just can’t help telling how clever it is and showing off its knowledge.
People speak of the uncanny valley in terms of appearance. I am getting this from Gemini. It’s sort of impressive but feels freaky at the same time.
No, there's an odd disconnect between the impressiveness of the multimodal capabilities vs the juvenile tone and insights compared to something like GPT-4 that's very bizarre in application.
It is a great example of what I've been finding a growing concern as we double down on Goodhart's Law with the "beats 30 out of 32 tests compared to existing models."
My guess is those tests are very specific to evaluations of what we've historically imagined AI to be good at vs comprehensive tests of human ability and competencies.
So a broad general pretrained model might actually be great at sounding 'human' but not as good at logic puzzles, so you hit it with extensive fine tuning aimed at improving test scores on logic but no longer target "sounding human" and you end up with a model that is extremely good at what you targeted as measurements but sounds like a creepy toddler.
We really need to stop being so afraid of anthropomorphic evaluation of LLMs. Even if the underlying processes shouldn't be anthropomorphized, the expressed results really should be given the whole point was modeling and predicting anthropomorphic training data.
"Don't sound like a creepy soulless toddler and sound more like a fellow human" is a perfectly appropriate goal for an enterprise scale LLM, and we shouldn't be afraid of openly setting that as a goal.
What an ugly statement. DeepMind has been very open with their research since the beginning because their objective was much more on making breakthroughs with moonshot projects than near term profit.
In contrast to the main video, this video that is further down the page is really impressive and really does show - the 'which cup is the ball in is particularly cool': https://www.youtube.com/watch?v=UIZAiXYceBI.
Other key info: "Integrate Gemini models into your applications with Google AI Studio and Google Cloud Vertex AI. Available December 13th." (Unclear if all 3 models are available then, hopefully they are, and hopefully it's more like OpenAI with many people getting access, rather than Claude's API with few customers getting access)