Watson could be described as a natural language search engine. This is no small thing. It's linguistic abilities were showcased on jeopardy, though it's wins might have had more to do with speed of processing and "buzzing in" than it did with being really smart. Watson is quite possibly the most sophisticated specific use natural language program to ever exist (as opposed to general use nlp, which is star trek level problem).
That said, the approach and subsequent utility might not live up to the hype that IBM is pumping out. It's one thing to search very quickly. Being able to discover patterns that lead to new levels of understanding and predictable relationships is another thing entirely. IBM is more search vs predict in part because they only have so much data to work with. All of the medical books in the world are a drop in the bucket in terms of algorithmic understanding. Watson has mastered working with all available information. Collecting and processing massive data sets is another challenge that IBM hasn't been willing to tackle yet.
IBM is billing Watson as the all singing, all dancing solution to the world's data problems. They're tackling a lot of problems in diverse areas. I hope it works out, the world needs as much help as it can get. But IBM has shifted their core mission to be consulting and I wonder if Watson's purpose will be to support that more than becoming a Super Siri type software project that could do the most good.
Regarding Watson's speed on Jeopardy, that was certainly a big advantage for it. However, consider that no matter how fast it is, a machine that "only" gets 50% of the questions right after it buzzes in (which would be an amazing accomplishment already) would lose the game horribly. That it won so solidly shows that it goes well beyond mere speed.
From the consumer's end, we expect computers to produce accurate calculations almost always. If a calculator produced the wrong answer to a basic mathematical function we would throw it out.
Humans are error prone, even when doing things they know and are good at.
Artificial intelligence is marketed as being a machine that is as smart as a human, but somehow we infer that because AI is a machine it will not make human mistakes. Mistakes are what produces learning.
The question becomes, do we only release AI for public use when it is assigned to a narrow range of problems and trained to 99.9% accuracy? Or does a consumer just throw AI at unknown, or even non trainable, problems and we take the result with a grain of salt? (Non trainable being something like predicting the value of the S&P 500 in 24 months.)
Perhaps a new words will be formed to describe AI, its behavior, accuracy, and experience? For now there is a lot of "one size fits all" and "holy grail" seeking. Big companies with armies of sales people seem to prefer this.
I think a search engine is the appropriate metaphor. Google's first result isn't always what you're looking for, but if you have no idea where to start, you can just type some words and Google will give you 10 pretty good ideas. You can then easily know if those 10 ideas are what you needed, or at least have a better idea of how to modify your query.
Watson, I think, will be the same way. Say for medical diagnoses - you don't just feed it observations and prescribe whatever it says. But if you want to ask about an unusual combinations of symptoms you've never encountered, it'll come back with something you can then go research. How you could make that connection before systems like Watson or Google (or similar medically-focussed systems if they exist) is beyond me - but they'll probably never replace the doctor's judgement. They're just a tool and should be treated that way.
> its wins might have had more to do with speed of processing and "buzzing in" than it did with being really smart.
It may only be better than humans at buzzing in, but being as good as humans at natural language search, but faster and more consistently (doesn't make mistakes when tired; works just as well in Kampala as New York; can be audited when it makes mistakes) is already better than humans.
> That said, the approach and subsequent utility might not live up to the hype that IBM is pumping out. It's one thing to search very quickly. Being able to discover patterns that lead to new levels of understanding and predictable relationships is another thing entirely.
I don't get this, Watson might not be able to provide new levels of understanding but it still does a much better job than current search engines, so why do you think it cannot live up to the hype? Why do you think that it is not a significant improvement? What makes you think the approach is wrong? I need some clarifications :)
If you query Google with a Jeopardy style question you won't have a valid answer as the first result, so at least for this particular application, yes it does a better job.
I'm sure there are plenty of reason to believe that this cannot be generalized to real world problem but no one here has given these reasons. So I am not really sure why people claim that it might not live up the the hype. (And I would genuinely like to understand).
Watson is trained for answering those questions ,google is not. this is why some one can pee all his urine into a small cococola bottle on the ground from upstairs while others can,t:he trained himself and practiced a lot. if you need Watson help you with other things, you need to train him again.that's why you need apps for: do the training. you will find there's nothing different with training a SAS or R program. you are still on your own. there's no help you can get from Watson
Watson is is now only a search engine you need to train and define your own domain models and logics in side the App. The bloom depends on wether there can be enough Apps on the platform. Remember: those Apps are enterprise level and will always be developed by companies. Why don't these companies just deploy their app in other clouds and connect with a 'siri' like voice interface? I can't see any values inside this platform. Vertical search and IP is not hard for developers today. Small companies can do that without IBM's help.
“allows business users to send natural language questions and raw data sets into the cloud, for Watson to crunch, uncover, and visualize insights; without the need for advanced analytics training. After analyzing the data, Watson will deliver results to its users through graphic representations that are easy to understand, interact with, and share with colleagues; all in a matter of minutes.”
This sounds a lot like my present job description.
Remember, no one cares about data, they want a story and the evidence to back that up. Watson may just make it easier for you to focus on the story telling and evidence rather than the crunching.
I'm currently working on an my own open source version of Watson/Siri/Google Now. (It can answer "What is the capital of Brazil" Yay!).
As part of that I've been leaning as much as I can about how Watson actually works.
The most useful information can be found by Googling "Deep QA" which is what IBM has dubbed their question answering pipeline.
A slide deck like [2] is a good place to start if you are interested in this.
[1] Yeah, I know that is kind of a crazy thing to work on. It's actually even more stupid than you may think, because I want it make it self-hostable, with the ability to keep your own data separately to the rest of the application (ie, enforcing privacy).
Not crazy at all. I have been dreaming on working on something similar even before watson was announced (though i was just a Freshmen back then). And was hellishly jealous of that team. So I can totally understand your motivations for this.
Do hit me up if you'd welcome help with this. Email is in HN profile
I really don't appreciate the media's blood thirst - the slayer of this and the killer of that. Why can't we just have something that contributes in a non-zero-sum game?
You never had journalistic training. Journalists frame stories to satisfy criteria of "newsworthiness," a combined measure of the story's importance, urgency, and entertainment value. Just like a good novel has a dramatic conflict, journalists are taught to report on stories with conflict (which are often more interesting to read than dull, peaceful hum-drum). And if the story doesn't have conflict built-in, they make conflict by framing the story to include a conflict narrative. Journalists refer to the way they frame a story as their "angle." Anything can be news with the right angle.
Real life example: in the 90s, journalists reported on the "Great Hacker War," a "virtual gang war" between two competing hacker groups, LOD & MOD. In reality, the event was a scuffle between some hackers in a chat room, which resulted in some minor hacking, name calling, and prank phone calls. But that didn't make for a great headline.
My field of research is in machine learning, and upon chatting with medical folks about recent machine learning breakthroughs that outperform panels of experts at making diagnoses, they are all extremely resistant and think that using AI in medicine is somehow immoral.
I'd personally want to be diagnosed by a panel of experts with access to said AI.
Problem is they might override the AI's (more-likely correct) judgment. See the table labeled "Predictive modeling vs. the experts" at the bottom of the article "Predictive Modeling Holds Promise of Earlier Identification, Treatment":
In the end, we'll rely on the AI. We'll _want_ to rely on the AI. Doctor's roles will be reduced to that of nurses. Care will improve. See it all described in the 2006 movie "Idiocracy":
"I'd personally want to be diagnosed by a panel of experts with access to said AI."
Exactly. Neither one nor the other alone, even though today the panel of experts seems the best bet
To see how the AI can go wrong, just try to diagnose something with flu-like symptoms using Google.
There's also an issue with incomplete tests. Oh the AI can improve the diagnostic using exam X but exam X is too expensive/too invasive/risky etc, someone needs to play "middle ground"
Some kind of machine/human integrated medical system is a common goal of current research. Where many AI people think the lack of uptake is coming from, besides just general resistance to AI diagnosis, is that current systems don't have great real-world usability.
A few issues: There is a lot of information available to the doctor in a typical diagnostic setting that is not currently codified in machine-readable form, and asking the doctor to do custom data entry per patient is not likely to improve uptake. Ideally the systems should integrate with other patient-information-management systems, and such patient-information systems might need to be augmented with new or differently coded data collection. Perhaps equally or more importantly, if the AI system is going to be a component of the diagnosis rather than handed over full trust to make the diagnosis, it should ideally produce "white-box" diagnoses with justification for its reasoning and human-readable explanation of what it thinks the situation is, not just black-box predictions.
Is the idea that i have an appointment, and the doctor / AI come to a diagnosis?
Or that an AI runs through a doctor surgery's collection of patient medical records and highlights patients that probably would benefit from an intervention?
Certainly agree for a panel of experts. Thinking about day-to-day use for the masses (which I don't know if they are targeting or not), Watson would certainly outperform most of the doctors I've been examined by in the last few years (and I'd trust it more than the doctors).
I would expect that Watson is used as a discovery, not a decision tool - something to point out the symptoms as matching an uncommon disease that the doctor had not otherwise considered.
There are around 500 000 biology and medical research papers published per year (http://www.stm-assoc.org/2012_12_11_STM_Report_2012.pdf), and it's completely impossible for a doctor to keep up with all of the latest knowledge. A search engine can help a lot in discovering otherwise unnoticed connections.
I'm not sure how insurance companies are going to approach this - but "the computer agrees with my decision" seems to be a good defense argument in a malpractice case. And if the end result is statistically better, then insurance companies will adjust their prices.
Kaiser makes use of an expert system when determining whether to schedule an appointment. I once called in with the the symptoms, "Excruciating pain in the chest, left arm just went completely numb at the same time" - and the admitting nurse (under the guidance of an expert system) determined with three quick questions that it wasn't urgent. (Any shortness of breath, are you light headed, if you press your fingernail down, how long does it take to return from white to red)
So, expert systems are already here and helping make decisions...
With that history you should have been seen urgently. There are a number of serious things that are possible with those symptoms.
Plus, the nurse was relying on your accurate reporting of symptoms. Things are often very different face to face than over a telephone. People very often under/overplay their symptoms.
Presumably those people who've developed the kaiser system have come to a different conclusion than you. It may be the case that there is no scenario in which "Excrutiating Chest Pain + Numb Left Arm" is urgent when you have a full return to red on when pressing your fingernail (blood pressure check?)
Also, presumably, if I was having any shortness of breath, it could be determined just by talking to me. They may have also taken my age into account (32) and decided that 32 year olds don't have heart attacks in the way I was describing.
I came in for a checkup a week later, and all was well (except for a RSI issue that was contributing to the completely numb left arm)
Oh yeah this would never be purely AI...think of Watson like the Tardis and the Doctor like ...The Doctor. They go hand in hand, Watson simply makes it so the Doctor doesn't have to waste time.
In 2012, Memorial Sloan-Kettering Cancer Center in New York began work on an adviser to recommend cancer treatments. Dr. Mark Kris, a Sloan-Kettering oncologist, said an early version of the Watson tool could be used on patients later this year if it passes tests.
At his office, he pulled out an iPad and showed a screen from Watson that listed three potential treatments. Watson was less than 32% confident that any of them were correct. "Just like cancer, it is much more complex than we thought," Dr. Kris said.
Watson may start really outperform humans when his strength - the ability to ingest and process/correlate huge amounts of, probably with not completely understood relevance, information, like full DNA sequence of a patient, etc... - will be fully utilized, and some previously unknown to humans patterns start to emerge.
Is there even a product to sell to medical? I thought Watson was just a proof-of-concept system. An actual product based on Watson would need to have some defined scope (i.e. only used for cancer treatments), would need some sort of FDA clearance which would probably require clinical trials. Is it there yet? Does IBM even want to take on the liability and expense of bringing it to medical? And is the market even big enough to interest IBM? Today, the "decision-support system" market is tiny.
Slow to adapt comes from many factors. We've been studying the history of medical innovation. The biggest hurdle is the training time required. We're talking a decade to get credentials. Then another decade to get experience. Those 20 years are hard work (schooling, residency, long hours, loans, meager pay). By the time they're ready to contribute, risks are not a good thing esp adding to the chance of doing harm to patients. So the status quo persists until the next generation comes along.
We're meeting mental health professionals who were trained in the 60s, 70s, and 80s who proudly claim they know little about the brain. I don't blame them entirely - when they were trained we only knew about the human brain from stokes and open head trauma. To get them up to speed requires a whole new training. That's very hard when you spend all week seeing patients - that's your livelihood.
For specialized tasks expert systems have been better than doctors since the 1970s (Mycin, infection/antibiotic assignment). So I would guess there is some resistance.
So far Watson for developers/business is 100% PR and 0% real. In November they announced the Watson API. Where is the API? Where is the documentation? Where are the examples? Google it and you'll get a torrent of PDF press release and incentives to call their sales team.
I'm afraid Watson is just a PR stunt. Was it oversold by IBM engineers to their executives? Or by the executives to the PR team? Or by the PR team to the press? I don't know. But they lost control of it.
Watson in its original incarnation could be already quite useful. It could provide workers with valuable input on what their manager has in mind when he babbles incomprehensibly throwing his favorite buzzwords at random.
At least 20% of full-stack programmers job is to figure out how people want the computer to behave and all we have to work with is chaos of words that flow from their mouths and fingers.
So Siri is for consumer devices and Watson is going to be for businesses. I dont think they are going to be killing each other since they are targeting different markets.
I asked Siri "Did Michigan win its bowl game?" and it gave me the right answer and said "Michigan lost to Kansas state in the Buffalo Wild Wings Bowl.
Wolfram Alpha, with the same query, just gave me information about the state of Michigan.
Bing gave me search results about Michigan Football, and Google's top results were an article about the actual bowl game. I don't have an Android phone to try Google Now.
I'd like to see that happen if only to push Google to open a Google Now API. To me this sounds like Wolfram Alpha for business more than Siri or Google Now though.
Comment 1 - Most likely it's not a full "Watson" but rather a network appliance that slots into a rack. Watson is powered by 2,880 8-core IBM POWER7 processors, which AFAIK haven't received a core bump or a die shrink since their introduction in 2011.
Comment 2 - POWER7 (which came out in 2009) was replaced by POWER7+ in 2012. IBM shrunk the lithography but kept the die size the same, so they used the extra space for more cache, a crypto accelerator, a compression/decompression accelerator, and some other goodies. There were able to bump up the clock speed as well.
Core for core, POWER7+ is about a 20% improvement, but you're right, no more cores per socket so there is no way they would see the kind of shrink described in the article if they kept the same amount of compute power.
IBM did come out with a new blade design (Flex Systems) with denser packaging, but that combined with the faster CPU will still only get them about 2/3rd of the way there (still impressive).
Honestly, look at the new MacPros. The computing power from even 10 years ago is amazing. WATSON will soon be software only for most standard servers. Looking at the SoftLayer acquisition in that light is very exciting.
It would be more convincing if Watson accomplished some real knowledge archievment such as finding a cure for a specific disease, or publish some papers enhance our understanding of some research topics ...
Unix starts small, it works. Google starts small, it helps us tremendously. Haven't seen something starts as a big business plan can success greatly? even Microsoft started small ...
That said, the approach and subsequent utility might not live up to the hype that IBM is pumping out. It's one thing to search very quickly. Being able to discover patterns that lead to new levels of understanding and predictable relationships is another thing entirely. IBM is more search vs predict in part because they only have so much data to work with. All of the medical books in the world are a drop in the bucket in terms of algorithmic understanding. Watson has mastered working with all available information. Collecting and processing massive data sets is another challenge that IBM hasn't been willing to tackle yet.
IBM is billing Watson as the all singing, all dancing solution to the world's data problems. They're tackling a lot of problems in diverse areas. I hope it works out, the world needs as much help as it can get. But IBM has shifted their core mission to be consulting and I wonder if Watson's purpose will be to support that more than becoming a Super Siri type software project that could do the most good.