Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why do you think there is no open and free search engine? I don't think it's because of hosting costs, because a distributed search engine could be run with local hardware.



A search engine is many things in a giant package. Most wholesale search companies have a database text/images and algorithm data that is in the terabytes if not more. They also have many servers for fast and efficient processing of queries and giving results.

This database could be made freely available (see CommonCrawl) but without a datacenter big enough to store and run queries on it, it's meaningless as a "open" search engine.

A distributed search engine runs into all of the scalability and quality of service problems that researchers of Peer-to-Peer Information Retrieval have been trying to solve since the early days of the internet.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: