Here I sit, in my small studio, with indexed archives on one panel, a cartoon on another panel for my son to watch, and yet another monitoring this weekends parsing progress. Search can take up so much time.
But yet it’s worth it I feel. Bigskybot has been around now for a while, and slowly I’ve watched our tech search index go from hundreds of sites to thousands.
Every website we index has to be viewed the old fashioned way. With human eyes. Pop-ups, redirects, and spamvertised pages are filtered out of the results for the most part, simply because it’s a true and direct experience we are hoping to achieve.
Our Tech Search is a niche index. Only tech related websites are parsed and indexed.
Sure, you can always search Google, or Bing for information technology websites, but why would you want to, when they list websites just for the sake of listing them? The quality of web search has gone down quite a bit over the last 10 years. Search, giving way to just being glorified ad agencies (never mind the results, just as long as we get those ads sold)
Websites that have advertisements strewn across the entire top quarter of the home page aren’t what we’re looking for. Things like Adsense and Adwords are fine, but using these and others like them to destroy the integrity of the homepage is a bit of a stretch, so we don’t list them.
The search industry is pretty stagnant. Everyone uses API’s these days, that draw from other established search engines. They put the API on, and declare it as their own search engine, when actually it is not.
I won’t use an API. I want the certain unique qualities that come with independent search. Our search draws from it’s own database(s) and has an algo all it’s own. If Google , or Bing, or Yahoo, ever went out of business, our search would still produce results.
Search can be a grind however .. especially if you are one to put it on independent of any other source.
Our Bigskybot is gaining the trust that is so very important in the search industry these days. It follows the established robots.txt protocol and obeys all no-index; no-follow tags that are found in web pages.
Our Tech Search isn’t anything new or groundbreaking. It doesn’t boast of all of the pages it has indexed, and it quietly goes about it’s business as an independent entity on the world wide web.
The only thing really groundbreaking (if it could be considered so) about our Tech Search, is that it doesn’t rely on the listings of other indexes to function.
Leave a Reply
Your email is safe with us.
You must be logged in to post a comment.