The Search Engine Experiment
Which engine really offers you the most relevance? Take the test and find out
Comments
Random Rotation
The webpage should rotate the search engines between the letters for those of us who choose to take it multiple times.
Re: Random Rotation
It does now.
earth
help!
whats this all about??
Strange link.... strange page... but has a ranking.... interesting when there is no real content in it as such!!
Science
rotation
Science
rotation
Science
rotation
Re: Science
hi guys,
the test worked for me in firefox.... not sure if i did anything wrong.
i was hoping i could as you guys to assist me in SEO. The main question i had was what importantance do search engines give to directory structures with keywords in them, are they as important as say the h1 tag?
any help would be appreciated. my web site is at http://www.webtrendz.co.uk
silver surfer
just doing a search on the comic book character.
- By Olof Edlund at Nov 20, 2005, 4:26 pm
- Reply
Search String
audi 100 turbo quattro
Looking for a result other than buying parts.
audi 100 turbo quattro
You want to find audi 100 turbo quattro? Try Cantufind.com and then if you want to expand the search to get rid of part as the car you chose is old then add 'audi 100 turbo quattro for sale' for example. You'll find you won't beat this search
It will walk all over Yahoo, MSN and even Google without a doubt. It's called Chinese technology and is better than the rest as you don't see China trying to enter the west's market do you!!!
Re: Search String
Chinese techonlogy, ha!! All results are redirected through yahoo.com. They also mirror yahoo's results perfectly!
Abstract Algebra Field Isomorphism
I'm taking a class in this right now so presumably, I know the material fairly well.
- By Mickey Kawick at Nov 20, 2005, 4:56 pm
- Reply
another very relevant search engine
I think you should definitely add del.icio.us search engine to your panel. For me, it is _by far_ the most relevant search engine today
Re: another very relevant search engine
Del.icio.us is great, but, very different to traditional engines; apples and oranges scenario, so I won't be adding it I'm afraid.
you should try
exalead !
It's a fast growing and very relevant search engine
Blood test results
I am looking For test results ordered by Dr Micheal Cecil
- By Lyndon Shaw at Apr 7, 2006, 11:32 pm
- Reply
Blood test results
I am looking For test results ordered by Dr Micheal Cecil
- By Lyndon Shaw at Apr 7, 2006, 11:32 pm
- Reply
Blood test results
I am looking For test results ordered by Dr Micheal Cecil Wk of 4-3-06
- By Lyndon Shaw at Apr 7, 2006, 11:36 pm
- Reply
Blood test results
I am looking For test results ordered by Dr Micheal Cecil Wk of 4-3-06
- By Lyndon Shaw at Apr 7, 2006, 11:42 pm
- Reply
results of blood work
I,m looking for test results of blood work ordered by Dr. Micheal Cecil week of 04-03-06
- By lyndon shaw at Apr 7, 2006, 11:50 pm
- Reply
results of blood work
I,m looking for test results of blood work ordered by Dr. Micheal Cecil week of 04-03-06
- By lyndon shaw at Apr 7, 2006, 11:51 pm
- Reply
Clusty
Has anyone found or liked www.clusty.com ?
Interface Mucking with Results
I did it a few times and clicked twice for each one, to see which was second. (Sometimes I clicked the bad one instead of the 2nd.) I imagine I"m not the only one to do that. You could probably get better results and better serve your participants by identifying all three engines when someone clicks on any.
Re: Interface Mucking with Results
The system only counts the first vote, so clicking around won't screw the test results.
I might add that feature for (as you suggested) participants benefit, though.
Potential Problem with the test
You need to consider the fact that most people have yahoo e-mail addresses, webpages, and all kinds of things so in some cases its only natural that yahoo would have better results. You shouldn't allow names in this test.
- By Jared Torrez at Nov 20, 2005, 5:23 pm
- Reply
Re: Potential Problem with the test
I don't agree. At the end of the day what counts is the results not the means they came to them.
Re: Re: Potential Problem with the test
I agree. Names or not results should be better. How you do it is why people are paying above $400 per share for...
Cool!
I tried doing this manually for some keywords my blog are on the top results so I can very well remember which search engine is performing better for me.
results:
I think that you should allow for the possibility of idempotent results.
1-2 | 2-3 | 1-3 | 1-2-3 are more or less equally good.
- By lee.doolan at Nov 20, 2005, 5:46 pm
- Reply
msn won
ugh
What the he..
One of the search engine results looks like relevant returns but when clicked take me to this site at the submit an article page. Huh?
Other Engines
MSN won on relevance in my estimation - I searched for myself. Of course, relevance isn't the only thing that's important - I built a list of a dozen criteria when I tried this sort of research for myself - I documented my efforts on June 25 & June 30 here: http://www.something.ivebeenthinkingabout.com/
I would appreciate it if you would add AllTheWeb to your test - that's the one I use by default now.
- By BloggerRuggles at Nov 20, 2005, 7:58 pm
- Reply
None Win
I know this is a test to see which one is most relevant, but what if none are in fact returning any results that you are searching for? Also, on the result page, or maybe in between this and the result page, there should be a "Reasoning" box for users to to explain why they chose one over the other.
Other than that, its perfect.
Thanks, Allan.
educaitonal software
Not much different results
- By Gary Bloom at Nov 20, 2005, 8:45 pm
- Reply
MSN won
I tried 5 different queries. MSN won hands down in all of them
interesting
Pretty neat idea. I've put it up on http://www.bsalert.com/
No go
I get the message: "One of the search engines isn't playing ball right now please wait a while, then try again."
Unfortunately, this message doesn't go away...
- By Unsuccessful User at Nov 20, 2005, 10:27 pm
- Reply
i would
like to know
Other search engines
Please add clusty.com and ask.com to the search engines!
It's broken
It doesn't work anymore
Broken
It does now work !
nice idea, but...
I agree with the idempotent guy. You should allow users to rank brand x, y, and z and cacluate success based on that. For example, a search engine that always gave the second most releavant set of results would appear to be doing much worse than engines that gave 1st 1/2 the time and 3rd the other 1/2 the time, when in fact it's at least as useful. Also, what if two engines give exactly the same results? You have to pick one or the other.
Also, are you limiting votes based on ip address? Does that mean that if someone from my company (who appears to you to have the same ip address as I do) has already done this, then my vote doesn't count?
I think this would be more valid if you had a large number of participants, each doing several tests on a range of subjects and ranking the results. Obviously you want to avoid people trying to skew the results, but the way you have it set up now seems pretty flawed.
- By Anonymous Coward at Nov 21, 2005, 3:34 pm
- Reply
Re: nice idea, but...
I agree, it probably could be a bit more scientific. The problem is that would involve asking a lot more thought process of participants, and unless they're being paid, I think that would put most people of doing the test.
We do only count one vote per IP, and I think this is absolutely necessary to prevent ballot stuffing, with the side-effect that some valid votes will not be counted.
biased?
google ... 4 times in a row :P
- By Anonymous Coward at Nov 21, 2005, 5:17 pm
- Reply
Randomness or not
yahoo seem to be right every single time for me...
interesting results
Google came out on top for me most of the time. But I think they need to show more results of each search to paint a more accurate picture.
- By some_young_guy at Nov 21, 2005, 11:48 pm
- Reply
Re: interesting results
I agree, I need a longer list to determine which one returns the most relevant results for my queries. 2/3 right is not enough to determine one engine is less capable while the other 2 are returning 3/3 right. A listing of 8-10 would be more conclusive.
- By Danielle Borisoff at Feb 22, 2006, 4:59 pm
- Reply
non-English search
When typing non-English words (in my case, Japanese), only Google seems to return the same results with my browser-search results.
I do not claim that you should fix it for all G/Y/M-supported languages, but the statistics could be biased to Google as some visitors might vote for Google just because other 2 returned results makes no sense.
Re: non-English search
Yep, we noticed this issue. To counteract any bias the system tries to recognize and then disregards all votes for non-English searches.
Note am ideal solution, but unfortunately getting the different engines to return all the correct results for all languages seems to be impossible.
Not enough results to compare
Good idea, but for my taste not enough engines and results to compare.
Try www.etools.ch where you can compare up to 10 search engines (after a search, check each data source in the 'Results Tool' window on the right side and play with 'advanced search' to adjust the number of results)
- By Stephan Schmid at Nov 22, 2005, 8:55 am
- Reply
Bug in code
Searching for "deed of postponement" produces incorrect link for item 2 of brand y I think due to a "'".
Re: Bug in code
We can't reproduce that bug. Can I ask which browser you are using? That might be the cause.
not working ...
hm, i recently blogged about your experiment and now it isn't working :-(
"one of the search engines isn't playing ball right now" ...
- By netzkobold at Nov 22, 2005, 12:57 pm
- Reply
Re: not working ...
Working again.
The Yahoo API restricts the amount of queries per 24 hours, so this can happen from time to time. We have multiple IPs to try counteract this issue.
What Yahoo and MSN are worth against Google - Ce que valent Yahoo et MSN compares a Google
Yahoo Search un an apres : enfin un vrai concurrent pour Google !
http://www.precisement.org/blog/article.php3?id_article=11
MSN Search : que vaut le nouveau moteur de recherche de Microsoft ?
http://www.precisement.org/blog/article.php3?id_article=70
- By Emmanuel Barthe at Nov 22, 2005, 7:34 pm
- Reply
discuss against google at http://www.againstgoogle.com
discuss against google at http://www.againstgoogle.com
- By againstgoogle at Dec 5, 2005, 6:34 am
- Reply
http://www.againstgoogle.com
http://www.againstgoogle.com
- By http://www.againstgoogle.com at Feb 28, 2006, 7:55 am
- Reply
rank instead of pick best?
how about 1-2-3 ranking, instead of just picking the best? I wanna know who #2 and #3 are... :P
Also, Ask Jeeves?
Yahoo & MSN Tied
Yahoo & MSN each returned different links for search on sales tax software. Both lists were relevant and each seemed to favor a specific software vendor (Yahoo featured Vertex, MSN favored Burr Wolf).
I also have had good results with AskJeeves
- By Ed O'Neill at Nov 23, 2005, 11:05 am
- Reply
Firefox???
This doesn't seem to work with a FireFox browser (v 1.0.6 on WinXP Pro).
Sloppy.
- By DavidFilmer at Nov 23, 2005, 9:51 pm
- Reply
Re: Firefox???
It does work with your version of Firefox and all other versions we've tested.
Maybe you tried during a very busy period, and so you would need to be a bit more patient for the results to load.
Re: Re: Firefox???
Um, I second the firefox not working comment - Mac 10.3.9; Firefox 1.0.7.
Re: Re: Re: Firefox???
Can you give the following information, to help me resolve this issue:
If you try again, then go to Tools -> JavaScript Console, what's the last error message (if any)?
Thanks
Yahoo results...
Yahoo's 2nd and 3rd result are prefixed by http://rdrw1.yahoo.com/click?u= and terminated by &y=021FCCAC05E42163&i= [Ed: Snip, to prevent horizontal scrolling]
-- which means it's not a "blind" test any more, and which means that under Firefox the column corresponding to Yahoo is much, much wider than the others
Re: Yahoo results...
What was the search query?
I think you'll find that this is case of Yahoo indexing its own redirect URLs. Try a different search, and you'll see that Yahoo's URLs are not tracked.
Yahoo results not the same as yahoo.com, at all
Hi,
When I look for my web sites with navigational queries like "joueb" or "viabloga", I don't get the same results at all on what you show as "Yahoo" and yahoo.com. The results are completely different. Are you scraping yahoo.com or using the API?
For almost any other query, the results are different as well.
Thanks,
Stephane
Re: Yahoo results not the same as yahoo.com, at all
Yes, there are issues due to way the way Yahoo API functions that mean non-English searches do not usually return as accurate results.
We do not count non-English searches towards the total, so this should not introduce bias.
Bug or Bias?
A couple of "obscure" searches resulted in Google being the only one that returned links to this "experiment" However, when the same search was performed via their respective homes...both MSN and Yahoo returned results. Something's flawed here.
Re: Bug or Bias?
There is no intentional bias, that's for sure. Can you give me some examples of these obscure searches? so that I can look into it.
true blind test
This is a true "blind" test. It awards the lowest rated search engine and punishes the highest rated search engine.
Why? it is very simple. Since many results are equally good among the three, it would then voted randomly to each service.
A scientific one should have votes counted for "three are the equal good" and "two are the equal good".
Re: true blind test
I agree. I voted almost at random, since the three results were basically identical, instead of searching again. This would overall tend to "flatten" the difference between search engines, awarding votes to an overall crappy engine.
More results
Seems like 3 results per engine is too little.... but a cool idea nonetheless.
Good one.
A very good test indeed.
But I got Yahoo as the best one.
Not bad.
Good one.
A very good test indeed.
But I got Yahoo as the best one.
Not bad.
Good tool
Good one.
A very good test indeed.
But I got Yahoo as the best one.
Not bad.
- By http://www.freewebs.com/telecomm at Jan 4, 2006, 5:16 am
- Reply
X vs Y vs Z
could you show statistics of the actual letter that was chosen? i.e. did perhaps "Y" receive more than "X" just because of location? This is only useful data assuming an equidistribution of search engines to letters.
- By Nick Wagers at Jan 26, 2006, 4:53 am
- Reply
Re: X vs Y vs Z
Which engins is x/y/z is rotated randomly, so that shouldn't be an issue
3 listings are not enough
I need a longer list to determine which engine returns the most relevant results for my queries. 2/3 right is not enough to determine one engine is less capable while the other 2 are returning 3/3 right. A listing of 8-10 would be more conclusive.
- By Danielle Borisoff at Feb 22, 2006, 5:02 pm
- Reply
duda
Cuantos de aqui estamos en clase de RAI?
almost
No difference.. what's "good" about this test?
hmm
doesn't seem to be working.
- By Aaron Pratt at Mar 18, 2006, 5:33 am
- Reply
Firefox 1.5.0.1 not working
I tried to paste the javascript console error message, but your comment field keeps rejecting it.
Seo Good
Good one.
A very good test indeed.
But I got Yahoo as the best one.
Not bad.
wallpapers
wow great!
- By wallpapers at May 2, 2006, 2:55 pm
- Reply
Doesn't work
I get "Fatal error: Cannot create references to/from string offsets nor overloaded objects in /var/www/html/lib/format/converter.php on line 136" as a JS error when I try to do a search.
- By Emmett Shear at May 13, 2006, 7:51 pm
- Reply
Long Loading
It takes some time to load. I just use http://z.multiz.com
- By Z.MultiZ.com Multi Search at May 23, 2006, 9:41 pm
- Reply
Doesn't work in real browsers
Doesn't work in Firefox, people don't browse anymore with IE, so why would they use IE to do this test?
- By Mad linuxer at May 31, 2006, 6:23 am
- Reply
Doesn't work in real browsers
Doesn't work in Firefox, people don't browse anymore with IE, so why would they use IE to do this test?
- By Mad linuxer at May 31, 2006, 6:23 am
- Reply
Doesn't work in real browsers
1.5.0.03 same sh*t
- By tweaktubbie at May 31, 2006, 6:42 am
- Reply
NS8
Not with Firefox, but it works with Netscape 8.
I won't use IE
I won't even test it with IE. I already tried with Opera and Firefox and it doesn't work. It is a shame, really.
Page Rank and Alexa Rank
Hello Everybody! I want to enquire you about the tips and tricks to increase the Page Rank, Alexa Ranking and Web Traffic of my website http://www.infohuge.com
kindly help me!
Thanks
- By Gurpreet Singh at Aug 9, 2006, 11:10 am
- Reply
Data Recovery Software
checking the results of that query
- By data recovery software results at Sep 19, 2006, 6:14 pm
- Reply
Search Engine Experiment
really good...
Re: ..
very interested theme
- By mieszkania warszawa at Oct 7, 2006, 11:26 am
- Reply
Submit Comment