art by =saintchase

Theoryland Resources

WoT Interview Search

Search the most comprehensive database of interviews and book signings from Robert Jordan, Brandon Sanderson and the rest of Team Jordan.

Wheel of Time News

An Hour With Harriet

2012-04-30: I had the great pleasure of speaking with Harriet McDougal Rigney about her life. She's an amazing talent and person and it will take you less than an hour to agree.

The Bell Tolls

2012-04-24: Some thoughts I had during JordanCon4 and the upcoming conclusion of "The Wheel of Time."

Theoryland Community

Members: 7653

Logged In (1): uausnaughbroo,

Newest Members:johnroserking, petermorris, johnadanbvv, AndrewHB, jofwu, Salemcat1, Dhakatimesnews, amazingz, Sasooner, Hasib123,

Theoryland Tweets

Forums

Home | Chat | Old Forums(Yuku)


Go Back   Theoryland of the Wheel of Time Forums > THEORYLAND STEDDINGS > Forum Archives > Archived - Non Wot Discussion Boards > Archived: Non WoT Related Discussion 09/09 - 9/10
User Name
Password

 
 
Thread Tools Display Modes
  #1  
Old 09-29-2009, 06:47 PM
JSUCamel's Avatar
JSUCamel JSUCamel is offline
Hero of the Horn
 
Join Date: Dec 2005
Location: The Windy City
Posts: 3,264
JSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant future
Send a message via AIM to JSUCamel Send a message via MSN to JSUCamel Send a message via Yahoo to JSUCamel
Default Any of you know anything about databases?

I'm working on a project and I need some assistance. I've got a database chock full of data, and now I need to develop a search engine type thing to search the database and return relevant results.

Here's an example:

SEARCH: France

RESULTS:

1. France is a country. (exact match)
2. Spain is next to France. (related match).


Now, that works. France is a country, and so it checks each field and determines that in the "country" field there is a record that contains an exact match of "France", and so it returns results correctly in that vein. If something is an exact match, it shows up at the top. If something is a close match, it shows up after the exact matches, and then it shows the matches that contain the search query in some capacity.


Now let's try something different:

SEARCH: France blue

RESULT:

No results.


Why does this happen? Well, it's checking for exact or close matches to "France blue". The problem is that "France" is a country while "blue" is a color. Unfortunately, no field in the database contains a record that reads "France blue".

If I simply split up the words and run separate queries, I still don't know how relevant this result for "France" is to the color "blue". In other words, I don't know how to order the results or to filter out irrelevant ones. For instance, "France" with "red" isn't relevant to this query, since we're specifically asking for "blue", but if I were to run separate queries, the program doesn't know to filter out the red result.

Basically, what I need is an algorithm to check the all search terms against the database and return the most relevant results.

Any ideas?
__________________
---

"Nice try, Jack!" - Robert Jordan (to Camel)
"Camel must be the coolest TLer ever, by now." -GLotD

A grilled cheese eaten more than five minutes later is just no good. -Crispin



  #2  
Old 09-29-2009, 07:15 PM
Ivhon's Avatar
Ivhon Ivhon is offline
Hero of the Horn
 
Join Date: Dec 2005
Location: The Blue Star Deep in the Heart of Tejas
Posts: 3,672
Ivhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond reputeIvhon has a reputation beyond repute
Send a message via MSN to Ivhon Send a message via Yahoo to Ivhon
Default

tltr.

Short answer. No.
__________________
I like your Christ, I do not like your Christians. Your Christians are so unlike your Christ. - Mahatma Ghandi

Proud owner of the first post on TL's VBulletin forum (sorry...I had ta )

Greeter Emeritus
  #3  
Old 09-29-2009, 08:23 PM
Dragon Thief's Avatar
Dragon Thief Dragon Thief is offline
Hero of the Horn
 
Join Date: Jan 2006
Location: Sunnyvale, CA
Posts: 828
Dragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond repute
Send a message via Skype™ to Dragon Thief
Default

Quote:
Originally Posted by Ivhon
tltr.
It's tldr, useless dolt.

And no, I don't know how to do search relevancy either like that, although I'm fairly sure it'll involve more than just SQL.

I'd run separate queries (and/or subqueries) to get all possible results, store them into an array. And then apply a relevancy algorithm to the array (thus re-sorting it), because I honestly don't know how to sort by relevancy from within SQL.

Not much help, I know.
__________________
Terez is old as hell, and it's still her fault. Has been all these 40 years too.
"Yeah... I'm my computer's physically abusive girlfriend" - Birgitte (on Facebook)
  #4  
Old 09-29-2009, 09:02 PM
Weird Harold's Avatar
Weird Harold Weird Harold is offline
Hero of the Horn
 
Join Date: Dec 2005
Location: "Lost Wages," NV USA
Posts: 5,008
Weird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant future
Default

Quote:
Originally Posted by JSUCamel
If I simply split up the words and run separate queries, I still don't know how relevant this result for "France" is to the color "blue". In other words, I don't know how to order the results or to filter out irrelevant ones.
I'd search for the first word -- France.

For each hit 'France', I'd search that record for the next word -- Blue.

For each hit on 'Blue," I'd tag the record as relevant.

If there were more search terms, I'd search the record for them in turn and increment a relvancy index for each hit.

If the order of search terms isn't relevant -- i.e. France Blue is the same as Blue France, I'd search for each term as each record is read and increment a "relevancy" variable for each match.

Of course, it depends on what sort of record you're used to dealing with -- I'm an old school Pascal programmer and prefer random access fixed length records to text markup data files.
__________________
Answers! I got lots of answers!

(Now if I could just figure out which questions they go to. )

I met you in a past life. You were wrong then, too.
  #5  
Old 09-29-2009, 09:13 PM
Dragon Thief's Avatar
Dragon Thief Dragon Thief is offline
Hero of the Horn
 
Join Date: Jan 2006
Location: Sunnyvale, CA
Posts: 828
Dragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond reputeDragon Thief has a reputation beyond repute
Send a message via Skype™ to Dragon Thief
Default

If I had to guess I say he has a MySQL database.

LAMP (Linux, Apache, MySQL, PHP) is all the rage these days, after all.
__________________
Terez is old as hell, and it's still her fault. Has been all these 40 years too.
"Yeah... I'm my computer's physically abusive girlfriend" - Birgitte (on Facebook)
  #6  
Old 09-29-2009, 09:42 PM
Weird Harold's Avatar
Weird Harold Weird Harold is offline
Hero of the Horn
 
Join Date: Dec 2005
Location: "Lost Wages," NV USA
Posts: 5,008
Weird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant futureWeird Harold has a brilliant future
Default

Quote:
Originally Posted by Dragon Thief
If I had to guess I say he has a MySQL database.

LAMP (Linux, Apache, MySQL, PHP) is all the rage these days, after all.
Probably, but thepoint I was making is that I don't know the fine points of that type of data record. The main differenc eis probably whether each field is fetched and parsed separately (as a line of text) or the entire record can be read with a blockread type of fetch.
__________________
Answers! I got lots of answers!

(Now if I could just figure out which questions they go to. )

I met you in a past life. You were wrong then, too.
  #7  
Old 09-29-2009, 10:00 PM
JSUCamel's Avatar
JSUCamel JSUCamel is offline
Hero of the Horn
 
Join Date: Dec 2005
Location: The Windy City
Posts: 3,264
JSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant futureJSUCamel has a brilliant future
Send a message via AIM to JSUCamel Send a message via MSN to JSUCamel Send a message via Yahoo to JSUCamel
Default

Yes, I am using a mySQL database. It's not the query code itself that I needed but the algorithm.

However, it turns out that mySQL has a built in function for relevancy in searches. Cool! All is well and I wasted about four hours of coding trying to figure this out manually.

Thanks for the help though
__________________
---

"Nice try, Jack!" - Robert Jordan (to Camel)
"Camel must be the coolest TLer ever, by now." -GLotD

A grilled cheese eaten more than five minutes later is just no good. -Crispin



 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 11:27 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.