Impressions / Clickthrough software

Dillenger

Hello,

I'm looking for a php/mysql script that will allow me to track my custom search engine on my site. What I need is:

Track what user searched for and which listings came up (impression for each listing)
Track if a listing gets a click through with that search

I have tried out Google Analytics, but unfortunately it doesn't get that specific... for instance, it can track the keyword, but it won't tell you which listings came up when that keyword was searched.

Can anyone suggest a good tracking tool, or a good method of coding this myself?

Thanks in advance.

dagon

i would write that myself, as you wrote the search engine 'just' add logging of every querry

1 - is basic logging can be done on the page that returns the results

2 - you will have to rewrite the urls with some tracker code in it - or go the otehr way and check the reffere at the destination (not so reliable)

bpat1434

Well, as for saving the search and results, that's just typical logging as dagon said. You can use a text-file, xml, or even a couple tables in the database to save the info 😉 Whichever you prefer and works out for you.

As for knowing what link was clicked, here's a good spot for sessions/cookies. You can set a session variable (like "referrer") and give it the value of say the search "id" 😉 Then on every page you check to see if $_SESSION['referrer'] is set. If so, log it as a "click-through" from your search; otherwise, ignore it and move on.

That's much more reliable than using $_SERVER['HTTP_REFERER'] since not all browsers are going to send it (since it's not required). So using a session and setting your own "referrer" code would be best. Obviously you'd clear it out once they've hit the next page.

Alternatively if you're looking for a type of "ranking" scheme for search results, you could do what many large sites do and ask the user if the returned result was helpful to them or not. Let them answer either yes or no, or on a scale from 1 to 5 or something and then calculate a % based on <helpful>/views. So then you can rank it in order of "helpfulness" 😉

Dillenger

I was going to start tracking with a database, my worry is that with a large site like ours is going to be, that the table would get SO huge.

For instance, if I want to track impressions on 1 search, and there are 20 results per page. That's 20 database entries every time you switch a page or result. I guess there is no real way around that though, I'm just wondering if there is a good way to break it down, like have a day table and then a month table then a year table? I just wouldn't know how to set it up.

bpat1434

Well, the easiest thing would be to make sure your tables have timestamp columns so you can get dates when necessary. Then you could run a cron every day, hour, week, whatever, which would grab the data in the tables and either move them to a new table, or back them up to a text-file.

Either way would suffice. And the table sizes wouldn't matter. If you use nothing but keys so that your table is holding nothing but say integers, then it won't be that "slow". Now, if your MySQL server is on the same server as your mail and web, then it might slow down since it can't devote 100% of the resources to the database. But it shouldn't be slow in the beginning.

mgscox

It depends upon the traffic and the storage database format; some database formats have a higher overhead than others so do some research and select the most appropriate for your server type. However, in general, the database has to get really huge before you'll see any noticeable perfomance hit for selecting a less-than-optimal type.
That aside, you can plan ahead for that - having multiple tables for days, months, years is Ok as long as your webscript always goes straight to the day table - you don't want to increase the script/ database processing time running queries just to figure out which table to store data into as it will degrade responsiveness for the user. Look at your webtraffic and then schedule a cron job to archive the data off to either another table/ database/ text-file when your sites usage is at its lowest. If you have a pretty steady flow of traffic then execute a more frequent cron that will move less data on each run to balance the load. To ease the load further you could run a stand-alone text-file (rather than a separate table) in parallel to the database that just stores the primary key of the insertion together with a timestamp and use that to get the indexes to be archived, which would reduce the query load on the server during archive runs. There are also plenty of tutorials on the web geared to explaining how to optimize your database server performance, so check these out - tweaking MySQL variables can have a massive difference for example. Running a separate database server to the web server will increase performance as long as the letancy between the two is low and your query results are optimised.