info about number of visits/page views vs db server capabilities

VisionsOfCody

I'm working on a site with a mysql database which is pretty small for the moment but will probably start getting a whole load of visitors quite soon

I've never had to face the problems of high traffic with a lot of simultaneous users and i would like to get adequate hosting to cope with that but obviously not something that's far more than i need (to keep costs down, lol)

has anyone seen any info about the kind of requirements needed as a site expands - what are the requirements at 100,000 pages viewed ? and what about at 500,000 or 1,000,000 ? The pages will be pretty average 50k-100k pages but there may be a lot of simultaneous connections.

I really can't work this out

can anyone point me to some useful sites ?

thanks

remnant

There are memory usage formulas for both MySQL and Apache (or any db / web server); however, it's important to note that the HTTP requests are served up very quickly by a web server. Once a page has loaded there is no additional demand on the web server until the user loads another page. That being said, even if you have 100,000 simultaneous users it's unlikely that they'd all be loading a page at the same time. Furthermore, even if they were all requesting at once, a queue builds and they're funneled through.

Your biggest problem might be the database server, but again a well optimized sql server can handle a pretty high workload. If your server begins to bottleneck, it's likely because you have poorly indexed tables and your queries are taking too long to execute.

Good practice would be to separate the web server and DB server, that way they're not sharing memory. Other than that, i would purchaes normal equipment and upgrade when you begin to grow out of it.

HTH

Sxooter

Once you get to 100,000 views a day, you'll likely be using multiple database servers on fairly hefty servers.

At that point, you'll need to figure out if you're CPU or I/O or network bound. normally, for database servers running efficient queries, you'll be I/O bound, and you'll want to move on to a fast RAID controller on top of a big RAID 1+0 array. Once the array is big and fast enough, you'll be CPU bound, and you'll want to use load balanced web servers.

The only way to really tell is to build your app and start load testing it. jmeter is a pretty good loadtester, for software written in the dreaded java language. It's quite versatile. apply load, examine your system, improve the performance of whatever subsystem you're working on.

Plus, what will your read to write ratio be? if you'll nave a fair number of writes (say > 10%) you'll want to use transactional tables (or a completely transactional database, like PostgreSQL or Firebird) because the "one big table lock" design of myisam tables isn't going to cut it in a heavily updated database.

VisionsOfCody

thanks for that information from both of you

Sxooter wrote:
At that point, you'll need to figure out if you're CPU or I/O or network bound. normally, for database servers running efficient queries, you'll be I/O bound, and you'll want to move on to a fast RAID controller on top of a big RAID 1+0 array. Once the array is big and fast enough, you'll be CPU bound, and you'll want to use load balanced web servers.

I didn't understand most of that bit ! - but I'm going to read up on it

The read/write ratio will be pretty low - i mean a low percentage of writes because 99.999% of the data will be written in by a team of site admins and the users will be just reading

thanks again for those pointers