You will need to add a LIMIT clause to to your SQL. For example:
SELECT *
FROM my_table
LIMIT 0,10
Where 0 is where you want to start from, and 10 is how many records you want.
To make this work on many pages, pass a variable via the URL, and use it to calculate the appropriate numbers for the LIMIT clause before running the query.