You are doing all of your sorting of the data after the query is performed. So when you add a limit to the query, you are then sorting some arbitrary subset of the data. To simplify it for illustration, suppose your query without a limit returned these values in this order: 5, 1, 9, 2, 10, 4, 7, 3, 8, 6. If you sort the whole set numerically, you get 1, 2, 3,...10. But if your query retrieves the data in two chunks of 5 based on a limit clause, each set then being sorted independently of the other, the two result sets would be 1, 2, 5, 9, 10 (the first 5 query values sorted) and 3, 4, 6, 7 8 (the last 5 query values sorted).
So your challenge is to either build all of the sorting logic you need into an ORDER BY clause of the query before the LIMIT clause (preferred solution), or else do the query with no limiting, do your sorting, and then grab a chunk of the sorted array of data based on your pagination offset/number-of-records requirements