Let's assume a couple of thing.
You have one single image, which is 1400x700 pixel.
Some of your users have 1440x800 or higher screen resolution and you serve them the full sized image (1400x700)
Most of your users have 1024x768 or lower screen resolution and need a resized image (980x490)
So, for some of your users you just read the image data from the image file, realize no conversion is needed and send image data to the browser.
But for most of your users, you have to read in image data, realize resizing is needed, make use of GD or Imagick to make the necessary resizing of the image, and then send the newly resized image to the browser.
Looking at the difference between these two cases, it becomes apparent that one part in the second case is extra work, compared to the first case: resizing the image. But since you resize to the same dimensions every time, why bother?
What you do instead is:
decide what image dimensions you need.
resize the original to all these dimensions.
store them in a way so that it is possible to retrieve them in the right format.
That is, you save the 1400x700 image, you also resize it to 980x490 and save this.
Now, for each incoming request you decide which image size the user should get, and then either serves the 1400x700 image or the 980x490 image. Every request for the smaller image is suddenly faster as it requires no additional image processing, just plain i/o: read from disk, send output to browser. The cost is additional memory usage: you now have two images on disk instead of one. But storage is cheap so it (probably) doesn't matter.
And the same obviously goes for thumbnails. Resize once (create thumbnail) and store on disk.
Edit: But if you happen to get a visitor with 640x480 screen resolution and don't want them to suffer scrolling, this might be an appropriate example case for on-demand resizing of the image. It happens so rarely that it doesn't matter if you do it on the fly or not.