Something in PHP most likely....in each of your foreach or while loops, add something that says (this is psuedo-code) "if the capitalized first-letter of the current name is equal to a new letter, write an anchor in". You'd probably want to use an array with all of the letters of the alphabet.
BTW, you have a badly wordly question. You wrote "Is this something I can put into the table?", which is a yes/no question. I could've easily replied with "No", and, while still being correct, absolutely no help at all. Kind of like Microsoft tech support 😃