First, you need to decide whether you want this function to work for everyone or just for people who have accounts on your site. If you want it to work for everyone, then you will have to set cookies. If you only want it to work for people who have accounts, then you can keep a record of the time of their last visit in your database. The database method is nice because the "New" tags still work even if the user moves from machine to machine.
The next thing you have to decide is what criteria you want to use to determine whether or not something is new. For example, may I visit the site on December 1 and there is a topic about basket weaving. I'm not interested in basket weaving so I don't click the link. What happens when I visit the site on December 5? Should it still say "New" (since I've never read it) or should it not say "New"? Or maybe new things should be labelled "New" until I read them or until ten days go by, whichever comes first.
Once you have these questions answered, then you will know how much data you need to store. A simple way to make this work is to store the TIME of the user's last visit. (You can store that in a cookie or in the database). Then you can store the most recent update time for every item on your site. (For example, the basket weaving site was updated 10 days ago. The "Photos" section was updated 5 days ago.) Then, when a visitor comes to the page, you can read the time of their last visit (either from the database or from a cookie) and display the "New" tag next to the items that have NEWER dates than their last visit. This is the simple way and it might be what you're looking for.
Another way to do it, which is more difficult, is to store the time of the last visit for every item on your site and for every user you have. This way, when the user visits the page, you can check the update time on each item and the "last viewed time" for that item for that user and then you will know if you should display the "New" tag next to each item. This is more work for you and increases the size of your database. But it will make a little more sense to the user. For example, if you had a category about PHP and a category about Perl, and I visited the PHP category on December 1 but NOT the Perl category... then the next time I visit, the Perl could still say "New" since I haven't read it while the PHP category does NOT say new since I read it since it's been updated.
As you can see, this is going to require a good deal of planning. You're also going to need to decide exactly how it should behave. That's really your first step.