Personally, unless you're running around looking for hidden easter eggs (such as runaway snitches) then a full day in each is probably overkill. However you mention 3 hour lines, which 99% of the time is a bad thing (except in the wwohp, because the line and scenery are half the fun) - if you want to avoid the lines, go off season. My wife and I have been twice (obviously only to hogsmead) - once in late april, and once in mid february, and neither time were lines longer than 10 minutes... usually just walk in and get on the next carriage that comes through.

If you are a potterhead, it's definitely worth it... The food in general is a bit of a let down for me, except for the butterbeer (which is amazing.) The extra bits in the background and scenery are really what makes all the difference. Seek out the unadvertised things around the area (chior, bathrooms, things of that nature) and enjoy the wizard in world... but take some time to explore other areas too... there's a lot to be done, and the parks are a ton of fun.