I have been visiting Apple.SO every day throughout the day including yesterday, and today I see the counter has gone from about 15 down to 1 :-( This is the second time it has happened (last time I was up to 13) so its beginning to bug me.
You may be falling on the wrong side of when the server rolls over between days - the day changes at midnight GMT (10 AM in Brisbane, 8 PM EST) so if you don't visit between 10 AM today and 10 AM tomorrow you won't have visited for 24 hours despite having visited once each day (eg. 9:30 AM today and say noon tomorrow).
Check out Consecutive Day count reset this morning on MSO for a similar situation.
Also, just visiting the front page isn't always enough to make it count.