I clicked the review tab, it showed there was 1 First Post, and 1 Early Answer to be reviewed.
When I clicked each, there was no task waiting for me to review.
Is this because someone else clicked it right before I did, or is this a bug?
This is a known behavior where others can take review tasks in the time between when you looked and when your request hit the server.
To check on this, you can look at the review history for the queue that just has shown no tasks to see who handled the items that were just recently outstanding.
This is far more common on large sites (like Stack Overflow) where dozens of questions arrive each minute and it's not uncommon to have hundreds of reviews happen within a 60 second interval.