1.Backlinks.This is by far the biggest factor, you need plenty of them, its fine having a lot of pr0, but you do need some nice pr and quality links,.edu links are very nice.
2.Having your keyword in your domain.This is a really good one not mentioned enough.If you want to rank number one for say flash games, then the domain flashgames.com would be easy to rank up there without much seo, however the name familyroom.com would be so so much ahrder to get up there.
3.domain age.This is a touch and go area.If there are two sites competiting at the top, and one is a month older, thats not really going to make a difference at all, however, if theres a site at the top, which is 10 years old, and 500 backlinks, it will be unlikely a site which is half a year old, and has 2000 backlinsk will out rank it.
page rank also works like this, sites around pr1,2,3 or 4 dont have much authority, but if you get pr6+ it can make a very good difference in serps