Brees first really good season came in the same year they drafted Rivers. Then they kept Drew for one more year, before letting Rivers take over. Even that decision was made easier because of a Brees' injury. They obviously didnt view Brees as their franchise Qb, prior to drafting Rivers. He was just an average QB at the time.
And again, Kurt was just a veteran stop-gap. Who hadnt had a good season in years.