Was there no difference in fragrance either? Saffron imparts a very distinct fragrance, more than taste. And that fragrance affects perceived taste. In larger quantities, it has a bitter-ish taste. I use it in rice puddings, greek yogurt (simplest recipe possible - yogurt, sugar, cardamom powder and few threads of saffron. Big difference with and without saffron).
I never noticed a difference in the odor of saffron rice and regular rice. But as I said in a different comment, I take this to mean that I've never had saffron rice that actually had saffron in it.