I forget where it was that I heard it - but olfactory memories are stronger than any other type in the human mind. (Something tells me it was Alton Brown, but I'm not positive today).
Hopefully all the memories are positive and the office scent isn't freaking you out too badly.

(if it is, febreeze works wonders!)