I tried exploring the Fourier series a bit more (to learn DSP well). I found two wonderful references, which I must share.
Eric W. Weisstein. "Fourier Series." From MathWorld--A Wolfram Web Resource. The discussion of orthonognal functions is particularly insightful and so is the discussion on the Gibbs Phenomenon.
The second one is R W Hammings book on Numerical Analysis. I wish all books were written in the same simple manner this book adopts. After all, the "Purpose of computing is insight, not numbers"
I remember some of these details from my engineering days, but I wish I had access to these resources then.