I find programming to fractally involve debugging all the time. So if I allowed warnings when debugging I'd be dead :)
You're right that there are exceptions. I think of warnings as something to indulge in in the short term. The extreme short-term; I try very hard not to ever commit a patch that causes warnings. It really isn't that hard in the moment, and the cost rises steeply thereafter.
Incidentally, I'm only this draconian with C/C++. Given their utterly insane notions of undefined behavior I think it behooves us to stay where the light shines brightest. Whether we agree with individual warning types or not, it's easier to just say no.
But with other languages, converting errors to warnings is a good thing in general. Go, for example, goes overboard by not permitting one to define unused variables.