If your answer is No, say No. Don’t try to explain yourself. You don’t owe anyone an explanation. No is a complete sentence.