You cannot completely eliminate the risk, but you can significantly reduce it. The most important step is to charge batteries in one fixed, controlled location. This helps you maintain oversight and reduces the chance of a fire spreading.