It has nothing to do with being worried about losing the game. Playing them always puts you at greater risk for getting hurt. No way you want that risk at the beginning of the year. The fact you also have to devote valuable practice time to their scheme and it won't help you the rest of the year also seems like a waste. Playing them in no way helps prepare you for the SEC season. I just don't see the benefit of scheduling them.