why did only Levi people refuse from sin with golden calf? what helped them or saved them?

ב"ה Shalom The Da'at Zekenim in his commentary on the Chumash (Shemot 32:26) deals directly with your question. Shevet Levy who was close to Moshe didn't even consider the possibility of replacing Moshe when according to the calculations, Moshe Rabenu had not come on time. There was loyalty to Moshe in other tribes but only Shevet Levy was totally loyal. There were 3 groups during the sin of the golden calf. One were those who only wanted to replace Moshe, the second group were those who actually worshipped the calf (the 3000 that were killed) and there was Shevet Levy which was totally dedicated to Torah and loyal to Moshe. All the best and have meaningful 17th of Tammuz fast