once voldemort found out that either harry or neville were going to be his downfall, the potters and longbottoms were in hiding. So he couldn’t just waltz into their house and find them. He decided that Harry was the bigger threat and put all his efforts into finding him and so when wormtail betrayed them he basically DID just waltz over to take care of business.
He killed james without blinking and would’ve done the same to lily but because snape had asked that she be spared, voldemort gave her the option of living. She refused and chose to die for Harry, her choosing death when she could’ve chose life put a powerful spell on Harry (the power of love pretty much) and it protected him from the threat. the charm left on him by his mothers sacrifice rebounded voldemorts killing curse (which would’ve killed harry just as easily as it had his parents otherwise) and so instead of hurting harry, voldemort got the AK spell and thats how his body was destroyed. (had he not had horcruxes to keep him alive, he would’ve been completely dead then)
it’s also why voldemort couldn’t touch him in philosophers stone, and why he wanted his new body to be made with harry blood in goblet of fire. so that the spell lily’s death left on harry wouldn’t affect him anymore due to their sharing blood.
This information is like so ingrained into my mind that I honestly didn’t realise that they don’t explain it in the movies o_o