Frozen yoghurt is generally considered safe during pregnancy as long as the ingredients yoghurt is pasteurized or made using pasterurized milk.
The only caution lies in the buying frozen yoghurt from Froyo parlours serving it in cups from soft serve machines. While they may be using pasteruized ingredients or premixes that are pregnancy safe, the machine itself may have a risk of contamination if proper hygiene is not maintained as the same spout serves cups of the desserts repeatedly with older servings staying on the spout at room temperature. So use your judgement while buying frozen yoghurt from a soft serve machine to ensure it is from a clean and hygienic facility.