ROME — The Vatican on Thursday said it had excommunicated top clerics in an archconservative Catholic sect with thousands of adherents in the United States and Europe for ordaining renegade bishops in defiance of the Holy See, triggering the most serious schism in decades within the world’s largest Christian faith.