The tax body had used travel data to conclude that thousands of parents had left the country permanently but many say they just went on holiday.