Ibister is a Scottish surname of non-Gaelic, non-English origin. It derives from the name of the locale "Isbister" on the island of South Ronaldsay, in the Orkney Islands just off the north coast of Scotland.
The Orkney group, inhabited since the Late Stone Age, were invaded and permanently settled by the Vikings in the 800s. As a result, the traditional culture, language and place names are derived from Old Norse. The Scots language of the mainland replaced Norn, the local dialect of Norse, by the early 1800s.
Most surnames in Orkney derived from the homeplaces of their bearers. I(s)bister as a placename could likely mean either Is+bister (icy farmstead) or Isa+bister (farmstead belonging to a man named Isa or Issa). I couldn't find a definite answer, but hope that helps.