Gandalf was the last king of the AlfR-clan which ruled over Alfheim, what is now the westcoast of Sweden. His sons were killed by Halvdan the Black, and later Gandalf was killed by the same person.
A note upon this king of small power can be found in the poetic Edda, where he is said to be a dwarf, i e with small power. You can learn more about the original Gandalf from "The Saga of Halvdan the Black".
I am sure that Tolkien had this priest-king in mind.