A famous bearer of the surname was James Madison (1751-1836), one of the authors of the American constitution who later served as president (and after whom Madison Avenue was named).
This also coincides with an Irish surname (an Anglicized form of Irish Gaelic Mac Searraigh), derived from the byname Searrach meaning "foal".