Not in South Africa, it isn't. Here it's exclusively feminine - a boy named Esme would have a lot in common with a boy named Sue - and we invariably pronounce it EZ-may. It's a bit dated now - Esmes tend to be older women - but I expect it'll be back.
We had an actress of that name - radio, TV and I thnk stage though I never saw her live. Esme Euvrard. Definitely a grandmotherly figure - at one time she advertised washing powder.