I don't do that, at least on the first read. If I'm re-reading it I like to put music that I think fits what I'm reading to some extent, mostly instrumentals and rarely pause that to put something with vocals on the key moments or fights XD
Did that for the ocarina of time manga before and it just worked.