You need to put in a timestamp tagging an empty line for the lyric-less sections. Have the timestamp be shortly after the singing from the previous line ends.