US Vice-President Mike Pence has skipped a dinner at which he was due to share a table with North Korea's ceremonial head of state Kim Yong-nam.