What you experience is the nature of network music performance in the public Internet without guaranteed packet delivery tinmes. Make sure that everybody is connected via cable to their router and avoid any additional cross traffic (Netflix, Internet Gaming etc.) in any of the involved home LANs. This will lower the risk of such dropouts but you cannot 100% ensure it won’t happen again. Increasing the jitter buffer helps but at some point will lead to too much latency.