stream instable when people connect to stream

What can I do if the stream gets a little unstable when people connect to a live stream?

I have around 60 live viewers and the stream stutters for people who are already watching when other  people connect to the stream.

Should I add ram or can I adjust some software settings?

Thanks.