GIF animation export out of Photoshop

You will loose quality on rich color images.  Gif File format only supports up to 256 mapped colors. 255 if you want transparency. If you want high quality render out and Image sequence or a MP4 video. 40K is quite small for three images the frame size will need to be small and you will loos quality if you need to reduce your current frame size.

Another thing to keep in mind is that as long as the pixels don't change, the file size will be smaller. For example, if you move an image from one frame to the next or change its layer opacity, there is no change in pixels. But if you rotate an image from one frame to the next, there will be a change in pixels.

For more suggestions, It would be helpful to see a screen capture of the animation.

You can reduce the file size of a GIF by reducing the number of colors. Since I don't know what your file looks like, I'm not sure that would be a good option for you. One saving grace of animated GIF is that, since it is animated and moving, loss of quality is less noticeable.