Did you try Pathfinder panel Unite?
From time to time, I too have experienced Shape Builder adding more points than necessary.
My go to is usually to use Live Paint.
We don’t know your original paths to start with before you used Shape Builder.
Here I recreated using pen tool and strokes.
I outlined the strokes. Dup and Flipped.
Next Live Paint.
Expand Live Paint.
Select Same color, Pathfinders>Unite.
Nice and clean. 58 points.
Thanks, it;s made from simple shapes and then filled with Create Shape tool. It seems like Live Paint doesnt solve the problem. 😞
Lukas, thank you for posting your screenshot today of all your paths. That explains a lot.
Whether using Shape Builders tool or Live Paint, your results will be similar.
As Monika stated previously, you have so many overlapping paths, thus resulting into a large amount of anchors points.
Up to you how you want to resolve.
Personally, I would use Pen Tool and strokes, outline, Live Paint as I showed you before.
With Scissor Tool, cut and delete all those overlapping, extraneous path segments BEFORE using Shape Builder or Live Paint.
As K already wrote: use live paint.
This tends to happen when you have duplicate paths (or duplicate segments of paths)
Also when you have too many paths selected and operating on them. In my experience if possible select only those paths you need. Then add more paths to the selection while you complete the piece.