We are sorry to hear you are having trouble using Adobe XD. Would you mind sharing the file with me over a private message so that we can escalate the issue to the team? It would be really helpful if you can share the OS & XD version of your machine.
I am also facing this problem Serkan. Does your design system include a lot of component states? Using states has caused my file size to increase considerably, especially in cases where I am using one type of component that has many different states throughout my project. I tried deleting the states from a common component as a test to see what happens and my file size went from over 100mb to 50mb. For me, this renders states unusable in most cases (not to mention all of the other bugs and issues with them).