This is very common with OpenStreetMap PDFs — they can contain tens of thousands of tiny paths, and the Layers panel tries to thumbnail/render all of them at once, which is why it goes black and then freezes.
Things that help in practice:
1) Before opening the Layers panel, switch to Outline view (View > Outline, Ctrl/Cmd+Y) and turn off GPU Preview if it is on (View > Preview on CPU). That takes a lot of load off while you reorganise.
2) In the Layers panel flyout > Panel Options, set Row Size to Small and untick 'Show Layer Thumbnails' (and thumbnails for layers/groups). This alone often stops the black panel.
3) Save a copy as a native .ai file straight away and work from that, not the PDF.
4) Trim the file down before detailed editing: zoom to only the map area you actually need, delete everything outside it, use Select > Same > Appearance to remove unwanted road/label groups in bulk, and flatten unnecessary sublayers (Object > Flatten Artwork / release clipping masks where safe).
5) If it still stalls, export a smaller map area from OpenStreetMap (it is usually the size/complexity, not your install, that is the problem).
How much RAM does your machine have, and roughly how large is the PDF? With 16GB or less, splitting the map into tiles/areas is usually the realistic fix.