Well, I still believe, it’s possible using GREP... with some assuming, and in three steps. Unfortunately, GREP has some limitation/feature/bug we need to 'hack’. Also it’s a good idea to create special Character style for your numbs (if you haven’t the one already).
So, here’s our initial text:

We’ll need two additional glyphs that are NOT used anywhere in the text. I’ll use:
broken bar: ¦
dagger: †
Find what:
[,\d]+
Change to:
¦$0†
Find format: color: Orange
Change format:
color: [Black] + position: normal

Here goes the main trick:
Find what:
(¦[,\d]+†)([[:punct:]])
Change to:
$2$1
Don’t forget to trash Find and Change formats before running this!

And finalizing:
Find what:
(¦)([,\d]+)(†)
Change to:
$2
Change format:
Character Style: my orange numbs
