Can You Switch between Regular Brushes and the Smudge Brush?

Not that I know of, (but could be proved wrong). Brushes and mixer brushes are in the same set, while the smudge tool isn't... BUT
There is no Smudge tool shortcut by default and could could easily make one going through the keyboard shortcuts (Edit>keyboard shortcuts)

The tools in the same set have the shortcut O, so I assigned this same one, but you could assign anyother one (that is not already taken)

Another solution is go to the tools bar manager

and drag the brush in the same set as the smudge tool, and AFTER, go and change the Brush shortcut to be the same as the smudge tool (or conversely drag and drop the Smudge tool in the brush set)
That way all you have to do is SHIFT+O (or SHIFT+B if you took the latter solution) to switch between tools.

To edit the toolbar, go and click on the 3 little dots at the bottom

Let me know if it helps!