As I (mis)understand it, you will be happier if you apply a difference in (outer) corner radius that is half the difference in rectangle size.
So with the larger being 12" x 18" and the smaller being 11" x 17", the difference is 1", so the difference in radius should be 0.5"; you can use 1" for the larger and 0.5" for the smaller (or 0.75" and 0.25" or similar; if you go down to 0.5" and 0" with a centre aligned stroke for the smaller you need to use Round Join).
I gave your outer shape a stroke (on the inside) of 30 pt to show what the rounding would become on the inside, a very tiny one for the red one 0,005 in (without the round corners effect).