Nice examples. On reflection, I can change my advice slightly: a that modifier, like other noun modifiers, should be placed as close as possible to the noun it's modifying, but may not always be touching.
1. Not correct: that seems to be modifying 'river'.
2. Not correct. Again, the position of the 'that' modifier is too far from 'dog'. I should emphasize that GMAT will never test you on comma usage; i.e. commas alone will never decide wrong from right answers.
3. This one seems okay. Sure, the 'that' is not touching 'walk', but 'by a river' is just such a small modifier that we're not going to get confused.
4. Commas seem unnecessary here, but since we're not being tested on them there's no need to discuss.
5. Fine.
I'd caution you not to invest too much time in uncovering the many nuances of grammar (unless it's your hobby, of course). However, you've tapped into the logic behind modifiers. We often want to present lots of information in a single sentence; using modifiers as 'extra information' is often an efficient way to do this. However, we need to make sure the meaning is clear. For this reason, in constructing a sentence we generally look for a nice solution. Take example 3: Sage took his dog on a walk by the river that was very exhausting. You want to give information about the walk, about the river and what Sage did. Although I think your solution is acceptable, this might be a simpler one: Sage took his dog on a very exhausting walk by the river.
Similarly, with 1 and 2, you miss a possible solution:
Sage took his dog, a pure breed, on a walk by the river.