Sometimes that is the case. We generally find affordable lodging or transportation then look at other things to see if we can make it work.
The Canada trip this year was just because I wanted to go. It was not cheap. I think I spent more than my pension payment for the month just in gasoline...