It's an idiom. A regular (or nautical) mile has a specific distance. A country mile doesn't, it just means "a very long way".
I'd assume it originated back when the size of miles was relative to the area, rather than a set length everywhere.
I actually didn’t know about it originally being a relative length. Learn something new every day.