Eventual deterioration and destruction would be the fate, unless someone was taking care of the upkeep, and it sounds like the owner was too far away for that! By removing and caring for these old artifacts, we are actually preserving a part of history for the future.
In cases where the premises are being cared for, I prefer that they are left alone for all the world to enjoy, but in 98% of the cases, this is not happening.
Don't feel bad about their removal, but take care of what you have obtained, and try to put it on display for others to enjoy, even if semi privately.
Don't be surprised when other gas pumps suddenly appear and want to be restored and cared for too! It happens every time a new collector finds a pump or two! Within a short amount of time, there are several in the yard, and no one ever seems to know why! LOL
Enjoy your find and start building that display!