I think that they all should pay any property tax, but that is it. When you have a small town and there are 50 storefront churches that 25 people each now go to, which are run by one guy (it is for all intents and purposes his business), a stable local tax base is gone.
I also think all non-profits be required to have an open and available annual accounting statement showing income and where it went (and many do, but too many do not) as well as an asset and debt financial statement if they want the government's involvement of being tax exempt in any way. Let people see it and then they can decide if the non-profit is worthy in their eyes to donate to. These ministries or other groups with huge fund raising budgets or land holdings or salaries need to show in my opinion what they are doing with donations for people to see where it is going. If they want that information private, then fine, it should no longer be tax-exempt, and allowable expenses should be severely limited.