One thing that I often see when folks start deciding to go off-grid is that they don't really mean it. By that, I mean that they want to do an alternative grid for their own use, but not go truly off-grid. For that, one would need to return to pre-grid technology, food production, and storage techniques, all easily duplicated, and actually less expensive than trying to duplicate the grid on one's own property. Also, the non-grid efforts are immanently sustainable!
Think in terms of the three necessities of life: Shelter, water, food. Think also of the "3"s for each, i.e., a man can live 3 hours without shelter from the elements and from whatever comes one's way, protection wise (which includes protective clothing), 3 days without water, and 3 weeks without food (all assuming that he just lays there in a lump, not working hard!).
Then, start thinking in terms of, "How do I provide for each of these necessities of life in turn, so that they are SUSTAINABLE, preferably without much effort (read that labor hours) on my part?"
Obviously, first choice is a piece of land that can be productive and provide the necessary ingredients for life. That choice alone eliminates a lot of potential places one might live, for much of what is now populated is done so only because of the grid and a self-sustaining effort to live there without grid support would be a death sentence. One easy clue is, "Where did the earliest settlers to America build and thrive before they moved on to settle other less hospitable places?" Find those spots -- villages or rural areas that have been settled since the early 1800s, and get to one -- do what you have to do to support yourself being there, for there is the only true hope of survival long term if the grid goes down.
Once you find that land, then I would be looking to make sure that there is a water and food source on it. I say this before shelter because one can always live somewhere else during the building phase and a sustainable food and water resource MUST be in place before moving in and starting with shelter plans. Is there a creek that runs all year on the property? How could you manage that to provide both water resources and if directed through some piping, a source of cooling for a true root celler? Or, conversely, is there a spring on the property? Could you build a spring house over it to help manage your water supply and also provide cooling for food items? If you need a well, know what you are up against and plan on how you will get water out of the ground without grid support (and that includes using batteries, which WILL fail eventually). Collecting rain water? That too is good, but is there enough and can you collect it clean enough to use? Plan on planting every sort of perennial (returns on its own year after year without re-planting) food plant that you can grow in your land, in double the quantity that you expect you will need to provide for your family. Excess goes to feed animals and in lean years to keep you and yours alive. This includes nut trees, fruit trees, asparagus, berries of all sorts, rubarb, herbs, and a host of other plants that return year after year. There are great books on the subject.
Then, finally, start thinking shelter. I'd think something with thick walls, plus insulation. The cordwood houses are attractive for non-grid homesteading as they offer low cost to build, and might even stop bullets. If clad outside in metal, they would also be rather fireproof. I would also probably insulate under the steel siding. The idea is that the thick walls help to heat and cool the building. They could be 24" thick! When building, plan on keeping windows in the proper place, not just to let in light, but also to add solar heating during the winter. Think "thermal mass" in the center of the home with winter-angled windows directed to shine on that thermal mass. That could be a big X huge stone fireplace (preferably with a true stove insert in place of an open fire!) and with the chimney ducted throughout the mass to warm it, thus radiating heat throughout the home. Think about how you are going to manage getting water inside the home during bad weather without grid-connected plumbing and definitely think about bathroom needs. How would you or your wife react to porta-potty smells inside your home every day of life? Most people who are accustomed to grid living cannot deal with the rawness of true natrual things like bathroom waste, so plan accordingly. There is a reason that people were early adopters of indoor plumbing, but it takes some means to make it continue to work.
There are, of course, a million other things -- difficult to teach things -- that one discovers first hand by actually living off-grid. Some can be fatal, prayerfully not...