Any well-designed long-term monitoring program should include definitions of what is "in" and "out" in a plot. For a good discussion of density, see Chapter 8 Section F in Elzinga et al's Measuring and Monitoring Plant Populations, particularly Figure 8.3 for discussions of boundaries.
http://www.blm.gov/nstc/library/pdf/MeasAndMon.pdf The protocol you are using is definitely unusual in counting stems that are rooted out of the plot. If this program is just starting out, by all means improve the protocol with definitions of stems and how to treat boundaries. However, it sounded to me like there is already a reasonably long history of data collection. If that is the case, I would attempt to analyze the data as is and keep the methods the same, as the value of a long-term dataset may outweigh an improved technique that would be unable to incorporate previous data. If the previous data are unusable because of the slop, then it is worth attempting to correct the problem. If you can use the old data but still want to improve the methods, one way to transition is to do both methods side-by-side for several years to get a "correction factor" that can be applied to previous years of data. As examples, the U.S. National Park Service Inventory and Monitoring program has many long-term monitoring protocols available that could be useful. https://irma.nps.gov/App/ProtocolTracking Try searching under Biological Integrity/Forest/Woodland Communities for examples.
