An algorithm and a computer program that implements the algorithm that performs recursive hierarchical segmentation (RHSEG) of data have been developed. While the current implementation is for two-dimensional data having spatial characteristics (e.g., image, spectral, or spectral-image data), the generalized algorithm also applies to three-dimensional or higher dimensional data and also to data with no spatial characteristics. The algorithm and software are modified versions of a prior RHSEG algorithm and software, the outputs of which often contain processing window artifacts including, for example, spurious segmentation-image regions along the boundaries of processing-window edges. The modification consists of the addition of an efficient subroutine through which pairs of regions are identified that may contain pixels that are actually more similar to other regions in the pair. Once these pairs of regions are identified, pixels in one region that are more similar to pixels in the other region are reassigned to the other region. The subroutine is computationally efficient because it focuses only on those regions that could potentially contribute to the processing-window artifacts. In addition, any adverse effect of the subroutine on the computational efficiency of the algorithm is minimized by executing the subroutine at a point in the algorithm such that switching of pixels between regions that are subsequently merged is avoided.