Because of overcultivation in the semiarid Loess Plateau during the past half century, soil erosion has become a serious problem in maintaining a sustainable ecosystem. To find an effective way to overcome soil erosion, we investigated the changes of soil pH, soil bulk density, soil aggregate distribution and stability, and soil organic carbon, total nitrogen, total phosphorus, and available phosphorus associated with soil aggregates when cropland was converted to grassland on the semiarid Loess Plateau of China. In 2003, a typical cropland was planted with (i) perennial legume species alfalfa (Medicago sativa Linn.) at a density of 22.5 kg ha−1, (ii) the biennial legume species sweetclover (Melilotus officianalis Linn.) at a density of 11.3 kg ha−1, and (iii) natural regeneration. By 2008, the soil pH and soil bulk density significantly decreased in all treatments. The number of soil aggregates in the 2 to 1-mm and 1 to 0.5-mm categories and the stability of aggregates significantly increased (P < 0.05) in all treatments. In different size categories of soil aggregates, soil organic carbon, and total nitrogen increased, but available phosphorus significantly decreased in all treatments. Soil total phosphorus was not significantly different between treatments. Overall, soil physical properties significantly improved after the cropland was converted to grassland in all treatments. The natural regeneration treatment showed the best improvement.