Recognized as an effective low impact development (LID) practice, rain gardens have been widely advocated to be built with urban landscaping for stormwater runoff reduction through the retention and infiltration processes; but the field performance and regional effect of rain gardens have not been thoroughly investigated. In this paper, we presented a four-year monitoring study on the performance of a rain garden on stormwater retention; hydrological models were proposed to predict the potential effect of rain gardens on runoff reduction under different storms and the future urban development scenarios. The experimental rain garden was constructed in a sub-humid loess region in Xi’an, China; it has a contributing area ratio of 20:1 and depth of 15 cm. During the study period, we observed 28 large storm events, but only 5 of them caused overflow from the rain garden. The flow reduction rate for the overflow events ranged from 77 to 94 %. The runoff coefficient from the contributing area (RC) was reduced to less than 0.02 on annual basis, and 0.008 over the four years average. Field observations also showed that infiltration rate remained stable during the operation period. The predictions based on the future landuse and storm variability of the study area showed that by converting a small fraction of the city land area into rain gardens, the negative hydrological effect from expansion of impervious area can be reduced significantly. The challenge, however, lies in how to plan and build rain gardens as an integral part of the urban landscape.