Based on the balanced panel data of 30 provinces in China from 2000 to 2018, this paper use DEA-Malmquist index, panel instrumental variable method and panel threshold model to analyze the impact and mechanism of water-saving irrigation infrastructure on the growth of grain total factor productivity. The results show that: (1) water-saving irrigation infrastructure can significantly improve the total factor productivity of grain production through technical progress and technical efficiency improvement, the impact on the technical progress is greater than the technical efficiency improvement, and the increasing effect of pipe irrigation and sprinkler irrigation on total factor productivity is higher than that of micro irrigation. (2) From the perspective of regional and crop heterogeneity, the increasing effect of water-saving irrigation infrastructure on total factor productivity in main grain producing areas is greater than that in non main grain producing areas. Water-saving irrigation infrastructure can significantly improve the total factor productivity of wheat, corn and rice, in which wheat and corn are dominated by technical progress, while rice is dominated by technical efficiency improvement. (3) The impact of water-saving irrigation infrastructure on grain total factor productivity has a non-linear characteristic, and its positive impact increases with rural human capital, per capita grain production scale and per capita GDP, and decreases with the increase of water resources adequacy. Therefore, strengthen water-saving irrigation infrastructure according to local conditions, coordinate with the development of external economic environment, combine policy support with market mechanism, and improve the property rights and management mechanism are the key to develop water-saving irrigation infrastructure for grain production efficiently and sustainably.