Fe3O4@SiO2 magnetic nanoparticles modified by grafting poly(1-vinylimidazole) oligomer (FSPV) was fabricated as a novel adsorbent to remove Hg(II) from water. Fourier transform infra-red spectroscopy confirmed the successful grafting of oligomer, and thermogravimetric analysis showed FSPV had a high grafting yield with organic content of 22.8%. Transmission electron microscopy image displayed that FSPV particles were polymer-coated spheres with size of 10–20 nm. With saturation magnetization of 44.7 emu/g, FSPV particles could be easily separated from water with a simple magnetic process in 5 min. The Hg(II) adsorption capacity of FSPV was found to be 346 mg/g at pH 7 and 25 °C in 10 mM NaCl. Moreover, the removal of Hg(II) by FSPV was not obviously affected by solution pH (from 4 to 10) or humic acid (up to 8 mg/L as TOC). The presence of seven common ions including Na+, K+, Ca2+, Mg2+, Cl−, NO3−, and SO42− (up to 100 mM ionic strength) slightly increased the adsorption of Hg(II) by FSPV. X-ray photoelectron spectroscopy analysis revealed that the N atom of the imidazole ring was responsible for the bonding with Hg(II), whereas the bonding of Hg with N did not result in cleavage of Hg–Cl bond in HgCl2 and HgClOH. The regeneration of Hg(II)-loaded FSPV could be achieved with 0.5 M HCl rapidly in 10 min, and the removal of Hg(II) maintained above 94% in five consecutive adsorption–desorption cycles. Therefore, FSPV could serve as a promising adsorbent for Hg(II) removal from water.