If mass conversion starts happening, it could be matter of concern: Rajnath Singh