We take decisions based on ground report, not by sitting in office: PM Modi