You can remove the index.php in the frontend URLs performing the following steps:
Go to the admin section of Magento.
In System -> Configuration -> Web -> Search Engines Optimizations, select YES.
In System -> Configuration -> Web -> Secure -> Use secure URLs in the frontend, select YES.
Edit the .htaccess and uncomment the line:
RewriteBase /magento/
