If you have a problem to login per ssh and key in the server you must check the auth.log file to see more detail.
Authentication refused: bad ownership or modes for directory /home/user/.ssh
When you see this message you have a problem with the permission of .ssh/authorized_keys
chmod 600 .ssh/authorized_keys chmod 700 .ssh/
to your sshd_config file you can also fix the problem, but thats not a good idea. Fixing the permission is the best way.