The actual process of installing an SSL certificate will vary by server. This is not a guide for how to install an SSL, period. This is for all the other stuff that can slowly break when you install an SSL certificate — stuff you might not notice immediately.
For SSL purchasing, I recommend Namecheap. I’ve used them as a domain registrar for a while now, and I’m very happy with them. They’re the only cheap SSL certificate ($10/year) I’ve found that seems reputable (i.e., other SSL certificates I’ve found for similar and lower prices have seemed to come from disreputable sites).
Insecure Content
Calling to absolute URLs in your website — e.g., literally linking to the file http://yourwebsite.com/website/file.jpg — can cause “insecure content” errors on some pages. That is — the website displays with the lock in the URL bar, but there’s a small warning icon, too.
You can usually fix this easily with WordPress HTTPs if you’re on WordPress, but on other site types, you might have to do a little bit more work. It’s important to check all pages on your website for insecure content, as it can happen on virtually any page. There are some tools to help you do this, such as Why No Padlock, so it’s not quite as tedious as it sounds.
301 Redirects
If you are migrating an existing site, it’s naturally important to make sure you’re 301 redirecting everything. In WordPress, you can do this with the above-mentioned plugin, WordPress HTTPs and by setting your WordPress URL to use https://. Otherwise, you’ll have to use a different plugin or do an .htaccess rewrite.
Google Webmaster Tools, Google Analytics
For both of these tools to function properly, you have change a few things around.
Google Analytics
There are two places in Google Analytics where you can set https:// — both in the View settings and the Property settings. Both should be set to https:// for all aspects of Analytics to function correctly. If you forget to set it in one or the other, most things will work — but the In-Page Analytics, at the least, will break.
Google Webmaster Tools
In Google Webmaster Tools, there’s actually no place to set https:// and http://. You might have been confused by this if you want looking for it. In the instance you already had http:// in your GWT account, you should add your https:// site as a secondary site. You will then (assuming your 301 redirects are in place and functional!) see all necessary data under the https:// version of your site.
Google AdWords and Other Campaigns
If you’re running Google AdWords campaigns, switch from http:// to https://. Your advertisements will still work if you don’t. However, http:// when you should be using https:// results in a noticeable delay as the browser redirects you to the secure site.
This delay, however slight it is, can hurt your conversions from advertisements. If everyone is leaving your site before it loads after clicking your ads because it’s taking too long to load, you’re just chucking money to the wind.