All great advice above. The key to hiring well is being clear on what you need, clear on what tradeoffs in a candidate you are willing to make, and then having a structured process to assess candidates.
With that in mind I have just a few things to add that may be helpful:
1) When drawing up a job description of what you need from the role be super clear on what must be done. To often I see a very long list in a job description only to find that only 3-4 things are absolutely required. Nail the absolutely required so you can assess well.
2) When thinking about who to hire to do what you listed in #1 above list the attributes of what you need in a person in a few buckets: technical skills you don’t want to teach, attributes of the person you don’t want to teach, technical skills you are willing to teach. I like this process because there is rarely a perfect candidate and you need to be clear about what you absolutely need in a successful hire. It also forces you to be clear about what you need in a candidate above and beyond just being able to do the job.
3) When you interview you should do a few things: 1) Give each interviewer a clear area of focus. You could use the exercise in #2 above to give people areas of focus; 2) Ask the candidate to give you examples of their work, e.g Tell me about a time you had to work under a tight deadline? The muse.com has a good starting list of behavioral interview questions if you need some.
4) Get people who have interviewed together to discuss their impressions based on their interviews and come to a conclusion.
5) Beware of people who are great technically but don’t fit your culture or have the wrong mindset for your business. They are the hardest hires to say no to.
Finally, all hiring has risk inherent with it. The suggestion to do KPI’s 6-10 weeks in is great as its the best way to make sure you know early on you made the right decision.
Hope that helps.