Have you checked out the information at https://www.business.govt.nz/starting-a-business/
There are some really helpful notes and also a 10 point checklist on how to start a business.
In New Zealand, it’s really really easy to register a company, which has the advantage of keeping your business affairs separate from your personal affairs in a legal sense. However, if what you want to do can easily be done as a sole practitioner, it’s probably even easier to just start as a self-employed person, and see how it goes.
Either way, you still need to keep your financials in order, making sure you keep good records of all income and outgoing costs, so you can pay the right taxes, and so on. There’s a threshold of income over which you must register for GST. The IRD website is also very helpful about what to do.
Hope that helps, and good luck. Starting a business is one of the most exciting, challenging and rewarding things you can do.
All the best.