Section 22 of the Companies Act prohibits registration of a company name that is identical or almost identical to another registered (or reserved) company name. This is applied quite narrowly – e.g. if a word in the middle of the name is different to existing names, the Companies Office will usually accept it. For further guidance se https://companies-register.companiesoffice.govt.nz/help-centre/starting-a-company/how-to-reserve-a-company-name/company-names-that-cant-be-reserved/. If you think the other company’s name identical or almost identical to yours, you could contact the Companies Office and ask them to review the name on that basis.
From a trade mark perspective, trade marks are usually restricted to specified goods and services – it is generally acceptable to use the same trade mark on different goods and services, although there are exceptions for extremely well-known brands e.g. Coke.
You may have to either live with it, or consider changing your own name.
If you need further assistance you could talk to a trademark specialist. Potter IP and Hudson Gavin Martin have people that could help.