Basically the visa is a label placed in your passport that tells the immigration officer at the border that you are allowed to enter New Zealand for the purpose outlined in the visa. The permit is stamped in your passport by the immigration office on arrival. In the case of a residence permit, it gives you the right to live and work in New Zealand. Whilst the permit is valid, you are legally in New Zealand.
All temporary entry permits like work and visitor permits will have an expiry date. So, in summary, a New Zealand visa indicates your New Zealand immigration status when you are outside of New Zealand and a permit indicates your immigration status when you are inside New Zealand.