Not knocking is quite rude and should be avoided.
As a courtesy, you should always knock or ask for permission to enter someone's closed office. It demonstrates you respect their privacy and that they can refuse your request, and it gives them control of the situation.
