They both involve actively doing something.
The difference is you could just be walking around and that's physical activity. Exercise is actually going and working out with that being your intention.
Hope this helps :)
