Does transferring money between accounts at different banks normally involve paying a fee?

Yes, usually it does. Moving money costs money because of the interbank networks at play here. Each party has a cost associated with moving money, so depending on where in the world you are, it could be free (subsidized) or might cost you a little bit.

There is no universal agreement to this. Within each country, the regulator assigns brackets for charges, known as schedule of charges, under which banks can charge/not-charge for services rendered. Money transfer between two different bank accounts is usually a charged item.