Does every central bank maintain foreign exchange reserves?

Yes. Pretty much.

Why? You ask.

If a country wants to trade with another country (and I cannot think of a single country that does not trade with someone else), you need a common currency to trade usually (if you cannot agree for a swap).

Let’s say you are CountryX and your currency is called Xollars. Now you want to buy some wheat and you can only export coconuts. CountryY will buy coconuts from you and give you Yollars for it, but you have no need for Yollars. You want to buy wheat from say Egypt and Egypt prefers the payment in US Dollars (because Egypt was to buy something else).

So how would you trade?

Option A: You can ask CountryY to give you Yollars, which are readily traded on the international Foreign Exchange market and you sell your Yollars for Dollars.

Option B: You can ask CountryY to pay you in Dollars directly.

Option C: You sell your Xollars and trade them with Yollars, sell your Yollars to buy Dollars and then you can buy wheat.

When you have a foreign exchange balance that is not yours, you pretty much have a foreign exchange reserve. As the world trades, almost everyone is holding on to a wee-bit to a whole lot of foreign exchange reserve of other countries because we are a trading world.