Kindly use a thermometer first to know whether the "hotness" is truly something to worry about or not. If the temperature is normal, that is between 36.5 - 37.4, there is no need to worry at all

Normal body temperature is between 36.5°c - 37.5°c with a digital thermometer irrespective of what you feel with your hands. But if above, it's fever. You can tepid sponge and give paracetamol and if it persists, take baby to a paediatrician.
The brain does a lot of work so their is a lot of blood flow to the head which can make it a little warm to touch.
However, that does not mean fever or sickness. As long as the temperature in your thermometer reads normal, their is really no need to worry.

Check the temperature with a digital thermometer first to really know if it is high. Temperature of 36.5 to 37.5 is normal. Anything higher than that give
paracetamol and see a doctor.

Kindly check the temperature with a
thermometer to be sure its hot. Temperature
between 36.5 - 37.5 is normal. Anything
above give paracetamol and take the baby
to hospital if it persists.

The most accurate way is to use a thermometer. The average temperature ranges between 36.5 -37.5degrees. Once their temperature is with this range nothing to worry about. Any temperature above 37.6 may indicate fever Please see a doctor for proper evaluation and treatment. Manage the temperature with tepid sponging and paracetamol.