Yes, and for babies under 6 months a baby sun hat matters more than sunscreen, since sunscreen is not generally recommended for very young babies and shade plus physical coverage becomes the main line of defence. A wide-brimmed sun hat protects the face, neck and ears, which are some of the most exposed and sensitive areas.
Health guidance for this age group generally recommends keeping babies out of direct sun altogether where possible, with a sun hat and light clothing as backup for the times you are outside, such as walks in the pram or short trips to the car.
A soft, breathable baby hats fabric matters here too, since younger babies cannot regulate their temperature as well as older children, so a hat that is too heavy or non-breathable can contribute to overheating just as much as no hat at all in the wrong conditions.
Even on overcast days, UV exposure in Australia can be significant, so a sun hat is worth keeping in the pram bag as a default for outings under 6 months, rather than something you only reach for on obviously sunny days.