Silly Billy by Anthony Browne

From the back of the book:
Billy worries about everything, especially when he spends the night away from home. On a visit to Grandma's, Billy is so worried that he can't sleep. But Grandma has just the thing for a boy like Billy - tiny, colourful worry dolls. He puts the dolls under his pillow and lets them do the worrying for him.