The wealthy Carey family is left destitute when the father dies suddenly. They are selling off their possessions and have nowhere they can afford to live.
Teen daughter Nancy (Hayley Mills) writes a letter to the owner of a beautiful yellow house, from a prior family vacation. To her surprise, the family gets a letter back inviting them to live there. The house is owned by Mr Hamilton, who has been gone for several years. Mr Popham (Burl Ives) is watching the house for Mr Hamilton. Popham ends up letting them live there rent free, and gives them free rein to paint, paper and do anything they want to the house.
Mrs Carey is unwilling to take advantage living rent free, so Popham tells them that Mr Hamilton just wants them to hold a party in his mother's honor to put up her portrait. Popham grabs a random hideous painting of an old lady to be "Mrs Hamilton".
The day of the party, Mr Hamilton returns. Hamilton is obviously upset about things. He goes to the party incognito with Popham. Hamilton is angry with Popham because of the hideous painting, and because the Carey's repeat various absurd lies about Hamilton that Popham had told them.
While everyone expected Hamilton to be old, he is actually a very young man who inherited the house.
At the party, Hamilton seems to be falling for Nancy. Hamilton reveals his identity to Nancy, and asks her to dance.
The movie leaves you hanging. It is not clear what Hamilton will do, if he will let them remain in the house.
Thanks Joseph C!