Waller's Oak was built in 1909 as a parsonage for the Curate of Coleshill. The house was built by the Elizabeth Bent Charity, of which the Rector of Amersham, the Rev. Briggs, was the leading trustee. It was designed by the London architect, George Ernest Nield, and built by George Darlington, of Amersham, at a cost of £929.1 It was named after the ancient oak tree in the garden under which the poet Edmund Waller is said to have composed his verses.
In 1919, the house was sold to the Samaritans Free Hospital for Women, Marylebone Road and became a convalescent home. Coleshill had a reputation of healthiness and the patients were said to improve with much rapidity after a short stay in the village.
In the 1950s, it was the home of the novelist and biographer, Lawrence Meynell and his wife Shirley Darbyshire, who was also an author.