Bakers Arms dates back to the about 1672 to 1680 when it was two cottages. In the second half of the 1700s another attached dwelling was built to the east of the original pair. Much of the property is an old quarry which is particularly noticeable in the north end of the 2+ acre gardens.
In 1938 the western of the two original cottages was a ruin, the western wall being about 10ft high. The eastern one was only partly habitable. Half of the adjoining cottage (the 1700s newer addition) was used as a pub. This originally had its entrance on the road but by the time the property was listed (in 1953) the original front door hasd been blocked off
Some time after 1938 the whole was rebuilt, probably to extend its use as a pub, using ham stone. The property remained a pub until 1957 when it was converted into a private house. The main entrance to the pub in 1953 was in the rebuilt western wall but this is now blocked off.
At the rear of the main house is a sunny courtyard. At the east side of this is a row of 3 small cottages which have been rebuilt. Little is known of their history, except that one was the village bakery: the old bread ovens remain as a decorative feature. This has been converted and used as a holiday let - The Old Bakery.
The second cottage (Jasmine cottage) had been used as an office but has now been fully converted to a holiday let.
The third cottage has been made almost weatherproof, but the inside has not been touched, so it is a 'barn' at present.
At the north side of the central courtyard, the ruins of an old skittle alley provide an attractive feature as well as one path through to the garden at the rear.
To the north stretch the gardens nearly 300 metres back from the road, arranged in a series of rooms. The gardens now require a lot of attention, having been little maintained (other than cutting the hedges and mowing the grass) over the past 20 years. These gardens are fully accessible for walking and exploring. Access is via a lawned path or via the ruins of an old skittle alley. Beyond is a private lawn for visitors. Then an area with the river Parrett bordering on the west side. Half way down the garden is a bridge where the river crosses the garden and borders the property on the east side. Cross a bridge to an open area with a small orchard and a spring then at the north end the garden is a lawned, wooded area with an intermittent spring and some splendid trees.
The area north of the bridge was clearly once a quarry for inferior oolite stone (essentially Portland stone) from which is built the house. This old quarry is very large, including not only the north section of Baker's Arms garden but also the area to the west of the garden, property of The Granary. This quarry is clearly very old and very extensive: it is suspected that it is the quarry from which the stone for Mohun Castle was extracted. Mohun Castle was demolished around 1650, some 30 years before the original cottages were built and it seems very likely that the stone for these cottages, and many other house in the village, was robbed from Mohun castle - Baker's Arms contains an old oak door which appears to have come from the castle.