In type it belongs to the same family as the late Elizabethan and Jacobean prodigy houses, reduced to miniature proportions. Fortunately, more information has emerged about the whole of Swarkestone following research in the Harpur-Crewe archive by Dr Howard Colvin and Philip Heath, and this has since been added to by local historians.
A new house (28 hearths in 1662) was built at Swarkestone in the 1560s by Sir Richard Harpur, a lawyer who rose to eminence under the patronage of the Earl of Shrewsbury and by his own marriage to Jane Findern, an heiress. In 1622 the estate was inherited by his son's ten year old grandson, John Harpur. This John formally took up his inheritance in 1630, and was knighted in the same year. At the same time a marriage was arranged for him with Catherine Howard, grand-daughter of the Earl of Suffolk, and step-daughter of William Cavendish. Accounts for 1631-2 record the expenses of `gloves, gauntlets and liveries' at their wedding, and for preparing Swarkestone Hall for their residence.
The arms of this young couple appear on the shields on the front of the Pavilion and it may have been built to celebrate the marriage. Entered in the same set of accounts for 1631-2 is a payment of £111 12s 4d to Richard Shepperd the mason for `New Buildynge', together with a sum for `Boardes' for the `Bowle Alley house'. There seems every reason to identify these entries with the Pavilion and to date it therefore to 1631-2. The Shepperds were well-known masons in the area at the time. Whether Richard Shepperd was also its designer is uncertain: he also built the `Gothic Survival' church at Staunton Harald but little else is known of him, although he describes himself in his will as ‘Architecter’. The Harpur accounts also name a Mr. Woolridge as the Bowl Alley Surveyor, and Mark Girouard has suggested that the Pavilion could be attributed on stylistic grounds to John Smythson. This is credible, given Smythson's service with William Cavendish and Sir John Harpur's marriage with Cavendish's step-daughter, the closest of several links between the two families.
The accounts also help to disperse the bloodthirsty mythology that has grown up around the Pavilion, and establish it as belonging not to the activities of the park but to those of the garden as a pavilion or banqueting house overlooking a bowling green, no doubt as part of a formal garden layout. It is worth noting that in estate papers of the last century it was commonly referred to in these terms, once even as the Bowling Green Pavilion. It may well have doubled as a banqueting house to which small groups could retire to enjoy the ‘banquet’ course of fine wines and sweetmeats, play cards or just enjoy the view of their host’s estate.
The interpretation of the Pavilion's surroundings is confused by the later history of the site. Sir John's branch of the family died with him in 1679. The estate passed to the Harpurs of Calke and the house was dismantled in 1746-8. Surviving high walls containing windows and doors and even a fireplace may reflect its outline at least in part. While an element of picturesque management has been involved in this structure's preservation, as in that of the Pavilion, a symmetry appears to exist between the two, in axis with the old door in the wall opposite. This fits too with a formal layout of gravel paths recorded during ploughing in 1988 in a paddock to the west of the walls; and with a rectangular pond, possibly a canal, which formerly lay to their east. However, in the early 19th century, Swarkestone was the scene of large-scale breeding and sale of livestock. It is possible that some of the walled enclosures, even that in front of the Pavilion, relate equally to this activity.
Although the cupola roofs were carefully repaired in 1844 after one was struck by lightning, the Pavilion fell into dereliction. It acquired some notoriety in 1968 when it was used by the Rolling Stones to promote their album Beggar’s Banquet, and another image from the same photoshoot was used on the back of a later compilation album, Hot Rocks 1964-1971.