The church of St Mary's is situated on the high ground leading to the castle headland overlooking the south bay of Scarborough and has commanding views of Scarborough and the coast towards Filey and Flamborough Head.
The church of St Mary belonged to the Abbey of Citeaux, but passed to Bridlington with the confiscation of the property of the Alien Houses. The twelfth century church was probably an aisle-less building and a much larger new church was begun around it 1180. The West front, formerly with two towers, is the earliest art of this structure and was followed by the nave arcades of which the arches sit irregularly upon cylindrical piers. This may mean that the bays and piers were inserted individually in the walls of the earlier nave. The western part of the South arcade has a thinner wall and a different types of pier rather later date than the rest.
The surviving south transept was built in the second quarter of the fourteenth century and late in the same century the barrel-vaulted chapels were added to the South aisle and a second aisle was added on the North. The aisled chancel was rebuilt about the middle of the fifteenth century.