An experiment to investigate and quantify reliability in collection condition surveys was carried out with 33 professional conservators assessing 20 objects of various materials. Data were analysed using several reliability indices, particularly Krippendorff's alpha. The quantified reliability level (0.372 alpha) was halfway between levels considered acceptable in other fields and chance agreement. Responses demonstrated, in addition to individual differences, the main reasons for low reliability; ambiguity in survey form definitions, the broadness of the term ‘condition’, high levels of deterioration, and the professional roles of surveyors. Some existing explanations for disagreement, particularly institutional differences, were not found to have a significant effect on reliability, and object complexity and experience in conservation were found to have only a limited effect. Three factors ‐ surveyors, objects, and survey forms ‐ and the relationships between them, are reviewed in order to determine evidence-based recommendations for increasing reliability.