A metric space (M,d) is said to have the small ball property (sbp) if for every ε₀ > 0 it is possible to write M as the union of a sequence (B(xₙ,rₙ)) of closed balls such that the rₙ are smaller than ε₀ and lim rₙ = 0. We study permanence properties and examples of sbp. The main results of this paper are the following: 1. Bounded convex closed sets in Banach spaces have sbp only if they are compact. 2. Precisely the finite-dimensional Banach spaces have sbp. (More generally: a complete metric group has sbp iff it is separable and locally compact.) 3. Let B be a boundary in the bidual of an infinite-dimensional Banach space. Then B does not have sbp. In particular the set of extreme points in the unit ball of an infinite-dimensional reflexive Banach space fails to have sbp.