An intense standardization process is favouring the convergence of grids and Service Oriented Architectures (SOAs). One of the benefits of such technological convergence is that grid resources and applications can be virtualized by services and offered through the SOA paradigm. In the broad and interoperable scenarios enabled by the SOA, involving the participation of several grid infrastructures across many administrative domains, service discovery can be a serious issue. In this paper we present a P2P-based infrastructure that leverages semantic technologies to support a scalable and accurate service discovery process. The key concept of the presented idea is the creation of an overlay network organized in several semantic groups of peers, each specialized in answering queries pertaining to specific applicative domains. Groups are formed by clustering together peers offering services that are semantically related. The architecture details of the proposed solution are presented. A system prototype has also been implemented and validated through a case study deployed on the PlanetLab testbed.