The complexity of developing and deploying context-aware pervasive-computing applications calls for distributed software infrastructures that unobtrusively support to contextual data collecting, information sharing and security privacy controlling. In this paper, we motivate a human-centric design for such an infrastructure to support context-aware applications. A novel peer to peer based semantic policy infrastructure for pervasive computing, PSPI, is presented. The core of PSPI is a scalable and self-organizing peer-to-peer overlay to dynamically support semantic policy management according to the context in the pervasive computing applications. And the semantic specification of policy and dynamic policy implementation are described. The experience and lessons learned when using PSPI to support Smart-conference scenario demonstrate that PSPI is effective to facilitate both the development and deployment of context-aware pervasive-computing applications.