Photometric stereo is well-established to estimate surface orientations for Lambertian surfaces, while it remains challenging for surfaces with non-Lambertian reflectance. This paper presents a new framework of photometric stereo (PS) using the dichromatic reflectance theory with color images in separating diffuse and specular components in order to better estimate local surface orientations for non-Lambertian surfaces. These local surface orientations can be further integrated via surface normal integration (SNI) to reconstruct the surface in 3D. This framework opens a door to tackle specularities separately as meaningful signals rather than outliers as most of the previously proposed techniques did. Experiments on synthetic and real images demonstrated the state-of-the-art performance.