The North American pawpaw (Asimina triloba [L.] Dunal) fruit is the largest edible fruit native to the United States. Over the years, the fruit has remained underutilized with limited published data on the properties of the fruit. In this study, the color of the skin and the pulp of the fruit, as well as the textural properties of unripe and ripe fruit harvested from wild trees were evaluated. The results show statistically significant differences in the textural properties of the unripe and ripe fruits. The ripe fruits had a hardness of 2.2 ± 0.5 kg-force, similar to the hardness of green ripe mangoes but harder than green ripe bananas, and the unripe fruits had a hardness of 68.2 ± 10.9 kg-force. Also, there were strong negative correlations between the fruit skin color a* values and the hardness (r = −0.87), chewiness (r = −0.86), and cohesiveness ratio (r = −0.73), and a strongly positive correlation with total soluble solids (r = 0.90). The skin hue angles had strong positive correlations with hardness (r = 0.86), cohesiveness ratio (r = 0.74) and chewiness (r = 0.86), and a strongly negative correlation with total soluble solids (r = −0.91). The fruit skin color a* values (degree of greenness), skin hue angle and total soluble solids content can be used as non-invasive indicators of pawpaw ripeness. The correlations established in this study provide new insights; farmers could use commercially available portable color and near-infrared Brix meters to determine the maturity of pawpaw fruits. These findings will help farmers and processors to harvest and process pawpaw fruits at the right time to minimize postharvest losses.