This paper aims at presenting the concept of fruit quality management, a system which determines the quality of fruit by its color, size and weight. Sorting tons of fruits manually is a time consuming, costly, and an inaccurate process. This sorting system is developed in order to increase the quality of food products made from fruits. The sorting process depends on capturing the image of the fruit and analyzing this image using image processing techniques to discard defected fruits. The main emphasis is to do the quality check with a short span of time so that maximum number of fruits can be scrutinized for quality in minimum amount of time. The absolute reference point is the way to perceives and interpret the quality of fruit. This system performs the sorting using MATLAB software and gives some advantages over traditional practices.