GrabCut in OpenCV
The function used here is cv2.grabCut()
Syntax: cv2.grabCut(image, mask, rectangle, backgroundModel, foregroundModel, iterationCount[, mode])
Parameters:
- image: Input 8-bit 3-channel image.
- mask: Input/output 8-bit single-channel mask. The mask is initialized by the function when mode is set to GC_INIT_WITH_RECT. Its elements may have one of following values:
- GC_BGD defines an obvious background pixels.
- GC_FGD defines an obvious foreground (object) pixel.
- GC_PR_BGD defines a possible background pixel.
- GC_PR_FGD defines a possible foreground pixel.
- rectangle: It is the region of interest containing a segmented object. The pixels outside of the ROI are marked as obvious background. The parameter is only used when mode==GC_INIT_WITH_RECT.
- backgroundModel: Temporary array for the background model.
- foregroundModel: Temporary array for the foreground model.
- iterationCount: Number of iterations the algorithm should make before returning the result. Note that the result can be refined with further calls with mode==GC_INIT_WITH_MASK or mode==GC_EVAL.
- mode: It defines the Operation mode. It can be one of the following:
- GC_INIT_WITH_RECT: The function initializes the state and the mask using the provided rectangle. After that it runs iterCount iterations of the algorithm.
- GC_INIT_WITH_MASK: The function initializes the state using the provided mask. Note that GC_INIT_WITH_RECT and GC_INIT_WITH_MASK can be combined. Then, all the pixels outside of the ROI are automatically initialized with GC_BGD.
- GC_EVAL: The value means that the algorithm should just resume.
Python Implementation: Foreground Segmentation and Extraction
Image used in this tutorial: link
The image is a scene from the video game Dragon Ball: Raging Blast 2 consisting foreground characters, Goku and Vegeta. Let’s remove them from the background.
Python3
# Python program to illustrate foreground extraction using GrabCut algorithm# organize importsimport numpy as npimport cv2from matplotlib import pyplot as plt # path to input image specified and # image is loaded with imread commandimage = cv2.imread('image.jpg') # create a simple mask image similar# to the loaded image, with the # shape and return typemask = np.zeros(image.shape[:2], np.uint8) # specify the background and foreground model# using numpy the array is constructed of 1 row# and 65 columns, and all array elements are 0# Data type for the array is np.float64 (default)backgroundModel = np.zeros((1, 65), np.float64)foregroundModel = np.zeros((1, 65), np.float64) # define the Region of Interest (ROI)# as the coordinates of the rectangle# where the values are entered as# (startingPoint_x, startingPoint_y, width, height)# these coordinates are according to the input image# it may vary for different imagesrectangle = (20, 100, 150, 150) # apply the grabcut algorithm with appropriate# values as parameters, number of iterations = 3 # cv2.GC_INIT_WITH_RECT is used because# of the rectangle mode is used cv2.grabCut(image, mask, rectangle, backgroundModel, foregroundModel, 3, cv2.GC_INIT_WITH_RECT) # In the new mask image, pixels will # be marked with four flags # four flags denote the background / foreground # mask is changed, all the 0 and 2 pixels # are converted to the background# mask is changed, all the 1 and 3 pixels# are now the part of the foreground# the return type is also mentioned,# this gives us the final maskmask2 = np.where((mask == 2)|(mask == 0), 0, 1).astype('uint8') # The final mask is multiplied with # the input image to give the segmented image.image_segmented = image * mask2[:, :, np.newaxis] # output segmented image with colorbarplt.subplot(1, 2, 1)plt.title('Original Image')plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))plt.axis('off')# Display the segmented imageplt.subplot(1, 2, 2)plt.title('Segmented Image')plt.imshow(cv2.cvtColor(image_segmented, cv2.COLOR_BGR2RGB))plt.axis('off')plt.show() |
Output:
Grabcut Algorithm
The GrabCut algorithm has successfully segmented the foreground characters, Goku and Vegeta, from the background. Here are some specific observations:
- Accurate segmentation of characters: The GrabCut algorithm has effectively separated Goku and Vegeta from the background with minimal errors. The edges of the characters are mostly smooth and well-defined.
- Partial segmentation of some objects: The segmentation is less accurate for objects with complex textures or transparencies, such as Goku’s hair and Vegeta’s aura. These areas appear partially segmented or blended with the background.
- Potential for further refinement: While the segmentation is good overall, some minor adjustments could improve accuracy, particularly around complex areas like hair and auras.
Why Grabcut is useful but lacks Perfection?
GrabCut is a valuable image segmentation tool, offering efficient foreground extraction by involving user-provided information. It relies on a user-defined bounding box to distinguish between foreground and background.
However, its reliance on manual input makes it susceptible to inaccuracies, especially in cases where the object boundaries are not well-defined or when there are complex structures.
GrabCut may struggle with subtle and intricate details, leading to imperfect segmentations. While it serves as a practical solution for many applications, its dependency on user guidance and potential challenges in handling complex scenarios contribute to its limitation and lack of perfection in certain cases.
Python | Foreground Extraction in an Image using Grabcut Algorithm
In this article we’ll discuss an efficient method of foreground extraction from the background in an image. The idea here is to find the foreground, and remove the background.
Contact Us