Advanced Smart Crop — Intelligent, Automatic Image Cropping
We have made significant improvements to our Smart Crop algorithm making automatic image crops even better. With improved face detection, feature detection, and different crop modes, the new algorithm provides higher cropping accuracy for both, facial and non-facial images. We have updated the text and examples to demonstrate this improvement.
Each image is different. Each device is different. Therefore, each image needs to be treated differently to build a perfect responsive experience across devices. Smart Crop is a feature that intelligently looks at the content of an image and automatically focuses on the most important part of an image for cropping.
What is Smart Crop?
When we resize images for different devices or placeholders, and the aspect ratio of the requested size does not match that of the original image, some cropping is bound to take place on the sides of the image. That is when we do not want to stretch or skew the image forcefully.
This can be seen from the graphic below, where we request a 1:1 aspect ratio output from a 4:3 aspect ratio input image -
Now, since cropping would take place unavoidably in these circumstances, if the subject of the image is not in the centre then it would get cropped out of the image.
For example, if I try to resize the image below to get a squared thumbnail, the sides of the image (the buildings) will get cropped away.
Smart Crop is a way of intelligently determining the most important part of the image and keeping it in focus while cropping the image. By using this feature you would be able to generate perfect thumbnails that are centred on the image's subject, or the most important feature with a high probability.
For example, for the image above, Smart Crop generates the following thumbnail:
As demonstrated, ImageKit's Smart Crop could intelligently determine that the buildings in the image are important and preserved them in the final thumbnail.
How do we determine the most important part of the image?
We take into account multiple factors while calculating the most important region of the image. The most obvious one being detecting a face (or multiple faces) in an image. If we find a face in the image, we try to centre the cropping on that face as closely as possible. If, however, we are not able to find a face in the image, we try to do a feature detection on the image. The centre of such non-facial features is then used to calculate the cropping dimensions on the original image.
Can we control how the cropping is done?
ImageKit offers two crop modes to use with smart cropping. One is the auto crop mode, and the other is the face crop mode.
In auto crop mode, we try to preserve as much of the image as possible, while centering on the most important part of the image. This is useful when you are trying to create an image gallery. Each thumbnail in that gallery will contain the most important part of the image. You need to use the "fo-auto" transform for this crop mode.
In the face crop mode, we try to extract out the largest face from the image, and the rest of the image is discarded. This is useful when you are trying to create profile pictures from a larger image. You need to use the "fo-face" transform for this crop mode.
How can I start using Smart Crop for images?
Like all other image transformations and optimizations in ImageKit, this feature too works in real-time by simply adding a parameter to the URL. In this case, we need to set the value of "focus" in the transformation string to "auto" (for auto crop mode) or "face" (for face crop mode). This means, you need to add fo-auto or fo-face to the transformation string of your thumbnail URLs.
For Smart Crop to work, you need to specify both, height and width of the output image. If you specify only the height or the width, then ImageKit maintains the aspect ratio without cropping. You can read more about different focus modes in our detailed documentation here.
If your normal thumbnail URL is
https://ik.imagekit.io/demo/img/tr:h-200,w-200/default-image.jpgTo enable Smart Crop with auto crop mode, this URL will change to
https://ik.imagekit.io/demo/img/tr:h-200,w-200,fo-auto/default-image.jpgTo enable Smart Crop with face crop mode, this URL will change to
https://ik.imagekit.io/demo/img/tr:h-200,w-200,fo-face/default-image.jpgExamples of Smart Crop in action (Auto-Mode)
Example 1
Example 2
Example 3
Example 4
Example 5
Example 6
Examples of Smart Crop in action (Face-Mode)
Example 1
Example 2
Example 3
Example 4
Best applications for this feature
Smart Crop will work well for any case where you need to create smaller images or thumbnails from a larger image, like creating user profile picture thumbnails from a larger profile picture using the face crop mode, or creating product image galleries where each image should have the product in the centre regardless of where it was in the original image.
Can I use the Smart Crop feature with other transforms
Of course, you can. You can combine smart cropping with any other transformation offered by ImageKit.
For example, you can combine smart cropping with responsive image tags to create a perfect responsive image experience across all devices for your users. You can also club it with a radius transform in a chain, to create rounded profile pictures of your users like this
https://ik.imagekit.io/demo/img/tr:w-200,h-200,fo-face:r-max/https://images.pexels.com/photos/874158/pexels-photo-874158.jpeg
Feedback related to Smart Crop
While we have tested the performance and results for a wide array of images, smart cropping may return inaccurate results for a small percentage of images. Smart cropping is a difficult problem to solve while keeping things near real-time. We are open to your feedback that will help us further improve the accuracy of the cropping algorithm. Please give it a try and do share your feedback in the comments below. Or write to us on developer@imagekit.io.
Like the idea? Want to try it out for yourself? You can try it for free!