Material Segmentation and Recognition in Cluttered Scenes

Research areas

Temporary Supervisor

Dr Cong Phuoc Huynh


Revealing object and surface materials in natural environments is beneficial to a broad range of applications such as robotic manipulation and navigation, visual quality control in manufacturing and waste sorting. For example, knowing an object’s material, a robot could adjust its grasping strategy according to the friction of its surface. For this reason, material categorisation from real-world images has attracted growing interests from both the theoretical and practical points of view. In this project, we aim to design and develop a method for material detection and classification in natural images. In practice, the robustness of such a method is challenged by variations presented real-world environments such as illumination conditions, viewpoints and occlusions. Recently, computer vision researchers have contributed a number of benchmark databases for material classification in the wild, such as the CUReT [1], KTH-TIPS [2], Flickr Material Dataset (FMD) [3], UIUC [4], UMD [5], Outex [6], and Drexel Texture Database [7]. However, these datasets are not realistic as a single material usually fills the entire image. To support applications of material recognition in realistic scenarios, there is a need for a dataset of images, each of which capturing multiple materials in cluttered backgrounds. The construction of such a dataset would open up opportunities to examine the extent of applicability of existing algorithms for material detection and classification in the wild.


• A real-world image dataset for material segmentation and classification. The images should be composed of multiple materials in a cluttered background. • Experimentation and performance reporting for the newly built dataset with combinations of existing segmentation and material classification methods.


This project is suitable for candidates with a strong undergraduate background in mathematics, computer science and software/computer engineering. The following skills are essential for a successful undertaking of the proposed project • A strong foundation in mathematics (linear algebra, calculus and optimisation). • Familiarity with computer vision, pattern recognition and image processing (desirable). • Good software design and programming skills (especially in C++ and Matlab).

Background Literature

1. 1. K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink. Reflectance and texture of real world surfaces. ACM Transactions on Graphics, 18(1):1–34, 1999. 2. B. Caputo, E. Hayman, and P. Mallikarjuna. Class-specific material categorisation. In ICCV, 2005. 3. L. Sharan, R. Rosenholtz, and E. H. Adelson. Material perception: What can you see in a brief glance? Journal of Vision, 9:784(8), 2009. 4. S. Lazebnik, C. Schmid, and J. Ponce. A sparse texture representation using local affine regions. PAMI, 28(8):2169–2178, 2005. 5. Y. Xu, H. Ji, and C. Fermuller. Viewpoint invariant texture description using fractal analysis. IJCV, 83(1):85–100, June 2009. 6. T. Ojala, M. Pietikainen, and T. Maenpaa. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. PAMI, 24(7):971–987, 2002. 7. G. Oxholm, P. Bariya, and K. Nishino. The scale of geometric texture. In European Conference on Computer Vision, pages 58–71. Springer Berlin/Heidelberg, 2012.


The student will gain a practical experience through working on real-world research problems with experienced researchers in the areas of computer vision and pattern recognition. He/she will receive technical support in terms of hardware equipment/software for data collection and processing. The student will gain the working knowledge applicable to the above and other related real-world applications.

Updated:  1 June 2019/Responsible Officer:  Dean, CECS/Page Contact:  CECS Marketing