Improving Melanoma Pathology Accuracy through Computer Vision Techniques
The IMPACT Study

This research is supported by the National Cancer Institute grant R01 CA200690, PI: Dr. Joann Elmore.

In this project, novel computational techniques are being used to analyze digitized slides for the purpose of assisting in the pathologic diagnosis of melanoma and related skin lesions. These techniques include the detection of both cellular-level and architectural features for use in feature-based classification, and exploration of deep neural networks that operate on raw pixel data for the difficult task of mitosis detection. In addition, a machine learning approach will be applied to the digitized slides to determine the histopathological characteristics associated with human diagnostic errors.

People:

Joann Elmore	Linda Shapiro	Shima Nofallah	Meredith Wu	Beibin Li	Sachin Mehta	Ximing Lu

Donald Weaver	Oliver Chang	Caitlin May	Stevan Knezevitch

Recent Foundational Model Work

• QUILT paper--NeurIPS 2023
• Quilt-LLaVA paper--CVPR 2024

Publications:

• Liu, K., Wu, W., Elmore, J.G., Shapiro, L.G. (2024). ``Semantics-Aware Attention Guidance for Diagnosing Whole Slide Images," In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. Lecture Notes in Computer Science, vol 15005. Springer, Cham. https://doi.org/10.1007/978-3-031-72086-4_56.

• K. Liu, B. Li, W. Wu, C. May, O. Chang, S. Knezevich, L. Reisch, J. Elmore, L. Shapiro, "VSGD-Net: Virtual Staining Guided Melanocyte Detection on Histopathological Images," IEEE Winter Conference on Applications of Computer Vision, 2023.

• S. Nofallah, W. Wu, K. Liu, F. Ghezloo, J. G. Elmore, L. G. Shapiro, "Automated Analysis of Whole Slide Digital Skin Biopsy Images," Frontiers in Artificial Intelligence, 2022.

• S. Nofallah, B. Li, M. Mokhtari, W. Wu, S. Knezevich, C. J. May, O. H. Chang, J. G. Elmore, L. G. Shapiro, "Improving the Diagnosis of Skin Biopsies Using Tissue Segmentation," Diagnostics, Vol. 12, 2022.

• F. Ghezloo, P. Wang, K. F. Kerr, T. T. Brunyé, T. Drew, O. H. Chang, L. Reisch, L. G. Shapiro, J. G. Elmore, ``An Analysis of Pathologists' Viewing Processes as They Diagnose Whole Slide Digital Images," Journal of Pathology Informatics, 2022. PMCID: PMC9576972

• S. Nofallah, M. Mokhtari, W. Wu, S. Mehta, S. Knezevich, C. J. May, O. H. Chang, A. C. Lee, J. G. Elmore, L. G. Shapiro, ``Segmenting Skin Biopsy Images with Coarse and Sparse Annotations using U-Net," Journal of Digital Imaging, April 2022.

• K. Marathe, C. Marasinou, B. Li, N. Nakhaeib, Bo Li, J. G. Elmore, L. Shapiro, W. Hsu, ``Automated quantitative assessment of amorphous calcifications: Towards improved malignancy risk stratification," Computers in Medicine and Biology, 2022.

• W. Wu, S. Mehta, S. Nofallah, S. Knezevich, C. J. May, O. H. Chang, J. G. Elmore, L. G. Shapiro, "Scale-Aware Transformers for Diagnosing Melanocytic Lesions," in IEEE Access, vol. 9, pp. 163526-163541, 2021, doi: 10.1109/ACCESS.2021.3132958.

• S. Mehta, N. Nuechterlein, E. Mercan, B. Li, S. Nofallah, W. Wu, X. Lu, A Caspi, M. Rastegari, J. Elmore, H. Hajishirzi, L. Shapiro,"Applications of the ESPNet architecture in medical imaging," in State of the Art of Neural Networks and their Applications, Vol. 1, 2021, pp. 117-131, Academic Press.

• X. Lu, S. Mehta, T. Brunye, D. Weaver, J. G. Elmore, L. G. Shapiro, "Analysis of Regions of Interest and Distractor Regions in Breast Biopsy Images," IEEE Conference on Biomedical and Health Informatics, 2021.

• K. Liu, M. Mokhtari, B. Li, S. Nofallah, C. May, O. Chang, S. Knezevitch, J. Elmore, L. Shapiro, "Learning Melanocytic Proliferation Segmentation in Histopathology Images from Imperfect Annotations," IEEE Workshop on Computer Vision for Microscopy, 2021.

• S. Nofalla, S. Mehta, E. Mercan, S. Knezevich, C. J. May, D. Weaver, D. Witten, J. G. Elmore, L. Shapiro, "Machine Learning Techniques for Mitoses Classification," Computerized Medical Imaging and Graphics, Vol. 87, 2021.

• M. A. Jafry, S. Peacock, A. C. Radick, H. L Shucard, L M. Reisch, M. W. Piepkorn, S. R. Knezevich, M. A. Weinstock, R L. Barnhill, D. E. Elder, K. E. Kerr, J. G. Elmore,"Pathologists' Agreement on Treatment Suggestions for Melanocytic Skin Lesions", J Am Acad Dermatol., 2020 Jun;82(6):1435-1444. doi: 10.1016/j.jaad.2019.12.020. Epub 2019 Dec 17. PMID: 31862403.

• C. I. Lee, N. Houssami, J. G. Elmore, D. S. M. Buist, "Pathways to Breast Cancer Screening Artificial Intelligence Algorithm Validation," The Breast, Vol. 52, 2020.

• J. G. Elmore, D. E. Elder, R. L. Barnhill, S. R. Knezevitch, G. M. Longton, L. J. Titus, M. A. Weinstock, M. S. Pepe, H. D. Nelson, L. M. Reisch, A. C. Radick, M. W. Piepkorn, "Concordance and Reproducibility of Melanoma Staging According to the 7th vs 8th Edition of the AJCC Cancer Staging Manual," JAMA Network Open, 2018.

• T. Onega, R. L. Barnhill, M. lW. Piepkorn, G. M. Longton, D. E. Elder, M. A. Weinstock, S. R. Knezevich, L. M. Reisch, P. A. Carney, H. D. Nelson, A. C. Radick, J. G. Elmore, "Accuracy of Digital Pathologic Analysis vs Traditional Microscopy in the Interpretation of Melanocytic Lesions," JAMA Dermatology, 2018.

• Presentation at 2023 REMI Meeting

• ICPRAM Mitosis Paper by Y. Li et al.

• Presentation at UBC 2020

• ITCR 2020