Application of Zero-shot object detection (ZSOD) Models to Image-Based Construction Documents - QC-729

Project type: Research
Desired discipline(s): Engineering - other, Engineering, Computer science, Mathematical Sciences
Company: BuildCheck AI, Inc.
Project Length: 4 to 6 months
Preferred start date: As soon as possible.
Language requirement: Flexible
Location(s): Montreal, QC, Canada; Vancouver, BC, Canada; Canada
No. of positions: 1
Desired education level: Master'sPhDPostdoctoral fellow
Open to applicants registered at an institution outside of Canada: Yes

About the company: 

BuildCheck AI’s mission is to automate tedious design compliance and coordination processes. Specifically, BuildCheck AI uses advance machine-learning algorithms to review specifications and drawings against contractual and regulatory requirements, flag non-compliant and vague items, offer value-engineering opportunities, and make useful recommendations. BuildCheck AI is the "spell-check" for project design. In the medium term, we see ourselves as the ubiquitous, AI-powered design and pre-construction management platform for the industry. We dream of a world where design professionals have one screen open on BuildCheck, the other on AutoCAD/REVIT, and their emails, Microsoft Excel, digital code books, standards and the like minimized in a distant tab.  BuildCheck was founded by three Stanford graduates in March 2023.

Describe the project.: 

What is the project about?

The project focuses on harnessing the latest zero-shot computer vision models to analyze image-based documents, such as construction drawings, shop drawings or specifications.

What is the main goal of the company (a final product, software, knowledge in a specific area, etc)?

The company's ultimate objective is to integrate a performing zero-shot computer vision model into its product to help quickly recognize objects that were not included in the dataset on which our machine-learning models were trained. The additional capability afforded by the zero-shot computer vision models will expedite the review of the construction documents.

What are the main tasks to be performed by the candidate?

• Advice: Provide insights on the required data labeling, model training, and define key performance metrics for such system.
• Design: Define the system's architecture, determining the essential components and how they will interact with one another.
• Model Development, Selection and Training: Choose an appropriate model architecture and train it to understand and process image-based construction documents effectively.
• Prototype: Build the necessary system components.

What methodology/techniques are to be used?

The project will leverage cutting-edge tools and techniques such as Grounding DINO, CLIP, ZSD-SCR, RRFS-ZSD, etc.

Required expertise/skills: 

The intern will play a crucial role in advancing our platform's capabilities through cutting-edge data science and computer vision techniques.  

Key expertise and skills required:

The ideal candidate for this research project should possess a strong foundation in computer vision skills coupled with proficiency in machine learning techniques. A robust understanding of advanced computer vision concepts, including zero-shot object detection, instance segmentation, and feature extraction, is crucial. Proficiency in programming languages like Python and experience with popular deep learning frameworks such as TensorFlow or PyTorch are essential for developing and fine-tuning the required zero-shot computer vision models.

Hands-on experience in image preprocessing, augmentation, and data manipulation will be pivotal in ensuring the accuracy and reliability of the models. The intern should be adept at translating theoretical computer vision principles into practical implementations that align with BuildCheck AI's vision of automating design compliance and coordination processes.

While a background in the construction industry is beneficial, a strong emphasis on computer vision skills takes precedence. The intern's ability to creatively tackle complex image-based document analysis challenges and adapt cutting-edge techniques to the unique context of construction documents will be key to the success of the project.