Presentation Title: Amenity Detection and Inventory Tracking using Detectron2
Presenter(s): Anisha Rao & Lina Louis
Abstract: Airbnb is a marketplace that allows hosts to upload images, list amenities, and provide location and pricing information. Guests are inquisitive about the amenities when choosing a stay. However, inventory and property tracking can be improved to mitigate risks and ensure accuracy. Current research has shown positive results in tasks like production and supply chain management. This project proposes the enhancement of Airbnb’s hosts’ and guests’ experiences by leveraging Detectron2 and implementing image and live video-based amenity detection and inventory tracking. Detectron2 model is trained using a custom dataset from OpenImagesV7, focusing on Airbnb relevant classes. Preprocessing, augmenting and synthesis of the dataset is done to enable quality and diversity. The Detectron2 object detection framework is used, configuring the architecture, selecting appropriate backbone networks, and fine-tuning hyperparameters. An improved model is deployed by overlaying predictions from the custom and pre-trained models, enhancing accuracy and performance. It also includes an inventory management system to track amenities in properties. Instance counting ensures effective inventory tracking mechanisms. Qualitative appraisal of annotations improves labeling, and a chatbot linked to the inventory database enhances user satisfaction. The deployed model ensembles, a custom Faster-RCNN with a pre-trained version. The tracker enhances host and guest experiences by identifying remaining supplies, voids, and mislaid items for restocking and accurate space availability. Hosts benefit from property maintenance support and quick property damage claim resolution. Guest View provides a user-friendly interface for guests to explore and visualize amenities, improving the booking experience leading to an efficient and sustainable Airbnb service.
Link to Recorded Presentation: https://www.youtube.com/watch?v=LuxwRh4hdJg
I follow machine learning innovations and liked the way you guys have used technique to ensemble computer vision models. It’s new and something that I’d like to try.
Thank you! We’re glad you’re interested in our ensemble computer vision models. Innovation drives us, and we’re happy to share more about our techniques. Let us know if you have any questions!
You’ve done a fantastic job integrating Generative AI and Large Language Models into the Airbnb application. This innovative approach will undoubtedly simplify the process for both hosts and guests, making it easier to manage and select properties with minimal effort.
Thank you for your feedback! We’re glad you find our integration of Generative AI and Large Language Models helpful.
I have query regarding your application. Can we use this approach to find cracks in the construction or any such domain?
Hi, sure it can be done if we curate a relevant dataset for the model to be trained on. Since it’s a specific domain such as construction the model can be fine tuned to detect different type of defects. One of the papers we referred did something similar for pothole detection. Hope this helps!
Hi, sure this can be done. The model needs to be trained on a dataset curated for this specific detection. Since it is a specific domain such as construction the results can be even better to detect different type of defects. A paper we referred fine tuned a model to detect potholes. Hope this helps!
This is an interesting concept for creating an ensemble model when the metadata is different.
Can this be used for non CV models?
Yes, the concept of ensemble modeling with different metadata can be applied to non-computer vision models as well. By combining predictions from different models, such as regression or classification, we can enhance performance. The key idea is to leverage the diversity of models to capture different aspects of the data and reduce the risk of overfitting. By combining the strengths of multiple models, we can often achieve better generalization performance and robustness. Hope this helps!
I enjoyed watching your presentation and learning how CV libraries can help reduce manual work of Airbnb hosts. If this feature could be integrated, I’m sure it would benefit hosts like myself. Thank you for sharing your research!
Thank you for your kind words! I’m glad you found the presentation informative.