Inspiration
The inspiration stemmed from a passion for promoting health and wellness through food. We were driven by the desire to address the prevalent lack of accessible and easy-to-understand nutritional information. It's a common struggle for people to decide what they can make with the ingredients that they have at home, oftentimes letting groceries they bought earlier in the week or month go bad in favor of the convenience of ordering food to their doorstep. With our app, we aid in minimizing food waste by giving people tasty and easy options to finish up what's left in the fridge, not letting a penny or ingredient go to waste. For many of us college students, leaving the comfort of home-cooked meals and having to sustain ourselves has left us making unhealthy and often not financially smart food decisions rather than cooking, so as to "save ourselves the hassle." But had we had an app like RecipeBud, we're sure that more students like us could see themselves buying groceries and cooking at home. Not only does this save us the extra buck, but contributes to a healthier lifestyle.
Furthermore, our team is formed of immigrants or children of immigrants. We are no stranger to the real struggles that come with learning another language and trying to place the names for common ingredients that come so naturally to us in our mother tongue, which is even tougher to spell out. Thus, our image-recognition software transcends and overcomes linguistic boundaries such as these so that we all may be able to participate in the universal language that is a comforting home-cooked meal.
What it does
RecipeBud is a culinary game-changer, seamlessly melding image recognition with nutritional insights. This innovative web application lets users snap ingredient pics, swiftly identifying them and delivering a spectrum of recipes with comprehensive nutritional profiles. Serving as a personalized nutritionist, it guides users in making informed meal decisions. Offering a diverse array of recipes while highlighting nutritional benefits, RecipdBud champions a healthier lifestyle for a more vibrant, inclusive community.
How we built it
We engineered our web app using Python and Streamlit, skillfully combining OpenAI's cutting-edge GPT-3.5 model with Microsoft's Resnet-50 model from Hugging Face. Furthermore, we made use of Edamam's comprehensive Food Nutritional Database API to ensure nutritional accuracy. Along with this, we harnessed the power of OpenCV for efficient image capture. Our app seamlessly marries AI-generated responses with accurate food image recognition, providing users with an enriched and insightful culinary experience
Challenges we ran into
Integrating two Language Models was challenging. We hosted a Microsoft image-recognition model locally alongside accessing OpenAI's API, which was challenging: while the local model had quick responses, constantly making API requests was expensive in terms of time. Streamlining this timing discrepancy was challenging. OpenAI's token limit sometimes resulted in incomplete outputs, making testing difficult. However, to be able to explore this field of engineering over the course of one weekend in the depth that we did was ultimately extremely rewarding despite the difficulties we faced. Additionally, as first-time users of Streamlit, navigating it initially was tricky. While Streamlit simplifies many aspects, customizations that may seem simple, like just changing the color of a button, posed complexities, as Streamlit does not support applying custom CSS styles to its element using external CSS files. However, the interface that resulted was elegant and extremely easy to interact with, which would not have been possible without Streamlit's functionalities and features.
Accomplishments that we're proud of
We take pride in having successfully developed a functional application that significantly contributes to users' journey towards a healthier, more nutritious lifestyle. This reinforces our commitment to making a positive impact on their well-being. In this day and age, due to advertising and toxic media, many people's relationships with food are becoming more and more negative and destructive. Furthermore, this age of fast food and cheap, low-quality foodstuffs has drastically affected our population's overall health. We know that RecipeBud will be of help to people who want to take steps in the right direction, one recipe at a time.
What we learned
Developing RecipeBud has been a tremendous learning experience. We gained valuable insights into effectively integrating multiple large language models into a cohesive application. Navigating the intricacies of OpenAI's API, as well as hosting a machine-learning model locally considering model's bias and weight storage, and managing its responses taught us important lessons in error handling and optimization. Additionally, our foray into Streamlit exposed us to the potential and challenges of rapidly building a user-friendly interface. We now have a better understanding of the nuances of balancing simplicity with customization in app development. Overall, RecipeBud has been a rich source of knowledge, preparing us for even more ambitious projects in the future.
What's next for RecipeBud
Our journey doesn't end here; in fact, it's just the beginning. We're diving into the future, brainstorming innovative ideas to take this app to the next level. One crucial aspect we're focusing on is enhancing accessibility for non-English speakers, ensuring our platform caters to a broader audience. Additionally, we're exploring automation features like seamless image capture and intuitive tools for comparing nutritional visuals across various recipes. We also hope to incorporate audio interaction, and for recipes to be played back for users to listen to as they are cooking for a more hands-free, immersive experience. These advancements aim to empower users, making informed, health-conscious choices effortlessly and effectively. Food is a universal language, as we've said before. It brings people together. We hope to also train our own model with data on recipes from around the world specific to less represented cuisines so that a person from one corner of the world can have a taste of the other side of an ocean or spanning across many continents, from the comfort of their own kitchen.