Before GSoC

Introduction

I am Samia Haque Tisha, a recent graduate in Software Engineering from Daffodil International University. I am passionate about Computer Vision, AI Agents, and open-source contributions. I am currently working on integrating Large Language Models (LLMs) with the DeepForest library for ecological analysis, which is a Python package for training and predicting ecological objects in airborne imagery.

How I Chose Organization and Project

At first, I reached out to some GSoC alumni and took notes on how they contributed. Following their instructions, I began by listing organizations and reviewing their projects and contributions from previous years. I created a tracker like this one. Anyone interested can also make a copy from here. I checked their current projects and proposals as well. My purpose was to choose a proposal that closely relates to my current goal.

For my undergraduate thesis, I worked on detecting congested regions in traffic and implemented a methodology to do so. I used an object detection model to detect vehicles and pedestrians, then used the object counts and other techniques to identify congested areas. Later, I decided to integrate an LLM with the system. I’ve been working on making it more generalizable, so that any object detection model can be plugged into my methodology along with the LLM. The system would then detect congestion, provide insights on the image, and allow interactive conversations about it. While looking for similar proposals, I came across one that also involved integrating an LLM with the DeepForest pre-trained object detection model. I thought it was an excellent opportunity.

Contribution, Communication and Crafting Proposal

I started by reading the README, documentation, and contribution guidelines. I also followed the contributions of some GSoC alumni for Pull Request formats. While doing that, I fixed small issues like broken links and minor documentation errors to get more comfortable with pull requests and contributions. Here are a few of my PRs: #963, #970, and #1017. I also explored the codebase and “good first issues” and began working on them. For my first pull request, I didn’t include test cases or documentation updates because I wasn’t aware of their importance. But with guidance from my mentor, I understood the expectations, updated the PR accordingly, and was listed as a co-author. In PR #976, I initially messed up in rebasing, but with support from my mentor, another contributor, and some thorough research, I learned how to rebase properly. That experience taught me a lot before the GSoC application, and it was a very fun experience. You can find more of my pull requests here.

After making sufficient contributions, I started discussing the proposal with my mentors. Initially, they gave me some excellent ideas, like resurrecting historical images. Later, they encouraged me to just focus on any LLM integration with the DeepForest model as a starter. I began implementing demo versions of my proposal to see how things worked. Initially, I used the DeepForest Bird Detector and the Gemini API. I utilized some prompt engineering where I passed the object detections from the Bird Detector, the query and the full image on the frontier model “gemini-2.5-pro-exp-03-25.“ I included these demos and some code snippets in my application. Even after submitting the application, I continued contributing. Besides these contributions, I have also been actively exploring other multimodal approaches along with reading relevant research papers to discover unique and innovative possibilities for this proposal.

More information on my project can be found here: https://summerofcode.withgoogle.com/programs/2025/projects/ItYAp7By