🚀 Introduction

The data job market is booming—but with rapid evolution comes confusion. Whether you're an aspiring data scientist, an experienced analyst exploring your next step, or someone transitioning from tech support to machine learning, navigating this landscape can be overwhelming.

That’s where my capstone project, AI Career Coach for Data Professionals, steps in.

Using Google’s free tier Gemini GenAI models, I built a solution that:

Extracts and analyzes job trends from real postings,
Identifies the most in-demand hard and soft skills, and
Provides personalized career advice from a user’s resume using structured GenAI output.

💡 The Use Case: Solving Career Confusion with Objective Clarity

Even seasoned professionals struggle with questions like:

“What skills am I missing for a promotion?”
“What tools are most in demand in my field?”
“How do I transition to a more advanced role in AI or data engineering?”

Traditional job boards give you unstructured chaos.

My solution turns it into actionable insights with a GenAI-powered career coach.

🤖 GenAI Capabilities Used

This project integrates three core GenAI capabilities:

1. 📄 Document Understanding

Using a dataset of 50 job postings Data Science roles (small sample size for efficiently, resource and time constraints in this capstone project however much larger samples could be used) we parse unstructured job descriptions to extract the most relevant hard and soft skills using Gemini-2.0-flash-lite:

          prompt = f""" 

          Extract the hard and soft skills from the following job description. 

          Format the result as a JSON object with keys: 'Hard Skills' and 'Soft Skills'.

          Job Description: 

          {job_description} 

          """

🔧 Parse job descriptions to extract in-demand skills: Top Skills and Frequency

🎯 Top Hard Skills:
  Python: 17
  SQL: 15
  Machine Learning: 8
  Data Analysis: 8
  Data analysis: 7
  Deep learning: 7
  Artificial Intelligence (AI): 6
  Machine Learning (ML): 6
  R: 6
  TensorFlow: 6

💡 Top Soft Skills:
  Collaboration: 41
  Problem-solving: 34
  Teamwork: 24
  Communication: 18
  Adaptability: 13
  Innovation: 12
  Leadership: 11
  Critical thinking: 9
  Project management: 7
  Decision-making: 7

📈 Additionally these insights are visualized using matplotlib and seaborn to help users quickly understand market demands.

2. ⚗️ Embeddings + Vector Search

To find the listings most similar to the skills described in the resume text, we use Gemini model text-embedding-004 and ChromaDB to embed text job descriptions and cluster them based on cosine similarity, revealing the top matches in job types and semantic similarity to a user’s resume:

3. ✍️ Few-Shot Prompting with Structured Output

The AI Career Coach uses structured prompts to analyze a resume and return career recommendations in clean JSON using both Gemini-1.5-pro-002 and Gemini-2.5-pro-exp-03-25 responses:

    ```json
    {
      "Current Role": "Senior Technical Support Engineer with extensive experience in troubleshooting complex systems and a strong recent focus on developing data-focused skills.",
      "Suggested Next Roles": [
        "Data Engineer",
        "Cloud Data Engineer",
        "Machine Learning Engineer",
        "Data Analyst"
      ],
      "Missing Skills": [
        "Strong experience with building and maintaining data pipelines in a production environment.",
        "Deep understanding of data warehousing concepts and experience with cloud-based data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).",
        "Practical experience with deploying and managing machine learning models (if targeting ML Engineer role).",
        "Proven experience in data visualization and storytelling for business stakeholders (if targeting Data Analyst role)."
      ],
      "Learning Resources": [
        "For Data Engineering: Datacamp's Data Engineer career track, AWS Certified Data Analytics - Specialty certification.",
        "For Cloud Data Engineering: Google Cloud Professional Data Engineer certification, Azure Data Engineer Associate certification.",
        "For Machine Learning Engineering: Andrew Ng's Machine Learning course on Coursera, TensorFlow Developer Certificate.",
        "For Data Analyst:  Tableau Desktop Specialist certification, Microsoft Power BI certification."
      ]
    }

```

Additionally the AI Career Coach currently returns tailored guidance in a downloadable .pdf format, as well as displayed in beautiful Markdown like this:

🎯 AI Career Coach: Personalized Career Advice for Data Professional

Current Role:
Senior Technical Support Engineer with extensive experience in troubleshooting complex systems and a strong recent focus on developing data-focused skills.

🔼 Suggested Next Roles:
Data Engineer, Cloud Data Engineer, Machine Learning Engineer, Data Analyst

🧩 Missing Skills:

Strong experience with building and maintaining data pipelines in a production environment.
Deep understanding of data warehousing concepts and experience with cloud-based data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
Practical experience with deploying and managing machine learning models (if targeting ML Engineer role).
Proven experience in data visualization and storytelling for business stakeholders (if targeting Data Analyst role).

📚 Learning Resources:

For Data Engineering: Datacamp's Data Engineer career track, AWS Certified Data Analytics - Specialty certification.
For Cloud Data Engineering: Google Cloud Professional Data Engineer certification, Azure Data Engineer Associate certification.
For Machine Learning Engineering: Andrew Ng's Machine Learning course on Coursera, TensorFlow Developer Certificate.
For Data Analyst: Tableau Desktop Specialist certification, Microsoft Power BI certification.

No more guessing—just clear next steps.

🛠️ Technical Architecture

Gemini API via google.generativeai
Python + Pandas for data processing
Matplotlib / Seaborn for skill visualization
Notebook output saved as both JSON and Markdown
Sleep throttling and retry logic for API quota handling
All the models used are part of the Google GenAI offerings accessible through Vertex AI:
- gemini-2.0-flash-lite
- text-embedding-004
- gemini-1.5-pro-002
- gemini-2.5-pro-exp-03-25

⚠️ Limitations

Rate Limits: Free Gemini API plans restrict the number of requests per minute. This was handled via time.sleep(4) per job description and retry logic.
Extraction Accuracy: While GenAI is powerful, its output may vary slightly across runs and sometimes require post-processing (e.g., removing ```json blocks).
Resume Variability: Different formats might yield inconsistent advice—structured resumes work best.

🌱 Future Possibilities

There are so many ways to leverage the methods and tools applied here to expand the concept’s usefulness. Here are a few ideas I am considering:

✅ Update database with more recent job postings and return live hyperlinks to matches
✅ Add RAG (Retrieval-Augmented Generation) to cross-reference job postings with latest industry reports
📺 Build out a web app where users upload their resume and get live recommendations
🧠 Integrate career personas and long-term roadmap suggestions (e.g., 6-month, 1-year goals)
✅ Integrate hyperlinks directly to learning resources
🔁 Automate weekly skill trend digests for subscribers
✅ Update with newer google cloud-ai VertexAI models
📈 Write a Bonus: Career Path Timeline using structured JSON to draw a visual timeline (e.g., matplotlib or plotly)

📣 Final Thoughts

The AI Career Coach capstone project shows how GenAI isn’t just for chatbots—it’s a career partner. By blending document understanding, structured output, and semantic search, we created a tool that truly empowers professionals to take charge of their career path.

💬 I'd love to hear your thoughts and feedback! Would you use this tool in your job search?

🥇 Built for the Google GenAI Capstone Competition

This blog accompanies my submission for the Google GenAI on Vertex AI Capstone Challenge, showcasing real-world value of GenAI across structured reasoning, analysis, and personalized output.

👉 If you found this interesting, please consider supporting my project on Kaggle or sharing this post!

📎 Resources

💻 View the Full Notebook on Kaggle

Gen AI Intensive Course Capstone 2025Q1

🤖 AI Career Coach 💼: Using GenAI to Decode Data Job Trends and Give Personalized Career Advice

Table of contents