Product Analyst - AI

About the job
Position: Product Analyst – AI
Location: Remote (India)
Experience: 0–2 Years
Type: Full-time

"Is this AI response actually correct? Is it helpful? Did it hallucinate? Is the research thorough? Is the code it generated safe to use? Someone has to answer these questions before our customers see it. That someone is you."
You'll be the human-in-the-loop for enterprise AI.

About Katonic
We are a Sovereign Enterprise AI Company. Founded in Sydney in 2020, we've grown into a profitable global operation powering AI infrastructure for enterprises and governments across 11 countries. Our platform runs entirely within customer infrastructure - zero data egress, zero vendor lock-in.

Our platform - 250+ AI models, 80+ pre-built agents, ISO 27001 certified. Used by enterprises who report up to 80% increase in workflow efficiency.

Role Overview
We're hiring 2 Product Analysts for our ACE team. Your job is to evaluate AI - ask it questions, test its research capabilities, check if it hallucinates, verify its code works, and ensure image generation is appropriate. You're the quality bar for AI outputs.

This role is perfect if you:
Are curious about AI and use tools like ChatGPT or Claude
Have strong critical thinking skills
Can spot when something is wrong, even if it sounds right
Write clearly and can explain complex issues simply
Care about accuracy and quality

What You'll Evaluate
ACE is our enterprise AI assistant. Here are its capabilities - all of which you'll test:
Ask AI: Conversational AI that answers questions from enterprise data
Knowledge Base Search: RAG-powered search across company documents
Web Browse: Real-time web search and summarization
Deep Research: Comprehensive research reports with citations
Code Generation: Write code in Python, SQL, JavaScript, and more
Image Generation: Create images from text descriptions

What You'll Do
• Ask AI questions across different topics and evaluate response quality
• Test Knowledge Base Search - upload documents, ask questions, verify answers are accurate
• Evaluate Deep Research outputs - are citations real? Is information current? Is analysis sound?
• Test Web Browse - does it find relevant information? Are summaries accurate?
• Review Code Generation - does the code run? Is it safe? Does it do what was asked?
• Evaluate Image Generation - is output appropriate? Does it match the prompt?
• Identify hallucinations - when AI makes up facts, names, or citations
• Test edge cases - confusing questions, ambiguous prompts, adversarial inputs
• Document issues with clear examples and expected vs actual behavior
• Suggest prompt improvements and guardrail recommendations

Who You Are

Must Have:
0-2 years experience (fresh graduates welcome)
No coding required - this is a non-technical role
Strong critical thinking and analytical skills
Excellent written communication - you can explain what's wrong clearly
Experience using AI tools (ChatGPT, Claude, Gemini, Copilot - any of them)
Attention to detail - you notice when facts don't add up
Curiosity - you'll ask "is this actually true?" constantly
Comfort reading and evaluating different types of content

Nice to Have:
Background in research, journalism, or content writing
Experience in consulting, analysis, or fact-checking roles
Basic understanding of how AI/LLMs work (conceptual, not technical)
Familiarity with enterprise software or B2B products

What You'll Become In 12 months:
Deep understanding of how LLMs work - strengths, limitations, failure modes
Expertise in AI quality evaluation and prompt engineering
Knowledge of enterprise AI use cases across industries
Skills in AI safety, guardrails, and responsible AI
Experience shaping AI products used by Fortune 500 companies
Foundation for roles in AI Product Management, AI Safety, or Trust & Safety

AI evaluation is a new and growing field. Get in early.

Soft Skills
Problem-solving mindset
Strong communication skills
Ownership and accountability
Ability to learn fast and adapt to new technologies

What we offer
Opportunity to work at the forefront of Generative AI and Agentic AI
Fully remote - work from anywhere in India
Health insurance
Learning budget for courses and certifications
Global exposure - collaborate with teams in Sydney, Singapore, Dubai

Please apply only if you match the criteria.
To apply, please fill out this form: https://shorturl.at/vfuds
Without filling out the form, your application is not complete.

Katonic AI is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all.


Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Research, Analyst, and Information Technology
Industries
Software Development